Newniks: Tips for new Unix users

• Background processing
• Command-line editing
• Cygwin & MinGW
• Debugging
• Environment variables
• File management
• Fonts
• Linking
• Login scripts
• Make
• Network setup
• Regular expressions
• Scripts
• Shells
• System administration
• Tar
• Telnet & ftp
• VMS/Unix equivalents
• Wildcards & patterns
• X windows

The term Unix is used here to refer to operating systems in the family of UNIX, OSF, GNU/Linux, FreeBSD, Mac OS X, etc.

The Art of Unix Programming by Eric Steven Raymond (2003) is a very interesting overall introduction to Unix.

This page is based on information that I started putting together for myself in 1991, as I started learning Unix (and csh) after years of using VMS. It started as a simple list of Unix equivalents for VMS commands but has grown somewhat beyond that. It is still intended mainly as an aide-mémoire for myself. For historical reasons, there may remain an indiscriminate mix of OSF and Linux, and of the csh and bash shells.

The sections are intended to be in order of increasing sophistication of the task. At the right is a table of contents with the section names sorted alphabetically. For Unix under MS Windows, see Cygwin & MinGW below.

Cautionary note

1. Command-line editing

See 'man cmdedit'. Non-default key bindings are defined in ~/.bindings

2. Wildcards and patterns

The pattern matching implemented in shells has some similarities to, but is not the same as, the use of regular expressions.

2.1 Pattern matching in the bash shell

Simple wildcards:

• * matches any string (including the null string)
• ? matches any single character
• [...] matches any one of the enclosed characters, with special meaning for - (range) and ! or ^ (not), and other fancy things

Use the built-in bash command shopt -s extglob to turn on extended patterns. Then, for example, ls *.*(cpp|h|ui) will list all .cpp, .h and .ui files.

2.2 Regular expressions

grep uses regular expressions. By default it understands basic regular expressions. To use extended regular expressions, use grep -E or egrep.

For example, to find lines of the file test.txt that contain either ‘aaa’ or ‘bbb’, use the command grep -E 'aaa|bbb' test.txt

3. File management

3.1 Making and deleting directories and files

If using the command line rather than a GUI:

• mkdir to make a new directory within the current directory
• rmdir to delete (remove) a directory
• rm to delete (remove) a file

3.2 Directory listings

• ls for command-line directory listings. Its sorting is controlled by LC_COLLATE; normally it is case sensitive, and sorts the numeric parts of file names as ordinary character strings.
• Nautilus is the GNOME file manager. Its sorting is case insensitive, and it sorts the numeric parts of file names in numerical order rather than as character strings (e.g., x9.tmp comes before x10.tmp).
• Konqueror is the KDE file manager. Its sorting can be either case sensitive or case insensitive; it sorts the numeric parts of file names as ordinary character strings. It can also act as a media player, Web browser, FTP client, etc.

3.3 Finding files

• find searches for files within a specified directory tree, and can perform various operations on the files that are found. The command takes various options, tests (e.g., file size, time of last file access, filename pattern or regular expression) and actions (e.g., print file name, execute command). For example, to find Fortran source files and search for a text string in each one,
  find ./ -name \*.for -exec grep -H string {} \;
• locate works from a database that is periodically updated, so it's faster than find. The database is updated by the programme updatedb, which may be run manually or, for example, by the find script in /etc/cron.daily/. If the given pattern contains a wildcard, then it should probably start with a wildcard to allow for the unknown directory path (e.g., locate *libglib*)

4. Background processing

4.1 Running after logout

The '&' at the end of a command provides background processing that continues after logout. The 'nohup' command can be used to ignore hangups, but is necessary only to ignore explicitly sent hangup signals, since the '&' effectively protects a process from being sent a hangup signal upon logout.

4.2 Reduced priority

The 'nice' command (a different 'nice' command is built into csh) can be used. However, it may not be necessary: in 'UNIX for VMS Users' (Bourne, 1990) it says that 'in most versions of Unix the shell automatically lowers the priority of processes running in background for a system defined period of time' (p. 818), although I haven't found further details about this. The 'renice' command can be used to lower the priority of an already-running job. Note the stupid convention that the 'priority' parameter increases with decreasing priority.

4.3 Scheduling

The at command provides scheduled running of a process. The related batch command runs jobs when the average system load level falls below some specified value. The jobs are run using the sh shell.

cron is a dæmon for executing scheduled commands. It searches its spool area (/var/spool/cron/crontabs) for crontab files, which are named after user accounts, and runs them; these crontab files should be set up using the crontab command. cron also processes the file /etc/crontab, which in turn invokes any files in /etc/cron.hourly/, /etc/cron.daily/, /etc/cron.weekly/ and /etc/cron.monthly/. (One can also put crontab files in /etc/cron.d/ but this is deprecated.)

4.4 Notification

Shell provides notification when background process terminates. Normally notification given just before system prompt reappears. The shell variable 'notify' (or the 'notify' command) will cause notification to be immediate.

5. Linking

Linking is the process of linking a compiled programme together with its libraries. It may also be referred to as building, and in Unix it is done by ld, the loader. In the case of shared libraries, the linking must take place both at link (build) time and at run (load) time.

The linker ld has a built-in search path for finding required libraries. The default search path seems to include only /lib/ and /usr/lib/. A directory can be added to the default search path by adding it to /etc/ld.so.conf and then running ldconfig. The command ldconfig -v can be used to see what libraries ld is aware of.

One can also specify additional search directories on the ld command line using the -L and -R flags.

There is also a mechanism involving the environment variables LD_LIBRARY_PATH (deprecated except for temporary kluges) and LD_RUN_PATH. See Russ Allbery's Shared library search paths for a good discussion.

The purposes of the various directories like /lib/ and /usr/lib/ are specified in the Filesystem Hierarchy Standard.

6. Debugging

The following are commonly used commands within dbx:

run	Run from beginning. Command-line arguments can be included as they would be when invoking the programme from the shell. Redirection of stdin and stdout can also be included in the usual way.
next (n)	Execute one line (do not enter procedures)
step (s)	Execute one line (do enter procedures)
stop in proc	Stop when procedure is called (set breakpoint)
stop at line	Stop when given line in current procedure is reached (set breakpoint)
delete all	Delete (disable) all breakpoints, trace events and record events
cont (c)	Continue
return	Execute until return from procedure
list (l) n1,nl	Lists lines in the current source file
print (p) expr	Print value of expression
where (t)	Display list of active procedures
use dir	Sets list of directories searched for source files (~ not supported, so must specify, e.g., /usr/users/name/src rather than ~name/src; separate paths by spaces)
quit (q)

7. Make

Running make recompiles only routines modified since the last make. In order to recompile everything, must use touch to fake the modification dates(!).

8. Environment & shell variables

8.1 Environment variables

The shell variables user, term, home & path are automatically mirrored in the environment variables USER, TERM, HOME & PATH.

8.2 Aliases

Environment variables & aliases may be def'd in .login or .cshrc

The mv and cp commands by default will overwrite existing files without warning. It is prudent to define aliases which invoke the -i flag, which causes a prompt before overwriting. With csh under Tru64 Unix, the -i in the alias can be overridden just by giving the command cp -f. With tcsh under Debian Linux, this doesn't seem to work, but the alias substitution can be avoided by giving the command \cp.

Note a nasty way of messing up using the cp command: if you say, e.g., cp *.dcl instead of cp *.dcl ., and if there are exactly 2 .dcl files, it will copy the first over the second without warning that the syntax is wrong.

9. Fonts

This sample set of font specifiers includes all combinations of serif or sans serif, medium or bold, upright (roman) or oblique (italic), and proportionally spaced or monospaced (fixed pitch):

For a serif font, italic is indicated by -i-; for a sans-serif font, oblique is indicated by -o-.

The DejaVu fonts are an extension of the very high-quality Vera fonts.

Use xfontsel, xlsfonts or xfd to explore available fonts.

10. Tar

Example of using tar to transfer files from VMS:

    On VMS:
      $ FUN TAR
      $ FUN TO GUT
      $ VMSTAR_U cvf GUT_FOR.TAR *.FOR
    On OSF:                     On VMS:
      % cd gut
      % ftp funvie                $ FTP PALACE
      ftp> cd fun_gut             FTP>SET DEF "gut"
      ftp> image                  FTP>SET TYPE IMAGE
      ftp> get GUT_FOR.TAR        FTP>PUT GUT_FOR.TAR
      ftp> quit                   FTP>EXIT
      % tar xvf GUT_FOR.TAR 

11. Scripts

A script is invoked by typing its name, like any other command.

A csh script must start with a comment (#) in order to be recognized when invoked directly. (It's unnecessary if it's invoked with a csh or source command.)

Script files must be made executable, e.g. chmod u+x name.

12. Shells

12.1 Alternatives

To find out which shell is being used, look at the $SHELL environment variable (e.g., echo $SHELL).

The following are alternative Unix shells. (References: John Shepherd's notes from 2004; Sven Mascheck's notes; Wikipedia article.)

Bourne-shell family
sh	Steve Bourne	early 70's	the original Unix shell	uses .profile
Rsh			Restricted version of Bourne shell
ksh	David Korn	early 80's	like sh + functions, history editing, ...
ash	Kenneth Almquist	late 80's	small and fast version of sh
zsh	Paul Falstad	late 80's	like ksh but many, many enhancements
bash	Ramey/Fox	early 90's	like ksh with extras; default Debian interactive shell	uses .bashrc & .bash_profile
dash	Herbert Xu, Debian	late 90's	POSIX; default Debian /bin/sh shell
C-shell family
csh	Bill Joy	late 70's	original C-syntax shell, + job control, history	uses .cshrc & .login
tcsh	Ken Greer	early 80's	like csh + extra interactive features
rc	Byron Rakitzis	early 90's	even more C-like syntax, from Plan-9

12.2 Login scripts

.cshrc executed when csh starts
.login executed by csh, after .cshrc, if running as a login shell

.profile executed when sh (Bourne shell) starts

Bash: As a login shell, executes ~/.bash_profile, ~/.bash_login or ~/.profile.
As a non-login shell, executes ~/.bashrc (which is typically also executed early by ~/.bash_profile).

13. Network setup

To set up a fixed IP address for an Ethernet connection, use System ▶ Preferences ▶ Network Connections. (If this item doesn't appear in the menu, make sure the package network-manager-gnome is installed.) In the Wired tab, select an existing connection and click Edit, or click Add. In the box that pops up, go to the IPv4 Settings tab. Under Method select Manual. Enter the IP address that has been assigned. Enter the netmask (e.g., 255.255.255.0), the gateway address, the IP addresses of some DNS servers, and the name of a domain to be searched.

14. Telnet & FTP

telnet:
  Use  set escape '^@'  to change escape character.
ftp:
  Transferring multiple files:
    In Unix:
      Use mget & mput; invoke ftp with -i to disable prompting.
      Use ;0 with mget from VMS if only latest versions are desired
        (version numbers will appear in Unix filenames as ;n).
      Attempting to mget/mput large numbers of files will cause
        errors creating data sockets.
    In VMS:
      Use GET/PUT with wildcards; /NOCONFIRM is default.
      Use ;0 with mput if only latest versions are desired
        (version numbers will appear in Unix filenames as .n).
      Large numbers of files can be transferred without errors
        (it pauses occasionally, perhaps to clean up sockets).

15. System administration

15.1 Mail

To change limit on message size, edit MaxMessageSize= in /var/adm/sendmail/sendmail.cf . (Funsan receives mail but puts it into INBOX's on Cortex, so the message size limit depends on the limit set on Cortex.)

15.2 System logs

To roll over the system log /usr/var/adm/binary.errlog (which can be read using uerf) do

• /usr/bin/kill -HUP pid

15.3 Web-server logs

To roll over the Web-server logs, do the following as su in /usr/local2/etc/httpd/logs:

• mv access_log access_log_old
• mv error_log error_log_old
• /usr/bin/kill -1 pid

15.4 Finding out what's going on

• cat /etc/debian_version - shows which version of Debian is running
• uname -a - shows system information: hostname, hardware platform, kernel version, etc.
• fuser - shows which processes are using specified file(s) or file system
• iostat - reports I/O statistics
• lsof - lists open files (must be run as su, so must say /usr/local/bin/lsof if /usr/local/bin is not in the path for su)
• top - for CPU and memory usage, etc.
• iotop - for disk i/o, etc.
• iostat - for disk i/o per device, etc. (part of sysstat package)
• apt-file search filename - find out which packages contain the specified file; first use apt-file update to update the lists of package contents. apt-file is in the package of the same name.

15.5 Start up

The /sbin/init programme initializes the system by creating and controlling processes.

On Funsan, the system is considered to run at one of four run levels:

• 0: a safe state to turn off power
• s: single-user
• 2: multi-user local
• 3: multi-user networked

For each run level n, a directory rcn.d/ contains links which point to scripts in init.d/. On Funsan the rcn.d/ and init.d/ directories are in sbin, on Fundus they're in /etc/.

On Funsan, there are scripts /sbin/rcn for bringing the system to level n. I don't know how (or if) they're used, or if they also exist on Fundus.

15.6 Login

When users log in, the welcome message is taken from /etc/motd ('message of the day').

16. Cygwin & MinGW

Cygwin and MinGW provide environments for running Unix under Microsoft Windows. I don't really understand the relationship between them.

Sometimes Cygwin gets itself into a condition in which things don't work very well any more and beastly error messages appear, such as

One way to start the X11 server under Cygwin is to do startxwin -- +bs (analogous to startx -- +bs in a 'real' Unix setup). Starting it directly from Windows with Start ▶ Cygwin-X ▶ XWin Server doesn't work well for me.

17. X

The X Window System (often called X11) is a client-server software system that provides low-level functions from which to build graphical user interfaces and applications for either networked or local computers. Perhaps confusingly, the X11 client is the computer (possibly remote) on which the graphical application is running, and the server is the local computer on which the display occurs.

To run a programme remotely and have its display appear on your local computer, just use the -X option with your ssh command when logging in to the remote computer. (This assumes that both computers are running Unix.)

18. Backups

Resources for Amanda:

• Using Amanda (book chapter by John Jackson)
• FAQ

19. VMS/Unix equivalents

VMS commands for which Unix equivalents are given:
copy, create, define, del, diff, dir, dump, help, inquire, lo, rename, search, set, show, submit, type

19.1 copy

19.2 create

19.3 define

set or setenv

19.4 del

19.5 diff

19.6 dir

Fields in -l directory listing: type, protection, no. links, owner, group, size, date, name.

Date shown is date modified; -lu for date used, -lc for date created.

19.7 dump

19.8 help

19.9 inquire

19.10 lo

19.11 rename

19.12 search

19.13 set

19.14 show

19.15 submit

Facilities for background processing in Unix are quite different from the VMS submit command.

19.16 type