Introduction to Linux for Programmers

Course Outline

Base Concepts

A Brief History of Linux

1. Proprietary Operating Systems
   1. Why operating systems were developed in the first place.
   2. Early standards focused on languages, not system software.
   3. Proprietary operating systems not an issue until minicomputers emerged.
   4. State of computing in 1975.
2. Origins of Unix
   1. Major purpose.
   2. Design philosophy:
      1. KISS!
      2. Design parity throughout.
      3. People act professionally.
   3. Function and performance goals.
   4. BSD fuels the growth of Linux.
   5. State of computing in 1985.
3. The POSIX® Standard
   1. Stands for Portable Operating System Interface.
   2. Currently managed by IEEE Portable Application Standards Committee (http://www.pasc.org).
   3. Various Subsets:
      1. System Interface Standard: IEEE Std 1003.1-1990 (ISO 9945-1:1990).
      2. System Interface Standard: IEEE Std 1003.1-1996 (ISO 9945-1:1996).
      3. Real Time Extensions : IEEE Std 1003.1b-1993 and Realtime Technical Corrigenda : IEEE Std 1003.1i-1995.
      4. Threads Extensions : IEEE Std 1003.1c-1995.
      5. Shell and Utilities : IEEE Std 1003.2-1992 (ISO/IEC 9945-2:1993).
      6. Test Methods for 1003.1 : IEEE Std 2003.1-1993.
      7. Ada Bindings for POSIX : IEEE Std 1003.5-1992.
   4. The IEEE maintains a team of lawyers (!) who interpret POSIX. See http://www.pasc.org/interps for details.
4. Development of Linux
   1. Richard Stallman, MIT, and the Xerox laser printer (1970).
   2. Free Software Foundation and HURD (1984).
   3. Linux Torvalds, Linux, and the "merger" (1990).
   4. Linux is a POSIX, not Unix, implementation.
   5. GNU General Public License (GPL) and the Lesser GPL (LGPL).

Internal Structures

1. Processes
   1. Protected memory space.
   2. Environment variables.
   3. Parents and children.
   4. Super and user privileges.
   5. Kernel and user space.
   6. Daemons and shells.
2. Threads
   1. Interaction of processes and the scheduler.
   2. Thread code is run independently by the scheduler.
   3. All threads share the same memory space as the host process.
   4. Differences in context switch time.
3. File System
   1. Volumeless tree starting at root.
   2. Subdirectory structure.
   3. Inode types and storage:
      1. Regular or executable file.
      2. Directory.
      3. Symbolic link.
      4. Sockets.
      5. Named pipe.
   4. File permission system: user, group, world.
   5. Virtual File System.
4. Device Drivers
   1. Parity between devices and files.
   2. Device driver directory.
   3. Driver is executable code that delivers data back to file system.
   4. Typically run in kernel space.

Command Shells

1. The login Process
   1. User and password collection.
   2. Authentication and configuration.
   3. Launching of the command shell.
   4. Initial configuration script processing.
   5. Differences for GUI based systems (X Windows).
2. Command Shell Primer
   1. Basic functions of a shell.
   2. Various types of shells:
      1. /bin/sh
      2. /bin/csh
      3. /bin/ksh
      4. /bin/bash
   3. Odd shell is the C shell: its scripting language looks like C.
   4. The base shell sh is useful for batch jobs due to low overhead.
   5. We will focus on bash in this course.
3. Special Characters
   1. Variable expansion: $
   2. Universal pattern match: *
   3. Literal interpretation of next character: \
   4. Command line ends here: ;
   5. Comment follows: #
4. Environment Variable Primer
   1. Purpose.
   2. Difference between environment and shell variables.
   3. Displaying all environment variables: env | more
   4. Displaying a specific environment variable: $<variable name>
5. Key Environment Variables
   1. HOME
   2. HOSTNAME
   3. DISPLAY
   4. MAIL
   5. PATH
   6. PWD
   7. SHELL
   8. TERM
   9. USER

Linux User Fundamentals

Command Line Processing

1. Parsing a Command Line
   1. Shell matches paired characters: ' " `
   2. Performs symbol expansion.
   3. Breaks resulting command line into word tokens.
   4. One line processed at a time.
   5. Backslash used as continuation character: \
   6. Semicolon breaks multiple commands on same line: ;
   7. If no errors, first token loaded and all tokens passed as arguments.
   8. Exception to above are shell built-in commands.
2. Symbol Expansion
   1. File name expansion: [<text>]*[<text>]
   2. Environment variable expansion: $<variable name>
   3. Shell command expansion: `<string>`
   4. Inhibit file name expansion: "<string>"
   5. Inhibit all expansion: '<string>'
   6. Class Exercise: Interpreting Shell Expansion.
3. Command Formats
   1. By definition first token, $0, is file loaded (command being executed).
   2. By convention, subsequent tokens prepended by hyphen are options
   3. By convention, tokens not starting with a hyphen are parameters
   4. Commands don't see most special characters after symbol expansion.
   5. By convention, commands needing a target use the last argument.
4. Keyboard Shortcuts
   1. File name and path completion with the TAB key.
   2. Command line recall with up and down cursor keys.
   3. Command line editing with left and right cursor keys, backspace, DEL.
5. Running Foreground and Background Jobs
   1. Commands can be executed in the background by appending an & as the last token.
   2. Foreground jobs can be paused by typing CTRL-Z.
   3. List all jobs running: jobs
   4. Start a paused job in the background: bg %<job identifier>
   5. Bring a background or paused job to the foreground: fg %<job identifier>

Essential Shell Navigation

1. Current Directory Concept
   1. Initially user is placed in home directory.
   2. Can use change directory command to move within tree: cd
      1. Provide relative path as argument.
      2. No argument places user in $HOME.
   3. Subdirectories are separated via forward slashes: /
   4. A single period indicates current directory and two periods parent.
   5. Subdirectory paths are relative to current.
   6. The path must be specified for an executable ($0) not in the PATH.
   7. Programs we write must reference PWD for current directory.
2. Listing Subdirectories: ls
   1. Long format: -l
   2. Single column: -1
   3. Turn off directory expansion: -d
   4. Turn off filtering and show all files: -a
3. Directory and File Manipulation
   1. Creating a new subdirectory: mkdir
   2. Removing an existing subdirectory, if empty: rmdir
   3. Copying files: cp
   4. Renaming (moving) files: mv
   5. Removing files: rm
   6. Forced removal option: -f
   7. Recursive removal option: -r
4. Reading and Setting File Permissions
   1. The permission bit structure:
      1. Three bits each for user, group, and others.
      2. One bit each for read, write, execute.
      3. Additional "sticky" bit
   2. View permissions with ls -a
   3. Set permissions with the chmod command:
      1. Set nine permissions bit by octal sum or symbolic constant.
      2. Sticky bit must be set by octal sum.
   4. Change owner with chown
   5. Change group with chgrp
5. Linking Files
   1. Difference between a link and a file.
   2. Hard versus soft links.
   3. Advantages of and restrictions on hard links.
   4. Creating links: ln [-s] <destination> <alias>
6. Pipes and Redirection
   1. By design, Linux commands use standard system files:
      1. stdin
      2. stdout
      3. stderr
   2. These can be redirected to other files:
      1. Redirection of stdin is via <
      2. Redirection of stdout is via >
      3. Redirection of stderr is via 2>
   3. Output of one command can be used as an input via a pipe: |
   4. Here is one example: env | more
   5. There is no limit to the number of allowed pipes except for system capacity.
   6. When using pipes or redirection, arrange commands to reduce data at each stage.

System Administration Primer

1. On Line Documentation
   1. Need a clue where to start? apropos <keyword> | grep "(1)" | more
   2. Quick reference manual pages: man <command>
   3. More in-depth reference material: info <command>
2. Viewing Configuration Information
   1. Who is logged into the system? users and who
   2. Which machine am I on? hostname
   3. What is my current user name? whoami
   4. What hardware and operating system am I running on? uname -a
   5. How is the file system mapped to disk hardware? df
   6. How much free memory does the system have? free
   7. What TCP/IP connections are active now? netstat -t
3. Disk Management
   1. Mounting CDs and hard disks: mount -<options&gt <device&gt <path>
   2. File systems:
      1. CD: -r -t iso9660
      2. Win 3.x/95/98/ME: -t vfat
      3. WinNT/2K/XP: -t ntfs
      4. OS/2: -t hpfs
   3. Mounting devices at system boot: /etc/fstab
   4. Removing CDs: eject cdrom
   5. Verifying the file system (DANGEROUS): fsck -N
4. X Windows
   1. X history and architecture:
      1. MIT Project Athena.
      2. Design goal of graphic display over network.
      3. Client and server defined (different from typical use).
      4. XFree86 is an open source implementation of X11.
   2. Configuring remote display:
      1. The DISPLAY environment variable: export DISPLAY=<node>:0.0
      2. Allowing other machines to use a local display: xhost +<node>
   3. Fixing installation or other configuration errors:
      1. Boot the system in text mode (or use CTRL-ALT-F1 if lucky).
      2. Edit the XFree86 configuration file: /etc/X11/XF86Config
      3. Scroll down to the Monitor section
      4. Delete the offending mode if you know it.
      5. Delete all modes except 640x480 if you don't.
      6. Edit a mode or monitor entry if you have a hardware manual and understand the specification.
      7. Avoiding this error in the first place: generic monitors.
   4. Opening X apps at a fixed position and size: -geometery <x>x<y>[+<width>+<height>]
5. Stuck Software
   1. Linux doesn't fail so much as take forever to figure there is a problem!
   2. System originally designed in days of 300 baud modems, so timeouts are long.
   3. All processes are assigned an identifier.
   4. Passing this identifier a signal stops the process.
   5. Finding the identifier:
      1. If you know the command name: ps -e | grep <command>
      2. To search the running commands: ps -e | more
      3. The process identifier is the number in the first column.
   6. Stopping the process:
      1. Gracefully: kill <process identifier>
      2. With force: kill -9 <process identifier>
   7. What does kill actually do?
6. IP Addresses
   1. Most computers use dyanamic IP addresses:
      1. DHCP stands for Dynamic Host Control Protocol
      2. Automatically assigns an IP address to a node from a server.
      3. Address may expire at some point in the future.
      4. Typical small network is 192.168.1.1 - 192.168.1.255
   2. Sometimes devices need a static IP address:
      1. Fixed for the device and never expires.
      2. Printers and servers are example devices for which static makes sense.
      3. Can be edited with system administration tool.
      4. Or file in the /etc/sysconfig/network-scripts directory.
   3. The file /etc/sysconfig/network controls IP forwarding.

Linux Programming Basics

Configuration and File Management

1. Packing and Unpacking Archives
   1. The .ZIP file format is used on PCs to "bundle" a directory tree.
   2. Linux uses a command line utiltity for (tape) archives: tar
   3. Building archives:
      1. tar -cvf <path> to create a new archive.
      2. tar -cvfz <path> to also zip.
      3. tar -uvf <path> to update an existing archive.
      4. tar -cvfz <path> to update a zipped archive.
   4. tar archives and expands relatively at or below the specified path.
   5. Unpacking archives:
      1. tar -xvf <bundle> to expand an archive.
      2. tar -xvfz <bundle> to first unzip the bundle.
2. Software Installation
   1. Obtain the appropriate archive via download or media.
   2. Create a directory:
      1. Software supplier should provide guideance.
      2. If not, consider under /usr/local if software for all users.
      3. Or under $HOME/bin for a single user.
      4. Follow guidance for modifying PATH and LD_LIBRARY_PATH.
   3. Unpack the archive with tar -xvf[z] <bundle>
   4. Look for and run an install script.
3. Red Hat Package Manager (RPM)
   1. Use on Red Hat systems where software supplier provides .rpm file.
   2. Type rpm -i <filename> to launch an install GUI.
   3. RPM guides choosing installation options.
   4. RPM automatically checks dependencies and installs software.
4. Version Control Systems
   1. Software products change over the development life cycle.
   2. In team development, person A may make a change that breaks person B's code.
   3. Version control systems help by adding a version path to the directory path.
   4. Typical systems are:
      1. Source Code Control System.
      2. Resource Control System.
      3. Conncurrent Version System.
      4. ClearCase (proprietary--and the best! ;-}).
   5. We will discuss version control concepts, but not these systems, in class.

Shell Scripting

1. What is a Shell Script?
   1. Automates typing at the command line.
   2. Ordinary text files can be given run permission (recall chmod).
   3. In such a case, shell will read lines of text and assume they are commands (recall stdin and redirection).
   4. Thus hundreds or even thousands of commands can be executed by typing one: the name of the script file.
2. A Simple Shell Script

   # .bashrc
# User specific aliases and functions
alias rm='rm -i'
alias cp='cp -i'
alias mv='mv -i'
# Source global definitions
if [ -f /etc/bashrc ] ; then . /etc/bashrc ; fi

3. Command Line Parameters
   1. During scripting, special environment variables exist to collect command line parameters.
   2. Positional parameters: $0, $1, $2, ...
   3. All parameters: $*
   4. All parameters, but qutoed: $@
   5. Number of parameters: #
   6. Others can be found in the bash man page.
4. The if Statement
   1. Format: if <test> ; then <actions> ; [ elif <test> then <actions&gt ; ] [ else <actions&gt ; ] fi
   2. Arguments for the test command:
      1. Logical expressions: raw, -a, -o, !
      2. Numeric expressions: -eq, -ne, -gt, -ge, -lt, -le
      3. String expressions: =, !=
      4. File operations: -f and others.
   3. Note if and elif and else are separate.
5. The for Statement
   1. Command format: for <variable> [ in <arguments> ] ; do <actions> ; done
   2. Arguments may be hard coded text, shell parameters, environment variables, or anything that resolves to a token.
   3. The in clause is optional and creates a variable expandable with dollar sign.
   4. The actions execute once for each argument supplied.
6. The while Statement
   1. Format: while <test> do <actions> ; done
   2. Test is performed.
   3. If true, actions are executed.
   4. If false, break out of loop.
   5. Lather. Rinse. Repeat.

Build Management

1. The make Command
   1. Used to automate building (compilation and linking) of software.
   2. By default uses a file named either Makefile or makefile.
   3. Inspects file for targets.
   4. Checks targets for dependencies.
   5. If a dependency is out of date, executes the associated build command.
   6. Can use files with other names via the -f option.
2. Structure of a Makefile
   1. Environment Variables
      1. Similar to shell environment variables.
      2. Whitespace separated list of tokens.
      3. All associated tokens assigned must be on one statement, usually terminated by end of line.
      4. Use \ for line continuation.
      5. If longer then one character must use parenthesis for expansion.
      6. Example: $(CC)
   2. Recursive and Static Assignment
      1. Recursive assignment with = keeps processing until expansion finishes.
      2. Static assignment with := only makes one pass through the tokens.
      3. Use static assignment to initialize a target.
      4. Use recursive assignment to build on a target.
      5. If in doubt, use recursive assignment.
   3. Defining Targets
      1. Targets are tokens that start in column one and are terminated with a colon.
      2. Tokens can be assigned to a target like an environment variable.
      3. These tokens should be relative path names to files, such as source and object.
      4. Make will execute a target's associated build command if any token is out of date.
      5. Matched token considered when generating actual command passed to shell.
      6. Extension matching, for example .c.o or .o.a, can be used as well.
3. Defining Build Commands
   1. Build commands immediately follow line after the target and start with tab.
   2. Commands must be one logical line long.
   3. Within a build command, one can use special environment variables.
      1. Prerequisite token: $<
      2. Target: $@
      3. Stem: $*
4. Chaining Makefiles
   1. The make command honors an include directive.
   2. Files are processed in order listed.
   3. Normally make processes the first target encountered (unless a target is specified).
   4. This also works with include directives: each file is processed using the same target.
   5. Technique to maintain independent Makefiles at subsystem level that are run at system level.