Chapter Four:

1
Guide To UNIX Using Linux Third Edition

• Chapter Four
• UNIX/Linux File Processing

2
Objectives

• Explain UNIX and Linux file processing
• Use basic file manipulation commands to create,
  delete, copy, and move files and directories
• Employ commands to combine, cut, paste,
  rearrange, and sort information in files

3
Objectives (continued)

• Create a script file
• Use the join command to link files using a common
  field
• Use the awk command to create a
  professional-looking report

4
UNIX and Linux File Processing

• Based on the approach that files should be
  treated as nothing more than character sequences
• Because you can directly access each character,
  you can perform a range of editing tasks this
  offers flexibility in terms of file manipulation

5
Reviewing UNIX/Linux File Types

• Regular files, also known as ordinary files
• Create information that you maintain and
  manipulate, and include ASCII and binary files
• Directories
• System files for maintaining file system
  structure
• Special files
• Character special files relate to serial I/O
  devices
• Block special files relate to devices such as
  disks

6
Understanding File Structures

• Files can be structured in many ways depending on
  the kind of data they store
• UNIX/Linux store data, such as letters and
  product records, as flat ASCII files
• Three kinds of regular files are
• Unstructured ASCII character
• Unstructured ASCII records
• Unstructured ASCII trees

7
(No Transcript)
8
Processing Files

• UNIX/Linux processes commands by receiving input
  from a standard input device (e.g. keyboard) and
  sending it to a standard output device (e.g.
  monitor)
• System administrators and programmers refer to
  standard input as stdin, standard output as
  stdout
• When UNIX/Linux detect errors, they send data to
  standard error (stderr, the monitor)

9
Using Input and Error Redirection

• You can use redirection operators (gt, gtgt, 2gt, lt,
  and ltlt) to retrieve input from something other
  than the standard input device and send output to
  something other than the standard output device
• Examples of redirection
• Redirect the ls command output to a file, instead
  of to the monitor (or screen)
• Redirect a program that receives input from the
  keyboard to receive input from a file instead
• Redirect error messages to files, instead of to
  the screen by default

10
Manipulating Files

• When you manipulate files, you work with the
  files themselves, as well as their contents
• Create files using output redirection
• cat command - concatenate text via output
  redirection
• without a command - gt filename
• After the command prompt, type gt newfile1 and
  press Enter. This creates an empty file called
  Newfile1
• touch command - creates empty files

11
Creating Files

• The primary purpose of the touch command is to
  change a files time and date stamp
• UNIX maintains the following date and time
  information for every file
• Creation date and time the date and time the
  file was created
• Access date and time the date and time the file
  was last accessed
• Modification date and time the date and time
  the file was last modified

12
Creating Files (Continued)

• Although the touch command cannot change an
  existing files creation date and time, it can
  alter the files access and modification dates
  and times

13
Manipulating Files (continued)

• Delete files when no longer needed
• rm command - permanently removes a file or an
  empty directory
• The -r option of the rm command will remove a
  directory and everything it contains
• Use the -i (interactive) option to have UNIX warn
  you before deleting the file
• Copy files as a means of back-up or as a means to
  assist with new file creation
• cp command - copies the file(s) specified by the
  source path to the location specified by the
  destination path

14
Copying Files

• Commands for copying different items
• cp reminder duplicates
• cp class_of_78 duplicates/classmates
• cp memo_to_boss memo.safe
• cp project1 project2 project3 duplicates
• cp designs/ duplicates

15
Manipulating Files (continued)

• Move files from directory to directory
• mv command - removes file from one directory and
  places it in another
• Moving and renaming a file are essentially the
  same operation
• You can also use the -i option with the mv
  command
• It causes the command to prompt you before it
  overwrites an existing destination file

16
Manipulating Files

• Finding a file helps you locate it in the
  directory structure
• find command - searches for the file that has the
  name you specify
• The command has the form
• find pathname - name filename
• Searches recursively beginning with the pathname
  directory
• The find command prohibits you from searching
  where you do not have system-level permissions
• find /home name phone1
• Although Linux does not use it, other UNIX
  versions require the -print option after the
  filename to display the names of fields the find
  command locates

17
Manipulating Files (continued)

• Combining files using output redirection
• cat command - concatenate text of two different
  files via output redirection
• paste command - joins text of different files in
  side by side fashion
• The paste command combines files line by line,
  whereas the cat command appends data to the end
  of the file
• When you use paste to combine two files into a
  third file, the first line of the third file
  contains the first line of the first file
  followed by the first line of the second file
• The paste command normally sends its output to
  the screen - To capture it in a file, use the
  redirection symbol
• Note the d option to substitute non-tab
  delimiter The paste command normally sends its
  output to the screen

18
Manipulating Files
cat product1 product2 gt product3
paste product1 product2
19
Manipulating Files (continued)
Extracting fields of a file using output
redirection the cut command removes specific
columns or fields from a file
20
Manipulating Files

• Extracting fields of a file using output
  redirection
• cut command - removes specific columns or fields
  from a file
• Syntax
• cut -f list -d char file1 file2 (field )
• or
• cut -c list file1 file2. (character )

21
Manipulating Files
To use the cut command to extract fields from
fixed-length records 1. After the command
prompt, type cut -c14-25, 26-35, 36 Corp_phones2
and press Enter
To use the cut command to extract fields from
variable-length records 1. After the command
prompt, type cut -f4-6 -d Corp_phones1 and press
Enter
22
Manipulating Files (continued)

• Sorting the contents of a file
• sort command - sorts a files contents
  alphabetically or numerically
• the sort command offers many options
• You can sort the contents of a file and redirect
  the output to another file
• Utilizing a sort key provides the option of
  sorting on a field position within each line

23
Manipulating Files (continued)
24
Using the Sort Command

• Here is an example of its use
• sort file1 gt file2
• Here is a more complex example
• sort .10 file1 gt file2
• This command specifies a sorting key
• A sorting key is a field or character position
  within each line
• The indicates that sorting does not begin at
  the first character position, but is offset
  elsewhere in the file
• The period (.) indicates that the offset is
  measured in characters

25
Using Script Files

• You can use the shells command-line history
  retrieval feature to recall and re-execute past
  commands
• But others who need to execute your commands
  cannot access them repeatedly
• MS-DOS users resolve this problem by creating
  batch files
• UNIX users create shell script files to contain
  commands that can be run sequentially as a set
  this helps with the issues of command automation
  and re-use of command actions
• UNIX users use the vi editor to create script
  files, then make the script executable using the
  chmod command with the x argument

26
Creating Script Files (continued)
27
Using the join Command on Two Files

• Sometimes you want to link the information in
  two files
• The join command is often used in relational
  database processing
• Relational databases consider files as tables and
  records as rows
• They also refer to fields as columns that can be
  joined to create new records
• The join command associates information in two
  different files on the basis of a common field or
  key in those files

28
A Brief Introduction to theAwk Program

• Awk, a pattern-scanning and processing language
  helps to produce professional-looking reports
• Awk provides a powerful programming environment
  that can perform actions on files that are
  difficult to duplicate with a combination of
  other commands

29
A Brief Introduction to theAwk Program
(continued)

• Awk checks to see if the input records in
  specified files satisfy a pattern
• If so, awk executes a specified action
• If no pattern is provided, awk applies the action
  to every record

30
Chapter Summary

• UNIX/Linux supports regular files, directories,
  and character and block special files
• File structures depend on data being stored
• UNIX/Linux receives input from the standard input
  device (keyboard, stdin) and sends output to the
  standard output device (monitor, stdout)

31
Chapter Summary (continued)

• touch updates a files time and date stamps and
  creates empty files
• rmdir removes empty directories
• cut extracts specific columns or fields from a
  file
• paste combines two or more files
• sort sorts a files contents

32
Chapter Summary (continued)

• To automate command processing, include commands
  in a script file
• join extracts data from two files sharing a
  common field and uses this field to join the two
  files
• Awk is a pattern-scanning and processing language
  useful for creating a formatted report with a
  professional look