awk An Advanced Filter

1

awk- An Advanced Filter
by
Prof. Shylaja S S
Head of the Dept. Dept. of Information
Science Engineering, P.E.S
Institute of Technology,
Bangalore-560085
shylaja.sharath_at_pes.edu
2
Session Objectives

• Arrays
• Functions.

3
Arrays

• An array is a variable that can store a set of
  values or elements.
• Each element is accessed by a subscript called
  the index.
• awk arrays are different from the ones used in
  other programming languages in the following
  aspects

4
Contd..

• They are not formally defined. An array is
  considered declared the moment it is used.
• Array elements are initialized to zero or an
  empty string unless initialized explicitly.
• Arrays expand automatically.
• The index can be virtually any thing. It can
  even be a string.

5
Array EX

• BEGINFS
• printf(\t\tBasic DA HRA Gross\n\n)
• /salesmarketing/
• da 0.256hra0.506 gp6dahra
• tot16tot2datot3hra tot4gp
• kount

6
Array EX

• ENDprintf(d d d,tot1/kcount,
  tot2/kount, tot3/kount, tot4/kount)
• awk f empawk3.awk empn.lst

7
Contd..
In the above program emp.awk, arrays are used to
store the totals of the basic pay, da, hra and
gross pay of the sales and marketing people.
Assume that the da is 25, and hra 50 of basic
pay. Use the tot array to store the totals of
each element of pay, and also the gross pay
8
Contd..
Execution awk f emp.awk emp.lst Output
Basic Da Hra Gross Average 8000
1790 3406 11921
9
Associative Arrays

• awk arrays are associative
• Information is stored as key-value pairs.
• The index is the key that is saved internally
  as a string.
• For example, consider mon1mon, awk converts
  the number 1 to a string.
• Note There's no specified order in which the
  array elements are stored.

10
EX
awk BEGIN gt direction N North
direction S gt direction E East
direction W West gt printf(N is s and
W is s \n, directionN, gt direction
W) gt mon1 Jan mon1 January
mon01 gtJAN gt printf(mon is s\n,
mon1) gt printf(mon01 is also
s\n,mon01) gt printf(mon\1\ is also s
\n, mon1) gt printf(But mon\01\ is
s\n, mon01)
11
Contd..
Output N is North and W is West mon1 is
January mon01 is also January mon1 is also
January But mon01 is JAN
12
Contd..

• Note
• Setting with index 1 overwrites the setting
  made with index 1.
• Accessing an array element with subscript 1 and
  01 actually locates the element with subscript
  1.
• Also note that mon1 is different from
  mon01.

13
ENVIRON The Environment Array

• To know the name of the user running the program
  or the home directory awk maintains the
  associative array, ENVIRON.
• It stores all environment variables.

14
Contd..

Example Shell variable , HOME and PATH, can be
accessed from inside an awk program as awk
BEGIN gt print HOME ENVIRONHOME gtpr
int PATH ENVIRONPATH gt Output HOME
/usr/home/isehod PATH/usr/bin/usr/local/bin
/usr/xyz/ise
15
Built-in Functions

• awk has several built in functions, performing
  both arithmetic and string operations.
• The arguments are passed to a function in
  C-style, delimited by commas and enclosed by a
  matched pair of parentheses.
• Although, awk allows use of functions with and
  without parentheses (like printf and printf()),
  POSIX discourages use of functions without
  parentheses.

16
Contd..

• Some of these functions take a variable number
  of arguments, and one (length) uses no arguments
  as a variant form.
• There are two arithmetic functions which a
  programmer will except awk to offer namely int
  and sqrt.
• int calculates the integral portion of a number
  (without rounding off).
• sqrt calculates square root of a number.

17
Contd..

• awk also has the string handling function. Some
  of them are
• length
• index(s1, s2)
• substr (stg, m, n)
• split(stg, arr, ch)
• system

18
Contd..

• length
• It determines the length of its arguments, and
  if no argument is present, the entire line is
  assumed to be the argument.
• length can also be used (without any argument)
  to locate lines
• Example1
• awk F length gt 2042 emp.lst

19
Contd..

• For example, the following program selects those
  people who have short names
• awk F length (2) lt 11 emp.lst
• index(s1, s2)
• It determines the position of a string s2 within
  a larger string s1.
• This function is especially useful in validating
  single character fields.

20
Contd..

• Example 2
• x index (abcde, b)
• If a field takes the values a, b, c, d or e you
  can use this function to find out whether this
  single character field can be located within a
  string abcde
• Output
• 2

21
Contd..

• substr (stg, m, n)
•  It extracts a substring from a string stg.
• m represents the starting point of extraction
  and n indicates the number of characters to be
  extracted.
• Example awk F substr(5, 7, 2) gt 45
  substr(5, 7, 2) lt 52 emp.lst
• This function can be used to select those born
  between 1946 and 1951.

22
Contd..

Output 2365 barun sengupta director
personel 11/05/47 3564 sudhir ararwal
executive personnel 06/07/47 4290 jaynth
executive production 07/09/50 9876 jai
sharma director production 12/03/50
23
Contd..

• split(stg, arr, ch)
• It breaks up a string stg on the delimiter ch
  and stores the fields in an array arr.
• Example
• awk F split (5, ar, /) print
  19ar3 ar2 ar1 empn.lst
• The above example converts the date field to the
  format YYYYMMDD

24
Contd..

• system
• system function is used to print the system
  information.
• For example, to print the system date at the
  beginning of the report.
• For running a UNIX command within a awk, system
  function can be used.

25
Contd..

Example BEGIN system(tput clear)
Clears the screen system(date)
Executes the UNIX date command

26
Contd..

Function Description int(x)
returns the integer value of x sqrt(x)
returns the square root of x
length returns the complete
length of line length(x) returns length of
x substr(stg, m, n) returns portion of string of
length n,
starting from position m in string
stg. index(s1, s2) returns position of
string s2 in string s1
27
Contd..

Function Description split(s
tg, arr, ch) splits string stg into array arr
using ch as delimiter, returns number of
fields. system(cmd) runs UNIX command cmd and
returns its exit status
28
Conclusion
In this session we had an insight into few
advanced topics of awk. The topics that we looked
at were Associative arrays Key-value pairs Key
treated as string
29
Conclusion
Built in functions for operating on numbers as
well as on strings Unix environment variables