CCPR Computing Services Workshop 1: Programming Basics, Unix, Remote Computing October 13, 2004

1
CCPR Computing ServicesWorkshop 1Programming
Basics, Unix, Remote ComputingOctober 13, 2004
2
Part 1 Programming Basics

• Motivation
• Before you start coding
• Programming Conventions
• Documentation
• Names, comments
• Directory Structure
• Basic Constructs
• Miscellaneous (debugging, cross-checking results)

3
Motivation

• Facilitate research
• Save time
• Cleaner code
• Easily share programs
• Basic Concepts
• MUCH better programming

4
Programming Conventions

• What are conventions?
• Examples
• Who cares?
• Readability of code
• Organization
• Transferring code to others
• Apply conventions consistently to
• variable names and function names
• comments
• directory structure

5
Before you start coding

• THINK
• WRITE down the problem
• WRITE down the algorithm in English (not code)
• Modularity
• Comments
• Create test (if reasonable)
• TRANSLATE one section to code
• TEST the SECTION thoroughly
• Translate/Test next section, etc.

6
Documentation - File Header
Laura Piersol (laurapiersol_at_ccpr.ucla.edu) HRS
project /u/socio/laurapiersol/HRS/ October 11,
2004 Python version 2.4 Stata version
8 Purpose Create and merge two datasets in
Stata, then convert data to SAS Input
programs HRS/staprog/H2002.do,
HRS/staprog/x2002.do, HRS/staprog/mergeFiles
.do Output HRS/stalog/H2002.log,
HRS/stalog/x2002.log, HRS/stalog/mergeFiles.
log HRS/stadata/Hx2002.dta HRS/sasdata/Hx200
2.sas Special instructions Check log files for
errors check for duplicates upon new data
release

• File header includes
• Name
• Project
• Project location
• Date
• Version of software
• Purpose
• Inputs
• Outputs
• Special Instructions

7
Naming Conventions

• Not a detail!
• Good names clarify your code
• Portray meaning/purpose
• Adopt a convention and BE CONSISTENT

8
Naming Conventions, cont.

• Use language standard (if it exists)
• If no standard, pick one and BE CONSISTENT
• Functions getStats, calcBetas, showResults
• Scalar variables scPi, scGravity, scWorkHours
• String variables stName, stCareer
• Global variables _Country, _Nbhd
• Be aware of language-specific rules
• Max length, underscore, case, reserved words

9
Naming Conventions, cont.

• Differentiating log files
• Programs MergeHH.sas, MergeHH.do
• Log files MergeHHsas.log, MergeHHsta.log
• Meaningful variable names
• LogWt vs. var1
• AgeLt30 vs. x
• Procedure that cleans missing values of Age
• fixMissingAge
• Matrix multiplication X transpose times X
• matXX

10
Commenting Code

• Good code is SELF-COMMENTING
• Naming conventions, structure, header explain 95
• Comments explain
• PURPOSE, not every detail
• TRICKS
• (good) reasons for unusual coding
• Comments DO NOT
• fix sloppy code
• translate syntax

11
Commenting Code - Stata example
SAMPLE 1 program def function1 foreach v of
varlist _all local x lower("v'") if "v'"'
! "x'"' rename v' lower("v'")' end
No conventions, comments, structure

Comments succinct and not overdone Names
lowerVarNames -action word for
program -distinct use of case LowName -descri
ptive -distinct use of case v -looping
variable and short scope -non-descriptive, but
does not detract from meaning! Structure-
indentations, parentheses lined up!

• SAMPLE 2
• Convert names in dataset to lowercase.
• program def lowerVarNames
• foreach v of varlist _all
• local LowName lower("v'")
• If variable is already lowercase,
• rename statement throws error.
• if "v'"' ! "LowName'"'
• rename v' lower("v'")'
• end

12
Directory Structure

• A project consists of many different types of
  files
• Use folders to SEPARATE files in a logical way
• Be consistent across projects if possible
• ATTIC folder for older versions

13
Miscellaneous Tips

• BACKUP! Weekly zip file stored externally
• README.txt file to describe folder
• BE ORGANIZED
• CROSS-VERIFY results
• Something not working?
• Remember the computer is following YOUR
  directions go back to your code

14
Programming Constructs

• Tools to simplify and clarify your coding
• Available in virtually all languages

15
Constructs - Looping

• Repeat section of code
• START value, INCREMENT, STOP value
• Example-
• convert uppercase to lowercase for each variable
  in a dataset

16
Constructs Looping Examples
C for loop Start with x1, Increment x1, Stop
when x10 for(x1 xlt10 x)
code
PERL while loop Start with count 1, Increment
count1, Stop when count11 count1 while
(countlt11) print "count\n"
count
STATA foreach loop Start first variable in
varlist, Increment next variable in varlist,
Stop last variable in varlist foreach v of
varlist _all local LowName
lower("v'") if "v'"' ! "LowName'"'
rename v' lower("v'")'
17
Constructs - If/then/else

• Execute section of code if condition is true
• if condition then
• execute this code if condition true
• end
• Execute one of two sections of code
• if condition then
• execute this code if condition true
• else
• execute this code if condition false
• end

18
Constructs - Elseif/case

• Elseif - Execute one of many sections of code
• if condition1 then
• execute this code if condition1 true
• elseif condition2 then
• execute this code if condition2 true
• else
• execute this code if condition1, condition2,
  condition3 are all false
• end
• Case- same idea, different name
• case condition1 then
• execute this code if condition1 true
• case condition2 then
• execute this code if condition2 true
• etc.

19
Constructs - And, or, xor
AND - BOTH conditions must be true results in
True OR - AT LEAST ONE condition must be true
results in True XOR - EXACTLY ONE condition must
be true for statement to be true
20
Constructs - Break

• Stop execution of program
• Examples
• Debugging. If particular error occurs then break.
• Parameters in function call are nonsensical.
  Print error and break.

21
Constructs - keywords

• Looping - for, foreach, do, while
• If statements if, then, else, case
• And/Or/Xor logical, and, or, xor, ,
• Break exit, break

22
PART 2 Unix

• Motivation
• Basic Commands
• Job submission and management
• Pipes
• Unix Shell
• Script files

23
Unix

• Motivation
• A quick history
• Unix variants
• (AIX, Solaris, FreeBSD, Linux)
• Where?
• Nicco (SSCs server)
• CCPRs linux cluster (coming soon)
• CCPRs data server (coming soon)

24
Unix Basic Commands
Getting Help
25
Unix Basic Commands
File Management
26
Unix Basic Commands
Directory Management
27
Unix Basic Commands
Using Previous Commands
28
Unix Basic Commands
Other Useful Unix Tools
29
Unix Basic Commands
Disk Usage
30
Editing Files in Unix

• Vi Editor and Emacs
• Neither are user-friendly for starters
• Look at CCPR internet when you start
• Best way to learn is to start editing a document
• Once you get used to them, theyre easy and fast
  to use.

31
Being nice

• Always run your jobs nicely
• Prevents interfering with other users
• Precede command with nice 19 (no quotes)
• user_at_nicconice 19 stata b jobfile.do

32
Job Submission in Unix

• Interactive
• user_at_niccostata
• Foreground jobs
• user_at_nicconice 19 stata b do jobfile.do
• user_at_nicconice 19 sas jobfile.sas
• Background jobs
• user_at_nicconice 19 stata b do jobfile.do
• user_at_nicconice 19 sas jobfile.sas
• Background jobs with logoff nohup
• user_at_nicconohup nice 19 stata b do
  jobfile.do
• user_at_nicconohup nice 19 sas jobfile.sas

33
Job Management
34
Pipes

• redirects
• command 1 output command 2 input

• Extends to more than 2 commands

35
Unix Shell

• Whats a Unix Shell?
• The Unix shell is the program that provides the
  interface between the user and the kernel
• Whats a shell script?
• A list of commands put into a file that can be
  interpreted by the Unix Shell.
• What are scripting languages?
• Generally easier to code but less efficient
• Shell scripts, Perl, Python

36
Remote Computing
See http//www.ccpr.ucla.edu/asp/compserv.asp,
XP users get latest version
37
Remote Computing
http//computing.sscnet.ucla.edu/training/tutor
ial_SSH.htm http//computing.sscnet.ucla.edu/tr
aining/tutorial_samba.htm
38
Finally

• Questions and Feedback