Advanced Unix

1
Advanced Unix

• Final Review
• December 6, 2005

2
IPSEC
3
Outline

• IPsec overview
• Alphabet soup being served
• Security Associations (SA) SPIs
• Authentication Header (AH) protocol
• Encapsulating Security Payload (ESP) protocl
• Internet Key Exchange (IKE)
• IPsec pitfalls
• IPsec vs tunneling (PPTP, L2TP)

4
IPSec Overview

• IPSec is a suite of protocols for securing
  network connections
• The details and variations are overwhelming
• One cause of the complexity is that IPSec
  provides a mechanism, not policy
• A framework that allows implementation possible
  that both ends can agree on

5
Virtual Private Network (VPN)

• Secure communications between two hosts or
  networks
• VPN, is the buzzword that solves all you problems
• IPsec is one of the more popular VPN technology's

6
What can IPSEC Provide

• Authentication
• Integrity
• Access control
• Confidentiality
• Replay protection (Partial)

7
Types of VPNs

• Host To Host
• Well do this in class
• Host To Security or Secure Gateway
• Secure Gateway To Secure Gateway
• Secure Gateway Firewall or VPN router
• Also referred to as Network To Network

8
Security Associations (SA)

• A group of security settings related to a
  specific VPN
• Stored in the SPD (Security Policy Database)
• Uniquely Identify IPsec sessions by
• SPI (Security Parameter Index) a unique number
  that identifies the session
• The destination IP address
• A security protocol or encryption method
• Normally AH or ESP
• A shared secret

9
Types of IPSEC Connections

• Transport Mode
• Does not encrypt the entire packet
• Uses original IP Header
• Faster
• Tunnel Mode
• Encrypts entire packet including IP Header (ESP)
• Creates a new IP header
• Slower

10
IKE (Internet Key Exchange)

• UDP port 500
• Negotiates connection parameters
• ISAKMP (Internet Security Association and Key
  Management Protocol)
• Oakley (Diffie-Helmen key exchange)

11
IPsec Pitfalls

• Complicated
• many different ways to configure
• Can be configured insecurely
• Client security is an issue
• Performance in IPv4 implementation

12
Advantages of IPSec

• Encrypts the entire packet, including IP Header
  (not just layer 4 and higher)
• Can Encrypt any protocol
• No Impact on users when using Secure Gateway to
  Secure Gateway
• Acts independent of IP address

13
IPsec Guidelines

• Always use
• 3des or blowfish
• SHA1 over SHA and MD5
• NEVER USE DES
• Tunnel Mode
• Main Mode
• AH and ESP together
• Certificates for production environments

14
OS Support for IPsec

• OpenBSD, FreeBSD, NetBSD
• Linux
• Solaris
• Windows 2000 (Native)
• Windows NT/95/98/Me (Add-on)
• Cisco IOS (PIX and Routers)
• Others as well....

15
Squid Proxy Server
16
Squid Features

• Its a caching proxy for
• HTTP, HTTPS (tunnel only)
• FTP
• Gopher
• A full-featured Web proxy cache
• Designed to run on Unix systems
• Free, open-source software

17
Squid Supports

• proxying and caching of HTTP, FTP, and other URLs
  
• proxying for SSL
• cache hierarchies
• ICP, HTCP, CARP, Cache Digests
• transparent caching
• extensive access controls
• HTTP server acceleration
• SNMP
• caching of DNS lookups

18
Other proxies (besides Squid)

• Commercial
• Netscape Proxy
• Microsoft Proxy Server
• NetAppliances NetCache (shares some code history
  with Squid in the distant past)
• CacheFlow (http//www.cacheflow.com/)
• Cisco Cache Engine

19
What is a proxy?

• Firewall device internal users communicate with
  the proxy, which in turn talks to the Internet
• Gateway for private address space (RFC 1918) into
  publicly routable address space
• Allows one to implement policy
• Restrict who can access the Internet
• Restrict what sites users can access
• Provides detailed logs of user activity

20
What is a caching proxy?

• Stores a local copy of objects fetched
• Subsequent accesses by other users in the
  organization are served from the local cache,
  rather than the origin server
• Reduces network bandwidth
• Users experience faster web access

21
How proxies work

• User configures web browser to use proxy instead
  of connecting directly to origin servers
• Manual configuration for older PC based browsers,
  and some UNIX browsers (e.g., Lynx)
• Proxy auto-configuration file for Netscape 2.x
  or Internet Explorer 4.x
• Far more flexible caching policy
• Simplifies user configuration, help desk support,
  etc.

22
How proxies work (user request)

• User requests a page http//www.rose.edu
• Browser forwards request to proxy
• Proxy optionally verifies users identity and
  checks policy for right to access
  uniforum.chi.il.us
• Assuming right is granted, fetches page and
  returns it to user

23
Samba
24
What is Samba

• Samba is an Open Source/Free Software suite that
  provides file and print services to SMB clients
• Samba current version 3.20b
• Samba Home Page
• http//www.samba.org

25
Prerequisites

• The following installs
• Samba
• samba-client
• samba-common
• system-config-samba
• samba-swat (optional)

26
Samba Utilities and Daemons

• net
• nmbd
• nmblookup
• smbclient
• smbd
• smbpasswd
• smbstatus
• smbtree
• swat (not part of samba)
• testparm
• testprns (deprecated and will be removed in a
  future Samba release)

27
Samba users, maps, passwords

• Usernames - /etc/samba/smbusers
• Passwords - /etc/samba/smbpasswd
• Demo
• /etc/samba/smbusers

28
Quick Start

• system-config-samba is used to configure samba
  server on linux computer
• Demo system-config-samba
• Samba users
• Linux shares

29
Sendmail and SMTP
30
Overview

• Introduction to Email
• Message Breakdown
• Sample Messages
• Extensions (MIME)
• MTAs and Mailbox Protocols

31
Email Statistics

• 31 billion emails are sent daily, expected to
  double by 2006
• Email generates about one billion Gigabytes of
  new information per year
• Spam accounts for about 40 of all email traffic
• http//www.spamfilterreview.com

32
SMTP

• Originated in 1982 (rfc0821, Jon Postel)
• Goal To transfer mail reliably and efficiently

33
SMTP

• SMTP clients and servers have two main components
• User Agents Prepares the message, encloses it
  in an envelope. (Eudora for example)
• Mail Transfer Agent (MTA) Transfers the mail
  across the internet

User Agent
Mail Transfer Agents
34
SMTP

• SMTP also allows the use of Relays allowing other
  MTAs to relay the mail
• Mail Gateways are used to relay mail prepared by
  a protocol other then SMTP and convert it to SMTP

35
What is Mail?

• Mail is a text file
• Envelope
• sender address
• receiver address
• other information
• Message
• Mail Header defines the sender, the receiver,
  the subject of the message, and some other
  information
• Mail Body Contains the actual information in
  the message

36
Return-Path ltJwatson_at_cis.udel.edugtDelivered-To
jwatson_at_cis.udel.edu Received by
mail.eecis.udel.edu (Postfix, from userid 62) id
17FBD328DE Wed, 5 Nov 2003 112702 Received
from mail.acad.ece.udel.edu (devil-rays.acad.ece
.udel.edu 128.4.60.10) by mail.eecis.udel.edu
(Postfix) with ESMTP id 5F41832893 for
ltJwatson_at_cis.udel.edugt Wed, 5 Nov 2003
112701 Received by mail.acad.ece.udel.edu
(Postfix, from userid 62)id 47509456C Wed, 5
Nov 2003 112701 Received from
stimpy.eecis.udel.edu(stimpy.eecis.udel.edu
128.4.40.17)by mail.acad.ece.udel.edu
(Postfix) with SMTP id 7C2943D79 for
ltJwatson_at_cis.udel.edugt Wed, 5 Nov 2003
112634 Message-Id lt20031105162634.7C2943D79_at_ma
il.acad.ece.udel.edugtDate Wed, 5 Nov 2003
112634 From Jwatson_at_cis.udel.eduTo
undisclosed-recipients MIME-Version
1.0This is a test.
Post Office Mailbox
Post office and mail route
Receivers Mailbox
37
How SMTP works

• The Essentials
• How about a Demo?

Keyword Arguments
HELO Senders Host Domain Name
MAIL FROM Email Address of sender
RCPT TO Email of Intended recipient
DATA Body of the message
QUIT
38
How SMTP works

• The Extras

Keyword Arguments
RSET
VRFY Name to be verified
NOOP
TURN
EXPN Mailing list to expand
HELP Command Name
39
Status Codes

• The Server responds with a 3 digit code that may
  be followed by text info
• 2 - Success
• 3 - Command can be accepted with more
  information
• 4 - Command was rejected, but error
  condition is temporary
• 5 - Command rejected, Bad User!

40
Connection Establishment
TCP Connection Establishment
41
Message Progress
42
Connection Termination
TCP Connection Termination
43
Problems with SMTP

• No security
• Authentication
• Encryption
• Only uses NVT (Network Virtual Terminal) 7-bit
  ASCII format

44
E-mails can be forged..

• HELO mail.rose.edu
• MAIL FROM carberry_at_rose.edu
• RCPT TO wrichards_at_rose.edu
• DATA
• From Dr. Art Zenner
• To Professor Richards
• Subject CIT 2243
• Professor Richards,
• By department decree all students in your CIT
  2243 Introduction to Unix class are hereby to be
  given automatic As.
• Thank you,
• Dr. Art Zenner
• .
• QUIT

45
Extensions to SMTP

• MIME Multipurpose Internet Mail Extensions
• Transforms non-ASCII data to NVT (Network Virtual
  Terminal) ASCII data
• Text
• Application
• Image
• Audio
• Video

46
MIME Headers

• Goes between the Email Header and Body
• MIME-Version 1.1
• Content-Type
• Content-Transfer-Encoding
• Content-Id
• Content-Description

47
MIME Headers

• Content-Type Type of data used in the body of
  the message
• Text plain, unformatted text HTML
• Multipart Body contains multiple independent
  parts
• Message The body is whole mail message, part of
  a message, or a pointer to a message

48
MIME Headers

• Image The message is a stationary image (JPEG
  or GIF)
• Video The message is an animation (Mpeg)
• Audio The message is 8 kHz standard audio data
• Application The message is a type of data not
  previously defined

49
MIME Headers

• Content-Transfer-Encoding The method used to
  encode the messages
• 7 bit no encoding needed
• 8 bit Non-ASCII, short lines
• Binary Non-ASCII, unlimited length lines
• Base64 6 bit blocks encoded into 8-bit ASCII
• Quoted-printable send non-ASCII characters as 3
  ASCII characters, , is the hex
  representation of the byte

50
Base64 Encoding

• Divides binary data into 24 bit blocks
• Each block is then divided into 6 bit chunks
• Each 6-bit section is interpreted as one
  character incurs a 25 overhead

11001100 10000001 00111001
110011 001000 000100 111001
(51)
(8)
(4)
(57)
(z)
(I)
(E)
(5)
01111010
01001001
01000101
00110101
51
Quoted-Printable Encoding

• Used when the data has a small non-ASCII portion
• Non-ASCII characters are sent as 3 characters
• First is , second and third are the hex
  representation of the byte

01001100 10011101 00111001
()
(9)
(D)
00111101
00111001
01000100
52
MIME Headers

• Content-Id Uniquely identifies the whole
  message in a multiple message environment
• Content-Description defines whether the body
  is image, audio, or video

53
A Multipart, Encoded MIME Message From
joe_luthier_at_plucknplay.comTo lchae_at_mfi.comSubje
ct Info on Gibson guitarMIME-Version
1.0Content-Type multipart/mixed boundary17 -
17Content-Type text/enriched
charset"us-ascii"Content-Transfer-Encoding
8bitContent-Description Greetings As promised,
I'm getting back to you about the Gibson Southern
Jumbo guitar you were Interested in. I've
enclosed a spec sheet on the guitar, which is in
Microsoft Word. I guarantee that you'll love
it! - 17Content-Type application/octet-streamC
ontent-Transfer-Encoding base64Content-Descripti
on Spec sheet saved as MS Word file - 17 -
54
MIME Example
Date Wed, 04 Apr 2001 001137 -0400 From
Meghna Naik ltmnaik_at_UDel.Edugt MIME-Version
1.0 To stoweg_at_hotmail.com Subject
?gb2312?B?1tDOxA? title Content-Type
text/plain charsetgb2312 Content-Transfer-Encodi
ng 7bit a body text, blah, blah
55
Mail Transfer Agents (MTA)

• MTAs do the actual mail transfers
• MTAs are not meant to be directly accessed by
  users.
• Other MTAs are
• Postfix
• Qmail
• MS Exchange
• CCMail
• Lotus Notes
• .etc.

56
Sendmail
It's been said that you aren't a real Unix system
administrator until you've edited a sendmail.cf
file.
It's also been said that you're crazy if you've
done so twice.
57
What is Sendmail?

• Definition Sendmail is the most widely used Mail
  Transport Agent (MTA) on the internet
• MTAs send mail from one machine to another.
• Sendmail is not a client program, which you use
  to read your email.
• Sendmail is one of the behind-the-scenes programs
  which move email over the Internet.
• Normally it runs as a background daemon
• Can even be run out of the super daemon (xinetd)

58
Implementations

• SMTP Gateway
• An SMTP gateway allows users on your network to
  communicate with others on the Internet without
  concern as to which local mail software package
  exists on your network.
• All incoming mail for your network will pass
  through this gateway which converts the message
  into the appropriate format specific to your
  local mail software.
• Similarly, all mail destined for the Internet
  from your network will pass through this gateway
  to be sent across the Internet via SMTP

59
Implemetations

• SMTP Relay Warning Will Rogers
• An SMTP relay is a machine that actually sends
  the mail across the Internet.
• A common misconception is that SMTP gateways are
  the same as SMTP relays. This is not always the
  case.
• There are SMTP gateways that act as relays
  themselves, but there are also many that do not.
  If the latter is the case on your network, you'll
  need to bounce your mail off one of the relays.

60
Installation Methods

• RPM installation
• Obtained from installation CDs
• Binaries (.tgz)
• Obtained from http//www.sendmail.org
• Source Code
• Obtained from http//www.sendmail.org

61
The Pieces

• The binary
• /sbin/sendmail
• The configuration file
• /etc/mail/sendmail.cf
• Supporting files
• /etc/mail/access
• /etc/mail/aliases
• and many more

62
More Pieces

• Email messages are stored in the directory
• /var/spool/mail
• There is a separate file for each user
• Email waiting to be sent
• ./var/spool/mqueue
• A log of Email sent and received
• /var/log/mail

63
Sendmail Features

• Sendmail uses DNS (Domain Naming System)
• But not 100 dependent Joe_at_192.168.1.1
• DNS provides Mail Exchange (MX) Info
• Sendmail can do a DNS double-tap
• Lookup up who the client says they are
• Sendmail default is mail relay off
• Realtime Blackhole Lists (RBL)
• Mail Relay checkers - - Open Mail Relay Db
• http//www.ordb.org/submit/

64
Sendmail Anti-Spam Enhancements

• Mailscanner
• Minimal anti-spam
• Anti-virus integration (scan in/outbound)
• http//www.sng.ecs.soton.ac.uk/mailscanner/
• Or http//www.mailscanner.info
• Spam Assassin
• Rule based heuristic
• Header and text analysis
• Blacklist (RBL)
• Vipul's Razor (http//razor.sf.net)
• http//www.spamassassin.org

65
Mail Access Protocols

• The MTAs place the email in the users mailbox
• The Mail Access Protocols are used by the users
  to retrieve the email from the mailbox
• POP3
• IMAP4

66
POP vs. IMAP
POP3
All Messages
Whole message
IMAP
Mr Smith
Friends
.
headers
67
Post Office Protocol v3

• Simple
• Allows the user to obtain a list of their Emails
• Users can retrieve their emails
• Users can either delete or keep the email on
  their system
• Minimizes server resources

68
Internet Mail Access Protocol v4

• Has more features then POP3
• User can check the email header before
  downloading
• Emails can be accessed from any location
• Can search the email for a specific string of
  characters before downloading
• User can download parts of an email
• User can create, delete, or rename mailboxes on a
  server

69
References

• RFCs
• RFC 821 - Simple Mail Transfer Protocol
• RFC0822 - Standard for the Format of ARPA
  Internet Text Messages
• RFC 1521 - MIME (Multipurpose Internet Mail
  Extensions)
• E-mail Explained
• http//www.sendmail.org/email-explained.html

70
Sendmail Configuration
71
Internal SMTP Issues

• Vrfy name
• Used to verify if a mailbox with the given name
  exists in an SMTP server
• Expn maillist-name
• Used to expand the members of the given maillist
  name
• Both sources of e-mail address for spammers
• Must be disabled

72
Sendmail

• An open source mail transfer agent
• Original version written by Eric Allman in 1980s
  at UC Berkeley
• Descendant of ARPANET delivermail
• Very flexible
• Supports different transfer and delivery
  protocols
• Very complicated
• Difficult to manage
• Configured using sendmail.cf, sendmail.mc
• Unfortunately, known for its bugs

73
Sendmail

• Security measures
• Sendmail restricted shell smrsh
• Standard security checks
• SMTP AUTH
• SMTP STARTTLS
• Rejecting SPAM
• Access database
• Anti-spamming relay features
• Validating senders

74
Sendmail

• Configuring sendmail
• /etc/mail/sendmail.cf
• Actual configuration file
• /etc/mail/sendmail.mc
• More user friendly configuration file
• Make sendmail.cf from sendmail.mc
• m4 /usr/local/share/sendmail/cf/m4/cf.m4
• /etc/mail/sendmail.mc gt /etc/mail/sendmail.cf

75
Sendmail

• Turning off exploitable features
• Find the line in sendmail.cf that contains
• O PrivacyOptions
• Add noexpn and novrfy
• O PrivacyOptionsnoexpn novrfy
• Most strict goaway
• Or set the confPRIVACY_FLAGS in sendmail.mc
• define(confPRIVACY_FLAGS, goaway, noexpn,
  novrfy, nobodyreturn)

76
Sendmail

• SMTP server banner
• May give away system info
• 220 192.168.1.1 ESMTP Sendmail 8.10.2Sun/8.10.2
  Tue,14 Jan 2003 09 2802-0500 (EST)
• Change SmtpGreetingMessage field in sendmail.cf

77
Sendmail

• Precautions against DoS attacks, in sendmail.mc
• Set confMAX_MESSAGE_SIZE to limit message size
• Set confMAX_DAEMON_CHILDREN to limit number of
  processes
• Does not prevent DoS attacks

78
Sendmail

• Controlled SMTP relaying in sendmail
  FEATURE(access_db)
• List the domains you are willing to relay from in
  /etc/mail/relay-domains
• FEATURE(relay_hosts_only)
• Hosts must also be listed
• FEATURE(relay_entire_domain)
• Relay all computers in domain
• FEATURE(access_db)
• Enables or disables access database
• FEATURE(blacklist_recipients)
• Also look up recipients in access database

79
Sendmail

• Controlled SMTP relaying in sendmail
• List the domains you are willing to relay from in
  /etc/mail/relay-domains
• FEATURE(dnsbl)
• Use realtime black hole list at mail-abuse.org
• 1.5.5.192.blackholes.mail-abuse.org 
• IN A 127.0.0.2
• FEATURE(accept_unqualified_senders)
• Allow users without domains
• FEATURE(accept_unresolvable_domains)
• Allow users with unresolvable domains
• FEATURE(relay_based_on_MX)
• Permit any relay directed to your host

80
Sendmail

• Following features make vulnerable to abuse
• FEATURE(relay_local_from).
• Allows relaying if the message claims to
  originate at your domain.
• FEATURE(loose_relay_check).
• turns off checking for explicit routing
• FEATURE(promiscuous_relay).
• turn off all checking for relaying.

81
Sendmail

• Access database
• In /etc/mail/access
• Allow access by individual domains
• Two-tuples key action
• Key
• Fully or partly qualified host name
• Network or subnetwork address
• Specific e-mail addresses
• Can also include FROM, TO, etc.

82
Sendmail

• Actions
• REJECT
• refuse connections from host
• DISCARD
• accept the message but silently discard it,
  sender will think message is accepted
• OK
• Allow access, overrides other checks
• RELAY
• Allow access including relaying
• ERROR arbitrary message
• Reject mail with customized message

83
Sendmail

• Example
• cyberpromo.com REJECT
• sendmail.org RELAY
• spam_at_buyme.com ERROR550 Spammers do not live
  here anymore
• Froma_at_b.com REJECT
• Toc_at_d.com REJECT
• 193.140 RELAY
• Generate database from map
• makemap hash
• /etc/mail/access lt /etc/mail/access

84
Sendmail smrsh

• The smrsh program is intended as a replacement
  for /bin/sh in the program mailer definition of
  Sendmail.
• It's a restricted shell utility that provides the
  ability to specify, through the /etc/smrsh
  directory, an explicit list of executable
  programs available to Sendmail.
• smrsh effectively limits Sendmail's scope of
  program execution to only those programs
  specified in smrsh's directory.

85
Sendmail smrsh

• The sendmail.cf is configured to run /bin/smrsh
  by default
• To prevent duplicate programs, and do a nice job,
  it is better to establish links to the allowable
  programs from /etc/smrsh rather than copy
  programs to this directory.
• For example
• cd /etc/smrsh
• ln -s /usr/bin/procmail /etc/smrsh/procmail

86
Sendmail

• smsrh
• Form an explicit list of executables that
  sendmail is allowed to execute
• sendmail.mc
• FEATURE(smsrh)
• Advised to be used in all sendmail versions

87
Sendmail

• Enhanced File Security
• Tight rules for opening files
• In general, all read directories should be owned
  by root
• No .forward in unsafe (group or world writable)
  directories

88
Sendmail

• Enhanced File Security
• If too restrictive, set the DontBlameSendmail
  option in sendmail.mc
• define (confDONT_BLAME_SENDMAIL,...)
• ForwardFileInUnsafeDirPath
• Allow .forward files in unsafe directories.
• ForwardFileInUnsafeDirPathSafe
• Allow a .forward file that is in an unsafe
  directory to include references to program and
  files.

89
Sendmail

• SMTP-Auth in sendmail
• Install an SASL library
• i.e. Cyrus SASL
• Compile sendmail with right options
• APPENDDEF(confENVDEF', -DSASL')
  APPENDDEF(conf_sendmail_LIBS', -lsasl')
• for Cyrus SASLv1
• APPENDDEF(confENVDEF', -DSASL2')
  APPENDDEF(conf_sendmail_LIBS', -lsasl2')
• for Cyrus SASLv2
• to site.config.m4

90
Sendmail

• Set options in sendmail.mc
• TRUST_AUTH_MECH(GSSAPI DIGEST-MD5')dnl
• define(confAUTH_MECHANISMS', GSSAPI
  DIGEST-MD5')dnl
• define(confDEF_AUTH_INFO', /etc/mail/auth/auth-i
  nfo')dnl
• DAEMON_OPTIONS(a')dnl
• Requiring SMTP AUTH
• Delete all other means of relaying

91
Sendmail

• To use as client,generate an info file
• client-info AuthInfoyour.isp.net "Uroot"
  "Ppassword
• Generate authentication database
• makemap hash client-info lt client-info
• Edit configuration file
• define(SMART_HOST',your.isp.net')
• define(confAUTH_MECHANISMS', DIGEST-MD5
  CRAM-MD5 LOGIN PLAIN')
• FEATURE(authinfo',hash /etc/mail/auth/client-inf
  o')

92
Sendmail

• SMTP STARTTLS in sendmail
• Allow relaying based on certificates
• Restrict incoming or outgoing connections
• Define following variables
• define(confCACERT_PATH', /etc/mail/certs/')
• define(confCACERT', /etc/mail/certs/CA.cert.pem'
  )
• define(confSERVER_CERT', /etc/mail/certs/my.cert
  .pem')
• define(confSERVER_KEY', /etc/mail/certs/my.key.p
  em')

93
Sendmail

• verify macro that keeps result of
  verification
• OK verification succeeded.
• NO no cert presented.
• NOT no cert requested.
• FAIL cert presented but could not be verified,
  e.g., the cert of the signing CA is
  missing.
• NONE STARTTLS has not been performed.
• TEMP temporary error occurred.
• PROTOCOL protocol error occurred (SMTP level).
• SOFTWARE STARTTLS handshake failed.

94
Sendmail

• Relaying based on certificates
• If sender not verified, usual relaying
• If verified, look up the domain of certificate
  issuer, and check access database for that domain
• If result is RELAY, relay
• If result is SUBJECT, look up the subject

95
Sendmail

• Example
• To allow relaying only for a subset of machines
  that have a cert signed by
• /CUS/STCalifornia/Oendmail.org/OUprivate/CN
  Darth20Mail2028Cert29/Emaildarth2Bcert_at_endma
  il.org
• use
• CertIssuer/CUS/STCalifornia/Oendmail.org/OUpr
  ivate/CNDarth20Mail2028Cert29/Emaildarth2Bc
  ert_at_endmail.org
  SUBJECT
• CertSubject/CUS/STCalifornia/Oendmail.org/OUp
  rivate/CN DeathStar/Emaildeathstar_at_endmail.org
  RELAY
• Received header
• (versiontls_version ciphercipher
  bitscipher_bits verifyverify)

96
Sendmail

• Deciding to continue communication
• Two-tuples in access map
• Key clients or servers
• Values
• VERIFY successful verification required
• VERIFYbits successful verification required
  cipher bits gt bits
• ENCRbits cipher bits gt bits
• TLS_Srv, TLS_Clt keywords

97
Sendmail

• Example
• TLS_Srvsecure.example.com ENCR112
• TLS_Cltlaptop.example.com VERIFY112
• E-mail sent to secure.example.com should be
  encrypted
• E-mail sent from laptop.example.com should be
  authenticated

98
Sendmail

• Known application bugs and exploits
• CERT advisories, www.cert.org
• Do not run sendmail as root
• Current versions do not
• Sendmail X new generation of sendmail
• Similar to Postfix architecture
• Not ready for Prime Time

99
Advanced Unix

• Apache Web Server
• November 29, 2005

100
Web Servers

• Tim Berners-Lee is credited with having created
  the World Wide Web
• he was a researcher at the European High-Energy
  Particle Physics lab, the Conseil Européenne pour
  la Recherche Nucleaire (CERN), in Geneva,
  Switzerland.
• A tool was needed to enable collaboration between
  physicists and other researchers

101
Web Servers

• Tim Berners-Lee wrote a proposal called HyperText
  and CERN in 1989
• an extension of the gopher concept but
  incorporated many new ideas and features,
  including
• HTML (HyperText Markup Language)
• HTTP (HyperText Transfer Protocol)
• Web browser client software program
• 1989 it was first installed at CERN
• 1991 it was fully operational

102
Web Servers

• Main type of web servers exist
• For Linux the primary server is Apache
• Fedora Core 3 comes with
• Apache
• Tux
• Stronghold
• Zope
• BOA
• Jigsaw, etc..

103
Apache Overview

• The A Patchy Web server
• put together over time by the Apache group
• Based on the National Center for Supercomputing
  Applications (NCSA) Web daemon.
• The NCSA was created by the National Science
  Foundation (NSF) and the state of Illinois in
  1986 at the University of Illinois
• Apache is free, open-source

104
Apache Overview

• Configured with Text files
• Dependable
• Available for numerous platforms,
• even Windows
• Netcraft.Com shows 76,000,000 web sites
• 70 are Apache
• 21 are Microsoft
• (http//news.netcraft.com/archives/web_server_surv
  ey.html)

105
Apache Overview

• There are two core versions of Apache
• Version 1.3.x
• Fast enough for most sites
• Particularly on 1 and 2 CPU systems
• Version 2.0.x
• More features
• filters
• threads
• portability
• Scales to much higher loads

106
Testing Apache

• Now if Apache is running
• Create two files
• index.htm
• phptest.php
• Save files in
• /var/www/html/
• Document Root Directory

107
Index,htm

• Looks like this

108
Phptest.php

• File looks like this

109
Testing Apache

• Open the web browser on the system that apache is
  configured.
• In the Address bar type in the IP Address of the
  system.

110
Testing Apache

• Now test Apache from another machine on the
  network.
• Open a web browser then type IP Address in the
  address bar.

111
PHP

• PHP is a script language for web sites
• Comes from Perl
• Great for databases and Content Management
  Systems (CMS)

112
PHP

• http//ltyour-ipgt/testphp.php
• Looks like this

113
Apache Configuration
114
Prefork MPM

• Apache 1.3 and Apache 2.0 Prefork
• Each child handles one connection at a time
• Many children
• High memory requirements
• Youll run out of memory before CPU

115
Prefork Directives (Apache 2.0)

• StartServers
• MinSpareServers
• MaxSpareServers
• MaxClients
• MaxRequestsPerChild

116
Worker MPM

• Apache 2.0 and later
• Multithreaded within each child
• Dramatically reduced memory footprint
• Only a few children (fewer than prefork)

117
Worker Directives

• MinSpareThreads
• MaxSpareThreads
• ThreadsPerChild
• MaxClients
• MaxRequestsPerChild

118
KeepAlive Requests

• Persistent connections
• Multiple requests over one TCP socket
• Directives
• KeepAlive
• MaxKeepAliveRequests
• KeepAliveTimeout

119
Apache 1.3 and 2.0Performance Characteristics

• Multi-process,
• Multi-threaded,
• or Both?

120
Prefork

• High memory usage
• Highly tolerant of faulty modules
• Highly tolerant of crashing children
• Fast
• Well-suited for 1 and 2-CPU systems
• Tried-and-tested model from Apache 1.3
• Youll run out of memory before CPU.

121
Worker

• Low to moderate memory usage
• Moderately tolerant to faulty modules
• Faulty threads can affect all threads in child
• Highly-scalable
• Well-suited for multiple processors
• Requires a mature threading library(Solaris,
  AIX, Linux 2.6 and others work well)
• Memory is no longer the bottleneck.

122
Important Performance Considerations

• sendfile() support
• DNS considerations
• stat() calls
• Unnecessary modules

123
sendfile() Support

• No more double-copy
• Zero-copy
• Dramatic improvement for static files
• Available on
• Linux 2.4.x
• Solaris 8
• FreeBSD/NetBSD/OpenBSD
• ...
• Zero-copy requires both OS support and NIC
  driver support.

124
DNS Considerations

• HostNameLookups
• DNS query for each incoming request
• Use logresolve instead.
• Name-based Allow/Deny clauses
• Two DNS queries per request for each allow/deny
  clause.

125
stat() for Symlinks

• Options
• FollowSymLinks
• Symlinks are trusted.
• SymLinksIfOwnersMatch
• Must stat() and lstat() each symlink, yuck!

126
stat() for .htaccess files

• AllowOverride
• stat() for .htaccess in each path component of a
  request
• Happens for any AllowOverride
• Try to disable or limit to specific sub-dirs
• Avoid use at the DocumentRoot

127
stat() for Content Negotiation

• DirectoryIndex
• Dont use wildcards like index
• Use something like this instead
• DirectoryIndex index.html index.php index.shtml
• mod_negotiation
• Use a type-map instead of MultiViews if possible

128
Remove Unused Modules

• Saves Memory
• Reduces code and data footprint
• Reduces some processing (eg. filters)
• Makes calls to fork() faster
• Static modules are faster than dynamic

129
Troubleshooting

• Common pitfalls
• and their solutions

130
Check your error_log

• The first place to look
• Increase the LogLevel if needed
• Make sure to turn it back down (but not off) in
  production

131
Check System Health

• vmstat, systat, iostat, mpstat, lockstat, etc...
• Check interrupt load
• NIC might be overloaded
• Are you swapping memory?
• A web server should never swap
• Check system logs
• /var/log/message, /var/log/syslog, etc...

132
Check Apache Health

• server-status
• ExtendedStatus (see next slide)
• Verify httpd -V
• ps -elf grep httpd wc -l
• How many httpd processes are running?

133
server-status Example
134
Other Possibilities

• Set up a staging environment
• Set up duplicate hardware
• Check for known bugs
• http//nagoya.apache.org/bugzilla/

135
Common Bottlenecks

• No more File Descriptors
• Sockets stuck in TIME_WAIT
• High Memory Use (swapping)
• CPU Overload
• Interrupt (IRQ) Overload

136
File Descriptors

• Symptoms
• entry in error_log
• new httpd children fail to start
• fork() failing across the system
• Solutions
• Increase system-wide limits
• Increase ulimit settings in apachectl

137
TIME_WAIT

• Symptoms
• Unable to accept new connections
• CPU under-utilized, httpd processes sit idle
• Not Swapping
• netstat shows huge numbers of sockets in
  TIME_WAIT
• Many TIME_WAIT are to be expected
• Only when new connections are failing is it a
  problem
• Decrease system-wide TCP/IP FIN timeout

138
Memory Overload, Swapping

• Symptoms
• Ignore system free memory, it is misleading!
• Lots of Disk Activity
• top/free show high swap usage
• Load gradually increasing
• ps shows processes blocking on Disk I/O
• Solutions
• Add more memory
• Use less dynamic content, cache as much as
  possible
• Try the Worker MPM

139
How much free memorydo I really have?

• Output from top/free is misleading.
• Kernels use buffers
• File I/O uses cache
• Programs share memory
• Explicit shared memory
• Copy-On-Write after fork()
• The only time you can be sure is when it starts
  swapping.

140
CPU Overload

• Symptoms
• top shows little or no idle CPU time
• System is not Swapping
• High system load
• System feels sluggish
• Much of the CPU time is spent in userspace
• Solutions
• Add another CPU, get a faster machine
• Use less dynamic content, cache as much as
  possible

141
Interrupt (IRQ) Overload

• Symptoms
• Frequent on big machines (8-CPUs and above)
• Not Swapping
• One or two CPUs are busy, the rest are idle
• Low overall system load
• Solutions
• Add another NIC
• bind it to the first or use two IP addresses in
  Apache
• put NICs on different PCI busses if possible

142
Virtual Hosts
143
Virtual Hosting

• Apache was among the first (the first?) web
  server to offer Virtual hosting.
• With Virtual hosting many URL's can be associated
  with one IP address
• this is useful as IP addresses are a limited
  resource.
• IIS as supplied free with W2K/XP does not support
  Virtual Hosting.

144
Many hosts, one IP

• Several Hosts may translate to the same IP
  address.
• IP addresses are a scarce reource.
• An Apache server listening on 193.111.200.150
  will read the Host field to see where to look
  for the page to serve.

145
Host field

• http//www.ollieclark.com/acronyms.html
• The HTTP request GET /acronyms.html
  HTTP/1.1. Host www.ollieclark.com
• Apache users the Host header to see which domain
  was requested
• this is only available in HTTP/1.1
• Apache checks its virtual hosts for the requested
  Host to see which page to serve or script to run.

146
An Example

• We want to give convenient access to some
  administrative functions at www.myfirm.co.uk site
• We want the URL http//admin.myfirm.co.uk/to
  run a script for administering the site.
• We add a virtual domain admin.myfirm.co.uk
• this is OK as registered .co.uk domain will be
  myfirm.co.uk.
• In fact 'www' indicates a subdomain

147
Adding Virtual Hosts

• NameVirtualHost directive specifies an interface
  on which Apache will accept virtual host
  requests.
• means all interfaces.
• can be several NameVirtualHost directives
• Virtual hosts on the loopback interface

148
Why?

• Why set up virtual hosts on your local computer?
• Use the Hosts file
• On XP in C\WINDOWS\SYSTEM32\DRIVERS\ETC
• also on Linux
• Add entries
• Then http//admin.myfirm.co.uk/ will go the
  local Apache instance which will process the
  Vhosts as it would in a real set up. Useful for
  constructing a website locally.

149
Security
150
Security small rant

• "Security" has three aspects
• Security. Data is not lost.
• Availability. Data is available to its owners
• Privacy. Data is not available to others
• It is trivial to achieve C on its own.
• The challenge is to achieve acceptable levels of
  A and C while allowing sufficient of B.
• Advice to keep an Apache web server secure is
  often just "Don't allow ".

151
Access (external)

• Security as regards visitors to websites hosted
  by Apache on the web-server.
• External security is managed by .htaccess files
• and in the main configuration files
• An .htaccess file is placed in a directory and
  manages access to that directory.

152
.htaccess

• An .htaccess file may be placed in any directory
• It controls many features of how Apache treats
  that directory
• security
• execute scripts
• use server-side includes
• .htaccess files only work if main configuration
  file has permitted them by an apprpriate
  AllowOverride directive

153
Authorization
Name of the password file
Simplest type of password
User must give password
Cannot GET or POST without authorisation
Displayed to user

• To protect a directory /htdocs/secure we place an
  .htaccess file in it
• This is a text file as above.

154
Order Allow Deny

• Three directives really order, allow, deny
• Allow directives specify who can access a
  resource
• Deny directives specify who cannot access a
  resource
• Order directive specifies the order in which the
  Allow and Deny directives are processed

155
Order directive

• Order directive takes a single argument which is
  one of
• Deny,Allow
• Allow,Deny
• Deny,Allow evaluates the Deny directives first
  and then the Allow directives. So the Allow
  directives can override the Deny ones. Any
  request which does not match any directive is
  allowed. So default is Allow access)
• Allow,Deny reverses the ordering. Default is to
  Deny access.

156
Allow/Deny directives

• "Allow from" location
• Recall that the address of the client is supplied
  
• location can be a domain name or partial domain
  name, an IP address or partial IP address
• Allow from comp.leeds.ac.uk would allow
  connections originating from within the School
• Allow from 129.11 would allow connections fro
  any IP address whose first two bytes were 129.11
• Allow from all is legitimate
• the Deny directive has the same syntax,

157
Example

• The example below is one way to allow access to
  clients in the School of computing

Order Deny,AllowDeny from allAllow from
comp.leeds.ac.uk
158
Access (internal)

• The security situation as regards other users of
  the web-server.
• A web-server has three relevant classes of users
• the administrators (root, wheel)
• users (ltusernamegt, users)
• Apache (nobody, users)
• Users manage websites.
• Apache needs access to the users directories to
  retrieve web-pages and execute cgi-scripts

159
A Typical Task

• We have a script that is going to create and
  modify the contents of a file.
• Visitors to the site will make these
  modifications
• We investigate
• file/directory permissions needed to make this
  work.
• how 'insecure' this leaves the files.
• Steps
• review file and directory permissions
• look at application

160
Permissions

• File permissions are 'r', 'w', 'x' set separately
  for owner, group, and other.
• processes run with user and group identity
• owner uses owner permissions onlygroup uses
  group permission onlyother uses other permission
  only

161
Octal

• Octal digits 0-7
• chmod command chmod access file(s)
• Access can be three octal digits
• 1st for owner, 2nd for group 3rd for
• 4 enables read, 2 enables write, 1 enables
  execute
• So 705 enables rwx for owner, no access group, rx
  for other, 777 enables everyone rwx, 700 enable
  rwx for owner but nothing for the group.

162
Paths

• To access a file referenced by a path you must
  have 'x' permission on every directory on the
  path.
• if 'x' is missing then you cannot list a
  directory even
• To read temp.txt requires 'r' a file

163
Create and delete

• To create files in a directory a process must
  have 'w' and 'x' permission on that directory
• If you can create a file you can delete any file
  in the directory
• unless the 'sticky bit' is set, then a process
  can only delete the files it owns (except the
  owner of the directory)

164
Application

• Web page visitors.html invites the user to add a
  comment.
• The work is done by visitors.py which opens the
  file visitors.txt, adds the comment and returns
  the current contents.
• See visitors.html, visitors.py

165
Sample permissions

• Set visitors.py permissions to 755
• Set visitors.html to 644
• Set visitors.txt to 666
• Ser directory of visitors.txt 777
• You see these permissions frequently suggested
• they will work whatever user and group Apache is
  running as.
• typically Apache runs as user nobody (group
  nogroup)

166
visitors.py

• The script opens visitors.txt for appending.
• if the file does not exist it is created
• Creation requires write permissions on the
  directory
• Creation permission on the directory carries with
  it delete permission
• so the script could delete the file if it wanted
  to.
• in fact any Apache script on that server can
  delete the file, not just your scripts.

167
Mitigation

• The malicious user needs to know the file system
  path to the writable directory.
• You only need set other permissions for the
  standard Apache set up. Thus 707, 606, 404 will
  do.
• you can set directory permissions to 705 on your
  home directory. Then other users cannot list your
  directories because they share your group (users,
  typically)
• Some server set ups allow Apache to run as the
  user who owns the file requested

168
Advanced Unix

• Linux Kernel
• December 1, 2005

169
Boot Process
170
Boot Process

• The basic input/output system (BIOS) starts and
  checks for hardware devices.
• Stored in the computers ROM and described as
  firmware.
• Finds the hardware devices (diskette drives,
  CD-ROM drives, and hard drives) needed by the
  boot process.
• Loads and initiates the boot program stored in
  the Master Boot Record (MBR, residing in the
  first sector of the device), and passes control
  to the boot program.

171
First Stage Boot Loader

• Two boot loaders are available Linux Loader
  (lilo) and Grand Unified Bootloader (grub)
• The first-stage boot loader
• reads in the partition table and looks for the
  second-stage boot loader on the partition
  configured as bootable (/boot partition).
• Launches the second stage boot loader.

172
Second Stage Boot Loader

• Presents the user with different OS kernels it
  has been configured to boot.
• Finds the kernel image in the /boot directory.
• The kernel binary is named
• /boot/vmlinuz-ltkernel-versiongt
• Places the appropriate initial RAM disk image,
  called an initrd, into memory. The initrd is used
  by the kernel to load drivers necessary to boot
  the system.
• Hands control to the kernel.

173
grub.conf

• grub.conf generated by anaconda
• Note that you do not have to rerun grub after
  making changes to this file
• NOTICE You have a /boot partition. This
  means that
• all kernel and initrd paths are
  relative to /boot/, eg.
• root (hd0,1)
• kernel /vmlinuz-version ro
  root/dev/hdb3
• initrd /initrd-version.img
• boot/dev/hdb
• default0
• timeout10
• splashimage(hd0,1)/grub/splash.xpm.gz
• title Linux Fedora (2.6.5-1.358smp)
• root (hd0,1)
• kernel /vmlinuz-2.6.5-1.358smp ro
  rootLABEL/ rhgb quiet
• initrd /initrd-2.6.5-1.358smp.img
• title Linux Fedora-up (2.6.5-1.358)
• root (hd0,1)
• kernel /vmlinuz-2.6.5-1.358 ro
  rootLABEL/ rhgb quiet

Specifies the default boot image will be the
first hard entry
Grub will wait for 10 seconds for input from the
user before continuing to boot.
The root partition is the second partition on the
first hard drive.
174
The Kernel

• Initializes and configures the computers memory
  and configures hardware attached to the system
  (processors, I/O subsystems, and storage
  devices).
• Decompresses and mounts initrd to load all
  necessary drivers.
• Mounts the root file system in read-only mode and
  frees any unused memory.
• Starts the init process by running /sbin/init.

175
Initialization Process

• Init parses the /etc/inittab file to determine
  the specifics of what programs to run and at what
  level.
• 0 used to halt the system. The system performs
  an init 0 command and the system is halted.
• 1 Puts he system into single-user mode.
• 2 Puts the system into a multiuser mode but does
  not support networking.
• 3 Puts the system into the standard full
  multiuser mode but does not automatically start
  X.
• 4 Unused.
• 5 X11 Puts the system into standard multiuser
  mode with a graphical (X-based) login.

176
Inittab

• id5initdefault
• Tells the init program what run level to use
  after a reboot.
• sisysinit/etc/rc.d/rc.sysinit
• Tells the init program to run the rc.sysinit
  script.
• Since the second field is empty, the script will
  run at boot time for all run levels.

177
rc.systinit

• Setting the path and the hostname, and checking
  whether networking is activated.
• Mounting the /proc file system
• Setting the kernel parameters
• Setting the system clock
• Loading keymaps and fonts
• Starting swapping
• Initializing the USB controller along with the
  attached devices.
• Checking the root file system.
• Remounting the root file system as read-write.
• Loading modules as appropriate.

178
Inittab (contd)

• Starts the /etc/rc.d/rc script with the
  appropriate run level.
• The rc script executes all of the scripts pointed
  to by the symblic links contained in the
  directory for that run level.
• For example, if the run level is 3, the scripts
  pointed to by the links in /etc/rc.d/rc3.d are
  run.

179
/etc/rc.d/rc3.d

• K01yum K35vncserver K74ypserv
  S12syslog S28autofs S90xfs
• K05saslauthd K36lisa K74ypxfrd
  S13irqbalance S40smartd S95anacron
• K10dc_server K45named K89netplugd
  S13portmap S44acpid S95atd
• K10psacct K50netdump K99readahead
  S14nfslock S55cups S97messagebus
• K12dc_client K50snmpd K99readahead_early
  S18rpcgssd S55sshd S97rhnsd
• K15httpd K50snmptrapd S00microcode_ctl
  S19rpcidmapd S56rawdevices S99local
• K20nfs K50tux S05kudzu
  S19rpcsvcgssd S56xinetd S99mdmonitor
• K24irda K50vsftpd S06cpuspeed
  S20random S80sendmail S99mdmpd
• K25squid K70aep1000 S08iptables
  S24pcmcia S85gpm
• K34yppasswdd K70bcm5820 S09isdn
  S25netfs S87IIim
• K35smb K74ntpd S10network
  S26apmd S90crond

• All the files here are only symbolic links to the
  actual scripts that exist in /etc/rc.d/init.d.
• The system first runs the scripts whose names
  start with K to kill the associated processes ?
  /etc/rc.d/init.d/ltcommandgt stop
• The system runs the scripts whose names st