Blacklist, Whitelist

1
Blacklist, Whitelist spamtrap

• Terena EQUAL Workshop
• Dec 9th 2009 amsterdam

2
Index

• SMTP Blacklist
• SMTP WhiteList
• Spamtraps

3
IRISRBL RedIRIS blacklist system
4
IRISRBL motivations

• Which/How many Blacklist to use ?
• SMTP traffic can be slowed with too much DNS
  checks
• But better results (more spam blocked)
• What can we do with the false positives ?
• How fast can a IP address be removed from a
  Blacklist system ?
• How can the NREN provide an additional service to
  their members ?

5
IRISRLB Motivations II

• Commercial Blacklist problems
• For the SMTP provider (listed in it)
• Sometimes outgoing SMTP servers are listed
• Bounce messages
• Infected users sending spam .
• Politics issues
• How to be removed from the list ?
• Need to pay money ?
• 48 hours delay
• To the user of the Black list
• Messages not received
• Manual removing of black list / white list
• No information about why this IP address is
  listed

6
Blacklist implementation I

• Based on part of a bigger product,
• Rks from Sandvine, http//www.sandvine.com
• Service only for own constituency
• http//www.rediris.es/servicios/irisrbl/
• Integrate different sources
• Several blacklist
• White List exceptions
• Events (Spamtraps)
• Only one query to DNS check the blacklist
• Small web interface to remove IP in the
  blacklists
• Only postmaster of the Blacklists (not IP owner)
  can remove IP addresses // false positives

7
Blacklist implementation RKS

• Custom DNS server based with a database backend.
• Incremental feed of information
• Server dont need to restart to add new IP
  addresses.
• Flexible policy to define which feeds to add and
  when a IP is listed.
• Support for different sources.
• Different operating system support.

8
IRISRBL Stats

• More than 60 of RedIRIS constituency is using
  IRISBL.
• About 350 DNS queries/second

9
Whitelist
10
White List

• 2004/2005.
• Lot of black listing problems between
  Universities ISP in Spain.
• SPF was not widely implemented
• Most of the mail providers, were using some kind
  of manual white list .
• No coordination .

11
Other White listprojects

• Some discussion in the E-COAT meetings, provide
  the initial jumpstart information.
• Dutch ISP WL. http//noc.bit.nl/dnsbl/nlwhitelist/
  
• DNSWL.org , http//www.dnswl.org

12
WhiteList motivations

• Our main motivation is to avoid problems with
  blacklisting of SMTP server.
• We only tried a minimum quality requirement for
  being listed in the whitelist.
• Its more important to receive the legal email
  from a blacklisted smtp server than dont receive
  any email at all
• You can use other filters (content filters, etc)
  after the blacklist to avoid this spam

13
WhiteList Vision button up

• Organizations usually exchange emails locally
  (country wide)
• SME partners and big local ISP are the main
  problem
• Including big ISP in the whitelist provide
  visibilit.
• Focus locally and exchange information with other
  similar initiatives.

14
White List format usage

• Two white list zones defined
• ESWL outgoing SMTP server of Abuses members.
• MTAWL White list with big international email
  providers, other organizations and similar
  initiatives.
• White list is provided in different formats
• DNS based (like blacklist)
• Configuration files for different SMTP servers.
• The files can be downloaded from the white list
  page.
• All the IP listed has a abuse/technical contact
  public address for troubleshooting

15
RedIRIS White List
RedIRIS white list Eswl y MTAwl
MTAwl
ESwl

• Yahoo,Gmail, Hotmail

• Goverment

Telecable
Hostalia
Telefónica
RedIRIS witoutSPF
Euskaltel
ONO
Sarenet

• Pymes

RedIRIS
Ya.com

• Agencias,

Hostalia

• Others

TusProfesionales

• zone high DNSwl.org

16
WL policy

• Dont spend too much time thinking how to
  implement it.
• Simple policy you are in the list
• Because you asked for this
• Someone added (mtawl )
• People using the WL, want to have you in the WL.
• WL , dont provide any kind of reputation good
  SMTP behaviour, only states that this is the
  address of an SMTP server that usually dont
  send too much spam.
• But also you provide contact information for
  abuse reporting.
• And our spamtrap system allow us to monitor IP
  address behaviour

17
Version 1.

• Simple Perl scripts .
• Manual processing of the information
• Ad-hoc scripts to add information from other
  White List
• Success
• Used by Universities Spanish ISPs
• Great interest from other groups
• Bank, local government
• Fix most of the black listing problems between
  ISP Universities.

18
Version 2.

• Web interface
• Registry of changes
• Most of the task can be done by the domain
  owners.
• Protocol to import information from other White
  List systems.

19
WhiteList soruces

• Spanish Universities ISP
• SME
• Big SMTP providers
• Feeds from other sources
• DNSWL
• trustedsource

20
Conclusions

• Use a white list to avoid problems caused by
  blacklist, not to provide any kind of email
  assurance.
• Whitelist are useful if people knows and use it,
  (and usually they want also to be there).
• Having different level of quality promotes
  postmaster to reach the high level , improving
  the email quality overall.

Edificio Bronce Plaza Manuel Gómez Moreno
s/n 28020 Madrid. España
Tel. [phone] / 25 Fax [phone]
35 www.red.es www.rediris.es
20
21
SPAMTRAP system
22
Spamtrap

• Fake emails accounts to receive spam.
• Provide information for
• Bad IP addresses that are sending spam(feed
  blacklist system)
• WL SMTP servers sending spam (compromise system,
  detection of bad usage or compromise)
• Early detect system of phising attacks.

23
Spamtrap features

• Use domains subdomains never used before. (ej,
  usr.rediris.es)
• Avoid collisions with real domains addresses.
• Redirect domains to a central machine to avoid
  parsing receive headers.
• Source IP address is always in the first received
  line.
• Publish email addresses in web pages for crawlers.

24
Spamtrap implementation

• Unix server SMTP server (postfix)
• Subdomains provided by universities.
• Simple script to generate fake email addresses
  for the domains
• Publish the information in a web page with a
  warning message.
• Parsing of the incoming emails to remove bounces
  from smtp servers.

25
Spamttrap implementation (II)

• Batch system to avoid system overload
• Real time check against different DNSzones
• Detection of Whitelisted servers sending spam
• URL binary extraction
• Extract malware from the files
• Store evidence for later use

26
Results of Spamtrap

• Blacklist IP addresses that sent spam are used
  to feed the blacklist reputation system in real
  time (5 minutes delay)
• WhiteList IP addresses are verified against
  whitelist to detect infected machine and SMTP
  problems in the whitelist member.
• Phising/trend reporting check some patterns to
  detect phising trends against some organizations
  in Spain.
• Provide information for security groups.

27
Expectations

• Blacklist
• Sharing of blacklist between NRENS
• Commercial agreement (SCS like) for Terena
  members ?
• Improve the tool
• WhiteList
• Sharing of information between different NRENs
• Spamtrap
• Improve the tool
• More robust sensor network.

28
Edificio Bronce Plaza Manuel Gómez Moreno
s/n 28020 Madrid. España
Tel. [phone] / 25 Fax [phone]
35 www.red.es www.rediris.es
28