A new Mailing List infrastructure at CERN

1
A new Mailing List infrastructure at CERN

• Ruben Gaspar Aparicio
• Michel Christaller Ruben Leivas Ledo
• IT - Internet Services Group
• CERN

2
Schedule

• What is Listbox?
• A bit of history
• New service architecture and design
• Externals External Lists
• Conclusions

3
What is Listbox

• Listbox is the Mailing list service at CERN
• It includes the Mailing list editor
  (http//simba.cern.ch) and the mailing list
  archive web site
• Numbers
• 3,300 lists
• Active lists 350 per day, 800 per week
• 8 Gb of web archives, with access control
• 60 users per list (avg), up to 6,000 users per
  list (max)
• 15,000 CERN users
• 35,000 addresses-without-CERN-account
• 1,500 list owners
• Traffic 2,000 (in) / 45,000 (out) messages
  distributed in 24 hours (avg)

4
List properties

• Name of the list (email address)
• Owners
• Members
• Description
• Subscription and unsubscription policies
• Posting restrictions
• Moderation
• Message size limit
• An Archive may exist  authorization settings
• Delivery policy
• Alias

5
Old Service
External internet
SMTP Gateways
Mailbox Stores
HR People database
Listbox4.cern.ch SUN computer
LDAP
CCDB Accounts database
AFS File System
6
Motivations for renewal

• 10 years of home grown software
• High maintenance cost
• Dedicated team and special skills necessary
• Single computer, non scalable architecture
• Service sometimes overloaded, delivery delays
• Flat file database, AFS and LDAP dependencies
• Difficult mirroring of flat files for majordomo
  and ldap repository for SIMBA
• Archiving problems
• Little monitoring and alarms (requires human
  monitoring)
• Missing functionalities
• anti-spam, anti-virus, anti-flood, expiration,
  invalid recipients removal, web archive
  management by owner,

7
Weaknesses Old System

• Weakness Listbox
• Info shared into Listbox4 and LDAP
• Problems of spurious synchronization.
• Membership 0.
• UNIX like
• All is in files.
• SIMBA deals with LISTBOX4 machine and LDAP
  server.
• Breaking Messages.
• Sometimes a message with a strange MIME type or
  with some awful expression inside (Perl can't
  cope with) is breaking "mhonarc" generating a
  core file in the Listbox4 archives directory.
• A lot of different perl scripts for guarantee the
  Majordomo features.
• Resend and bounce messages.
• Member addresses resolutions.

8
New Service
New Listbox Service (load balanced array of
PCs) ____ Mail Distribution (Exchange 2003
Gateways Spam, AntiVirus, Flood Check) ____ Web
Archives (Exchange 2003 Public Folders)
External Lists
New Simba (ASP .NET Web interface)
HR / CCDB databases
List definitions (Active Directory)
9
New Service
List definitions (Active Directory)
Mail Distribution
Exchange Infrastructure
HR People database
CCDB Accounts database
Public Folders (Web archive)
Simba (ASP.NET Web interface)
External Lists
10
New Service
MailBox Public Folders Exchange 2000/2003
External internet
HR database
SMTP Gateways (Spam, AntiVirus, Flood
Check) Exchange 2003
List definitions Active Directory
Externals Lists
New Listbox Service (load balanced array of PCs)
New Simba (Web interface)
11
New Listbox infrastructure

• Infrastructure shared with Mail Service
• All machines Windows 2000/2003, Exchange
  2000/2003, Dual Xeon 2.0Ghz, Hyperthreading on, 1
  Gbit/s network card
• Public folders
• 2x 4U servers, 4GB memory. Mainly Lists Archives
• Databases are replicated
• Front End Servers
• 4x 2U servers, 2GB memory. IMAP, POP, MAPI over
  HTTP and HTTP (webmail) gateway
• Store Servers
• 12x 4U servers, 3-4GB memory, 2 SRCU32 Intel RAID
  controllers with each 1xRAID1 (2x70GB/SCSI),
  1xRAID5 (3x120GB/SCSI).
• SMTP Gateways
• 6x 2U servers, 2GB memory. Windows Load
  Balancing, Symantec Antivirus for Exchange and
  CERN made C Protocol Event Sink
• Spam Content filtering servers (CERN SpamKiller)
• 4x 2U servers, 2GB memory.

12
New Listbox design

• User Objects (CERN accounts or Externals)
• Global security groups
• Owners
• Members
• List Public Folder with email _at_
• Message sent to the PF
• Restriction on who can send messages
• Restriction on message size
• Forwards to the group of members, possible
  delivery in the folder
• Web archive through OWA
• Archive Permissions
• Owners ? Editor
• Members ? Non Editing Authors
• Owners can have more freedom for managing the
  archive.
• delete and modify possible

13
Archive access from Outlook
14
From users perspective

• Almost the same interface for managing lists
• No change in email addresses
• Almost the same functionalities
• What was changed
• Bounced mail goes to sender instead of owners
• No more Majordomo mail commands
• Every subscription must be authenticated
• No more AFS archives / AFS lists
• WEBDAV access / External lists web service

15
CERN External Accounts

• People without a CERN mail account can register
  into Listbox
• Implemented as special user accounts with a mail
  address
• Access restricted to
• SIMBA interface
• Web archives (OWA)
• With a group policy which denies access from the
  network / logon locally on all CERN computers but
  a few servers
• Logon with the mail address (userPrincipalName
  property)

16
CERN External Accounts

• Account is created disabled when address is
  inserted in a list
• Account must be validated with a mail response
• Unused accounts are reclaimed
• MemberOf property
• altrecipientBL
• Web interface to manage account properties
  (validation, password.. ) http//cern.ch/externals
  web
• External Accounts can be reused by other CERN
  Services

17
Externals notification
18
External Lists

• Replaces AFS lists
• Membership is provided through a web service as a
  file
• Email addresses are extracted and matched to user
  objects
• Web interface shows the original file content
  (retains comments)

19
DEMOhttp//simba.cern.ch/
20
Simba web site
21
Owners View
22
Members View
23
Web archive
24
Conclusion

• New Listbox integrated with new Mail
  infrastructure deployed at CERN last year
• Benefits from
• Infrastructure redundancy
• Antivirus
• SPAM fight
• Flood control
• Integrated Web archive
• Integrated repository Active Directory
• Smooth migration undergoing
• New interface in production (combines search on
  the old-new system)
• 450 lists already in the new system
• shorter time in delivery for list in the new
  system
• Owners will have more functionality (manage
  archives, add/delete in bulk)
• Less SPAM hassle for owners (bounces,
  subscription)

25

• Questions