Webbased file systems and WebDAV gateway services to CERN DFS file system

1
Web-based file systems and WebDAV gateway
services to CERN DFS file system

• Alexandre Lossent, Alberto Pace

2
Agenda

• Cross platform File sharing issues
• (from 2 years ago)
• What is WebDAV ?
• WebDAV software overview
• Servers
• Clients
• WebDAV at CERN

3
Agenda

• Cross platform File sharing issues
• (from 2 years ago)
• What is WebDAV ?
• WebDAV software overview
• Servers
• Clients
• WebDAV at CERN

4
Introductory remindera slide from a
presentation made 2 years ago (June 2001)
5
Present values of the AFS file sharing services

• Native to all UNIX platforms
• Fully integrated in the environment, in the shell
• Seen by all applications as a native file
  system
• It is global on the internet
• Files natively shareable across institutes
• Good technical implementation (cache) minimizes
  internet traffic

June 2001
6
AFS is a success in HEP

• Lot of information is currently shared across HEP
  sites using AFS
• An AFS file path, identifies universally a file
  or a directory content
• AFS has become the standard for file and data
  sharing in the UNIX/HEP world
• HEP users are used to share data across the
  internet using AFS

June 2001
7
What we would like for the future

• Native access, including
• Shell access, Access Control, Source control ,
  versioning, offline files
• Global internet sharing
• Consolidated Standard
• Multiple vendors, strong support,
  interoperability, outsourceable
• Cross platform
• Windows, Linux, Mac OS X
• Free
• At least the client software
• Better if source code is available

June 2001
8
A good news

• A web based solution does no longer imply
  non-native
• If the web based file system is implemented at
  the OS level, applications can access it natively
  !

Back to October 2003
9
The Web is part of the solution

• Standard extensions to the HTTP protocol allow
  managing files on web servers as if these would
  be part of the local file system
• HTTP Extensions for Distributed Authoring (WebDAV
  IETF RFC 2518) have been widely adopted on all
  major OS
• Several commercial and public-domain
  implementations exists

10
Agenda

• Cross platform File sharing issues
• (from 2 years ago)
• What is WebDAV ?
• WebDAV software overview
• Servers
• Clients
• WebDAV at CERN

11
WebDAV

• Web Distributed Authoring and Versioning
• IETF RFC 2518 (February 1999)
• http//ietf.org/rfc/rfc2518.txt
• An extension to the HTTP protocol
• New verbs (PROPFIND, MKCOL, LOCK...), headers and
  status codes
• Uses XML to format information
• Initially designed as a way to author web sites
• Redundant with FPSE in the Windows world
• Versioning is limited to file locking (check
  in/out)
• Can be used as a low-end network filesystem
• WebDAV Home page
• http//webdav.org
• See it also for related open-source projects

12
WebDAV today

• File access
• Create / delete files and folders
• Read / write files
• Copy / Move / Delete / rename files and folders
• Document locking
• prevent the overwrite problem, where two or more
  collaborators write to the same resource without
  first merging changes
• Allow implementation of offline folders
• Properties
• XML properties provide storage for arbitrary
  metadata

13
On Windows
Nautilus- Gnome On Linux
14
Windows Console
SSH session On Linux
15
WebDAV tomorrow ?

• Access control
• Set / View / Modify Access Control lists using
  http
• Versioning and Configuration Management
• The V in WebDAV means Versioning
• Document check-out, check-in
• Retrieval of the history list
• Offline files and folders
• Other advanced features
• Symbolic links
• Ordered collections
• Aggregated operations

16
For the advanced features

• The standard is being discussed since few years
• Without real consensus on a common standard
• Proprietary implementations already available
• Example Microsoft Sharepoint services

17
Is this a solution ?

• WebDAV is currently implemented at the
  application level
• This means that applications that are WebDAV
  enabled can see files on web servers, otherwise
  notThis is not a solution. The WebDAV client
  must be done at the operating system level as a
  file redirector. This means that any
  application is able access WebDAV servers. This
  feature is already available on MacOS X, Windows
  XP and partially Windows 2000. There are open
  source implementations on Linux being worked
  at.(see http//webdav.org news)

18
Agenda

• Cross platform File sharing issues
• (from 2 years ago)
• What is WebDAV ?
• WebDAV software overview
• Servers
• Clients
• WebDAV at CERN

19
WebDAV servers

• Supported by all common web servers
• Apache module mod_dav
• WebDAV package in PHP PEAR
• Built-in support in IIS 5 and 6
• Need to activate appropriate HTTP verbs PUT
  (write setting), PROPFIND (directory browsing
  setting)
• Permissions are managed by NTFS ACLs
• Microsoft adds a header to the WebDAV protocol
  for a HTTP GET to return a scripts output or its
  source (source access setting)

20
WebDAV clients

• Clients exist on all major platforms
• See again http//webdav.org
• 3 flavors of clients with different usage
• Basic WebDAV clients or libraries
• GUI or command-line to browse/download/upload
  files (but not edit them in-place) similar to
  FTP
• E.g. IE5, Nautilus, Perl / Java libraries...
• WebDAV-enabled applications
• Can access and edit files over WebDAV
• E.g. Adobe, Macromedia, MS Office...
• Operating system drivers
• Allow to mount a WebDAV resource in a file system
• Enables in-place edition for all applications
• E.g. WinXPs mini-redirector, Mac OS X built-in
  client, Linux davfs...

21
WebDAV clients Windows (1)

• Office Web Folders (msdaipp.dll)
• Included in Office 2000/XP and Windows 2000 but
  NOT in XP/2003 (unless Office is installed)
• Allows in-place edition of Office documents on
  web servers
• Allows Internet Explorer to browse WebDAV and
  FPSE-enabled web sites (like FTP sites)
• Supports HTTPS and all authentication methods
• Implemented as an OLE DB data provider

22
WebDAV clients Windows (2)

• WinXP/2003 WebClient (WebDAV redirector)
• Youll find it in Administrative Tools
  Services
• A network file system driver
• WebDAV folders transparently mounted as file
  system and made available to all applications
  using UNC paths !
• Folders are viewed in Windows Explorer like usual
  folders, with same functionality
• Limited to HTTP on port 80 and basic or Kerberos
  authentication (!)
• We are in discussion with MS to void the http
  limitation

23
WebDAV clients Windows (3)
MS Office
Internet Explorer
Other applications...
Office Web Folders / MSDAIPPShips with MS Office
and W2k
WebClient / WebDAV redirectorShips with Windows
XP/2003
FPSE server
WebDAV server
Windows XP/2003 only
24
WebDAV clients Windows (4)
WebDAV client used by Internet Explorer when a
WebDAV location is opened
25
WebDAV clients Macintosh

• Built-in filesystem driver in OS X
• Similar to XPs WebDAV redirector
• Supports only HTTP and basic authentication (!)
• Works ok will all Mac Applications tested so far,
  including Office
• Open-source Goliath project (Mac OS gt 8.6)
• Supports HTTPS
• GUI interface to check in/out files
• Files are edited in a local folder
• Then use the GUI to check out modified version
• Plus an wide number of commercial applications
  with built-in WebDAV support (as for Windows)
• Example Adobe, Macromedia,

26
WebDAV clients Linux

• Command-line and GUI utilities (FTP-like)
• E.g. Cadaver is available in standard CERN
  distributions
• Filesystem driver
• Davfs (again, limited to HTTP and basic
  authentication) (!)
• But source code available and HTTPS prototype
  exists
• Many open-source libraries
• KDE, PerlDAV, Jakarta Slide Project...
• Plus all commercial applications with built-in
  WebDAV support (as for Macintosh and Windows)
• Generally, not well integrated yet
• The Davfs filesystem being the exception
• This might change with KDE/Gnome projects

27
WebDAV on Windows
28
WebDAV on Macintosh (1)
29
WebDAV on Macintosh (2)
30
WebDAV on Linux
31
Agenda

• Cross platform File sharing issues
• (from 2 years ago)
• What is WebDAV ?
• WebDAV software overview
• Servers
• Clients
• WebDAV at CERN

32
The WebDAVDFS gateway

• Built using IIS 6 / Windows 2003
• Basically a WebDAV-enabled virtual folder that
  points to \\cern.ch\dfs
• All script mappings are disabled
• no need for the Microsoft-specific SourceAccess
  flag
• Basic authentication
• The only method widely supported
• https//dfs.cern.ch/dfs

33
The WebDAVDFS gateway (2)

• Current issues and limitations
• Files Locked in WebDAV are also locked in DFS
• If the client does not explicitly unlock
  them,restart IIS to free the lock (!!)
• Some clients abusively create additional
  files/folders
• Mac OS X
• Permissions cannot be managed not an issue for
  CERN see below
• Internet access limited to https
• A web interface has been added
• https//dfs.cern.ch/dfsExplorer
• Enables access to the DFS from any browser
• Complete file management capabilities
• including permission management
• Aggregate operations

34
Future web site editing

• Replace insecure FTP by a WebDAV service for web
  site editing
• May Run on different port so that the security of
  the main site is not compromised
• Remember the source access setting ?
• Configuration similar to the gateway
• Authentication mandatory
• No script mappings
• Or cross mount local Web storage into DFS

35
WebDAV access to Exchange

• WebDAV is the protocol for exchange / mail
  workflow automation
• Many native Exchange clients use http instead of
  MAPI
• Ximian connector on Linux
• Outlook 2003 uses http instead of MAPI
• After the Blaster Worm MAPI on the internet is
  dead (port 135 closed everywhere)
• At CERN, prototype project to integrate Official
  Leave requests automatically in the Exchange
  calendar using WebDAV

36
WebDAV access to Exchange
37
Summary

• Use of WebDAV as interoperable network filesystem
  possible today
• Can be applied to collaborative tools as well
  (Exchange)
• Takes advantage of HTTP and XML ubiquity
• Excellent level of interoperability for file
  access
• Really reachable from any device / anywhere
• Very simple to implement
• But...
• Still few implementation glitches
• https support is still limited
• Not a high-performance file system
• Not a replacement for native file system (eg
  NTFS)
• Permission management still require custom
  implementations