Software Project

1
Software Project

• Brent Fultz
• California Institute of Technology

• ARCS Project
• Intellectual Coherence
• Distributed Computing
• Tasks Underway
• Coordination between Caltech and SNS

2
ARCS Spectrometer
Moderator
Shutter
Guides
Choppers
Sample
Detector Array
Beamstop
3
ARCS Spectrometer
Headstock
Tailpiece (Whammy Optional)
Frets
Neck
Pickups
Solid Body
Volume/Tone
4
ARCS Baseline Review

• Hardware
• - Detectors inside vacuum
• - Shielding
• Project Management
• - Project Execution Plan
• - Project Baseline Plan
• - Memorandum of Agreement (Caltech-SNS)
• - Budget Authority
• Software
• - Plan

Dates Review March Committee
Report May DOE Instructions September
ARCS Response November
5
Preamp Prototype
6
Hardware Issue 2 Shielding Design

• Effort on
• Beamstop design
• Crosstalk between instruments
• Background from proton transport line and choppers

7
Project Schedule
8
Software ProjectPlan
9
Software Roadmap v. 1.0
10
Software Survey
. . .
10 responses, from serious users. very detailed
comments.
11
User Ratings of Software Packages

User Opinion (2 to 2) Strength of Opinion (0 to
3) Number of Respondents Rating UOSO/N
12
Software Survey Results

• Interesting
• CS ISAW, Unisoft, Computational
  Crystallographic Toolkit
• User ISAW, DAVE, Mslice, McStas, Gulp, Chop
• Problems
• Code Quality and User Opinion are
• not necessarily correlated (McStas is an
  exception)
• 2. We are still looking at details, but we can
  already see that there will be more rewriting
  than expected

13
Issues in Code Development

• X hours to develop a code for yourself
• 3 X hours to make it convenient for others
• 9 X hours to make it run for others on several
  platforms
• Adding personnel to a late software project makes
  the project later. (Brooks Law)

14
Architectural Coherence
Left Saint-Sernin, Toulouse, Romanesque
Pilgrimage Church, c. 1080-1120 Right
Notre-Dame, Amiens, French Gothic Cathedral,
begun 1220
Inconsistencies in Architecture Cathedrals ---
Charming -) Software --- Annoying -(
15
Enforce Coherence -- Write the Manual Now
(Brooks, Mythical Man Month)

• Define needs and scope
• Reference for developers
• Identify and show interrelationships
• Standardize notation
• Documentation

16
Experimental Inelastic Neutron Scattering

Contents
Theory of neutron scattering
Theory of excitations in condensed matter
Data analysis procedures
Software architecture
Reference manual

Book Project Plan
We write it as we learn it
LATEX
Presently 130 pages, on www
Intellectual follow-on to diffraction textbook
Publish electronically?
17
library.caltech.edu
Caltech Library System Caltech Collection of
Open Digital Archives (CODA) Coda Mus. A
passage of more or less independent character
introduced after the completion of the essential
parts of a movement, so as to form a more
definite and satisfactory conclusion. -- Oxford
English Dictionary Caltech CODA uses the Open
Archives Initiative - Protocol for Metadata
Harvesting (OAI-PMH). Archives in production are
registered as OAI Data Providers. As registered
Data Providers, they may participate in federated
searching interfaces and other end user services
offered by a growing number of OAI Service
Providers. Several OAI Service Providers are
listed below.  Caltech CODA runs on freely
available electronic archiving software. The
Caltech ETD collection runs on ETD-db software
developed at Virginia Tech all other collections
run on EPrints software developed at the
University of Southampton. In Production
18

• Python Modern language with clear syntax
• Open source
• Object-oriented
• Interpreted
• Very high level data types
• Easy interface to C
• C Industry standard language with cryptic
  syntax
• Low-level control allows optimum machine
  performance
• Compiled and dangerous, but the only choice
• XML Syntax for organizing data (does no work
  itself)
• Java Some similarities to Python. The choice
  for browser applets. Java 1.4 integrates
  XML-RPC scripting.

19
Key Concept Data Analysis as a Web Service

• Data analysis is a service
• controlled by a web browser
• Computation is arranged by the server
• users web browser issues commands
• and receives results

20
XML and the Network

• XML-RPC is an open standard for remote procedure
  calls
• The users web browser issues commands to a
  server
• The server distributes the work to appropriate
  computers

XML-RPC
Database Server
XML-RPC
XML-RPC
XML-RPC
Web Server
Users Web Browser
Beowulf Cluster
21
The User Interface

• Users navigate the web site with the left frame
  (content is generated dynamically by the web
  server).
• Users select data and calculational tools, and
  add them to the Java applet on the right

22
The User Interface (Labview-like)

• The user wires the boxes together to represent
  data flows
• We intend to allow users to insert new code in
  empty boxes, and archive sessions and procedures

23
Data Analysis Execution

• User hits Run
• Applet translates wiring diagram into XML-RPC
  commands
• Server receives commands, arranges Python
  script, starts data processing.

24
Data Analysis as a Web Service

• The server can provide access to the best
  combination of hardware and software
• Experimental data and analysis codes reside on
  the servers, so little data bandwidth is needed
• Platform independent (a huge savings in effort!)
• Computing resources can be changed without
  affecting the user
• Clean separation of GUI from analysis code
• One web portal for all SNS instruments?

25
Software Project

• Decisions Made
• Python (for users and instrument scientists)
• C (core modules)
• Web service using remote procedure calls
• Immediate Tasks to Test Architecture
• Demonstration web portal (Java/XML -
  XML/Python)
• Rebinning software (Python/C)
• Lattice dynamics (Python/C)
• Contingencies minimized after we write some test
  modules
• A baseline plan is possible by January, 2003.

26
Software Roadmap v. 1.0
27
Born - von Kármán Lattice Dynamics
Simplicity Complexity

• Undergrowth of indices for tensor quantities
• Crystal structure

lattice, l, basis, k
allowed elements depend on symmetry
28
J. M. Ziman, Electrons and Phonons
29
Data Rebinning

• Account for incident flux
• Remove background
• Convert from time to energy
• Correct for detector efficiency
• Bin into rings of constant scattering angle
• Convert from angle to momentum
• Subtract multiphonon and multiple scattering
• Correct for absorption

30
Data Rebinning Tims Test

• Tested rebin procedure in several forms.
• 300 seconds with IDL
• 60 seconds with IDL using a DLL compiled from
  C
• 2 seconds with C Rebinner class

31
First-Order Approx. for SNS Caltech Interaction

• Universities codify science
• National Labs generate science

Detectors Dispersions
SNS Caltech
?
32
Model Fails in Second Order

• Project Phase of SNS
• User interface requires tremendous care from SNS
  and Caltech
• Common standards and documentation
• Analysis software and hardware now at Caltech,
  but must migrate to SNS, ORNL resources
• Operations Phase of SNS
• Role of Caltech and broad community in future
  software maintenance or development?

33
Electronic Publishing
library

documentation
?
?
?
user
codes
server
instrument and data
34
Discussion on SNS - Caltech Interactions

• Coordination of software projects at Caltech and
  SNS? Scope and scope management.
• Expectations of users for software support by SNS
  and instrument scientists.
• Consistency of GUI for SNS instruments? Single
  web portal?
• Who maintains the code?
• Standards for maintainable code with "open source
  coalition"?
• Issues with distributed computing. Lab policies,
  security, graphics, user permissions.
• Issues with releasing software to run on users'
  machines
• Status of storage and archiving of raw data by
  SNS?
• Institutional arrangements with the ORNL
  supercomputing center?
• On-line control of a neutron spectrometer.
  Technical and policy issues.

35
SNS - Caltech Interactions

• Communication is the key. Thanks for coming!
• Please visit the ARCS wiki
• http//viz.cacr.caltech.edu8000/arcs/1

36
(No Transcript)
37
(No Transcript)
38
Agenda
800 AM Breakfast or coffee 830 AM
Introductions 840 AM Overview (Brent
Fultz) 920 AM Directions (Michael Aivazis) 1010
AM Break 1030 AM Demo of web portal for data
analysis (Jonny Lin, Tabitha Swan-Wood) 1050
AM Physical concepts as software classes (Tim
Kelley) 1200 Lunch at the Athenaeum
(hosted) Discussion who what why when
how NeXus classes for analysis classes (Ray
Osborn) 6 PM Heritage Wines? 7 PM Dinner at Cafe
Santorini (no host)