IPY, eGY and Data Management Mark A' Parsons cochair IPY Data Policy and Management Subcommittee

1
IPY, eGY and Data ManagementMark A.
Parsonsco-chair IPY Data Policy and Management
Subcommittee
World Data Center for Glaciology, Boulder
Facilitating the international exchange of snow
and ice data
American Geophysical Union Baltimore, Maryland,
USA 24 May 2006
2
(No Transcript)
3
(No Transcript)
4
0 11 11 01 00 00 11 01 10 11 1
5
(No Transcript)
6
What is a Utility?

• Simple
• Predictable
• Reliable
• Extensible
• Accessible, i.e. usable
• Durable
• It is infrastructure

7
IT is Infrastructure

• The core functions of ITdata storage,
  processing, and transport are becoming costs
  of doing business that must be paid for by all
  but provide distinction to none
• Nicholas G. Carr, IT Doesnt Matter 2003

• We need to start thinking about software in a
  way more like how we think about building
  bridges, dams, and sewers Dan Bricklen,
  Software that lasts 200 Years 2004

8
Timeline of a Utility
Users or Units
eGY / IPY?
Unit Cost
9
What does this mean?

• It can guide our thinking about
• Interface design
• Interoperability
• Transfer mechanisms
• Communication protocols
• Usability
• Software design
• Cost models
• Data preservation
• Distributed vs. centralized data management

10
The Challenge!

• Will you be able to find all the data relevant to
  your research and see relationships between data
  sets. Access
• Will you be able to merge and integrate different
  data sets across experiments and disciplines and
  assimilate them into your model? Interoperability
• Will you be able to subset, visualize, and
  transform the data? Usability
• Will your students be able to retrieve and
  understand IPY4 data in 2050? Preservation

11
Some common themes

• Outreach to the scientific community
• Data discovery and access
• Interoperability
• Explore new technologies and techniquesontologies
  , virtual observatories
• Long term preservation

12
Vision a cultural shift in science
You are not finished until you have done the
research, published the results, and published
the data, receiving formal credit for everything.
Preserve or Perish
13
18 May 2006
14
Interoperability

• What few things must be the same so everything
  else can be different. - Eliot Christian
• A few good standards
• ISO19115 metadata
• Open Geospatial Consortium
• Explore advanced semantic approaches

15
The People Part
A striking proportion of project difficulties
stem from people in both customer and supplier
organisations failing to implement known best
practice.
Oxford University/Computer Weekly survey of
public and private sector IT projects (emphasis
added)
However, people are much more able to adapt to
change, uncertainty, and messy systems

• Service counts.

16
Design for Durability
Fold knowledge into data, so program logic can
be stupid and robust.
TransparencyInteroperabilityExtensibilityStorag
e economy
17
Documentation

• Documentation (metadata) goes on forever
• Describe uncertainty
• Challenge your assumptions to speak to you
  audience

We must not start from any and every accepted
opinion, but only from those we have defined
those accepted by our judges or by those whose
authority they recognize. Aristotle c. 350 BC
18
A scholars positive contribution is measured by
the sum of the original data that he contributes.
Hypotheses come and go but data remain.
Santiago Ramón y Cajal (Nobel Prize
winner,1906) in Advice to a Young Investigator
(1897)