Events Kernel Sequence Component ESQ One of three components of the Ekernel Subsystem

1
Events KernelSequence ComponentESQOne of
three components of the E-kernel Subsystem

• June 2004

2
EK/ESQ Purpose

• The SPICE E-kernels sequence component (ESQ) is
  concerned with mission event data
• Which events?
• Events relevant to current mission operations
  activities
• Events relevant to providing scientists a
  complete understanding of science data returned
  from the spacecraft, both during the mission and
  years later
• Events could be predicted or actual
• The EK/ESQ subsystem provides standards and
  mechanisms for
• browsing event data interactively
• conveniently accessing event data within
  application software
• documenting event data
• transmitting event data
• archiving event data

3
EK/ESQ Implementation

• The ESQ component uses a SPICE database
  mechanism, called DBK (Database Kernel), for
  storing events and related data
• Suitable for recording sequences of commands,
  spacecraft configuration, state, or other highly
  structured, high-volume data
• Examples
• Galileo, MGS, or Stardust sequences of events
• Galileo observation plans
• Clementine image catalog
• DBK is a simple, stand-alone relational database
  system developed by NAIF
• References to DBK in the remainder of this
  tutorial are applicable to the EK/ESQ

4
DBK Subsystem Features

• DBK characteristics
• All DBK software is provided in the SPICE Toolkit
• No licenses or third-party software involved
• Data are stored in binary files
• Direct access (fast compared to sequential text
  file access)
• Portable to all platforms supported by NAIF
• Provides room for metadata (comments), as in
  other binary kernels
• Utilizes operating-system-independent buffered
  access
• Read and write access is provided by SPICE
  subroutines
• SPICE Utilities/Apps that work with DBK files
• Use INSPEKT to browse DBKs interactively
• Use COMMNT for comment area access

5
DBK Database Organization - 1

• Relational Model
• A DBK database is a set of tables
• Each table is made up of rows and columns
• Tables are rectangular same number of rows in
  each column
• Column Attributes
• Data type
• Character (fixed or variable length)
• Integer
• Double Precision
• Time (similar to DATE type in SQL)
• Dimension (not supported in standard relational
  model)
• Fixed or variable
• Null values are allowed
• Data values may be indexed

6
DBK Database Organization - 2

• Tables and files
• A single table can be spread across multiple DBK
  files
• A single DBK file can contain multiple tables
• A single DBK file can contain pieces of multiple
  tables
• SPICE DBK software can load (access) multiple DBK
  files at one time
• If a table is spread across multiple loaded
  files, this is transparent to INSPEKT and to the
  high-level API routines

7
DBK Query Language - 1

• The DBK query language is used to select data
  from DBK tables
• A query selects entries in specified columns
  belonging to rows that satisfy the querys
  constraints.
• DBKs may be queried via the DBK API or INSPEKT
• The DBK query language is closely related to the
  widely used SQL language
• Restrictions
• Read access only
• Use API calls to write or update a DBK
• Subqueries are not supported
• Arithmetic operations in queries are not
  supported
• Extensions
• Recognizes SPICE time formats
• May retrieve data from array-valued columns

8
DBK Query Language - 2

• Query syntax
• SELECT ltcolumn listgt
• FROM lttable listgt
• WHERE ltconstraint listgt
• ORDER BY ltorder-by column listgt
• Example
• SELECT event_time, command_name, parameter_name,
  parameter_value
• FROM sequence
• WHERE command_name like 36 and ((event_time
  between
• 1985 Jan 1 and January 1, 2000) or
• ( event_time gt Jan 1, 2002))
• ORDER BY event_time DESCENDING, command_name

9
DBK API - 1

• DBK API Components
• DBK query interface
• Principal interface is a suite of routines
  supporting query-and-fetch operations. These
  are
• EKFIND --- issue a query
• EKGC, EKGD, EKGI --- fetch character, double
  precision, and integer data, respectively. EKGD
  is also used to fetch data of type TIME.
• EKNELT --- return the number of elements in a
  specified column entry
• Needed because DBK columns may be array-valued .
• EKPSEL --- parse the SELECT clause of a query
• Returns, attributes of columns in SELECT clause.
• Needed to handle arbitrary queries at run time
  enables calling application to determine on the
  fly which fetch routines to call.
• To make DBK data visible to the query interface,
  one loads DBK files of interest via FURNSH.
  DBK files can be removed from the system by
  calling UNLOAD.

- continues-
10
DBK API - 2

• Low-level DBK reader interface
• Supports reading specified column entries from
  DBK files
• Routines EKRCEC, EKRCED and EKRCEI are used to
  read column entries from character, double
  precision, and integer columns, respectively.
  EKRCED reads data from columns of type TIME as
  well.
• The column entry readers are file-oriented and
  record-oriented the caller reads an entry from
  a record specified by its ordinal position in a
  specified DBK file.
• This capability is an extension of the relational
  model.
• Supports retrieval of schema descriptions from
  DBK files
• Routines EKNSEG and EKSSUM are used to retrieve
  these data.
• Routine EKOPR is used to open a DBK file for
  access by the low-level interface. EKCLS is used
  to close a DBK opened by EKOPR.

- continues-
11
DBK API - 3

• DBK record-oriented writer interface
• Supports creation of tables record-by-record.
• Records may be added in random order.
• Data may be taken from an input stream no need
  to stage large amounts of data.
• File creation is relatively slow.
• To create a DBK using the record-oriented writers
• Open a new DBK for write access using EKOPN.
• Use EKBSEG to start a new segment.
• DBKs consist of one or more collections of data
  called segments. Each segment contains data
  belonging to only one table.
• Add new, empty records to a segment using EKINSR
  (insert record) or EKAPPR (append record).
• Add column entries to records using EKACEC,
  EKACED, and EKACEI for character, double
  precision, and integer data, respectively.
  EKACED is also used to write TIME values to
  records.
• Normally, each new record is filled in with data
  before a new record is started, but this is not
  required.
• When all segments are done, complete the DBK by
  calling EKCLS.

- continues-
12
DBK API - 4

• DBK fast load writer interface
• Much faster ( roughly 100x) than the
  record-oriented interface.
• Typically requires calling application to buffer
  large quantities of data in memory.
• Builds one segment at a time.
• Builds segments by writing the data for each
  column in one shot.
• To use the fast loader routines
• Prepare a new segment for the fast load
  routines by calling EKIFLD.
• Call EKACLC, EKACLD, EKACLI to add character,
  double precision, or integer columns to the
  segment. Use EKACLD to add TIME columns.
• Finish fast load by calling EKFFLD.
• When all segments are done, complete the DBK by
  calling EKCLS.

- continues-
13
DBK API - 5

• DBK update interface
• Allows updating of individual column entries.
• Use EKUCEC, EKUCED, or EKUCEI to update
  character, double precision, or integer column
  entries. EKUCED updates TIME entries as well.
• Allows deletion of records.
• Delete records using EKDELR.

14
DBK API - 6

• Example query-and-fetch with SELECT clause and
  DBK schema known in advance.
• Use EKFIND to issue a query. EKFIND returns the
  number of matching rows, an error flag, and an
  error message.
• Loop over the matching rows, using EKGC, EKGD,
  and EKGI to fetch data from each column. All
  entries are assumed scalar and non-null.

QUERY SELECT CHR_COL, INT_COL, DP_COL,
TIME_COL FROM TABLE1 CALL EKFIND ( QUERY,
NROWS, ERROR, ERRMSG )
DO ROW 1, NMROWS CALL EKGC ( 1, ROW, 1,
CDATA(ROW), NULL, FOUND ) CALL EKGI ( 2, ROW,
1, IDATA(ROW), NULL, FOUND ) CALL EKGD ( 3,
ROW, 1, DDATA(ROW), NULL, FOUND ) CALL EKGD (
4, ROW, 1, TDATA(ROW), NULL, FOUND ) END DO
15
DBK Data Access Using Inspekt

• INSPEKT is an interactive browsing program for
  DBK files
• Command line interface
• Command files
• Symbols
• Command editing
• Extensive formatting options
• Log files
• Create customized reports
• Uses DBK query language, plus various extensions
• Read access only
• There is currently no generic DBK writing program
• (NAIF re-uses a modestly flexible ESQ writing
  program designed to work with JPL sequence
  products - PEF2EK)

16
DBK Limitations

• The DBK subsystem does not support concurrent
  file access by multithreaded or multitasking
  applications.
• Multiple users may read a DBK file
  simultaneously, but there is no built-in
  mechanism to ensure data integrity when a file is
  read and written asynchronously.
• Various SQL features are not supported
• Commands providing write access to DBKs.
• Subqueries
• Expressions in SELECT clauses
• Views