new GetLums.py program and LumDB Status

1
new GetLums.py program and LumDB Status

• Elizabeth Gallas
• Fermilab Computing Division /
• D0 Computing and Analysis Group

Luminosity/Database supersize combo meeting (Get
more for your time) _at_ D0 Collaboration
meeting March 8, 2006
2
Outline

• NEW! GetLums.py program
• Purpose / Uses
• Current Operation
• Setup Logic
• Input
• Current Output
• Input Examples
• Output Examples
• Future plans
• Luminosity Database Status
• Conclusions

3
GetLums Purpose/Uses

• Uses
• Initial request came from user (Kristian
  Harder)
• interested to know how much data is available in
  various bins of instantaneous luminosity
• Global Monitor lmTrigger monitoring
• needs expected rates by trigger for a reference
  run
• Provides both content AND timing cross check for
• Operations Reports
• lmAccess
• Could never replace Reports or lmAccess

4
GetLums.py program Current Operation

• Setup/ Input (always case insensitive)
• in d0cvs package trigdb_xmlclient/GetLums.py
• setup python_dcoracle, oracle_client
• Run, Date, Store (number or range), Trigger List
  
• Logic
• Compile a list of Runs satisfying ALL INPUT
  criteria
• Impose standard Run criteria to reduce excess
  queries by LBN
• an official PHYSICS run using an official Trigger
  List
• with recording ON, during a STORE ...
• Query Lum,Trigger, Run DB to obtain OUTPUT.
• Current Output (for tick 0)
• Total Delivered Luminosity
• Delivered luminosity in bins of instantaneous
  luminosity
• Default 50 bins of instantaneous luminosity from
  0 to 200.

5
./GetLums.py Setup/Run Examples

• Run criteria for one Run, comma sep list or
  range
• Run 208120,208121,207875-207879,208037
• Date criteria for one Date or a date range
  (DD-MON-YY)
• Date 12-Jul-0515-Jul-05
• Store criteria for one Store or set of Stores
• Store 4200-4255
• Trigger List criteria by List name and/or List
  version
• tlname calmuo
• tlversion 14.9
• COMBINED !
• tlversion 14.93 -Run 215295-215548
• Additional optional input
• HELP, DEBUG, FILE
• Change the binning of the Integrated Luminosity
• ./GetLums.py Run 208037 -ilbins 1,101,20

6

• 
  
• Program Input
• ./GetLums.py -debug -Store 4255 -ilbins
  0,80,20
• Mode DEBUG
• Input Store Number 4255
• Input Instantaneous luminosity (min,max,bins)
  (0,80,20)
• Finished reading input.
• 
  
• Input Store list "4255"
• List of Runs based on Store input
• 208033, 208034, 208035, 208036, 208037, 208038,
  208039, 208040, 208041, 208042, 208043, 208044,
  208045, 208046
• Certifying Runs in the input list
  (CertifyRunList)
• Get the LBN Range for good Runs (LBNRangeByRun).

7
Store 4255
8
2. Output (Total Delivered and by Run)

• ./GetLums.py -debug -Store 4255 -ilbins 0,80,20
• Total Delivered Luminosity (in physics runs)
  "2407.50388544"
• Delivered Luminosity by Run number
• Run 208033 "63.4745652029"
• Run 208034 "12.5059345279"
• Run 208035 "40.6773378499"
• Run 208036 "393.123571464"
• Run 208037 "587.512820469"
• Run 208038 "436.72214245"
• Run 208039 "325.171381038"
• Run 208040 "2.54520907788"
• Run 208041 "101.981441656"
• Run 208042 "151.341442274"
• Run 208043 "180.331618514"
• Run 208044 "11.4104579857"
• Run 208045 "17.7370000868"
• Run 208046 "82.9689628446"

9
3. Output (Delivered in bins of InstLum)

• Delivered Luminosity in 20 Instantaneous
  Luminosity bins from 0.00 to 80.00
• Bin 1 (0.00 to 4.00) 0.00 nb-1
• Bin 2 (4.00 to 8.00) 0.00 nb-1
• Bin 3 (8.00 to 12.00) 70.78 nb-1
• Bin 4 (12.00 to 16.00) 307.37 nb-1
• Bin 5 (16.00 to 20.00) 283.42 nb-1
• Bin 6 (20.00 to 24.00) 259.75 nb-1
• Bin 7 (24.00 to 28.00) 232.61 nb-1
• Bin 8 (28.00 to 32.00) 215.87 nb-1
• Bin 9 (32.00 to 36.00) 194.16 nb-1
• Bin 10 (36.00 to 40.00) 181.87 nb-1
• Bin 11 (40.00 to 44.00) 164.40 nb-1
• Bin 12 (44.00 to 48.00) 150.46 nb-1
• Bin 13 (48.00 to 52.00) 136.44 nb-1
• Bin 14 (52.00 to 56.00) 98.48 nb-1
• Bin 15 (56.00 to 60.00) 63.54 nb-1
• Bin 16 (60.00 to 64.00) 48.34 nb-1
• Bin 17 (64.00 to 68.00) 0.00 nb-1
• Bin 18 (68.00 to 72.00) 0.00 nb-1

10
GetLums Future Plans

• New (additional) Input Criteria ?
• Trigger Name (and optionally Trigger Version)
• OUTPUT Recorded Luminosity or Rates by trigger
• LBN or LBN Range
• Operations criteria is imposed but there is
  currently no Quality or Status flag criteria
• Group based Quality criteria ?
• Luminosity Database Status criteria ?
• (Unsure how usable this is in a command line
  program)
• Current limit on number of good runs (1000)
• Oracle limit on WHERE IN CLAUSE

11
Switch to Luminosity Database andOffline
software ApplicationStatus
12
LumDb Short Term Issues

• Loading early data (2 categories)
• April 19, 2002 through August 2002
• pre April 19, 2002 (we need a justifiable reason
  why/not)
• Data Quality (DQ by LBN is waiting)
• final testing cannot start until early data is
  loaded
• TRANSITION_RECORDS
• 38 duplicate records cannot impose unique keys
  until fixed
• TRANS_TIME
• date okay (not null)
• time always 000000
• can we update TRANS_TIME with PVALUE ?
• PVALUE
• Currently python time
• should be data associated with the PNAME
• like the Store number for begin and end of
  store
• loading of fuzzy or other times (transition
  records)
• LBN.FUZZY_TIME ?
• needed for Takas study of flying wires

13
LumDB Long Term Issues

• Procedures for updating the database
• luminosity constant change
• discrepancies found
• establish on case by case basis
• Need program! check database vs online Scalar
  files
• at regular intervals and especially after updates
  (1.)
• Online data loading
• will require maintenance/supervision
• Thanks Vladimir for help so far !

14
Conclude with Recent Progress

• Lots of user input means we are successful
• complaints mean have a product people need/use ?
• size of the luminosity group collaboration of
  testers
• More and better Data Loading (understanding
  content)
• new lmAccess features, documentation
• new Operations Reports
• March 2, 2006, more complete, 10 x faster (short
  Runs)
• Need to
• understand operation and discrepancies
• address performance

15
The Luminosity Database Schema