bID Status

1
bID Status Plans

• G. Watts for the bID Group

(I do very little of the work!)
2
bID Group

• Responsible for developing the algorithms
• It is really the analyzers who do this
• Has worked best when done in partnership with
  physics analyzers.
• Produces the large data sets required for
  certification
• Extensive systematic error studies produced
• Generates Tag Rate Functions for application to
  MC
• Shepards all results through an EB
• Documentation, etc.
• Delivers all along with code to collaboration for
  general use.

Develops and Certifies algorithms used to
identify jets containing b quarks
3
Current Results
Three Certified Algorithms
SVT
Reconstructs a secondary vertex using tracks,
measures the decay length from the primary vertex.
CSIP
Counts tracks with a large impact parameter
Forms a probability distribution based, in part,
on tracks impact parameter.
JLIP
Very Close
SLT
Looks for a soft muon in a jet. Tag Rate
Functions are sample dependent.
4
For Each Algorithm
SVT
At least 3 well defined, tuned operating points
with varying fake rates
5
And
Code
Which is capable of being run in TMB, TMBTree,
and top_tree environments. See btags_cert package
for running pre-defined operating points. See
d0root_example cvs package for example usage of
btags_cert and also coding at the raw level.
Tag Rate Functions
Predict tagging rate in MC, derived from
data TRFs for b-quarks, c-quark, light-quarks
(with and w/out muons in jet). Scale factor to
translate from MC tagging rates to Data tagging
rates.
6
And
Documentation
Certification Documentation All plots and scale
factors, and procedures that went into the
certification process. Usage Documentation Minimal
documentation on how to run and build in your
analysis environment can be found in
d0root_example/doc. There is almost no
documentation on proper usage in an analysis.
7
This is a lot of work!
8
A New World Order
(can we go back to the old one on Tuesday?)
Context
The New bID Editorial Board
Certification
9
The Food Chain
Tracking Hits
Jets
Tracks
bID Algorithm
Analyzer
Before bID
Primary Vertex

• Tracking Algorithms
• Jets and Jet Energy Scale
• PV Finding and Errors

10
Organization
(the analyzers perspective)
This slide is not worth it, but it helps to
motivate turbulence ahead
bID EB
Analyzers with bID Algorithms
bID Group
What you really want to know is
11
How do I get my Cool new bID Algorithm Public!?
Physics Group
bID Group
Good enough to Certify!
I have an Idea!
Certify Me!

• Systematic Errors
• Efficiency, Fake Rates
• Documentation
• Code for Public

bID Group EB
12
But Each Physics Analysis Already Has One EB!?
Why are we adding a second one?
13
But Each Physics Analysis Already Has One EB!?
Certification is Hard!!
This is the task list from the p14 certification,
a second generation effort!
14
I cant read that
(these are the p14 certification guidelines)

• Provide 3 operating points (background of 0.3,
  0.5, and 1.0)
• Code and Documentation for collaboration (TMB and
  TMBTree)
• Performance on Data (eff and fake)
• Performance on MC
• Data/MC Ratios
• Systematic Errors for eff and fake

• Create TRF macros (about 6 of them) for
  collaboration to summarize the above results
• Test on common samples and provide results for
  implementation cross checks
• Web Page and DZERO Note

5-7 million events were used
15
The Motley Crew
EB 33
Mark Strovink (former bID Godparent)
Frank Filthaut (chair, former bID Convener)
Christophe Clement (ttbar production convener)
Avto Kharchilava (former Higgs Convener)
16
Who Develops The Algorithms?
Im not aware of any bID algorithm not developed
by someone doing a physics analysis!
Often work on new algorithm starts in a physics
sub group.
(Combining taggers is the hot topic now)
Come talk to bID group early!

• People in group can offer a lot of support
• Techniques
• Data samples suited to task
• Pitfals weve learned to avoid the hard way

In the end
You will have to go through bID certification to
get your algorithm out
17
P14 Certification Postmortum

• Took gt 3 months to complete Certification
• Another month for documentation
• 2.5 months for over committed Godparent to finish.

(new tracking in p14)
We were really late
3 Major Issues Identified
(supposed to be fixed before publication)

• Negative Tag Rates in Different Samples are
  different for no apparent reason.
• Performance in different MC samples also
  different for no apparent reason.
• Different groups used different efficiency and
  systematic calculation methods.

?
gt6 months to settle these ?
?
18
What Are The Plans?
Short term
P14 Data, Pass2, JES 5.3
December
Do it better
19
Basic Approach
No Major Improvements to the Algorithms Planned
Though several on the decks
Pass 2 contains no major upgrade to tracking
Hoping a major re-tuning will not be required!
But
Hoping a major re-tuning will not be
required! Gives a chance for some new people to
come up to speed. Get automated tools in place
So that if improvements/tunes do come along we
have a chance
20
Who Is Doing What?
SVT
Daniel Boline(BU), N. Narin, Jonas
These are the people who told us explicitly of
their plans There may be more And there is
plenty more to do!
JLIP
B. Clement, D. Bloch
CSIP
F. Rizatdinova, A. Khanov
Datasets
TMBTree - Herb
TopTree Top group
Automated Tools
Me Anyone who will help me
SLT
We need someone to work on this!
21
How Are We Doing?
Data Sets
TMBTree test samples generated Learning how to
put results in SAM (disk space crunch) top_tree
understood, many samples availible
Tagging
New people are learning how to use their
tools. Old-hands are tuning things up. Systematic
Studies and new Methods are ongoing Hopefully can
incorporate some of this
Automated Tools
Produces most of required MC plots and MC
TRFs Has just started to produce data-based fake
rate plots and TRFs Runs on TMBTree and top_tree
(for comparison)
22
Automation
Goal Produce all Plots and TRFs required for
Certification

• Use same interface collaboration uses
• btags_cert
• Generate TRFs that can be used with btags_cert
• Uses certification samples
• Good cross check of all certification efforts
• If works well, perhaps can be moved to in future.
• Probably not good enough to tune an algorithm.

Aimed at the experienced non-bID analyzer
23
Things To Do Better This Time
Documentation!
How to use in an analysis! More guidence on using
in an analysis (MC or Data Tagging)
Different Operating Points
Not clear the Collaboration uses all operating
points Some desire for even looser operating
points.
Smaller Systematic Errors
Larger Data Samples Increased and uniform MC
samples Better tuning for System 8
24
What is on Deck?
(This is what Im aware of)
Adaptive Vertexing
Waiting For Optimization and Vertex Group
Certification!
A. Schwartzman
There is also beam-spot constrained vertexing
25
What is on Deck?
Combination Tools
Large group, many people!
Waiting For Proposed Algorithm, someone to do
the certification.
26
Questions To bID Users
(we will send this list to the convenors)
What analyses use bID? How is MC using bID
treated? Flavor-dependent TRFs or Data/MC Scale
factor? Any bID development occuring in group?
Espeically if it hasnt been shown in bID
group? What aspects of bID limit you most?
(Efficiency, background, systematic errors). If
you had to prioritize bID improvements What
versions of JES you plan to use and when?
27
Plans For Farther Out

• Beyond fixed operating points
• TRF parameterization using new variables
• PV Error, Number of Tracks, etc.
• Systematic Studies
• Charm eff
• Luminosity Dependence
• NIM paper
• After this round of certification?
• How do things fit in with CAF?
• BID branch in the root tree that runs all
  necessary algorithms.

28
Cant Quite Do It Alone

• A single data format
• CAF should reduce a number of the difficulties we
  had with the last certification.
• New JES
• The JES needs to be availible significantly
  before bID certification.
• We have to TMBTree all of our data
• Over Smearing of MC Jets
• This factors into how MC is treated, and the
  Data/MC scale factor and MC TRF.
• People to Help!

29
Conclusions

• p14 Certification is finished.
• Few outstanding issues will be resolved in next
  round of certification.
• Moving from Godparent to EB
• Will likely mean faster turn-around for
  certification
• But more eyes, more cross checks.
• Next Certification pass 2, JES 5.3
• Many small improvements.
• Automation
• Several larger ones are on deck
• Combination, new PV
• Improve collaboration Interaction
• Easier to use tools from bID (documentation)!
• bID work going on in physics groups also reported
  on in bID group!
• Longer Term
• New ideas
• Enable collaboration to generate new operating
  points