GLAST Large Area Telescope

1
GLAST Large Area Telescope
LAT Pre-Shipment Review LAT Level Requirement
Verification Compliance Rich Baun SLAC
Systems Engineering
2
Agenda

• Verification Compliance Summary
• Requirements Verification Overview
• Requirement Verification Process
• LAT Level Test Program Overview
• LAT Level Requirement Sell-off Status
• Interface Verification
• Plan to Compliance Closure
• Requirement Issues
• LAT Compliance

3
LAT Verification Compliance Summary

• Verification Summary
• 334 of 459 VPs have been executed to date, as per
  plan
• Verification reports for 329 of the executed VPs
  have been submitted for customer approval
• 5 requirements have issues that require
  resolution
• Preliminary analysis for 62 of the VPs has been
  completed and compliance is fully expected
• 219 of the verification reports submitted have
  been approved
• Compliance Summary
• LAT compliance with all valid requirements is
  fully anticipated, except those noted below
• Science Requirements - Background Rejection
• Details provided in Science Requirements
  Verification presentation
• EMI Requirements - Conducted Radiated
  Susceptibility
• Details provided in EMI/EMC Test Results
  presentation

4
Requirement Verification Overview

• The LAT Level verification program encompasses
  the sell-off of 459 requirements
• 363 Level 2a/2b and 124 Level 3 requirements
  verified at the LAT Level
• 197 requirements allocated for sell-off by LAT
  Level Test in the 36 test cases documented by the
  Performance Ops Test Plan, LAT-MD-02730
• 204 requirements allocated for sell-off by the
  various Analysis Plans
• 22 LAT Level requirements are sold by lower
  level analysis and test
• The remaining 36 requirements are sold by
  inspection
• Classical verification approach utilized
• Each requirement assigned one of four
  verification methods
• Test Test whose results require analysis to
  determine success
• Demonstration - Test with a simple Go/No Go
  success criteria
• Analysis Simulations or models used to verify
  requirement, results often based on or correlated
  to test data
• Inspection Visual inspection to verify
  compliance
• Each requirement assigned a verification level,
  (i.e. device, unit, subsystem, etc.)
• Detailed verification plan written for each
  requirement

5
Environmental Requirement Sell-off

• The environmental reqts provided in the EMI Spec
  and the MAR define the environments in which the
  LAT is tested
• The LAT Performance Spec requires compliance to
  these environments either directly or by
  reference
• Sell-off of environmental requirement VPs from
  the LAT Performance Spec shows compliance to the
  environmental requirements
• The environmental test plans also insure these
  reqts are met by implementing the environments
  specified in the LAT Environmental Specification,
  LAT-MD-00778
• The plans specify when, at what levels, and how
  to apply the environments
• LAT performance through the environments is
  verified by the electrical tests specified in the
  Perf Ops Test Plan
• Electrical tests are performed before, during,
  and after the environments

6
Requirement Verification Flowdown
7
Requirement Verification Flowdown Discussion

• All NASA Level 2A reqts in the SRD, MSS, and IRD
  and the ICDs are flowed to the Level 2B LAT Perf
  Spec (LAT-SS-00010)
• All LAT reqts are flowed to Subsystem specs
  (Level 3) from the LAT Performance Spec
• DOORS is used to document the reqts and reqts
  flowdown
• NASA Systems and all LAT team members have fully
  reviewed the requirements and the associated
  flowdown
• All LAT Perf Spec requirement Verification Plans
  (VPs) are documented by the Verification Cross
  Reference Matrix (VCRM)
• Each LAT requirement is traced to a VP in the
  VCRM
• VPs and the verification traceability is
  documented DOORS

All Level 2A and ICD requirements have been
reviewed, flowed, have a VP, and are in the VCRM
8
Verification Cross Reference Matrix (VCRM)

• The LAT VCRM, LAT-MD-07658, documents the VPs and
  the verification methods, levels, locations, and
  results
• Test and demo VPs are logically collected to form
  test cases as defined by the Perf Ops Test
  Plan, LAT-MD-02730
• http//www-glast.slac.stanford.edu/documents/cyber
  doc.asp?lat_search02730
• Analysis VPs are allocated to the appropriate
  analysis plan and any LAT level data collection
  is specified
• Inspection VPs are documented in DOORS and the
  VCRM
• The VCRM can be found on-line at
• http//www-glast.slac.stanford.edu/systemengineeri
  ng/Verification2020Test/LAT20Level20VCRM20v2
  3.xls
• All VPs have been approved by the LAT team and
  NASA Systems

All VPs have been reviewed approved by the LAT
team and NASA
9
VCRM Example
A complete VCRM is provided in the LAT EIDP
10
Requirement Verification Documentation

• Each reqts verification is documented by a
  Requirement Verification Report (RVR), test
  report, or analysis report
• Each RVR, test report, and analysis report is
  reviewed and approved by Systems Engineering and
  the customer
• RVRs and test and analysis reports often combine
  verifications for like or related requirements
• For example, most Thermal reqts are sold by the
  TV Test Report
• The LAT VCRM summarizes the verification results
• Verification data and RVR, test report, and
  analysis report doc numbers are all provided in
  the VCRM
• LAT compliance to each requirement is documented
  in the VCRM

All requirement verification results are
summarized in the VCRM
11
Requirement Sell-off Process
12
Test Case Run Buy-off

• Test Case Run Buy-Off
• Once IT indicated a test run was complete,
  Systems, requirement owners, and assigned experts
  reviewed the run data
• Each test run had at least two reviewers
• Systems Engineering reviewed all data for every
  run/test case
• Reqt owners/experts reviewed all data for their
  assigned runs/cases
• If a reviewer found the run sufficient, they
  signed off on the hardcopy Test Case Review Log
• Test cases with more than one reviewer were not
  sold until all sign-off
• If an anomaly was found or the run was
  insufficient, it was documented in an NCR and
  corrective action was taken

The LAT verification test program was extremely
thorough and the data review was extensive and
complete
13
LAT Level Test Program (1 of 2)
14
LAT Level Test Program (2 of 2)
15
LAT Level Verification Summary

• Verification Terminology
• Requirement Identified Reqt documented in Level
  2B and/or Level 3 specification
• Flow Down Complete Flow down to Level 3 of
  Level 2A/2B requirement complete
• Draft VP Draft VP has been written and submitted
  to LAT team and customer for review
• Final VP Customer, Systems, and reqt owner have
  agreed on the contents of the VP
• VP Executed Verification plan has been executed
• Report Submitted Draft RVR or report has been
  provided to the customer for review
• Requirements Sold RVR or report approved by
  customer and released into LAT Docs
• Expect Compliance RVR not submitted, but
  preliminary analysis indicates LAT will comply
• VP Deferred Plan execution deferred beyond
  post-ship
• Requirements Issue Stale requirement or waiver
  required

16
Requirement Sell-off Status

• Aside for the requirement issues, reports for all
  VPs planned for pre-ship execution have been
  submitted to the customer
• No issues are anticipated in the remaining
  requirements
• Only a paperwork exercise remains to sell the
  remaining VPs

RVRs have been submitted for all Pre-Ship VPs
17
Interface Verification Methodology

• LAT/SC Interface verified primarily by test and
  demonstration
• Interface demonstrated by SIIS power up of the
  LAT in LAT141
• All SC/LAT prime, redundant, and cross-strapped
  interfaces extensively tested by test case
  LAT13x, using the VSC
• Some unit level test and analysis of interface
  and interface data also utilized
• All verification details are provided in the
  associated RVRs and the results summarized in the
  VCRM

The hardware performance of all LAT/SC
Interfaces has been verified
18
Interface Verification Status

• Aside for requirements issues, reports have been
  submitted to the customer for all interface VPs
• 67 of all interface requirements have been sold
• No issues are anticipated for the remaining
  requirements
• Only a paperwork exercise remains to sell the
  remaining VPs

All Interface requirements will be verified
19
Post-Ship Interface Compliance Plan

• Deferred Interface VPs 7 VPs total
• All deferred interface VPs are related to LAT/GBM
  messaging requirements from the LAT/GBM ICD
• All physical interfaces have been verified
• ECD 15 Dec 06
• Expect Compliance Interface VPs 10 VPs total
• All VPs related to TV analysis
• Preliminary analysis indicates the LAT will
  comply with margin
• ECD 1 Nov 06
• Requirements Issue Interface VPs 1 VPs total
• VP related to Symmetrical Common Differential
  Mode impedance
• Current requirement is stale
• Plan forward
• Analysis of measured data from LAT and SC has
  indicated no issues
• IRD/ICD requirements need to be deleted
• IRD change to be documented by CCR
• CCR in coordination with SASS
• ECD 15 Nov 06

20
Post-Ship Compliance Plan (1 of 2)

• Deferred VPs 63 VPs total
• Gamma Ray Burst (GRB) FSW upgrades provided by
  FSW B1.0.0 account for 60 of the deferred VPs
• FSW awaiting GRB algorithm definition from
  scientific community
• Target of Opportunity (TOO) and Automatic Repoint
  Request (ARR) functional requirement also
  included in this set
• ECD 15 Dec 06
• Remaining 3 VPs are those related to Observatory
  environments
• Observatory shock tests
• Four additional thermal cycles acquired by LAT at
  Observatory TV
• ECD 30 days following Observatory TV

21
Post-Ship Verification Plan (2 of 2)

• Expect Compliance VPs 62 VPs total
• 51 VPs related to TV analysis and LAT Thermal
  Model correlation
• ECD 1 Nov 06
• 6 VPs related to the just executed Mass
  Properties tests
• ECD 15 Oct 06
• 5 VPs related to EMI/EMC Testing
• ECD 15 Oct 06
• Preliminary analysis indicates the LAT will
  comply with margin
• Excluding EMI, see the EMI/EMC Test Results
  presentation

22
Requirements Issues

• Requirements Issues 5 VPs total
• 2 VPs related to Symmetrical Common
  Differential Mode impedance
• Current requirements are stale
• Plan forward
• Analysis of measured data from LAT and SC has
  indicated no issues
• IRD/ICD requirements need to be deleted
• ECD 15 Nov 06
• 1 VP related to Radiator Cut-outs venting in the
  wrong direction
• In coordination with SASS initial assessment No
  issue
• ECD 15 Nov 06
• 2 VPs require internal waivers no impact to LAT
  performance or Observatory level test
• Both waivers to TDF Level 3 Spec, LAT-SS-00019
• CAL TACK Latency LAT-XR-08615
• Subsystem Deadtime Contribution Measurements
  LAT-XR-08616
• ECD 15 Nov 06

23
LAT Compliance

• All Level 2A ICD reqts have been flowed down
  with traceability
• All VPs have been reviewed and approved by the
  customer
• All VPs planned for execution prior to shipment
  have been executed
• RVRs for 98.5 of the VPs planned for execution
  prior to ship have been submitted to the customer
  
• Requirement issues are all understood, only the
  paper trail remains to be completed, i.e. there
  is no risk
• A plan to full requirements sell-off has been
  provided
• Due to thorough test, data review, and analysis
  programs, no issues are anticipated for the
  remaining requirements to be sold
• All performance reqts in this group have been
  verified by preliminary analysis, i.e. Thermal,
  Mass Props, and EMI
• All other requirements are functional, i.e. GRB,
  TOO, and ARR

Full LAT compliance is expected
24
GLAST Large Area Telescope
LAT Pre-Shipment Review LAT Level Requirement
Verification Back-up Charts Rich Baun Systems
Engineering
25
Performance Operations Test Plan

• All test cases are defined in detail in the
  Performance and Operations Test Plan
  (LAT-MD-02730-05)
• The plan identifies 6 categories of test cases to
  be run
• 13 test cases for the LAT CPT
• 6 LAT LPT test cases
• 12 SVAC/ETE test cases
• 3 LAT calibration test cases
• 2 LAT Level test cases
• 1 one-time hardware verification test
• All requirements sold by Test and Demo are
  allocated to test cases by the Perf Ops Test
  Plan
• All functions to be validated are also allocated
  to test cases by the Perf Ops Test Plan
• The plan was reviewed by NASA Systems and
  approved by LAT Systems, ACD, CAL, ELX, FSW, IT,
  MECH, Thermal, TKR

26
LAT Level Test Case Summary (1 of 2)
Item Test Case ID Test Case Name Redundancy Configurations Redundancy Configurations Redundancy Configurations Redundancy Configurations Redundancy Configurations Redundancy Configurations Redundancy Configurations Redundancy Configurations Redundancy Configurations
Item Test Case ID Test Case Name 1 2 3 4 5 6 7 8 9
1 LAT00x LAT Power On X X X X X X X X X
2 LAT01x LAT Power Off X X X X X X X X X
3 LAT02x LAT Reinitialization X X X X   X      
4 LAT031 LAT Electrical Power Subsystem Performance X                
5 LAT04x Establish Science Operations Configuration X X              
6 LAT05x LAT Register Test X X              
7 LAT06x SIU/EPU Hardware Functional X X              
8 LAT071 LAT Energy Measurement Calibration X                
9 LAT12x LAT Science Modes X X              
10 LAT13x LAT/Spacecraft I/F Functional X X X X X        
11 LAT141 SIIS LAT/Spacecraft I/F Functional X                
12 LAT15x LAT Ambient TCS Test X X              
13 LAT16x LAT Ambient Survival Htr Test X X              
14 LAT17x LAT Conducted Radiated Emissions X P              
15 LAT18x LAT Conducted Radiated Susceptibility X                
16 LAT20x LAT Science Performance Diagnostics X X              
17 LAT21x LAT Timing Measure Adjust X X              
18 LAT22x LAT Science Ops Demo X X X X X X      
19 LAT23x LAT GRB Handling X X X X          
27
LAT Level Test Case Summary (2 of 2)
Item Test Case ID Test Case Name Redundancy Configurations Redundancy Configurations Redundancy Configurations Redundancy Configurations Redundancy Configurations Redundancy Configurations Redundancy Configurations Redundancy Configurations Redundancy Configurations
Item Test Case ID Test Case Name 1 2 3 4 5 6 7 8 9
20 LAT30x LAT ACD CPT X              
21 LAT31x LAT ACD LPT X X              
22 LAT40x LAT CAL CPT X              
23 LAT41x LAT CAL LPT X X              
24 LAT50x LAT TKR CPT X              
25 LAT51x LAT TKR LPT X X              
26 LAT521 LAT Config 1 Light Tight Test X                
27 LAT651 TDF False Triggers X                
28 LAT661 TDF Data Transport Errors X                
29 LAT70x LAT SVAC Flight Config on Gnd X X              
30 LAT711 LAT Config 1 SVAC Muon Calibration X                
31 LAT801 LAT Config 1 SVAC Condition Scan (27 V) X                
32 LAT811 LAT Config 1 SVAC Condition Scan (29 V) X                
33 LAT821 LAT Config 1 SVAC Nominal Rate CR X                
34 LAT831 LAT SVAC Nominal Rate CR Data Volume X                
35 LAT841 LAT SVAC Nominal Rate Condition Scan CR X                
36 LAT85x LATSVAC Nominal Rate Condition Scan CR X X              
28
LAT Redundancy Configurations
29
Test Case Run Review

• Test Case Run Review
• Test cases are run by IT in virtually any order
• Systems/Commissioner should review plans to
  insure all case interdependencies are considered
• After each test case is run, IT reviews the data
  and notifies Systems that a run was valid
• A valid run is one in which there were no script,
  database, test set, test software, configuration
  errors, or operator errors that invalidate the
  test
• This review must be within 24 hours of test
  execution
• Obviously, any issues determined to be anomalies
  must be reported through the appropriate system
  (JIRA or NCR)
• Test script, software, and config file errors
  must be documented
• IT must establish and maintain a Test Case
  Review Log that contains the following minimum
  information
• Test case name and number, script, start time,
  run number, LAT configuration discrepancies, IT
  reviewer, and the numbers of any NCRs or JIRAs
  written during the test case
• Log should allow space for expert/Systems sign-off

30
Test Case Run Buy-off

• Test Case Run Buy-Off
• Once IT has indicated a test run is valid,
  Systems, the Commissioner and/or assigned experts
  will review the run data
• Buy-off Process
• Each test case will have at least one assigned
  reviewer
• Systems Engineering will produce a matrix of who
  is responsible to review test case/requirement
  data
• Requirement owners will also review the data
  collected in each test case that is pertinent to
  their requirements
• Reviewers and requirement owners will sign-off on
  the Test Case Review Log when review is complete
• If the reviewer finds the run sufficient, they
  note the test case as complete on the Test Case
  Review Log
• Test cases with more than one reviewer are not
  sold until all sign-off
• If a reviewer detects an anomaly or that the run
  was insufficient, they will report this through
  appropriate documentation (NCR)
• A NCR is needed to justify modification or a
  re-run of the test
• Buy-off must be complete within one week of test
  execution or before test exit, which ever occurs
  first

31
Requirement Sell-Off (1 or 2)

• Requirement Sell-Off, High-level approach
• Only functional requirements will be sold in the
  Baseline LAT Test (or at the earliest
  opportunity)
• All performance requirements will not be sold
  until after all environments have been completed,
  that is following TV
• Performance requirement owners will be expected
  to look at performance data from all test phases
  and perform an early assessment
• Environmental requirements will be sold only
  after testing in the appropriate environment is
  performed
• For example, vibe related requirements will not
  be sold until after Sine-Vibe
• Requirement Sell-Off, Concept of Operations
• Nominally, each requirement owner will have 1
  week after test execution to review the test data
• Owners requiring longer time periods will need to
  negotiate with Systems
• For most requirements, simply executing the test
  case and certifying by review the test was
  successful is sufficient for sell-off as reported
  in an RVR
• For each requirement, reviewer names, when the
  data was reviewed, test case numbers, scripts,
  run times, and run numbers will be logged in the
  VCRM

32
Requirement Sell-Off (2 of 2)

• Requirement Sell-Off, Concept of Operations
  (continued)
• The requirement owner will have 2 weeks after
  test execution to produce a Requirement
  Verification Report (RVR)
• Owners requiring longer time periods will need to
  negotiate with Systems
• Once an RVR is written, it will be reviewed and
  approved by Systems and submitted to the customer
• Submit RVRs to customer for review and approval
• Incorporate customer comments if any
• Release RVRs into LAT Docs with customer approval
• Test Reports can replace RVRs
• Meetings to review RVRs can be held at customer
  request
• The customer, the Commissioner, Systems, and the
  appropriate requirement owners attend
• The VCRM documents all requirement sell-off and
  provides traceability from requirement to RVR

33
Requirements Verification Reports

• For an agreed to subset of LAT requirements only,
  not all reqts
• Must have an assigned LAT Docs number
• Minimum RVR contents
• Requirement owners name
• Requirement source, text, and number
• Verification Plan text and number
• Supporting test case name and number (if
  applicable)
• Script name(s) used to run test
• Date, time, and run number when supporting test
  case(s) were run
• If data was collected from multiple runs, each
  run must be noted
• Compliance statement, i.e. the report must state
  definitively the LAT complies with the
  requirement
• Margin to the requirement must be stated where
  appropriate
• Supporting documentation of test data, analysis,
  and results
• Deviations from the verification plan and the
  reasons for the deviations must be stated
• Any NCRs or JIRAs associated with the requirement
  must be listed