Senior User Group

1
Senior User Group

• 23rd July 2008

2
Introduction Previous Actions

• Nigel Williams

3
Key Messages Actions from Last SUG

• Action 1 NW to arrange for a response to AWM
  questions DONE, but need to discuss further
• Action 2 AMcI to issue workaround document
  DONE, see later
• Action 3 CA to confirm requirements to RDAs and
  MA DONE, further update today
• Action 4 All RDAs to ensure that their Annex XII
  statements are consistent with this version of
  MCIS RDA to update based on implementation date
  and feedback
• Action 5 All RDAs to ensure AA compliance of
  their Annex XII before submitting an aggregated
  claim.
• Action 6 All RDAs to identify what security
  measures are being put in place DISCUSS STATUS
• Action 7 RDAs to submit details of their planned
  ERDF implementation activities
• Action 8 MCIS Team to explain latest Data
  Dictionary and required changes for the XML
  upload - Done for YF, to be planned for other
  RDAs

4
Systems User Testing

• Mark Swan, Jemma Cornish

5
System User Testing, Topics

• System Test coverage results
• User Testing
• Plans for Core User Testing

6
Current System Testing Status

• Release 1.7.6 Full and Core System Users, as
  at 11 July
• Function
• Programme Set-up and maintenance Success, no
  high / critical defects raised
• Aggregate Project Set-up and maintenance of
  priority axis data as above
• Aggregate Claims (Includes generation of claim
  via MCIS and XML)
• Generating Claims Full - Passed Core 2
  highs
• Validation Full - Passed Core passed
• Grant calculation Full - Passed Core -
  passed
• Submission of Claims Full - Passed Core 3
  highs, resolved
• Certification of Claims Full - Passed Core -
  passed
• SAP Payments Interface manual workaround agreed

7
Current System Testing Status (Contd)

• Release 1.7.6 Full System Users, as at 11 July
• Function
• Intermediate Body Project Passed
• Electronic Funding Agreements Passed
• Preparation of Project Claim Passed
• Includes workflow, variance detection, routing
  function, validation rules
• Project Grant Calculation
• Interim Claims Passed
• Restricted Passed
• Submit Project Claim
• 3 stage Approval Process Passed
• Suspend function Passed
• User Visibility Passed
• Comments Passed
• Amending Project Claims Passed
• Risk Rating of Projects Passed

8
Current System Testing Status (Contd)

• Release 1.7.6 Core Full System Users, as at
  11 July
• Miscellaneous Functions
• Government Gateway Passed
• Access Control Passed

9
Test Script Coverage Statistics Last SUG, 25/06
10
Test Script Coverage Statistics This SUG, 23/07
11
Test Script Coverage and Statistics Comparison

• of resolved and closed issues as a of total,
  as at last SUG, 25/06
• Defects 49.1
• Questions 42.1
• Requirements 33.3
• Total 44.7
• of resolved and closed issues as a of total,
  as at this SUG, 23/07
• Defects 72.8 A lot better
• Questions 68.2 A lot better
• Requirements 31.3 Being collated for future
  releases
• Total 64.8 A lot better

12
Test Script Coverage Statistics This SUG, 23/07
13
Test Script Coverage and Statistics Trend
Analysis
14
User Testing

• Jemma Cornish

15
User Testing 17/18 July

• Overall objectives for session
• to check that there are no critical and high
  defects outstanding
• to provide opportunity for RDA to operate their
  own business scenarios and tests
• to provide opportunity for representatives of CA,
  MA and AA to increase their understanding of MCIS
  and ensure it meets their requirements
• to log any new defects found
• Performed by
• One NorthEast
• Certifying Authority
• Managing Authority
• Audit Authority
• MCIS Project Team (support role)

16
User Testing 17/18 July

• Specific objectives for ONE
• Re-run of testing carried out at ONE in April
• Test any amends made to ONE Business Process
  since April
• Review of all issues logged in SharePoint from
  previous testing
• Run through of ONE MCIS Guidance and desk
  instructions
• Improve MCIS expertise to provide effective
  internal help desk support

17
User Testing 17/18 July

• Key Areas Tested
• System administration - viewing of Operational
  Programme data, organisation and user set-up and
  maintenance, use of the Government Gateway and
  passwords
• Setting up and amending applicant projects
• Appraising, approving and making applicant
  projects live
• Submitting, approving, certifying, authorising
  for payment, and paying applicant claims
• Submitting, approving, certifying, authorising
  for payment, and paying aggregate claims

18
User Testing 17/18 July

• Conclusions
• And the voting from the ONE jury.
• We go-live on 28th July
• No show stoppers were found
• Previous issues retested and proved to have been
  fixed
• There are issues still for us to resolve but
  these will not impact on go-live (e.g. issues for
  Requirements User Group to consider)

19
User Acceptance Testing in YF

• Mark Swan

20
Plans for Core User Testing

• 1 August next visit to YF, to cover
• Integration testing of YFs MIS with MCIS
• Representative sample of project types
• Business scenarios
• YFs implementation of the Data Validation rules
• Use of MCIS test scripts and data, as required
• Recording of further defects / observations via
  SharePoint
• Correction of Critical and Highs, as
  necessary
• At a later date, t.b.a., to cover
• Involvement of MA, CA, AA
• End-to-end processes

21
Acceptance Go-Live

• Nigel Williams, Anne Smith

22
Acceptance Go Live, Topics

• Acceptance Approach (see document)
• Acceptance Report
• OGC Gateway Review

23
Acceptance Roll Out Criteria

• Functional Testing
• Non Functional Testing, especially performance
• Security Accreditation, ISO 27001
• Security Testing
• End to End User Testing, Full Core System
• Gateway 4 review, prior to full roll out
• Post Go Live Development Plans
• Implementation Plans for remaining offices

24
Test Acceptance Strategy
25
Acceptance Report

• The Acceptance Report formally records the basis
  on which the Acceptance Panel sign-off that the
  MCIS IT System is ready for live use by CLG and
  in RDAs
• It describes the testing and checking activities
  undertaken to provide the Managing Authority
  (MA), Audit Authority (AA), Certifying Authority
  (CA), and the RDA businesses with confidence that
  the Go-live release of MCIS is complete and fit
  for purpose
• The Acceptance Panel meet Thursday 24/07/08 to
  review the Acceptance Report for the initial
  Go-live release of Full MCIS. A further version
  will be issued after UAT of Core MCIS
• Acceptance Panel
• Managing Authority Margaret Lees
• Certifying Authority Kim Humberstone
• MCIS Senior Responsible Owner Richard Goodwin
• MCIS RDA Full User Jemma Cornish
• MCIS RDA Core User Rocco Volpe

26
Acceptance Report

• Purpose of the Acceptance Report is to document
  that
• The required scope has been delivered
• The system has been through Functional
  Non-Functional System Testing
• Pre-defined Exit Criteria for System Test have
  been met, with any caveats
• Independent User Acceptance tests (performed by
  CLG and the early adopter RDAs) have checked the
  system is complete and contains no critical
  defects
• Remaining defects are known, understood, and are
  not an obstacle to Go-live. Plans for addressing
  these are in place
• Operational processes, resources and environments
  are in place to support live use
• Software is under configuration management.
  Documentation needed for operational support is
  up-to-date, with timescales agreed for addressing
  any gaps
• IT security accreditation (full or interim) in
  place
• Plans for next release are in place

27
OGC Gateway review 4

• Purposes of the OGC Gateway Review 4
• Check that the original projected business
  benefit is likely to be achieved
• Confirm that all necessary testing is done (e.g.
  commissioning of buildings, business integration
  and user acceptance testing) to the clients
  satisfaction and that the client is ready to
  approve implementation
• Check that there are feasible and tested business
  contingency, continuity and/or reversion
  arrangements
• Ensure that all ongoing risks and issues are
  being managed effectively and do not threaten
  implementation
• Evaluate the risk of proceeding with the
  implementation where there are any unresolved
  issues
• Confirm the business has the necessary resources
  and that it is ready to implement the services
  and the business change
• Confirm that there are management and
  organisational controls to manage the project
  through implementation and operation
• Confirm that contract (service?) management
  arrangements are in place to manage the
  operational phase of the contract
• Confirm that all parties have agreed plans for
  training, communication, rollout, production
  release and support as required
• Confirm that all parties have agreed plans for
  managing risk
• Confirm information assurance accreditation/certif
  ication
• Confirm that defects or incomplete works are
  identified and recorded
• Check that lessons for future projects are
  identified and recorded

28
End of Morning Agenda Items Any Other Business?

• All

29
Lunch

• All!

30
Requirements Systems Documentation Mark Swan,
Alex McIntosh
31
Requirements System Documentation, Topics

• Functional Specification
• Feedback on workarounds
• CA Requirements
• MA Requirements
• Data Dictionary status discussion on whether to
  release or not?

32
Current System Testing Status

• Functional Specification
• Analysis
• Regulations
• User Manual
• Managing Authority
• Certifying Authority
• Regional Development Agencies

33
Current System Testing Status (Contd)

• Functional Specification
• Contents
• Overview of System
• System Features
• Workflow
• User Roles Access Control
• Audit Trail
• Programme Functionality
• Interface with SFC07 (including automated
  programme declaration)
• Programme Monitoring
• Programme Closure
• Projects Functionality
• Project Set-Up Maintenance
• Electronic Funding Agreement
• Funding Profile

34
Current System Testing Status (Contd)

• Functional Specification
• Contents Continued
• Project Monitoring
• Visits
• Irregularities
• Project Clawback
• Project Risk Assessment
• Aggregate Claim
• Generate Aggregate Claim Core Full
• Aggregate Claim Grant Calculation
• Certification of the Aggregate Claim
• Automatic Generation pf Payment Files
• Euro Conversion Financial Table Update
• Management Reporting
• Regulatory Reporting
• Programme Monitoring Reporting
• Ad-Hoc Reporting

35
Current System Testing Status (Contd)

• Go-Live Release Work Arounds Document
• Project Categorisation Capture
• Visits, Irregularities Clawback
• Expenditure Forecasting Workbook
• Funding Agreement Schedules
• Intermediate Body to Beneficiary Payment
  Processing

36
Data Dictionary Mark Swan
37
Data Dictionary To Release or Not To Release?

• Release It the Pros
• Early sight of changes since version 1.3 may
  improve understanding of the way the rest of the
  system works
• Overall elapsed time to deliver solution probably
  less
• Release It the Cons
• Inevitable change gt frustration levels?
• Higher initial effort to deliver overall solution

38
Data Dictionary To Release or Not To Release?

• Changes required
• Claims Schema 12 category headings of change,
  driven primarily by the need to record Progress
  Only claims (zero value, but reporting progress
  on targets and indicators), and feedback from
  Systems Testing activities
• Project Schema 7 category headings of change
  categories, primarily driven by the need to
  define match funding organisations more
  precisely, and feedback from Systems Testing
  activities
• Data Dictionary underpinning all of the above.
  167 logged changes in all, (at the detailed
  level), with documented reasons for the change
• Validation Rules optional to apply to RDA
  systems, but is a statement of the checks that
  will be applied to incoming data streams by MCIS.
  So, advised

39
Roll-out PlanPlan for Next Releases Anne Smith
40
Plans, Topics

• Implementation Schedule
• Requirements of MCIS Support in regional
  implementation activity
• Future stages releases

41
Roll-out proposal Core

• Core roll-out should proceed once testing with YF
  has reached the point where
• Upload and onwards processing has been proven for
  a wide range of YF project and claims files
• YF business staff have conducted end-to-end
  scenario testing
• YF business staff are ready to invite CA, MA and
  AA to participate in a UAT Event with the aim of
  signing-off Core MCIS as ready for live use
• We dont know exactly when that point will be
  reached.
• Our working assumption is final week of August,
  based on
• resuming testing with YF at around 1st August
• allowing 2 weeks for file upload testing with YF
  IT
• plus one week scenario testing with YF business??

42
Roll-out proposal Core

• Process
• 1-day visit from MCIS Team to walkthrough data
  design documentation
• 2-3 weeks for RDA to align their source systems,
  and prepare end-to-end scenarios, test data, work
  instructions etc. ready for user acceptance
  testing
• enough?
• 2 weeks for RDA to
• System test prove projects and claims xml
  upload and onwards processing, for a wide range
  of data (project / IT staff, not business people)
• enough?
• Involve CA, MA AA in witnessed end to end
  testing
• Make system available.
• Post go live, internal proving of Management
  Control System.
• How much support do RDAs need?
• Constraints
• Max 2 RDAs at any one time for implementation,
  core or full
• Others?

43
Roll-out proposal Full

• Full roll-out should proceed once Full MCIS has
  been signed-off for live use by ONE, CA, MA
• This point has now been reached!
• Allow 2 weeks for training and familiarisation
• MA, CA, AA will witness end to end testing at
  each RDA
• Then system will be available and made live
• It is RDAs responsibility to prepare for, and
  execute Model Office business simulation

44
Fit with RDAsanticipated business need
45
Release 1 Delivery Plan
46
High Level Plan
47
Functionality to be delivered in further
releases.

• Project Categorisation
• Automatic generation of electronic Funding
  Agreement or contract variation letter
• Fixes and changes required following UAT and
  Model Office phases and ONE and YF
• Medium and low severity defects outstanding from
  System Testing
• Visits
• Irregularities
• Clawbacks

48
Functionality to be delivered in further
releases.

• Project Funding Pot (a useful calculation that
  prevents projects claiming more ERDF than their
  remaining grant minus the value of any
  unconcluded irregularities)
• Expenditure Forecasting
• Real-time push-button reports
• Project Claim Report
• Project Report (printer-friendly snap-shot of a
  project)
• Aggregate Project Clawbacks (clawback from the
  funding available to a Priority Axis e.g. as a
  penalty for a systemic irregularity)

49
Functionality to be delivered in further
releases.

• Electronic interface with SFC07 for
• Maintenance of OP data
• Submission of Declaration
• Payment notification from SFC07
• MCIS-SAP interface for
• Receiving drawdown
• Paying Aggregate Claims
• On line Expressions of Interest

50
Functionality to be delivered in further
releases.

• Functionality that currently cannot be progressed
  
• Global Grants - unclear whether RDA businesses
  will operate this way?
• Article 55 Revenue Generating Projects - EC
  requirements substantially unclear
• Programme Closure Pack - dependent on Article
  55

51
End of Afternoons Business

• All

52
Test Script Coverage Statistics Outstanding
Items