ETSI Testing Projects ICT-OSA/Parlay Workshop Brazil, March 2006

1
ETSI Testing Projects ICT-OSA/Parlay
WorkshopBrazil, March 2006

• Anthony Wiles
• Manager
• ETSI Protocol and Testing Competence Centre

2
Some Major ETSI Testing Projects

• A few words on the ETSI PTCC
• IPv6
• 3GPP UMT
• HiperAccess/WiMAX
• VoIP
• IP Testing Open Library
• And a few words on the ETSI PlugtestsTM Service

3
The ETSI PTCC

• Protocol and Testing Competence Centre
• In-house team of experts providing direct support
  to ETSI Technical Bodies (for over 12 years)
• Development of protocols and profiles and the
  application of modern specification techniques
  (35)
• Standards Engineering
• Validation of standards (5)
• E.g., testbed activities (IPv6, SIP/H.323)
• Development of test specifications (60)
• Conformance tests
• Interoperability tests
• Network Integration tests
• Others (PHY, performance )
• http//www.etsi.org/ptcc

4
Specialist Task Forces (STFs)

• PTCC STFs produce test specifications
• Led and managed by PTCC
• Which projects are decide by the ETSI Board
• Experts are seconded from the ETSI membership
• Anything from 2 10 experts in an STF
• Short projects
• e.g., 2 man-months maintenance of VoIP tests
• Long projects
• e.g., UMTS testing 58 man-months per year over
  3-4 years
• PTCC STF Budget 2006
• From Funded Work program (0,6 M)
• 3GPP contribution (1M)
• eEurope (value 0,8M)
• 15 20 PTCC STFs per year

5
Typical ETSI Test Specifications

• Cellular GSM and 3G (UMTS) terminals
• Now includes IMS
• WiFi HiperMAN, HiperACCESS, WiMax
• VoIP H.323, SIP, SIGTRAN
• Service Creation OSA/Parlay (API, IDL)
• IPv6 Core, Security, Mobility, v4-v6
• Cordless phones DECT
• Radio communications TETRA, DMR
• Access terminals FSK, SMS
• Broadband ISDN, DSL
• Smartcards Readers, cards, security modules
• Intelligent Transport Systems (ITS) DSRC
• Future More Security, more ITS, GRID ...

6
IPv6 Test SpecificationsSTF 276

• Joint ETSI eEurope project
• Supported by the ETSI membership
• Mainly vendors
• Require IPv6 test specifications using ETSI
  methods and test languages
• Designed to be part of 3GPP and TISPAN NGN IPv6
  testing strategy (where applicable)
• European Commission
• Mandate for IPv6 device interoperability
• Amplify Europes voice in IPv6 testing
• Liaising with the IPv6 Logo Program
• For more info visit www.ipt.etsi.org

7
Two Phases (2005 2007)

• Phase 1 Core IPv6
• Total value approx. 0,5M
• Jan Dec 2005
• Phase2 IPSec, Mobile IPv6, IPv4-6 Transitioning
• Total value approx. 1,0M
• Jan 2006 May 2007
• Provided as
• Funding
• Provision of experts
• Hosting costs
• Travel
• All technical work done at ETSI by STF276
• Currently 11 participants in the STF

8
Outputs

• IPv6 Testing Framework
• Published (update due end 2006)
• IPv6 Requirements Catalogue
• On line as part of IPT Library
• Test specifications
• Interoperability test specifications
• Conformance test specifications
• Library of TTCN-3 test code freely available
• On-line, extensible and reusable
• All tests are validated (probably 30 of the
  effort)
• On ETSI IPv6 Testbed
• By member companies
• For more info visit www.ipt.etsi.org

9
Results Phase 1

• Over 1100 Core IPv6 Requirements identified
• Including 3GPP
• 300 Core IPv6 conformance tests
• fully validated
• 120 Core IPv6 Interoperability Tests
• fully validated
• All these test specifications including TTCN-3
  code are free and open source

10
STF276_at_Work
11
3GPP UMTS (terminal) Test Specifications (STF 160)

• Nobody can make money without interoperable
  handsets (UE)!
• UE Interoperability towards access technologies
  is the key issue for mobility
• Conformant UE Interoperability (of UE in 3G
  networks)
• Good experience from GSM
• Solution conformance testing
• Very detailed test specifications
• Based on the 3GPP core specs
• Before commercial products enter the market
• Voluntary declaration, but certified
• One of the most important UE quality benchmarks

12
Methodology for 3G Testing
Test Cases (TTCN) 34.123-3, 51.010-5
PICS (capability applicability) 34.123-2, 51.010-2
Test Purposes,34.123-1 Descript (prose), 34.108,
51.010-1
Core specs 25.331, 25.321, 2 24.008, 44.018,
44.060
13
3G Test Industry Commitment is Vital

• The TTCN test cases are implemented in the System
  Simulators (SS) on different platforms
• Must secure the commitment of the SS vendors
• SS vendors organise verification and regression
  testing
• Each and every test case is validated
• Bug-free tests have a high real value
• Important to ensure that a TC will execute
  correctly even after updates
• Early identification of clients who deploy the
  tests
• Prioritising TCs
• Scheduling the deployment
• Certifications

14
Funding

• Test specs developed at ETSI by STF160
• Duration of several years
• Shared costs to ensure the funding
• 3GPP funding from Organisation Partners
• Main source, approx 1M (61 mm) /year)
• Voluntary contributions from the marketing
  partners
• Range from 65k - 150k (5 12 mm) /year
• Man-power (free-of-charge)
• Range from 15 - 18 mm /year

15
Typical 3GPP UMTS UE Tests

• Protocol and signalling tests via radio interface
• Largest part of the testing process
• Full conformance tests
• USIM functional tests
• Identification, PIN, account charging, network
  priority, phonebook
• UICC tests (SIM electrical test)
• power on/off, 1.8v/3v detection, presence
  detection
• bit/character transmitting duration, command
  processing and error handling
• Acoustic tests
• Loudness rating, idle noise, echo
• Radio Frequency Radio Resource Management tests
• Characteristics and performance tests of
  transmitter receiver

16
Users of 3GPP Tests

• Global Certification Forum (GCF)
• PCS Type Certification Review Board (PTCRB )
• GCF PTCRB are the major clients for W-CDMA
  (FDD) GSM/GPRS test specs
• TD-SCDMA Industry Alliance (TDIA)
• TDIA is the major clients for TD-SCDMA (LCR TDD)
  test specs
• GCF WI-10 selected 400 TCs as high priority
• Divided in 4 packages
• Progressing sequentially one by one (milestone
  80)
• UMTS mobile certification will be started if 80
  of each package are validated

17
Approval Process

• Test cases are approved by e-mail
• No quality compromise
• Two week consultation period
• TCs are approvedone by one
• Each test must run on at least one SS platform
  against two independent commercial mobiles
• Log files and radio conditions as support
  information for approval
• Reason for this
• TTCN TC, SS and UE are developed at the same time
• None of them is perfect

18
Results

• 400 TCs in 8 ATSs, 25 MB code
• Platform independent
• Running on gt 4 platforms
• We have reached the GCF 80 validated milestone
• 3G mobile certification starting at beginning of
  2005
• The light is green for mass 3G mobiles entering
  the markets
• 4 types of 3G mobiles are ready to go
• Begun the real 3G test business
• 4 validated 3G SS platforms available in the
  markets
• 4 test houses in Europe opening 3G test services
• 3G TTCN experts becoming hot in the labour market
• We know the price of each 3G coded TC
• 10 k /TC
• 3G investment for 4 years 4 mil. for 400 TCs
• All TTCN code can be downloaded free of charge
• Code is continuously maintained and delivered
• Every three weeks for 3G members
• Every three months for everybody

19
VoIP-related Test SpecificationsSTF 269, STF 270
and others

• SIP (RFC 3261)
• Over 400 conformance test cases
• Fully validated (executables available on
  commercial tools)
• STF 270 is doing the maintenance
• H.225
• About 300 conformance test cases
• Fully validated (executables available on
  commercial tools)
• No ongoing maintenance
• SIGTRAN
• Test purposes for M3UA, M2UA, SCTP
• TTCN-3 available commercially (not ETSI)
• IMS SIP (together with SDP)
• 3GPP RAN5 (STF 160)
• TTCN-3 will be used
• Work started March 2006
• Based on the generic SIP test suite (see above)
  augmented to concentrate on 3GPP SIP specific
  requirements

20
ETSI WiMAX Test Specifications STF 252

• WiMAX Forum Board meeting in January 2005
• Agreed to harmonize the WiMAX/HiperMAN base
  specifications
• Agreed to harmonize the WiMAX/HiperMAN test
  specifications
• Agreed to exploit ETSIs testing expertise
• Agreed to accelerate the work using the ETSI STFs
• Agreed that validation of the test specifications
  needed
• complex interplay of all parties involved
• WiMAX Forum, ETSI BRAN, equipment vendors, ETSI
  STF, test tool developers, test labs, etc.
• ETSI WiMAX Co-operation agreement signed in 2005

21
Results

• Wave 1 Joint test specifications approved and
  published
• Completed Dec. 2005
• PICS Protocol Implementation Conformance
  Statement
• TSSTP Test Suite Structure and Test Purposes
• ATS Approx 60 test cases implemented, validated
  and used in certification
• First WiMAX products have been certified
• Wave 2 New drafts reaching approval
• PICS new updated version with corrections
• TSSTP new draft with close to 1400 Test
  Purposes
• ATS new draft with close to 300 additional test
  cases developed and entering validation

22
IPT LibrarySTF 295

• IP Testing Library
• Developed by ETSI TC MTS
• All MTS IP and IP-related conformance and IOP
  tests
• PICS, TPs, Test Suites etc.
• Due to be published April 2006
• SIP, IPv6, H.323, H.248
• Includes IPv6 Requirements Catalogue
• Linked to TPs and TTCN-3 code
• Powerful search and selection possibilities
• E.g., allows test suite tailored to specific
  search criteria

23
Screen Shot of IPT Library
24
PlugtestsTM ETSI Interoperability Service

• Created in 1999, specialized in the organization
  oftest events
• Open to all organizations
• Experts from ETSI and other technical communities
• Skilled ETSI Staff managing test events

25
Why?

• Provides useful feedback to the standardization
  process
• Accelerates time to market via quick product
  debugging
• Improves both quality and features of
  implementations
• Helps to ensure end users satisfaction
• Reduces deployment costs
• Allows networking with the market key players

COST TIME SAVING!
26
Upcoming events in 2006
Q1 2006 SIM/Handset Lawful Interception 3rd WiMAX Plugfest Digital Identity Q2 2006 IPv6 Asia 4th SQTE EU round NGN Mobile Application NGN_at_Home
Q3 2006 OSA Parlay UWB SIGTRAN Q4 2006 Human Factors (HF) IPv6 French Summit Interop (FR) Powerline RFID ITS ecall
27
Thank You!
Phone [phone] e-mail
ptcchelp_at_etsi.org Website www.etsi.org/ptcc

• Anthony Wiles
• ETSI Protocol and Testing Competence Centre