Benchmarking%20Methodology%20WG%20(bmwg)%2065th%20IETF%20

1
Benchmarking Methodology WG (bmwg)65th IETF
Dallas, Texas, USA

• Tuesday, March 21, 2006, 1300-1510 (Miro)
• Chairs
• Kevin Dubray (kdubray_at_juniper.net)
• Al Morton (acmorton_at_att.com)
• If you are not subscribed to the BMWG mailing
  list and would like to be, put an asterisk () by
  your name when you sign the blue attendance
  sheet.  

2
BMWG Agenda

• Note-Taker(s), IPR, and Blue Sheets
• Extended WG Status (Chairs, 10 min)
• IGP-Dataplane Convergence I-Ds. (Poretsky, 15
  min)
• Core Router Accelerated Life Testing I-Ds
  (Poretsky, 15 min)
• Traffic Control Methodology (Poretsky, 15 min)
• IPsec Benchmarking Terms New Methods (Kaeo, 15
  min)
• Milestones Status (Chairs, 15 min)
• New Work (Chairs, 35 min)
• Protection Bechmarking (Vapiwala)
• IPv6 Bechmarking (Dugatkin)
• Kevin gets the last words today

3
BMWG Activity

• AD/IESG Review
• ltdraft-ietf-bmwg-benchres-term-06.txtgt
• ltdraft-ietf-bmwg-dsmterm-12.txtgt, begin WGC
  shepherding
• WG Last Call
• ltdraft-ietf-bmwg-igp-dataplane-conv-term-10.txtgt
  LC ended 10/26
• ltdraft-ietf-bmwg-igp-dataplane-conv-meth-10.txtgtX-
  Area Comments
• ltdraft-ietf-bmwg-igp-dataplane-conv-app-010.txtgt
• ltdraft-ietf-bmwg-hash-stuffing-05.txtgt WGLC 2/3,
  Comments
• ltdraft-ietf-bmwg-ipsec-term-08.txtgt will proceed
  with Meth
• I-Ds
• ltdraft-ietf-bmwg-ipsec-meth-01.txtgt new
• ltdraft-ietf-bmwg-acc-bench-term-08.txtgt Ready for
  WGLC?
• ltdraft-ietf-bmwg-acc-bench-meth-04.txtgt Ready for
  WGLC?

4
BMWG Activity

• New Work proposals.
• LDP Convergence expired, but
• draft-poretsky-protection-term-00.txt (adopt?)
• draft-poretsky-mpls-protection-meth-04.txt
  (adopt?)
• draft-vapiwala-bmwg-frr-failover-meth-00.txt
  (adopt?)
• draft-popoviciu-bmwg-ipv6benchmarking-00.txt
  (adopt?)
• Expected I-Ds
• ltdraft-ietf-bmwg-bgpcmeth-00.txtgt expected soon?

5
BMWG Activity

• Expired BMWG I-Ds
• ltdraft-ietf-bmwg-acc-bench-meth-ebgp-00.txtgt,
• ltdraft-ietf-bmwg-acc-bench-meth-opsec-00.txtgt,
• ltdraft-ietf-bmwg-fib-meth-03.txtgt, Remove frm
  Goals
• ltdraft-ietf-bmwg-benchres-method-00.txtgt Pending
  term
• RFC Editor none.
• Charter Update
• Revise Milestones, and more discussion.
• New RFC

6
Revised Milestones

• Past Due Milestones
• Done Net Traffic Control Benchmarking
  Terminology to AD Review
• Apr 06 IGP/Data-Plane Terminology I-D to AD
  Review
• Apr 06 IGP/Data-Plane Methodology and
  Considerations I-Ds to AD Review
• July 06 Hash and Stuffing I-D to AD Review
• Dec 06 Router Accelerated Test Terminology I-D
  to AD Review
• Dec 06 IPsec Device Benchmarking Terminology to
  AD Review
• Dec 06 Router Accelerated Test Method. and
  Consid. I-Ds to AD Review
• Dec 07 Net Traffic Control Benchmarking
  Methodology to AD Review
• Jul 07 Basic BGP Convergence Benchmarking
  Methodology to AD Review.
• New
• Dec 06 IPsec Device Benchmarking Methodology to
  AD Review
• Dec 07 Router Accelerated Test Method for EBGP
  to AD Review
• Dec 07 Router Accelerated Test Method for
  Operational Security to AD Review

7
Recap on Protection Proposal

• Proposed in 2002, Yokohama
• Multiple Proposals Emerged, 2003, Vienna
• Harmonized Proposal, 2005, much interest
  (off-list)
• 2006 More Contributors, More comments on-list,
  new members joining to work it
• Room made by progress on other items

8
Hey, Hey My, My 10 Years in the BMWG

• Kevin Dubray

9
10 years, so what?
igp conv app
igp conv term
igp conv meth
ippm frame
ipsec meth
ipsec term
lanswitch
mcast term







bgp control meth
bmwg call
bgp con term
Dsmterm
Ethsw
Fibmeth
fibterm
firewall term
Framerelay
fr term















framework
salt meth
bgp salt
salt opsec
salt term
atm meth
atm abr
benres meth
benres term

mcast meth
lanswitch meth
secperf meth
router framewk
ospf conv cont app
ospf conv cont term
ospf conv cont meth hash
10
Lots of I-Ds

• 35 topics
• 197 I-Ds (over 5,000 pages of reading material
  for those sleepless nights!)
• 18 RFCs (another 6 or so in the pipe)
• Had 2 RFCs in 96

11
Lots of ADs

• Alvestrand
• Bradner
• Bush
• Curran
• Kessens
• Kostick
• ODell
• Wijnen
• Spin-in/Spun out WG! (IPPM)
• Co-chair Al Morton in March of 03

12
Lots of Support

• 250-350 subscribers to the BMWG reflector.
• Over 950 attendees to BMWG meetings.
• New influx of editor interest!

13
Lots of Spam!

• 73,000 pieces of spam scraped off the reflector.
  (estimated since introduction of mailman archive
  in 98).

14
Lots of Use!

• BMWG benchmarks cited in provider RFI/RFPs.
• Cited implementation from test instrument vendor
  community.
• Used by networking gear vendors or public testing
  forums to describe device capability.

15
Lots of Challenges!

• Role of Benchmark
• A test is not always a meaningful test
• A meaningful test is not always a good benchmark.
• Time Place for Everything
• Know when the train is leaving the station.
• But maybe that place isnt the BMWG!
• If you want a more general purpose test forum,
  start a BOF!
• Recruit ACTIVE BMWG supporters!