Welcome to CREST SEBASECREST Project Overview

1
Welcome toCRESTSEBASE/CRESTProject Overview

• Mark Harman

2
CREST Members
Chris McCulloch CREST Manager
Mark Harman Director
Nicolas Gold Deputy Director
Yuanyuan Zhang PhD Student
SEBASE
Shin Yoo PhD Student
Afshin Mansouri Research Assistant
Chris McCulloch Project Operations Manager
Laurie Tratt Research Assistant
EvoTest
Youssef Hassoun Research Assistant
Kiran Lakhotia PhD Student
CONTRACTS
Zheng Li Research Assistant
Kiarash Mahdavi Research Assistant
Tao Jiang PhD Student
Independent PhD Students
Kostas Adamopoulos PhD Student
Ghassan Dukes Part time PhD Student
Jaeeun Yi PhD student
Kazhaw AlKhaffaf Part time PhD Student
Marian Mohr PhD Student
Connie Bao PhD Student
Independent PhD Students
3
Search Based Software Engineering

• Overview of project management
• project structure
• staffing appointments
• meetings and internal training
• headline early results
• industrial involvement and outreach
• non standard dissemination
• service to community

4
What is SBSE?

• In SBSE we apply search techniques to search
  large search spaces, guided by a fitness function
  that captures properties of the acceptable
  software artefacts we seek.
• Genetic Algorithms, Hill climbing, Simulated
  Annealing, Random, Tabu Search, Estimation of
  Distribution Algorithms, Particle Swarm
  Optimization

5
The Eight Queens Problem
6
The Eight Queens Problem
7
The Eight Queens Problem
Perfect
8
The Eight Queens Problem
Perfect
Score 0
9
The Eight Queens Problem
10
The Eight Queens Problem
11
The Eight Queens Problem
Two Attacks
12
The Eight Queens Problem
Score -2
Two Attacks
13
The Eight Queens Problem
14
The Eight Queens Problem
15
The Eight Queens Problem
Three Attacks
16
The Eight Queens Problem
Three Attacks
Score -3
17
That was easy
18
Generate a solution
Place 8 queens on the board
so that there are no attacks
19
Scale up Generate a solution
Place 44 queens on the board
so that there are no attacks
20
Scale up Generate a solution
Place 400 queens on the board
so that there are no attacks
21
Checking vs Generating

• Task One
• Write a method to determine which is the better
  of two placements of N queens
• Task Two
• Write a method to construct a board placement
  with N non attacking queens

22
Checking vs Generating

• Task One
• Write a method to determine which is the better
  of two placements of N queens
• Task Two
• Write a method to construct a board placement
  with N non attacking queens

23
Checking vs Generating

• Search Based Software Engineering
• Write a method to determine which is the better
  of two solutions
• Conventional Software Engineering
• Write a method to construct a perfect solution

24
Checking vs Generating

• Search Based Software Engineering
• Write a method to determine which is the better
  of two solutions
• Conventional Software Engineering
• Write a method to construct a perfect solution

25
Checking vs Generating

• Search Based Software Engineering
• Write a method to determine which is the better
  of two solutions
• Conventional Software Engineering
• Write a method to construct a perfect solution

26
Checking vs Generating

• Search Based Software Engineering
• Write a fitness function to determine which is
  the better of two solutions
• Conventional Software Engineering
• Write a method to construct a perfect solution

27
Checking vs Generating

• Search Based Software Engineering
• Write a fitness function to guide a search
• Conventional Software Engineering
• Write a method to construct a perfect solution

28
Checking vs Generating

• Search Based Software Engineering
• Write a fitness function to guide automated
  search
• Conventional Software Engineering
• Write a method to construct a perfect solution

29
Temporal Testing
optimal point of time for triggering the airbag
igniter
deceleration
tmin
t
tmax
30
Insertion
Individuals
optimal point of time for triggering the airbag
igniter
Mutation
Test data
Duration
Testexecution
Monitoring
Fitness values
Recombination
End?
deceleration
Selection
tmin
t
tmax
31
Evolution is not random
32
SBSE is so generic

• Testing
• Fitness function coverage, time,
• Representation input vector
• Near neighbour similar input
• Restructuring
• Fitness function cohesion and coupling
• Representation mapping nodes to clusters
• Near neighbour clusters are one move apart

33
SBSE is so generic

• Testing
• Fitness function coverage, time,
• Representation input vector
• Near neighbour similar input
• Restructuring
• Fitness function cohesion and coupling
• Representation mapping nodes to clusters
• Near neighbour clusters are one move apart

34
SBSE is so generic

• Testing is a search problem
• Fitness function coverage, time,
• Representation input vector
• Near neighbour similar input
• Restructuring is a search problem
• Fitness function cohesion and coupling
• Representation mapping nodes to clusters
• Near neighbour clusters are one move apart

35
SBSE Applications

• Transformation Cooper, Ryan, Schielke,
  Subramanian, Fatiregun, Williams
• Requirements Bagnall, Mansouri, Zhang
• Effort prediction Aguilar-Ruiz, Burgess, Dolado,
  Lefley, Shepperd
• Management Alba, Antoniol, Chicano, Di Pentam
  Greer, Ruhe
• Heap allocation Cohen, Kooi, Srisa-an
• Regression test Li, Yoo, Elbaum, Rothermel,
  Walcott, Soffa, Kampfhamer
• SOA Canfora, Di Penta, Esposito, Villani
• Refactoring Antoniol, Briand, Cinneide, OKeeffe,
  Merlo, Seng, Tratt
• Test Generation Alba, Binkley, Bottaci, Briand,
  Chicano, Clark, Cohen, Gutjahr, Harrold,
  Holcombe, Jones, Korel, Pargass,
• Reformat, Roper, McMinn, Michael, Sthamer,
  Tracy,
• Tonella,Xanthakis, Xiao, Wegener, Wilkins
• Maintenance Antoniol, Lutz, Di Penta, Madhavi,
  Mancoridis, Mitchell,
• Swift
• Model checking Alba, Chicano, Godefroid
• Probe distion Cohen, Elbaum
• UIOs Derderian, Guo, Hierons
• Comprehension Gold, Li, Mahdavi
• Protocols Alba, Clark, Jacob, Troya
• Component sel Baker, Skaliotis, Steinhofel, Yoo

36
SBSE Applications in which SEBASE is active

• Transformation Cooper, Ryan, Schielke,
  Subramanian, Fatiregun, Williams
• Requirements Bagnall, Mansouri, Zhang
• Effort prediction Aguilar-Ruiz, Burgess, Dolado,
  Lefley, Shepperd
• Management Alba, Antoniol, Chicano, Di Pentam
  Greer, Ruhe
• Heap allocation Cohen, Kooi, Srisa-an
• Regression test Li, Yoo, Elbaum, Rothermel,
  Walcott, Soffa, Kampfhamer
• SOA Canfora, Di Penta, Esposito, Villani
• Refactoring Antoniol, Briand, Cinneide, OKeeffe,
  Merlo, Seng, Tratt
• Test Generation Alba, Binkley, Bottaci, Briand,
  Chicano, Clark, Cohen, Gutjahr, Harrold,
  Holcombe, Jones, Korel, Pargass,
• Reformat, Roper, McMinn, Michael, Sthamer,
  Tracy,
• Tonella,Xanthakis, Xiao, Wegener, Wilkins
• Maintenance Antoniol, Lutz, Di Penta, Madhavi,
  Mancoridis, Mitchell,
• Swift
• Model checking Alba, Chicano, Godefroid
• Probe distion Cohen, Elbaum
• UIOs Derderian, Guo, Hierons
• Comprehension Gold, Li, Mahdavi
• Protocols Alba, Clark, Jacob, Troya
• Component sel Baker, Skaliotis, Steinhofel, Yoo

37
Overall Methodology
WP1 Test Case Design (KCL)
WP2 Searching Design Spaces (York)
WP3 Software Eng Management (KCL)
WP4Multi Criteria Dependable Design (York)
WP5 Non-functional Under change (York)
WP6 Refactoring (KCL)
38
Overall Methodology
WP1 Test Case Design (KCL)
WP2 Searching Design Spaces (York)
WP3 Software Eng Management (KCL)
WP4Multi Criteria Dependable Design (York)
WP5 Non-functional Under change (York)
WP6 Refactoring (KCL)
WP7 Characterisation (Bham)
WP8 Algorithms (Bham)
39
Overall Methodology
40
Advantages of SBSE

• Scalability
• Robustness
• Insight
• Feedback

41
Advantages of SBSE

• Scalability

• Naturally Parallel
• Anytime Algorithm
• Often prepared to wait

42
Advantages of SBSE

• Scalability

• Naturally Parallel

43
Advantages of SBSE

• Robustness

• Explore sensitivity
• Stable areas of space
• Good Neighbourhood

44
Advantages of SBSE

• Insight

• Landscape structure
• Metric improvement

45
Advantages of SBSE

• Insight

• Landscape structure
• Metric improvement

46
Advantages of SBSE

• Feedback

• Generations
• Near neighbours

47
SEBASE Staff

• Kings College London
• Chris McCulloch Project Operations Manager
• Shin Yoo PhD Student
• Laurence Tratt 3-year RA
• Afshin Mansouri 5-year RA
• University of York
• Paul Emberson, 3-year RA
• Simon Poulding, 5-year RA
• David White, PhD Student
• University of Birmingham
• Per Kristian Lehre, 5-year RA
• Ramón Sagarna, 3-year RA
• Andrea Arcuri, PhD Student
• SEBASE Additional Research Students
• Yuanyuan Zhang, KC Wong Award PhD Student, Kings
  College London.
• Peter Laurens, IBM CASE Award PhD Student,
  University of York.

48
SEBASE Staff

• Kings College London
• Chris McCulloch Project Operations Manager
• Shin Yoo PhD Student
• Laurence Tratt 3-year RA
• Afshin Mansouri 5-year RA
• University of York
• Paul Emberson, 3-year RA
• Simon Poulding, 5-year RA
• David White, PhD Student
• University of Birmingham
• Per Kristian Lehre, 5-year RA
• Ramón Sagarna, 3-year RA
• Andrea Arcuri, PhD Student
• SEBASE Additional Research Students
• Yuanyuan Zhang, KC Wong Award PhD Student, Kings
  College London.
• Peter Laurens, IBM CASE Award PhD Student,
  University of York.

49
KCL Staff
Laurie Tratt Research Assistant
Mark Harman Project Director
Chris McCulloch Project Operations Manager
Shin Yoo PhD Student
Yuanyuan Zhang PhD Student
Afshin Mansouri Research Associate
50
KCL Staff
Laurie Tratt Research Assistant
Mark Harman Project Director
Chris McCulloch Project Operations Manager
Shin Yoo PhD Student
Yuanyuan Zhang PhD Student KC Wong funded
Afshin Mansouri Research Associate
51
York Staff
David White PhD Student
John Clark Principal Investigator
Iain Bate Co-investigator
Peter Laurens PhD Student
Kamran Ghani PhD Student
Simon Poulding Research Associate
Paul Emberson Research Assistant
52
York Staff
David White PhD Student
John Clark Principal Investigator
Iain Bate Co-investigator
Peter Laurens IBM CASE Award PhD Student
Kamran Ghani PhD Student
Simon Poulding Research Associate
Paul Emberson Research Assistant
53
Birmingham Staff
Ramón Sagarna Research Assistant
Xin Yao Principal Investigator
Per Kristian Lehre Research Associate
Andrea Arcuri PhD Student
54
Additional Collaborators

• Phil McMinn, Sheffield
• Massimiliano Di Penta, Sannio

55
Visiting Fellows

• Darrell Whitley
• visiting in July
• Marc Shoenauer
• Successful Dagstuhl proposal for April 2008
• Hans Paul Schwefel

56
Additional Potential Visiting Fellows

• Myra Cohen, UNL, USA
• Lionel Briand, Oslo, Norway
• Enrique Alba, University of Malaga, Spain
• Bogdan Korel, IIT, USA

57
First publications emerging already

• ISSTA paper accepted (Shin Yoo)
• GECCO paper accepted (Yuanyuan Zhang, Afshin
  Mansouri)
• TAIC PART paper submitted (Andrea Arcuri)
• CEC paper UIOs accepted (Xin Yao and Per Kristian
  Lehre)
• CEC paper EDAs for testing accepted (Andrea
  Arcuri and Xin Yao)
• CEC paper on memetic testing accepted (Andrea
  Arcuri and Xin Yao)
• GECCO paper accepted (Laurie Tratt)
• ICSM paper submitted (Antoniol, Di Penta, Harman)
• ISSTA paper accepted (Harman, McMinn)
• SBSE keynote at SQS, ICPC, AST and FoSE at ICSE
  (Mark Harman)

58
Meetings

• September 2006 2 days Full all three partners
  and industry
• February 2007 2 days York, Birmingham at
  Birmingham
• March 2007 2 days Birmingham, Kings at Kings
• March 2007 2 days York, Kings at Kings
• May 2007 1 day Management team meeting at
  Birmingham
• May 2007 1 day Steering committee meeting
• July 7th 2007 GECCO workshops
• July 8th 2007 SEBASE collaborators workshop at
  Kings
• July 30th, 31st Aug 1st 2007 Summer School at
  Birmingham
• September 18th and 19th 2007 Steering committee
  plus EPSRC

59
Meetings

• September 2006 2 days Full all three partners
  and industry
• February 2007 2 days York, Birmingham at
  Birmingham
• March 2007 2 days Birmingham, Kings at Kings
• March 2007 2 days York, Kings at Kings
• May 2007 1 day Management team meeting at
  Birmingham
• May 2007 1 day Steering committee meeting (now)
• July 7th 2007 GECCO workshops
• July 8th 2007 SEBASE collaborators workshop at
  Kings
• July 30th, 31st Aug 1st 2007 Summer School at
  Birmingham
• September 18th and 19th 2007 Steering committee
  plus EPSRC

60
Internal Project Training

• July 30th, 31st Aug 1st 2007 Summer School at
  Birmingham
• Instrumentation planning group
• Shin Yoo, Kamran Ghani, Andrea Arcuri, Kiran
  Lakhotia
• Empirical and Statistical Methods Tutorials and
  Workshop
• To be run by Simon Poulding
• EDAs tutorials and workshop
• To be run by Per Kristian Lehre

61
Industry

• Existing Partners
• Joint work with Motorola published
• Joint work with Daimler submitted
• Joint work with IBM awaiting revised agreement
• New Partner Outreach
• Emtex visited by Shin Yoo and Youssef Hassoun
• GRID Tools visited by Laurie Tratt and Kiran
  Lakhotia
• Legal General visited by Shin Yoo, Kiran
  Lakhotia and MSc
• Daimler (Planned) visit by Kiran Lakhotia,
  Andrea Arcuri

62
Some work at KCL

• More next week at GECCO and ISSTA
• Heres a brief overview

63
Motorola Cell Phone Requirements
64
Search Based Regression Test Selection
65
Search Based Refactoring - JHotDraw
66
Search Based Refactoring -
67
Search Based Refactoring -
68
Motorola Mobile Phone Requirements
69
Community
70
Community
71
Community
72
Non Standard Dissemination

• Schools outreach
• Not yet started
• Trade press
• IEEE Software Article planned
• Contact with IT week, Computing magazines
• Popular Science
• Meeting with New Scientist
• Special Interest Groups
• Visit to BCS SIGIST

73
Service to Community

• SBSE publications repository
• 180 papers
• 90 Full Text
• Author links to websites
• Whos Who
• Statistical Analysis
• Survey paper

74
Service to Community

• SBSE publications repository
• 180 papers
• 90 Full Text
• Author links to websites
• Whos Who
• Statistical Analysis
• Survey paper

75
Conclusion

• Good start to project
• Great staff recruited
• Following stated plans
• also activities and goals
• Great initial publication success
• Leading the world in SBSE