Research Software Doesn't Have to be Buggy: A Model-Driven, Test-Driven and Agile Process Applied in a University Research Team

1
Research Software Doesn't Have to be BuggyA
Model-Driven, Test-Driven and Agile Process
Applied in a University Research Team

• November 2013
• Timothy C. Lethbridge

2
Motivations for This Talk

• I was explaining to researchers at a conference
  how I have been able to maintain Umple
• They were interested in the process I followed
• It has been a very positive experience to
  experience real software development
• And not get bogged down it
• While maintaining quality
• And getting good feedback from students

3
Research Software Often Short Life or Not
Production Quality

• Software developed by computer science
  researchers often only lasts as long as one
  students PhD
• Perhaps a little longer
• It usually gets messy
• The next generation of students often wants to
  rewrite it
• The principal investigator wants to move on to
  other things
• The software becomes a drag
• Lack of resources for maintenance
• Grants are for research, not development

4
Why would we want research software to last for
the long haul?

• To really get it right
• To act as infrastructure for countless
  experiments and idea exploration
• To allow the professor and students to get really
  good at real software engineering

5
My Vision

• Prodution quality software that persists for 20
  or more years
• Continual improvement of the process
• Top-notch training, education, and research
  platform

6
My Approach

• Adopt software engineering best practices in the
  software used in the research
• Blend and mode-switch research approaches
• Design based research
• Action research
• Grounded theory
• Empirical experimentation

7
The Platform Umple

• Model-oriented compiler
• Adds UML and patterns to Java, C etc.
• Textual UML
• High quality code generator for class and state
  diagrams
• Web-based command line Eclipse
• Main site http//www.umple.org
• Web app http//try.umple.org
• Manual http//manual.umple.org
• Open source
• Google http//code.umple.org
• Mirrors on Github and SourceForge

8
Robustness maintained because of / despite
contributions by gt30 HQP

• 2 completed PhDs ( 7 underway)
• 4 completed Masters ( 1 underway)
• 22 undergraduate capstone projects ( 6 underway)
• 3 postdocs
• UCOSP Students from UBC, Simon Fraser, UAlberta,
  USask, URegina, Laurentian, Waterloo, Wilfred
  Laurier, Guelph, Bishops, Sherbrooke, UNB,
  Dalhousie
• International contributions from Brazil.
• Countless students taught in classrooms
• Umple helps students learn UML (CSEET 2011)

9
Essential elements of the processRed ones are key

• Open source
• Multi-level test driven development
• Model driven development
• Carefully categorized issues list
• Continuous integration
• Lean live documentation for users and developers
• Low management overhead
• Marketing and publication

10
Open Source

• Open to the world as early as possible
• If there are spinoffs, let them be based on
  service, not software

11
Multi-level test driven development

• Use test cases as specifications
• Multiple levels in the layered architecture
• For Umple Parsing, metamodel instance,
  generation, execution
• In Umple, the compiler itself is a big test
• Note We have not applied this yet to the user
  interface
• http//qa.umple.org

12
Model driven development

• Always start with the model and generate code
  from it
• It is possible
• Requires indoctrination, and students sometimes
  have to be asked to start again!
• Simplifies code, reduces volume, enforces
  abstraction
• That is what Umple is about, but we have also
  developed other systems in Umple
• Key Umple benefits compared to other MDD tools
• Model is code traditional code embedded in mode
• No need to edit generated code
• Bidirectional traceability

13
Carefully managed issues list

• Very-easy, easy, moderate, hard, very-hard
• Defect, enhancement, undergraduate project,
  graduate project
• Priority Critical, Vhigh, High, Med, Low, Vlow
• Ownership of issue, but not code
• http//bugs.umple.org

14
Continuous integration

• Single master trunk for all work
• Server builds then posts results
• http//cc.umple.org
• Test result browser
• http//qa.umple.org
• 100 test pass rate maintained 95 of the time
• 100 test pass rate within an hour of a failure
  99.9 of the time
• Only two backouts in the last year (failed
  tests that could not be quickly fixed)

15
Lean Live Documentation for Developers and End
Users

• Code commenting is key
• But comments transfer into documentation
• Auto-generated diagrams are a result of MDD
• http//metamodel.umple.org
• http//grammar.umple.org
• User manual is generated http//manual.umple.org
• If someone doesnt understand something they
  update the wiki
• http//wiki.umple.org
• Exploration of requirements done in issues and
  wikis
• Developers log experiences and progress for all
  to see

16
Low Management Overhead

• As a professor, no time for much project
  management
• About 2 hours a week direct interaction with
  students actively working on the software
• I manage issues, set up policies, refine
  documents
• Periodically I work on an issue
• Code sprint (20 hours) once a semester
• Code review of patches and commits
• Focus on checking the tests
• Delegation of inspection to others
• Some refactoring

17
Marketing and Publication

• Not just journals and conferences
• We need users and feedback from users of our
  tools
• Live mirrors Google Code, Github,
• Facebook, Google, Twitter, Email lists, blogs

18
Use of our software in teaching

• Several professors have taught software
  engineering concepts using our work
• Introduction to software engineering
• Advanced software design
• Software engineering graduate course
• Score project at ICSE
• Others

19
Introducing students to Umple

• 1. Train in the use of the software
• 2. Development environment setup
• 3. Coaching on process
• 4. Code sprint
• 5. Have them create several patches for easy to
  moderate problems in several architecture areas
• 6. Make them a committer (happens for most
  students)

20
Conclusions

• Make your academic software high quality by using
• TDD
• MDD
• Continual intregration
• Careful agile process
• It works!
• Benefits
• You and your students get better at software
  engineering
• You can do better research because your
  infrastructure is better

21
Questions?

• This talk is available at
• http//bit.ly/1fMkhzZ
• Or
• http//www.eecs.uottawa.ca/tcl/presentations/CSER
  -Nov2013-Talk-RshSWNotBuggy.ppt