The Shipt Left Strategy - Clear Sky

1
THE SHIFT LEFT STRATEGY

• OVERCOMING THE DOWNWARD SPIRAL OF DOOM

Darian Rashid darian_at_clearskytestautomation.com
2
(No Transcript)
3

• Late?
• Too many defects?
• Not enough content delivered?

• How did we get here?

4
LETS REWIND
Design
Requirement
Development
Deploy
Test
Create Test Cases
First time we validate requirements

• Development has most likely overrun its time
• Test capacity is constrained
• Not enough people and time to run System,
  Integration and Regression effectively

5
(No Transcript)
6

• Fix easy issues first
• Cannot replicate
• Test Coverage
• Technical debt mountain
• How many issues can be differed?
• How many issues from previous releases continue
  to be ignored?

7
AMBIGUITY IN REQUIREMENTS?

• WELL, YES AND NO

• Ambiguous requirements
• Test cases created after development has begun

• Leads to missed expectations and rework
• Discovered well into development cycle
• Each reworked item could
• Impact delivery schedule
• Incur fines

• Best case Spend time arguing that the delivery
  met the requirements
• Worst Case Implement CRs during testing

8
We just finished up our sprints. Now were gonna
test.
BUT..
9
THE CODEBASE
10
THE CODEBASE

• Brittle code
• Deeply coupled, interconnected web
• Usually new features on top of legacy code
• Refactored last in the Regan era

11
THE TEAM
12
(No Transcript)
13
(No Transcript)
14
EMERGENCY PATCHES

• Havent accounted for these in the release
• Are already overcommitted for the next release
• Cannot drop deliverables
• RESULT A compressed time frame for the next
  release

15
THE DOWNWARD SPIRAL OF DOOM
EACH RELEASEWORSE THAN THE LAST
16
(No Transcript)
17
THE PARADIGM SHIFT
CONTAIN ISSUES
Create Test Specifications
Create Customer Requirements
Prove Delivery
Run Manual Tests
Create Development Design Documents
Develop Code/ Execute Tests, Daily

• PROVE SOFTWARE DELIVERY BY
• Creating test specifications BEFORE development
  begins
• Execute ALL test cases DAILY, even during
  development
• Measure test coverage against requirements DAILY

18
CONVENTIONAL GHERKIN
Create Adult Customer Profile with Preferred Phone as Home Phone 1.Launch the eCustomer Application 2.Click on or register link GIVEN I am at the Home pageWHEN I click on the Register Now LinkTHEN I will be taken to the Register screen AND I will see the following
3.Enter the following Mandatory Details (a) enter the valid 10 digit Home phone number (b)Preferred Phone Home (c)Preferred Contact Method Email (d)Enter the other details given in the default demographics section of test case initialization table WHEN I enter the following THEN I will be taken to the Contact Information Screen AND I should see the following
Title First Name Middle Name Last Name Answer
(blank) (blank) (blank) (blank) (blank)
First Name Middle Name Last Name Answer
Miss Bethany Jones 9021
What other details? Which details are valid
details? Which are invalid?
First Name Middle Name Last Name Answer
Miss Bethany Jones 9021
What should I see when I click on this link? How
do I know which page Im taken to? How do I know
that data loaded that page correctly. If this
page isnt right, the rest of the test cannot
proceed.
Each cell is a test case. The number of output
values to check against will depend on the test
case.
COMPARE LEFT AND RIGHT
19
TEST SPECIFICATIONS

• Represent the behaviors of the user and the
  system
• Created in a simple, human-readable manner
• Objectively measurable
• Used as the primary customer and internal
  acceptance criteria
• Become the requirements for the development
  organization
• Proves delivery

20

• PROVE DELIVERY
• Creating OBJECTIVELY MEASURABLE test
  specifications
• Execute EVERY test cases EVERY day
• Measure test coverage against requirements DAILY

• END RESULTS
• Maintain quality
• Accurately show delivery
• Foresee Risk

21
Execute EVERY Test EVERY Day?
22
BUT

• Average Test Execution Time 2 minutes

Number of Tests 1 VM
1000 8.3 Hours
3000 25 Hours
5000 41.6 Hours
10,000 83.3 Hours
23
THE ANSWER
HYPERSCALE ON THE CLOUD
24
MAXIMIZE TEST THROUGHPUT THROUGH MASSIVE-SCALE
PARALLEL TESTING USING THE CLOUD
CLEAR SKY CORE
RUNNERS
25
SYSTEM CANT TAKE THE HEAT? KEEP SCALING!
CLEAR SKY CORE
LOAD BALANCER
RUNNERS
26
THE DEMO
27
THE CONCLUSION

• Create unambiguous test specifications, used as
  requirements
• Run EVERY test EVERY day to create a safety net
• Use hyperscaled automation on the cloud to scale
• Test case
• Instances under test

KEEP QUALITY CONSTANT DAILY