Improving Code Quality with VSTS Testing Capabilities for Developers

1
Improving Code Quality with VSTS Testing
Capabilities for Developers

• Adam Gallant
• Developer Solutions Technical Specialist
• Microsoft Canada, Co.
• Adam.gallant_at_microsoft.com

2
Agenda

• VSTS Overview (Quick!)
• Code Analysis
• Unit Testing
• Performance Testing
• TFS Source Control Usage
• More Information

3
ChallengesQuality

• Critical to Project Success
• Software quality errors cost U.S. businesses over
  59 billion annually
• The bulk of this cost is absorbed by the end
  users of the software

How do you improve software quality?
Source NIST, The Economic Impacts of Inadequate
Infrastructure for Software Testing 2002
4
Some Concerns with Quality Today

• Testing starts late in development cycle
• Quality tough to enforce
• Multiple IDEs affects productivity
• Testing Cycles Ineffective
• Code churn
• Cycles too long
• Repetitive process

5
Visual Studio Team SystemApplication Life Cycle
Management (ALM) Solution
6
Visual Studio Team System Quality Upstream
Team Foundation Server
Visual Studio Team System Client
Source Control
Unit Testing and Code Coverage
Bug Tracking
Static and Dynamic Analysis
Build Server
Checkin Policies
Quality Metric Reports
Web and Load Testing
7
Dev/Test Teamwork

• A test is a test is a test
• Testers use Dev unit tests as Build Verification
  Tests or in functional test pass
• Devs use Tester tests as Checkin tests
• Devs use Tester tests to view test results and
  repro bugs
• Integrated bug tracking
• Build server for smooth handoffs
• Seamless quality reporting

8
Visual Studio Team SystemTeam Edition for
Software Developers

• Static code analysis
• Support for managed and native code
• Locate security vulnerabilities and coding errors
• Code profiling
• Identify performance bottlenecks
• Reduce memory usage
• Integrated unit testing and test harness
• Code coverage
• Determine effectiveness of tests

9
Team Edition for Developers

• Writing Quality Code
• Guidelines for Conducting Design and Code Reviews
  (11 peers of Designer Reviewer)
• Guidelines for Writing Secure Code (CA)
• Create a threat model an review it!
• Guidelines for Checking in Quality Code (TFS!)
• Insisting quality check-in (passing CA Unit
  Testing)
• Guidelines for Debugging
• Guidelines for Using Code Analysis Tools (CA)
• Guidelines for Working with Unit Tests
• Guidelines Analyzing Application Performance

10
Static Code Analysis (FxCop)for managed code (or
C/C)

• Analyzes and reports violations of the
  programming, security and design rules
• Microsoft .NET Framework Design Guidelines
• http//msdn.microsoft.com/library/default.asp?url
  /library/en-us/cpgenref/html/cpconNetFrameworkDesi
  gnGuidelines.asp

11
SCA Warnings Topics

• Design Warnings
• Proper library design according .NET Framework
  Design Guidelines.
• Globalization Warnings
• World-ready libraries and applications.
• Interoperability Warnings
• Warnings that support interacting with COM
  clients.
• Maintainability Warnings
• Warnings that support library and application
  maintenance.
• Naming Warnings
• Naming conventions of the .NET Framework Design
  Guidelines.
• Performance Warnings
• High performance libraries and applications.
• Reliability Warnings
• Library and application reliability, e.g. memory
  and thread usage.
• Security Warnings
• Warnings that support safer libraries and
  applications.
• Usage Warnings
• Warnings that support proper usage of the .NET
  Framework.

12
Automated Unit Testing

• Based on xUnit Framework
• Introduced as a core concept of eXtreme
  Programming in 1998
• Automated unit tests are
• Structured.
• Self-documenting.
• Automatic and repeatable.
• Based on known data.
• Designed to test positive and negative actions.
• Ideal for testing implementation across different
  machines.
• Examples of configuration, implementation, and
  execution.

13
VSTS Unit Testing Assertions

• Assert Class
• AreEqual, AreNotEqual, AreNotSame, AreSame,
  EqualsTests, Fail, GetHashCodeTests,
  Inconclusive, IsFalse, IsInstanceOfType,
  IsNotInstanceOfType, IsNotNull, IsNull, IsTrue
• StringAssert Class
• Contains, DoesNotMatch, EndsWith, Matches,
  StartsWith
• CollectionAssert Class
• AllItemsAreInstancesOfType, AllItemsAreNotNull,
  AllItemsAreUnique, AreEqual, AreEquivalent,
  AreNotEqual, AreNotEquivalent, Contains,
  DoesNotContain, IsNotSubsetOf, IsSubsetOf

14
Common Unit Test Attributes

• TestClass
• TestMethod
• AssemblyInitialize
• ClassInitialize
• TestInitialize
• TestCleanup
• ClassCleanup
• AssemblyCleanup
• Description
• Ignore
• ExpectedException

15
Example Attribute Driven
namespace BankAccountDemo.Business.Tests
TestClass() public class BankAccountTest
TestInitialize() public static void
Initialize() TestCleanup() public
static void Cleanup() TestMethod() p
ublic void ConstructorTest() float
currentBalance 500 BankAccount target new
BankAccount(currentBalance) Assert.AreEqual(cu
rrentBalance, target.CurrentBalance,
"Balances are not equal upon creation")
TestMethod() public void
DepositMoneyTest()
16
Code Coverage

• Key metric for unit testing determining how much
  of the code is tested when the unit tests run.
• Percentage of the blocks of code tested over the
  total blocks of code.

17
Dynamic code testing

• Analyzing application/code performance based on
  collected data
• Measure time, hit count, numbers, etc..
• Evaluate why?
• Target where?
• Profiling (collecting data)
• Sampling
• Instrumentation
• IDE or command line

18
Sampling

• Sampling
• application is periodically interrupted
• low overhead
• the application behaves closer to how it would
  behave in the real world.
• Sampling to determine
• what function is running
• incrementing the sample count of that function
• storing information about the call stack
• The drawback
• only relative performance data for the sampled
  functions.
• Not all functions will get sampled and therefore,
  no overall information is available about them.

19
Instrumentation

• Gathering exact performance data for specific
  sections.
• During instrumentation, enter and exit probes are
  inserted into the application's functions.
• These probes report back to the data collection
  infrastructure
• Allow users to capture exact amounts of time and
  other metrics that a function took to run.

20
Unit TestingCode CoverageCode Analysis
21
How do I get started?

• Get free Basics Training!
• Modules, Hands-on Labs, fully configured Virtual
  Image
• http//www.microsoft.com/downloads/details.aspx?fa
  milyid527E2A17-1DEA-4D0B-9484-6AE43D00E570displa
  ylangen
• My Slide Decks
• http//adamga.officeisp.net
• My Blog
• http//blogs.msdn.com/adamga
• Other VSTS Webcasts
• Search http//blogs.msdn.com/adamga for
  Accelerate

22
Other Resources

• Visual Studio Team System home page
• http//msdn2.microsoft.com/en-us/teamsystem/defaul
  t.aspx
• Evaluate the Team Editions today
• Trial Editions
• http//msdn2.microsoft.com/en-us/teamsystem/aa7186
  67.aspx
• Visual Studio Team System community portal
• http//vstsrocks.net