COMS W4156: Advanced Software Engineering - PowerPoint PPT Presentation

Loading...

PPT – COMS W4156: Advanced Software Engineering PowerPoint presentation | free to view - id: 8058af-ZTg5M



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

COMS W4156: Advanced Software Engineering

Description:

COMS W4156: Advanced Software Engineering Prof. Gail Kaiser Kaiser+4156_at_cs.columbia.edu http://york.cs.columbia.edu/classes/cs4156/ – PowerPoint PPT presentation

Number of Views:79
Avg rating:3.0/5.0
Slides: 82
Provided by: GailK155
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: COMS W4156: Advanced Software Engineering


1
COMS W4156 Advanced Software Engineering
  • Prof. Gail Kaiser
  • Kaiser4156_at_cs.columbia.edu
  • http//york.cs.columbia.edu/classes/cs4156/

2
What is Open Source Software?
  • Open source usually refers to a program in which
    the source code is available to the general
    public for use and/or modification from its
    original design free of charge.
  • Open source code is typically created as a
    collaborative effort in which programmers improve
    upon the code and share the changes within the
    community.
  • The rationale for this movement is that a larger
    group of programmers not concerned with
    proprietary ownership or financial gain will
    produce a more useful and bug-free product for
    everyone to use.
  • The concept relies on peer review to find and
    eliminate bugs in the program code, a process
    that commercially developed and packaged programs
    do not employ.

3
Technical Case
  • Central part of engineering tradition, part of
    working method almost by instinct, for Internet
    and Unix hackers.
  • The running gears of the Internet are
    astonishingly reliable relative to their nearest
    commercial equivalents.
  • TCP/IP, DNS, sendmail, Perl, Apache,

4
Economic Case
  • There are companies making money programming
    open-source software right now.
  • If having a program written is a net economic
    gain for a customer over not having it written, a
    programmer will get paid whether or not the
    program is going to be free after it's done.

5
Economic Value
  • The use value of a program is its economic value
    as a tool.
  • The market value of a program is its value as a
    saleable commodity.
  • The monopoly value is the value you gain not just
    from having the use of a program but from having
    it be unavailable to your competitors.

6
Open-Source Doomsday (1)
  • The market value and monopoly value of software
    goes to zero because of all the free sources out
    there.
  • Use value alone doesn't attract enough consumers
    to support software development.

7
Open-Source Doomsday (2)
  • The commercial software industry collapses.
  • Programmers starve or leave the field.
  • Doomsday arrives when the open-source culture
    itself (dependent on the spare time of all these
    pros) collapses, leaving nobody around who can
    program competently.

8
Shaky Assumption 1 Programming will collapse if
software has no market value
  • Proportion of all code written in-house at
    companies other than software vendors gt75.
  • Includes most MIS the financial- and
    database-software customizations
  • Also includes OEM software like device drivers
    and embedded code for our increasingly
    microchip-driven machines.

9
Shaky Assumption 1 (cont) Programming will
collapse if software has no market value
  • Most vertical code is integrated with its
    environment in ways that make reusing or copying
    it very difficult.
  • This is true whether the environment is a
    business office's set of procedures or the
    fuel-injection system of a combine harvester.

10
Combine Harvester
11
Shaky Assumption 1 (cont) Programming will
collapse if software has no market value
  • Thus, as the environment changes, there is a lot
    of work continually needed to keep the software
    in step.
  • Maintenance makes up the vast majority of what
    programmers get paid to do.
  • And it will still need to be done, even if/when
    most software is open-source.

12
Shaky Assumption 1 (cont) Programming will
collapse if software has no market value
  • Between originating, customizing and maintaining
    vertical code (and related tasks like system
    administration and troubleshooting), the use
    value of software would still support the
    millions of good jobs in that 75 even if all
    horizontal or standalone software were free.

13
Shaky Assumption 2 Open-source software has no
market value
  • Red Hat (among others) has built a flourishing
    business selling software you can download for
    free from Red Hat's own web site!
  • What you're really buying from them is
    handholding and support for the free stuff they
    sell - a single place to go when you have
    problems.

14
Shaky Assumption 3 Open-source software has no
monopoly value
  • Adopting or even just studying someone else's
    software is not a costless, frictionless process
    you need to dedicate skilled time to it.
  • As product cycle times drop, coattail-riding gets
    less attractive, because the payoff period
    shrinks relative to the time you had to dedicate.

15
Shaky Assumption 3 Open-source software has no
monopoly value (cont)
  • And time your skilled people spend studying
    someone else's monopoly code is time you're
    spending getting to where the competition used to
    be (rather than where they are now).

16
Business Case 1
  • High reliability
  • Open-source software is peer-reviewed software
    it is thus more reliable than closed, proprietary
    software.
  • Mature open-source code is as bulletproof as
    software ever gets.

17
Business Case 2-N
  • Development Speed
  • Lower Overhead
  • Closeness to the Customer
  • Broader Market
  • Grab Mind Share (e.g., for startups)

18
Investor Case 1-2
  • Support Sellers give away the software product,
    but sell distribution, branding, and after-sale
    service.
  • Loss Leaders give away open-source as a
    loss-leader and market positioner for closed
    software.

19
Investor Case 3-4
  • Widget Frosting a hardware company goes
    open-source in order to get better drivers and
    interface tools cheaper.
  • Accessorizing selling accessories -- books,
    compatible hardware, complete systems with
    open-source software pre-installed.

20
Customer Case 1
  • Open source model applies even to internally
    developed software.
  • You are your developers customer!
  • Freedom from legal entanglements such as tracking
    copies and usage.
  • Very hard to do accurately.

21
Customer Case 2
  • Higher Security
  • Security through obscurity just does not work.
  • Closed sources create a false sense of security.
  • The bad guys will always find the holes, but the
    good guys will not find holes and fix them.
  • It is harder to distribute trustworthy fixes when
    a hole is revealed.

22
Marketing Case
  • Why not call it, as we traditionally have, free
    software?
  • The term free software has a load of fatal
    baggage to a businessperson, it's too redolent
    of fanaticism and flakiness and strident
    anti-commercialism.

23
Marketing Case (cont)
  • In marketing appearance is reality. The
    appearance that we're willing to climb down off
    the barricades and work with the corporate world
    counts for as much as the reality of our
    behavior, our convictions, and our software.

24
Free Software Foundation
  • The Free Software Foundation (FSF) is dedicated
    to eliminating restrictions on copying,
    redistribution, understanding, and modification
    of computer programs.
  • Free software is a matter of liberty, not
    price.
  • Think free speech, not free beer.

25
Free Software Tenets
  1. The freedom to run the program, for any purpose.
  2. The freedom to study how the program works, and
    adapt it to your needs. Access to the source code
    is a precondition for this.

26
Free Software Tenets (cont)
  1. The freedom to redistribute copies so you can
    help your neighbor.
  2. The freedom to improve the program, and release
    your improvements to the public, so that the
    whole community benefits. Access to the source
    code is a precondition for this.

27
Example Open Source License
  • Gnu General Public License (GPL), developed by
    Richard Stallman and the Free Software Foundation
    starting in 1985
  • Certified by OSI

28
Open Source Initiative (OSI)
  • http//www.opensource.org/
  • Open source is a development method for software
    that harnesses the power of distributed peer
    review and transparency of process. The promise
    of open source is better quality, higher
    reliability, more flexibility, lower cost, and an
    end to predatory vendor lock-in.

29
Open Source Initiative (OSI)
  • The Open Source Initiative (OSI) is a non-profit
    corporation formed to educate about and advocate
    for the benefits of open source and to build
    bridges among different constituencies in the
    open-source community.
  • One of our most important activities is as a
    standards body, maintaining the Open Source
    Definition for the good of the community. The
    Open Source Initiative Approved License trademark
    and program creates a nexus of trust around which
    developers, users, corporations and governments
    can organize open-source cooperation.

30
Open Source Definition
  • Open source doesn't just mean access to the
    source code.
  • The distribution terms of open-source software
    must comply with the following criteria
    upcoming slides

31
Open Source Definition
  • 1. Free Redistribution
  • The license shall not restrict any party from
    selling or giving away the software as a
    component of an aggregate software distribution
    containing programs from several different
    sources. The license shall not require a royalty
    or other fee for such sale.

32
Open Source Definition
  • 2. Source Code
  • The program must include source code, and must
    allow distribution in source code as well as
    compiled form. Where some form of a product is
    not distributed with source code, there must be a
    well-publicized means of obtaining the source
    code for no more than a reasonable reproduction
    cost, preferably downloading via the Internet
    without charge.

33
Open Source Definition
  • 2. Source Code
  • The source code must be the preferred form in
    which a programmer would modify the program.
    Deliberately obfuscated source code is not
    allowed. Intermediate forms such as the output of
    a preprocessor or translator are not allowed.

34
Open Source Definition
  • 3. Derived Works
  • The license must allow modifications and derived
    works, and must allow them to be distributed
    under the same terms as the license of the
    original software.

35
Open Source Definition
  • 4. Integrity of The Author's Source Code
  • The license may restrict source-code from being
    distributed in modified form only if the license
    allows the distribution of "patch files" with the
    source code for the purpose of modifying the
    program at build time.

36
Open Source Definition
  • 4. Integrity of The Author's Source Code
  • The license must explicitly permit distribution
    of software built from modified source code. The
    license may require derived works to carry a
    different name or version number from the
    original software.

37
Open Source Definition
  • 5. No Discrimination Against Persons or Groups
  • The license must not discriminate against any
    person or group of persons.

38
Open Source Definition
  • 6. No Discrimination Against Fields of Endeavor
  • The license must not restrict anyone from making
    use of the program in a specific field of
    endeavor. For example, it may not restrict the
    program from being used in a business, or from
    being used for genetic research.

39
Open Source Definition
  • 7. Distribution of License
  • The rights attached to the program must apply to
    all to whom the program is redistributed without
    the need for execution of an additional license
    by those parties.

40
Open Source Definition
  • 8. License Must Not Be Specific to a Product
  • The rights attached to the program must not
    depend on the program's being part of a
    particular software distribution.

41
Open Source Definition
  • 8. License Must Not Be Specific to a Product
  • If the program is extracted from that
    distribution and used or distributed within the
    terms of the program's license, all parties to
    whom the program is redistributed should have the
    same rights as those that are granted in
    conjunction with the original software
    distribution.

42
Open Source Definition
  • 9. License Must Not Restrict Other Software
  • The license must not place restrictions on other
    software that is distributed along with the
    licensed software. For example, the license must
    not insist that all other programs distributed on
    the same medium must be open-source software.

43
Open Source Definition
  • 10. License Must Be Technology-Neutral
  • No provision of the license may be predicated on
    any individual technology or style of interface.

44
Open Source Licenses
  • Open Source Licenses (by name or by category)
    comply with the Open Source Definition and are
    listed here after going through the approval
    process. We also track the approval status of
    licenses.

45
The Cathedral and the Bazaar
  • Eric Raymond (esr)
  • first presented May 1997, ongoing revision
    through September 2000
  • http//www.firstmonday.org/issues/issue3_3/raymond
    / or
  • http//www.catb.org/esr/writings/cathedral-bazaar
    /cathedral-bazaar/

46
The Cathedral
  • Draws an analogy between traditional closed
    source development and a cathedral, in which
    there is a rigid hierarchy among developers,
    managers, testers, etc.

47
The Cathedral (cont)
  • esr originally believed that the most important
    software (operating systems and really large
    tools like the Emacs programming editor) needed
    to be built like cathedrals, carefully crafted by
    individual wizards or small bands of mages
    working in splendid isolation, with no beta to be
    released before its time.

48
The Bazaar
  • Likens open source projects to Middle Eastern
    bazaars, where numerous merchants hawk their
    wares loudly to passersby.
  • Little hierarchy among contributors.
  • Contributors compete to have their modifications
    inserted into the next release, bringing
    recognition and reputation.

49
Based Primarily on Linux
  • Describes Linus Torvalds' style of development
    as release early and often, delegate everything
    you can, be open to the point of promiscuity.

50
Open Source Lessons (1)
  1. Every good work of software starts by scratching
    a developer's personal itch.
  2. Good programmers know what to write. Great ones
    know what to rewrite (and reuse).
  3. Plan to throw one away you will, anyhow. (Fred
    Brooks, The Mythical Man-Month, Chapter 11)

51
Open Source Lessons (2)
  1. If you have the right attitude, interesting
    problems will find you.
  2. When you lose interest in a program, your last
    duty to it is to hand it off to a competent
    successor.
  3. Treating your users as co-developers is your
    least-hassle route to rapid code improvement and
    effective debugging.

52
Open Source Lessons (3)
  1. Release early. Release often. And listen to your
    customers.
  2. Given a large enough beta-tester and co-developer
    base, almost every problem will be characterized
    quickly and the fix obvious to someone.
  3. Smart data structures and dumb code works a lot
    better than the other way around.

53
Open Source Lessons (4)
  1. If you treat your beta-testers as if they're your
    most valuable resource, they will respond by
    becoming your most valuable resource.
  2. The next best thing to having good ideas is
    recognizing good ideas from your users. Sometimes
    the latter is better.
  3. Often, the most striking and innovative solutions
    come from realizing that your concept of the
    problem was wrong.

54
Open Source Lessons (5)
  1. Perfection (in design) is achieved not when there
    is nothing more to add, but rather when there is
    nothing more to take away.
  2. Any tool should be useful in the expected way,
    but a truly great tool lends itself to uses you
    never expected.

55
Necessary Preconditions for the Bazaar Style (1)
  • One cannot code from the ground up in bazaar
    style.
  • One can test, debug and improve in bazaar style,
    but it would be very hard to originate a project
    in bazaar mode.
  • Your nascent developer community needs to have
    something unable and testable to play with.

56
Necessary Preconditions for the Bazaar Style (2)
  • To start community-building, what you need to be
    able to present is a plausible promise.
  • Your program doesn't have to work particularly
    well. It can be crude, buggy, incomplete, and
    poorly documented.

57
Necessary Preconditions for the Bazaar Style (3)
  • What it must not fail to do is (a) run, and (b)
    convince potential co-developers that it can be
    evolved into something really neat in the
    foreseeable future e.g., through strong,
    attractive basic design.

58
Bazaar Project Coordinator
  • It is not critical that the coordinator be able
    to originate designs of exceptional brilliance,
    but it is absolutely critical that the
    coordinator be able to recognize good design
    ideas from others.
  • Open-source community's internal market in
    reputation exerts subtle pressure on people not
    to launch development efforts they're not
    competent to follow through on.

59
Bazaar Project Coordinator (cont)
  • Must have good people and communications skills.
  • In order to build a development community, you
    need to attract people, interest them in what
    you're doing, and keep them happy about the
    amount of work they're doing.

60
What Happened to Mythical Man Month?
  • Fred Brooks The Mythical Man-Month observed
    that programmer time is not fungible - adding
    developers to a late software project makes it
    later.
  • He argued that the complexity and communication
    costs of a project rise with the square of the
    number of developers, while work done only rises
    linearly.

61
What Happened to Mythical Man Month? (cont)
  • This claim has since become known as Brooks'
    Law and is widely regarded as a truism.
  • But if Brooks' Law were the whole picture, Linux
    would be impossible.

62
Egoless Programming
  • Gerald Weinberg The Psychology Of Computer
    Programming supplied a vital correction to
    Brooks.
  • His discussion of egoless programming observed
    that in shops where developers are not
    territorial about their code, and encourage other
    people to look for bugs and potential
    improvements in it, improvement happens
    dramatically faster than elsewhere.

63
But
  • That cant be the whole story

64
Who Invented Open Source?
  • No one knows
  • Some say Linus Torvalds, initial developer of
    Linux (c. 1992)
  • Some say Richard Stallman, founder of GNU Project
    (c. 1985)
  • But lots of earlier software is public domain,
    e.g., original implementations of TCP/IP, DNS,
    sendmail, various other networking software

65
Linus' Law according to Eric S. Raymond
  • Given enough eyeballs, all bugs are shallow

66
Linus' Law according to Linus Torvalds
  • "Linus's Law says that all of our motivations
    fall into three basic categories. More important,
    progress is about going through those very same
    things as 'phases' in a process of evolution, a
    matter of passing from one category to the next.
    The categories, in order, are 'survival', 'social
    life', and 'entertainment'."
  • Himanen, Pekka Linus Torvalds, Manuel Castells.
    The Hacker Ethic. Random House, 2001. ISBN
    0-375-50566-0. 

67
Why Didnt Open Source Happen Earlier?
  • Well, it did
  • Thats where Unix and the Internet came from
  • But in a relatively small academic-oriented
    circle
  • And the pace was very slow

68
Why Didnt Open Source Happen Earlier?
  • Legal constraints of various licenses, trade
    secrets, and commercial interests (e.g., wrt
    Unix)
  • The Internet wasn't (yet) good enough egoless
    programming could only work in geographically
    compact communities.

69
Web and ISP Industry
  • Linux was the first project to make a conscious
    and successful effort to use the entire world as
    its talent pool.
  • The gestation period of Linux coincided with the
    birth of the World Wide Web, and Linux left its
    infancy during the same period in 1993-1994 that
    saw the takeoff of the ISP industry and the
    explosion of mainstream interest in the Internet.

70
An Opposing View Robert Glass, Open Source and
Hype, 7 July 2003
  • http//www.stickyminds.com/sitewide.asp?ObjectId6
    535FunctionDETAILBROWSEObjectTypeCOL

71
Open Source and Hype
  • Most of what I dislike about the open source
    movement can be summed up in one word Hype.
    Unfortunately, and perhaps surprisingly, the
    advocates of open source are no better in this
    regard than their proprietary colleagues.

72
Best People
  • The claim is frequently made that open source
    programmers are the best programmers around.
  • Attempts to define Programmer Aptitude Tests,
    which evaluate the capabilities of subjects to
    become good programmers, have historically been
    failures.
  • Although some programmers are better than others,
    nothing in the fields' research suggests that we
    have found an objective way of determining who
    those best people are. 
  • Since we can't identify who the best people are,
    there is no way to study the likelihood of them
    being open source programmers.

73
Most Reliable
  • The claim is also frequently made that open
    source software is the most reliable software
    available.
  • Open source advocates claim that a study
    identified as the "Fuzz Papers" produced results
    that showed that their software was more reliable
    than proprietary alternatives. 
  • But the Fuzz Papers have virtually nothing to say
    about open source software, one way or the other,
    and their author (according to Glass) agrees with
    that assessment.  

74
The Fuzz Papers
  • http//www.cs.wisc.edu/bart/fuzz/fuzz.html
  • Fuzz testing a simple technique for feeding
    random input to applications.
  • The input is completely random.
  • Fuzz testing does not use any model of program
    behavior, application type, or system
    description.

75
The Fuzz Papers
  • The reliability criteria is simple if the
    application crashes or hangs, it is considered to
    fail the test, otherwise it passes.
  • Note that the application does not have to
    respond in a sensible manner to the input, and it
    can even quietly exit.
  • Fuzz testing can be automated to a high degree
    and results can be compared across applications,
    operating systems, and vendors.

76
Back to Open Source and Hype More Secure
  • There are many claims that open source is more
    secure than proprietary.
  • There is very little evidence on either side
    regarding open source software and security.
  • Security holes have been found in proprietary
    software.
  • Security holes have been found in open source
    code.
  • It is all too easy for programmers to leave
    holes, independent of how the code is written

77
Robert Glass Concludes
  • So where do I stand on open source? I see
    nothing in particular, wrong with its fundamental
    ideas and ideals. But I see plenty wrong with the
    hype surrounding it. Not that it's any worse than
    its proprietary brethren in this respect. It's
    just that I expected more from this particular
    group! Yes, I do expect more from the open source
    advocates. 

78
Open Systems ? Open Source
  • Open systems concept predates OSI and FSF and
    may be more palatable to conventional software
    vendors
  • Concerned with system integration
  • Standard interfaces and protocols
  • Includes but not limited to component model
    frameworks
  • Industry consortia The Open Groups Single Unix
    Spec, Object Management Groups CORBA, W3Cs XML

79
Next Assignment
  • Code inspection week Tuesday November 20th
    through Thursday November 29th
  • Second Iteration Progress Report due Friday
    November 30th

80
Upcoming Deadlines
  • Code inspection week Tuesday November 20th
    through Thursday November 29th
  • Second Iteration Progress Report due Friday
    November 30th
  • Demo week Monday December 3rd through Monday
    December 10th
  • Second Iteration Final Report due Tuesday
    December 11th

81
COMS W4156 Advanced Software Engineering
  • Prof. Gail Kaiser
  • Kaiser4156_at_cs.columbia.edu
  • http//york.cs.columbia.edu/classes/cs4156/
About PowerShow.com