The subtle art of etriage

1
The subtle art of e-triage

• Professor Ken Birman
• Dept. of Computer Science
• Cornell University

2
Last Time

• We learned that complex e-commerce systems are at
  e-risk
• We saw that e-risks take many forms
• System complexity
• Failure to plan for failures
• Poor project management
• Can we do better?

3
E-triage

• In a hospital emergency room, nurse sees a bunch
  of sick people

4
E-triage

• In a hospital emergency room, nurse sees a bunch
  of sick people
• Cant deal with everything at once, so she
  decides what problems are most urgent
• We can use the same idea to triage risks to an
  e-commerce project

5
Example

• Your e-company will host the ultimate web site
  for womens cosmetic products
• Scan in pictures of your clients
• Software on the web site makes cosmetic
  recommendations
• Depicts results right on the image
• Plan is to be the worlds largest direct sales
  channel for cosmetics

6
Business Roles?

• You write the checks, but also set the tone
• You need to learn enough about the choices to
  guide your technology people to make the right
  ones
• Your role is triagebut of managerial type

7
Reduce to a Technical Problem

• Option A
• Hire a bunch of hackers
• Explain the vision, trust them
• They start to mumble about Oracle, Sybase,
  Informix, VRML, HHTPS, XML, IE 5.0 versus
  Netscape 3.2, scalable cluster servers
• You zone out
• Problem your fate in their hands

8
Reduce to a Technical Problem

• Option B
• You bring in the experts
• IBM shows up Monday, HP on Tuesday
• Their marketing guys want money
• Their technical guys are incomprehensible
• You zone out
• Problem your money in their hands. Didnt IBM
  blow that air-traffic project?

9
Reduce to a Technical Problem

• Option C
• You decide to manage the project in an effective
  way
• Before bringing in technical people, can you sort
  out the big issues?
• Problem you arent a technical person. (But
  they know that, and yet they look to you for
  guidance because you run the show)

10
Basic Maxims

• We cant do everything
• We need to focus on what really matters
• Each technical feature adds complexity
• Can we achieve a spare, elegant solution that has
  precisely the mechanisms we need to excel?
• On each choice, ask why does this matter?
  What will be the downstream costs?

11
Pieces of the system

• The Internet customers use it to talk to your
  company
• Your web server hosts the content
• A database of customer information
• Her face, her age, past purchases, etc
• A database of cosmetic products
• A database of advertising materials
• Back-office system clears transactions

12
Terminology

• A database is a big collection of information
  organized for easy access
• Internet technologies provide a way for client
  systems (browsers) to issue some form of database
  request to your system
• Usually the request occurs by fetching a
  computed web page
• Internally, your company may have information
  spread over multiple sites and hidden behind a
  firewall at each location

13
Complexity!

• All of this takes an initially simple idea and
  makes it complex
• The data may not be at one place
• Perhaps many systems will need to cooperate to
  satisfy your customers
• With lots of customers we need to worry about how
  the model scales

14
The System
15
Choices

• How to build the web site itself
• Well focus on BEA Weblogic
• This technology is a hot product that many feel
  does it all
• There are lots of other similar projects, but
  Weblogic is typical and most successful
• But picking the technology doesnt really solve
  the business problem

16
Roll Your Own?

• With Weblogic you
• Use their tools
• But your own people build the actual web site
• Their tools provide
• Basic functionality for web sites on which people
  can do shopping-cart style purchases
• Database to track inventory and customer
  histories
• They even handle credit card transactions

17
Why is Weblogic a Success?

• Very easy to get slick web sites off the ground
• Built in solutions for most of the things you
  might need to do
• And BEA has related products to link the web site
  you build to your other systems within your
  company
• All of this makes for a good story the whole
  story for building web sites

18
All About Weblogic

• The basic architecture
• On the client side, cookie tracks the contents
  of the shopping cart
• Requests are received by the Weblogic
  load-balancing proxy
• Behind it, a cluster of servers run the web pages

proxy
server
19
Weblogic Limits

• For extremely popular web sites, the approach
  Weblogic takes wont scale
• Remote users will get slow response
• Hard to customize content so that each user sees
  a private set of web pages
• And the database itself is built by other
  companies, forcing you to make a choice
• High availability a big issue at this level
• Yet their product literature claims that Weblogic
  scales extremely well.

20
Business Tradeoffs

• These limitations are serious worries
• Near term, they wont impact you
• Building a great site will come down to hiring
  great graphics people
• But long term, they point to problems scaling
  your business model
• You might not easily have discovered the problem
• In effect, the technology choice may dictate the
  business model and price point at a stage when
  you didnt even know you were buying into them!

21
Project Creep

• A common phenomenon
• You make sensible initial decisions
• And the technology seems to work
• Your company gets off the ground
• But then hit limits
• These force you to hack solutions
• Your business suffers downtime unreliability
• Ultimately, your people do more and more hacking
  and may have to rebuild from scratch or worse!

22
Avoiding Project Creep

• The key to sensible business decisions in
  technology settings is scalability
• To be a success, your company needs to see
  revenue growth increase without requiring
  commensurate skilled labor
• Focus your business decision making on
  scalability questions about the technology as
  your company will be using it
• The mantra scalable from start to finish

23
Scalability

• We use this term to talk about
• Ability of system to continue providing high
  quality services to its users
• Even as numbers of users rises, and/or
• Size of network rises, and/or
• Load on the system increases, and/or
• Amounts of data it manages increase

24
Scalability is Hard!

• In small scale settings
• Conditions are easily controlled
• We dont tend to see failures and recoveries
• Things that can fail include computers and
  software on them, network links, routers
• We are not likely to come under attack

25
Fundamental Issues of Scale

• Suppose a machine can do x business transactions
  per second
• If I double the load to 2x how big and fast a
  machine should I buy?
• With computers, answer isnt obvious!
• If the answer is twice as big we say the
  problem scales linearly.
• Often the answer is 4 times as big or worse!
  Such problems scale poorly perhaps even
  exponentially!
• Basic insight bigger is often much harder!

26
Does the Internet Scale?

• It works pretty well, most of the time
• But if you look closely, it has outages very
  frequently
• Butler Lampson won the Turing Award
• (to paraphrase) Computer scientists didnt
  invent the world-wide-web because they are only
  interested in building things that work really
  well. The Web, of course, is notoriously
  unreliable. But the insight we, as computer
  scientists, often miss is that for the Web
  doesnt need to work well!
• A reliable web an example of an oxymoron?
• Internet scales but has low reliability

27
How do technologies scale?

• One of the most critical issues we face!
• The bottom line is that, on the whole
• Very few technologies scale well
• The ones that do tend to have poor reliability
  and security properties
• Scale introduces major forms of complexity
• And large systems tend to be unstable, hard to
  administer, and fragile under stress

28
Web scaling issues

• A very serious problem for popular sites
• Most solutions work like this
• Your site offloads data to a web hosting
  company
• Example is Akamai, or Exodus
• They replicate your pages at many sites worldwide
• Ideally, your customers see better performance
• Second approach Digital Island
• They focus on giving better connections to the
  Internet backbone, avoiding ISP congestion

29
Akamai Approach

• They cut deals with lots of ISPs
• Give us room in your machine room
• In fact, you should pay us for this!
• Well put our server there
• And it will handle so much web traffic
• That your lines will be less loaded, since
  nothing will need to go out to the backbone
• And this will save you big bucks!

30
A Good Idea?

• Akamai approach focuses on rarely changing data
• Example pictures used on your web pages
• Non-example the pages themselves, which are
  often constructed in a customized way
• Pre-Akamai Your web site handles all the traffic
  for constructing pages and also handing out the
  pictures and static stuff
• Post-Akamai You hand out the main pages but the
  URLs for pictures point to Akamai web servers

31
Pre- and Post-Akamai
lthtmlgt ltheadgt ltMETA nameVI60_defaultClientScript
contentJavaScriptgtlttitlegtPokéMansion - The Big
House of Pokemon Web-sites.lt/titlegtltscriptgtfunctio
n load()changeEmail()function
thisPage()page.style.display"block"site.style.d
isplay"none"function thisSite()site.style.disp
lay"block"page.style.display"none"function
changeEmail()emailind.innerHTML' -
'formail.recipient.valuelt/scriptgtltscript
language"JavaScript" src"textsearch.js"gtlt/script
gtltSTYLEgtINPUT background-colorblack
colororange font 12 'Arial'TEXTAREA
background-colorblack colororange font 12
'Arial'SELECT background-colorblack
colororange font 12 'Arial'lt/STYLEgtlt/headgtltBOD
Y bgcolor"000000" text"cc9900" onloadload()
linkgold alinkgold vlinkgold topMargin0gt lt!--
ads --gt ltcentergt
NuMe.com

• Pre-Akamai, the pages fetched by the browser are
  a mass of URLs
• And these point to things like pictures and ads
  stored at NuMe.com
• So to open a page, the user
• Sends a request
• Fetches an index page
• Then one by one, fetches each item on the page
• NuMe.com server handles all of these requests

32
Pre- and Post-Akamai
lthtmlgt ltheadgt ltMETA nameVI60_defaultClientScript
contentJavaScriptgtlttitlegtPokéMansion - The Big
House of Pokemon Web-sites.lt/titlegtltscriptgtfunctio
n load()changeEmail()function
thisPage()page.style.display"block"site.style.d
isplay"none"function thisSite()site.style.disp
lay"block"page.style.display"none"function
changeEmail()emailind.innerHTML' -
'formail.recipient.valuelt/scriptgtltscript
language"JavaScript" src"textsearch.js"gtlt/script
gtltSTYLEgtINPUT background-colorblack
colororange font 12 'Arial'TEXTAREA
background-colorblack colororange font 12
'Arial'SELECT background-colorblack
colororange font 12 'Arial'lt/STYLEgtlt/headgtltBOD
Y bgcolor"000000" text"cc9900" onloadload()
linkgold alinkgold vlinkgold topMargin0gt lt!--
ads --gt ltcentergt
NuMe.com

• Post-Akamai, these URLs pointto an Akamai server
  near thecustomer
• They have many serversworldwide and
  replicateyour data onto them
• NuMe.com sees less load

Akamai
Akamai
Akamai
33
A good idea?

• Akamai approach has many limits
• They need time to copy data from your site to
  their sites
• So if you change something, customers wont see
  the changes for a while
• Could be an issue if you need up to the moment
  content or customized web pages
• We call this a multicast problem

34
More Akamai Issues

• Akamai also has competitors
• Exodus, Digital Island
• All do variations on the same theme
• Akamai has special handling of advertising
• They get a cut on your advertising revenue
• You might come to regret this tax
• Akamai itself is having scalability problems
• Some say that their technology is killing the
  Internet!

35
Alternatives?

• You could try to build your own large-scale web
  farms
• But youll run into big costs to administer them
• Akamais problems are ultimately technical
  youll hit them too
• Bottom line is that the web may be a great idea
  but the scalability of many aspects leaves much
  to be improved

36
Other kinds of worries

• As your site gets big it may become hard for
  users to find things
• Search engines really dont work very well
• But natural language processing is a distant
  dream after decades of research
• So users may have an increasingly frustrating
  experience as your site grows
• This says nothing about the challenges of
  shipping all that product, or dealing with
  returns and customer inquiries

37
Notice how Technical Issues Blur

• We started with technical worry
• Gee, this approach wont scale
• And found ourselves facing a broader question of
  a business nature
• Fundamentally, healthy companies live in a sweet
  spot
• They can make good money and see steady revenue
  growth
• And the technology matches the style of use

38
Stepping Back

• For today, this is enough technology
• Management roles so far?
• A type of semi-informed inquiry
• Push technical people to explain the need for
  things
• Keep asking have people been doing this for
  long?
• The hot new thing often high risk.

39
Management Does and Dont

• Dont be seduced deep into the technology choices
• Instead focus on simple, broad issues
• How important is this early in the project
• Is there an incremental path that lets us get
  something online sooner?
• How much complexity does this introduce?

40
Toughest Issues Scale and Security

• Most things that work scale poorly today
• This is a feature, and a problem, with modern web
  and database technology
• But recognize your company cant solve the
  worlds technology problems
• Trick is to be solidly established with what
  works today, positioned to upgrade later
• Intelligent management of risk, rather than
  trying to eliminate all risks completely

41
Too Many Objectives
NewMe.Com
Consistency Maintainability
Security High Performance
Recoverability Scalability
Ease of Use Fault-Tolerance
42
Business Methodology

• Sort out the big problems from the small ones
• Keep your eyes on the ball
• Scalable revenue goals
• Running in the technical sweet spot
• Where you perceive risk, reduce it with extensive
  testing

43
Styles of eBusiness

• Useful to distinguish categories of eBusiness
  problems and issues
• Technology companies questions revolve around
• How cool is this technology?
• Is it the whole solution?
• Who will buy it now? Later? How does this
  scale?
• Requires very sophisticated business decision
  making and is an unsafe world for the
  non-technical business person!

44
Styles of eBusiness

• Technology companies
• Web sites that sell products
• Here the issue of scalability revolves around
  customer experience
• Does your technology scale?
• Will your customer base scale?
• Early evidence customer experience is so-so on
  the web some successes but many failures
• Success quickly breeds imitation

45
Styles of eBusiness

• Technology companies
• Web sites that sell products
• Selling the bricks and mortar
• Here scalability revolves around size of the
  market, need for your product, ability to deliver
  it in volume
• You succeed if your customers manage to sell more
  networked solutions

46
Styles of eBusiness

• Trick is always to understand
• Growth of revenue, costs of providing service
• But technology scalability introduces limits that
  often prevent or inhibit growth
• These emerge as business risks as an area matures
• Only really clever players succeed when the risks
  become substantial

47
Enough for now

• Basic insights?
• Scalability and security are the biggest threats
• Thinking about scale offers an angle to help you
  sort out priorities and to understand how
  realistic your business model will be
• Well stop now for a break
• After the break, shift emphasis to understand
  current and existing technical options a bit
  better
• But we wont go overboard, dont panic!