Online Algorithms and Decision Making with Partial Information

1
Online Algorithms and Decision Making with
Partial Information

• Sandy Irani
• UC Irvine
• Collaborators
• Rajesh Gupta
• Xiangwen Lu
• Dinesh Ramanathan
• Amelia Regan
• Sandeep Shukla

2
Online Algorithms

• Input is revealed to the algorithm incrementally
• Output is produced incrementally
• Some output must be produced before the entire
  input is known to the algorithm
• How to make decisions with partial information?
• Unknown information the future.

3
Applications

• Resource Allocation
• Scheduling
• Memory Management
• Routing
• Robot Motion Planning
• Exploring an unknown terrain
• Finding a destination
• Computational Finance

4
Methods of Analysis

• Competitive Analysis (Worst Case)
• For any input, the cost of our online algorithm
  is never worse than c times the cost of the
  optimal offline algorithm.
• Pros can make very robust statements about the
  performance of a strategy.
• Cons results tend to be pessimistic.

5
Methods of Analysis

• Probabilistic Analysis
• Assume a distribution generating the input.
• Find an algorithm which minimizes the expected
  cost of the algorithm.
• Pros can incorporate information predicting the
  future.
• Cons can be difficult to determine probability
  distributions accurately.

6
Online Problems

• Dynamic State Management
• Dynamic Vehicle Traveling Repair
• Online Load Balancing with Multiple Resources

7
Dynamic State Management

• A system must be prepared for an action at some
  unknown time in the future.
• The system has a choice of residing in one of a
  finite number of states.
• When the system is called to act, it must be in
  the Active state.
• Each state i is described by two constants
• ?i Cost to maintain state i as a function of
  time. (Constant of proportionality is ?i for
  state i).
• ?i Cost to transition to the Active state when
  a task must be performed.

8
Dynamic State Management

• Offline Algorithm
• Knows length of time t until the action must be
  performed and will pick the optimal state (i.e.
  the state that minimizes (?i t ?i ).
• Online Algorithm
• Described by a set of thresholds T1 ,T2, ,Tk.
• Ti is the time the algorithm transitions from
  state i-1 to stat i.
• States are ordered such that ?i-1 ? ?i.

9
2-State SystemThe Ski Rental Problem

• Start to learn to ski.
• Dont know in advance the number of times you
  will want to go skiing.
• Should you rent or buy skis?
• Cost to buy 300.
• Cost to rent 30.

10
The Ski Rental Problem

• Omniscient strategy (if you know in advance you
  will ski x times
• If x lt 10, optimal policy is to rent.
• If x gt 10, optimal policy is to buy the first
  day.
• If x 10, both policies are the same.
• An Online strategy is described by a threshold T
• Rent until the T-th ski trip, then buy.

11
The Ski Rental Problem

• Online policy (Balance)
• Rent until the cost of renting is equal to the
  cost of buying.
• Balance never spends more than 2 times the
  optimal policy, regardless of number of times
  skiing.
• Balance is 2-competitive.
• Cant do better for any online strategy, its
  possible it may spend at least twice the optimal
  cost.

12
Ski Rental Problem

• Length of time until action needed is generated
  by known distribution with density function p(t).
• Choose threshold T to minimize cost
• Theorem Karlin, Manasse, McGeough and Owicki
• For any distribution p(t), the expected cost of
  the above algorithm is within e/(e-1) of the
  optimal cost. Furthermore, there is a
  distribution for which no algorithm can be better
  than e/(e-1) times optimal.

13
Multi-State System Lower Envelope Idea
Irani, Shukla, Gupta
Time
For each state i, plot
14
Deterministic Lower Envelope Algorithm

• The Lower Envelope Defines an ordering of the
  states.
• Throw out states that do not appear on lower
  envelope
• Given this ordering, only need to determine
  thresholds
• When to transition from state i to state i1.
• Lower Envelope Algorithm Transitions from one
  state to the next at the discontinuities of the
  lower envelope curve.
• THEOREM Irani, Shukla, Gupta Lower Envelope
  Algorithm is 2-competitive.

15
Probabilistic Lower Envelope Algorithm

• Use same order of states as determined by lower
  envelope function.
• Our approach
• Determine threshold for transitioning from state
  i-1 to state i by solving the optimization
  problem where i-1 and i are the only states in
  the system.

16
Probabilistic Lower Envelope Algorithm

• Can show that
• THEOREM Irani, Shukla, Gupta The
  Probabilistic Lower Envelope Algorithm is
  e/(e-1)-competitive.

17
Pubs and Future Directions

• Current model only handles cost to transition
  back to active state
• Can incorporate costs to transition down to less
  ready states if these costs are additive.
• Future work
• Develop algorithms for more general transition
  costs.
• Publications
• Competitive Analysis of Dynamic Power Management
  Strategies for Systems with Multiple Power
  Savings States. Sandy Irani, Sandeep Shukla,
  Rajesh Gupta. Design Automation and Test In
  Europe, 2002.
• An Analysis of System Level Power Management
  Algorithms and Their Effects on Latency. Dinesh
  Ramanathan, Sandy Irani, Rajesh Gupta. IEEE
  Transactions on CAD.

18
Dynamic Travelling Repair Problem

• Set of clients distributed over a wide
  geographic area.
• Servers move around this area distributing goods,
  personnel, service to these clients.
• At any point in time a client can place a request
  with a deadline.
• GOAL plan the routes of the servers so as to
  serve as many requests as possible by their
  deadline.
• Can incorporate latency from traffic, adverse
  road conditions, rough terrain, into a more
  abstract metric space.

19
Competitive Analysis

• Online algorithms learn about a new request only
  when it arrives.
• The Optimal Offline algorithm (OPT) knows the
  entire sequence in advance and serves it
  optimally.
• is the benefit obtained by algorithm A
  on request sequence
• Algorithm A is c-competitive if there is a
  constant d such that for every ,

20
A Few restrictions

• Single Server
• Uniform Window Size
• Length of time between request arrival and
  deadline. (Normalized to 1).
• Bounded Diameter Metric Space.
• Diameter b. Results expressed as a function of
  b.
• Service time is 0.

21
Notification Time

• At what point is an online algorithm committed to
  accepting or rejecting a particular request?
• Parameter a ( ).
• Must decide within time a of arrival of a job.
• a 0 Must accept or reject immediately upon
  arrival of request
• a 1 Decision made only when request is served
  or interval has passed.

22
BATCH Algorithm

• Divide time into intervals of length 1/2.
• At the beginning of an interval jobs that arrived
  in the previous interval are considered.
• Follow path that serves the maximum number of
  these requests in the coming interval given the
  initial location of the server.
• Note BATCH must solve an instance of he prize
  collecting TSP for each interval.

23
BATCH requires that a is at least 1/2.
24
Results for BATCH

• TheoremIrani, Lu, Regan For any arrival
  sequence, the number of jobs served by BATCH is
  at least times the optimal
  offline algorithm.
• This analysis is tight for BATCH
• Note BATCH requires that b lt 1/2 to be
  competitive.

25
Algorithm DOUBLE-GAIN

• State of Algorithm
• Current location
• Set of unexpired outstanding requests
• New Rout for Time t Route which serves the
  maximum number of requests given the algorithms
  state at time t.
• N(t) set of requests served on the new route
  for time t.
• At any time t, the algorithm has a route which it
  is currently following Current Route for Time t.
• C(t) - set of requests to be served in the
  current route for time t.

26
Algorithm DOUBLE-GAIN

• At any point in time, DOUBLE-GAIN will adopt the
  new route for time t as its current route if
• DOUBLE-GAIN changes route if the number of
  requests gain is at least twice the number of
  requests lost.

C(t)
N(t)
27
Results for DOUBLE-GAIN

• Features of DOUBLE-GAIN
• Well defined for non-uniform window lengths.
• Can incorporate additional information about the
  future.
• Theorem Irani, Lu, Regan For any sequence of
  arrivals, the number of requests served by
  DOUBLE-GAIN is at least
  times optimal
• Note DOUBLE-GAIN is competitive for any b lt 1.
  
  
  

28
Lower Bounds

• TheoremIrani, Lu, Regan For any blt1. There is
  no algorithm which can guarantee on all metric
  spaces to serve more than
  
• times the optimal.

Upper bound on ratio for DOUBLE-GAIN within
constant factor of best possible
29
Pubs and Future Work on DTRP

• Handle non-uniform window lengths (time from
  arrival to deadline).
• More general cost function incorporating soft
  deadlines.
• Previous work which minimizes total latency
  allows for some jobs to be ignored.
• Deadlines with charge for lateness?
• Publications
• An Asymptotically Optimal Algorithm for the
  Dynamic Traveling Repair Problem. Xiangwen Lu,
  Amelia Regan and Sandy Irani. Proceedings of the
  Transportation Research Board Annual Meeting,
  2002.
• On-Line Algorithms for the Dynamic Traveling
  Repair Problem. Sandy Irani, Xiangwen Lu, Amelia
  Regan. Symposium on Discrete Algorithms, 2002.

30
Load Balancing

• Set of tasks to be executed. Each arrives with
  its work requirements.
• Tasks arrive one at a time and each must be
  assigned to one of m agents upon its arrival.
• Assign tasks in such a way as to minimize the
  maximum load on any agent.
• The load on a agent is the sum of the work
  requirements of all the tasks to which it has
  been assigned.

31
Competitive Ratio

• Let J be a set of jobs.
• Let cost cA(J) of algorithm A on instance J is
  the load of the most heavily loaded agent.
• cOPT(J) is the cost of the optimal offline
  algorithm on instance J.
• GOAL find online algorithm A which minimizes

32
Greedy Algorithm

• The GREEDY Algorithm places each incoming task on
  the most lightly loaded agent.
• TheoremGraham65 The cost of GREEDY is at most
  twice the optimal offline algorithm for any set
  of jobs J.

33
Multiple Resource Load Balancing

• Each task j is described by a vector
• There are r resources. Each entry in the vector
  indicates how much of each resource this task
  needs.
• (For example, drain on personnel, use of
  different types of goods or equipment, etc.)
• Load on a agent i with respect to resource k
  under algorithm A for instance J is
  which is the sum over all jobs assigned to agent
  i of the required amount of resource k for that
  job.

34
Multiple Resource Load Balancing

• Competitive Ratio for resource k on instance J
• Competitive Ratio for the problem

35
Results

• Algorithm select a machine so as to minimize
• TheoremIrani This algorithm achieves a
  competitive ratio of 2r, where r is the number of
  resources.

36
Open Problem

• Is Balance the best possible?
• Lower bounds?
• What about randomized algorithms?