Title: Markets in Uncertainty: Risk, Gambling, and Information Aggregation
1Markets in UncertaintyRisk, Gambling, and
Information Aggregation
- a tutorial by
- David M. Pennock Michael P.
Wellman pennockd_at_yahoo-inc.com
wellman_at_umich.edu - dpennock.com
ai.eecs.umich.edu/people/wellman - presented at the 19th National Conference on
Artificial Intelligence, July 2004, San Jose, CA,
USA
MP1-1
2Outline
- Overview tour 15 minWhat is a market in
uncertainty? - Background 30 min
- Single agent perspective
- Subjective probability
- Utility, risk, and risk aversion
- Decision making under uncertainty
- Multiagent perspective
- Trading/allocating risk
- Pareto optimality
- Securities Markets in uncertainty
3Outline
- Mechanisms, examples andempirical studies
45 min - What how Instruments mechanisms
- Real-money markets Examples evaluations
- Iowa Electronic Market
- Options
- TradeSports Effects of war
- Horse racing, sports betting
- Play-money markets
4Outline
- Lab experiments and theory 20 min
- Laboratory experiments, field tests
- Theoretical underpinnings
- Rational expectations
- Efficient markets hypothesis
- No-Trade Theorems
- Information aggregation
5Outline
- Characterizing information 20 minaggregation
- Market as an opinion pool
- Market as a composite agent
- Market belief, utility
- Market Bayesian updates
- Market adaptation, dynamics
- Paradoxes, impossibilities
- Opinion pool impossibilities
- Composite agent non-existence
6Outline
- Computational aspects 60 min
- Combinatorics
- Compact securities markets
- Combinatorial securities markets
- Compound securities markets
- Market scoring rules
- Dynamic pari-mutuel market
- Policy Analysis Market
- Distributed market computation
- Legal issues miscellaneous 5 min
- Discussion, QA 15 min
71. Overview tour
- What is a market in uncertainty ?
8A market in uncertainty
- Take a random variable, e.g.
- Turn it into a financial instrument payoff
realized value of variable
US04Pres Bush?
2004 CAEarthquake?
9Aside Terminology
- Key aspect payout is uncertain
- Called variously asset, security, contingent
claim, derivative (future, option), stock,
prediction market, information market, gamble,
bet, wager, lottery - Historically mixed reputation
- Esp. gambling aspect
- A time when options were frowned upon
- But when regulated serve important social roles...
10Why? Reason 1
- Get information
- price ? expectation of random variable(in
theory, lab experiments, empirical studies,
...more later) - Do you have a random variable whose expectation
youd like to know?A market in uncertainty can
probably help
11Why?Reason 1 Information
- Information market financial mechanism
designed to obtain estimates of expectations of
random variables - Easy as 1, 2, 3
- Take a random variable whose expectation youd
like to know - Turn it into a financial instrument (payoff
realized value of variable) - Open a market in the financial instrument
- ? price(t) ? EtX (in many cases, ... more later)
12Getting information
- Non-market approach ask an expert
- How much would you pay for this?
- A 5/36 ? 0.1389
- caveat expert is knowledgeable
- caveat expert is truthful
- caveat expert is risk neutral, or RN for 1
- caveat expert has no significant outside stakes
13Getting information
- Non-market approach pay an expert
- Ask the expert for his report r of the
probability P( ) - Offer to pay the expert
- 100 log r if
- 100 log (1-r) if
- It so happens that the expert maximizes expected
profit by reporting r truthfully - caveat expert is knowledgeable
- caveat expert is truthful
- caveat expert is risk neutral, or RN
- caveat expert has no significant outside stakes
logarithmic scoring rule, a proper scoring
rule
14Getting information
- Market approach ask the publicexperts
non-experts alikeby opening a market - Let any person i submit a bid order an offer to
buy qi units at price pi - Let any person j submit an ask order an offer
to sell qj units at price pj(if you sell 1 unit,
you agree to pay 1 if ) - Match up agreeable trades (many poss. mechs...)
15Getting information
- Market approach ask the publicexperts
non-experts alikeby opening a market - If, at any time, for any bidder i and ask-er j,
pi gt pj, then ij trade min(qi,qj) units at price
?pj,pi - In equilibrium (no trades)
- max bid pi lt min ask pj bid-ask spread
- bounds aggregate public opinion of expectation
16Aside Mechanism alternatives
- This is the continuous double auction (CDA)
- Many other market auction mechanisms work
- call market
- pari-mutuel market
- market scoring rules
- CDA w/ market maker
- Vegas bookmaker, others
- Key Market price aggregate estimate of
expected value
Hanson 2002
17(Real) Great expectations
- For dice example, no need for market Ex is
known no one should disagree - Real power comes for non-obvious expectations of
random variables, e.g.
I am entitled to
1 if
0 otherwise
I am entitled to
x if interest rate x on Jan 1, 2004
18I am entitled to
max(0,x-k) if MSFT xon Jan 1, 2004
call option
I am entitled to
f(future weather)
weather derivative
I am entitled to
Bin Ladencaptured
1 if
0 otherwise
I am entitled to
1 if Kansas beats Marq.by gt 4.5 points 0
otherw.
19http//tradesports.com
20(No Transcript)
21Play moneyReal expectations
http//www.hsx.com/
22http//us.newsfutures.com/
http//www.ideosphere.com
Cancercuredby 2010
Machine Gochampionby 2020
23Does it work?Yes...
- Evidence from real markets, laboratory
experiments, and theory indicate that markets are
good at gathering information from many sources
and combining it appropriately e.g. - Markets like the Iowa Electronic Market predict
election outcomes better than pollsForsythe
1992, 1999Oliven 1995Rietz 1998Berg
2001Pennock 2002 - Futures and options markets rapidly incorporate
information, providing accurate forecasts of
their underlying commodities/securitiesSherrick
1996Jackwerth 1996Figlewski 1979Roll
1984Hayek 1945 - Sports betting markets provide accurate forecasts
of game outcomes Gandar 1998Thaler
1988Debnath EC03Schmidt 2002
24Does it work?Yes...
- E.g. (contd)
- Laboratory experiments confirm information
aggregationPlott 198219881997Forsythe
1990Chen, EC-2001 - And field tests Plott 2002
- Theoretical underpinnings rational
expectationsGrossman 1981Lucas 1972 - Procedural explanation agents learn from
pricesHanson 1998Mckelvey 1986Mckelvey
1990Nielsen 1990 - Proposals to use information markets to help
science Hanson 1995, policymakers, decision
makers Hanson 1999, government Hanson 2002,
military DARPA FutureMAP, PAM - Even market games work! Servan-Schreiber
2004Pennock 2001
25Why? Reason 2
- Manage risk
- If is horribly terrible for
youBuy a bunch of and if
happens, you are
compensated
26Why? Reason 2
- Manage risk
- If is horribly terrible for
youBuy a bunch of and if
happens, you are
compensated
I am entitled to
1 if
0 if
27The flip-side of prediction Hedging (Reason 2)
- Allocate risk (hedge)
- insured transfers risk to insurer, for
- farmer transfers risk to futures speculators
- put option buyer hedges against stock drop
seller assumes risk
- Aggregate information
- price of insurance? prob of catastrophe
- OJ futures prices yield weather forecasts
- prices of options encode prob dists over stock
movements - market-driven lines are unbiased estimates of
outcomes - IEM political forecasts
28Reason 2 Manage risk
- What is insurance?
- A bet that something bad will happen!
- E.g., Im betting my insurance co. that my house
will burn down theyre betting it wont. Note we
might agree on P(burn)! - Why? Because Ill be compensated if the bad thing
does happen - A risk-averse agent will seek to hedge (insure)
against undesirable outcomes
29E.g. stocks, options, futures, insurance, ...,
sports bets, ...
- Allocate risk (hedge)
- insured transfers risk to insurer, for
- farmer transfers risk to futures speculators
- put option buyer hedges against stock drop
seller assumes risk - sports bet may hedge against other stakes in
outcome
- Aggregate information
- price of insurance? prob of catastrophe
- OJ futures prices yield weather forecasts
- prices of options encode prob dists over stock
movements - market-driven lines are unbiased estimates of
outcomes - IEM political forecasts
30Examples
- I buy MSFT stock at s. Im afraid it will go
down. I buy a put option that pays Max0,k-s k
is strike price. If s goes down below k, my
stock investment goes down, but my option
investment goes up to compensate - Im a farmer. Im afraid corn prices will go too
low. I buy corn futures to lock in a price today.
31Examples
- I own a house in CA. Im afraid of earthquakes. I
pay an insurance premium so that, if an
earthquake happens, I am compensated. - I am an Oscar-nominated actor. Im afraid Im
going to lose. I bet against myself on an
offshore gambling site. If I do lose, I am
compensated. (Except that the offshore site
disappears and refuses to pay?)
32What am I buying?
- When you hedge/insure, you pay to reduce the
unpredictability of future wealth - Risk-aversion All else being equal, prefer
certainty to uncertainty in future wealth - Typically, a less risk-averse party (e.g., huge
insurance co, futures speculator) assumes the
uncertainty (risk) in return for an expected
profit
33On hedging and speculating
- Hedging is an act to reduce uncertainty
- Speculating is an act to increase expected future
wealth - A given agent engages in a (largely inseparable)
mixture of the two - Both can be encoded together as a maximization of
expected utility, where utility is a function of
wealth, ... more later
34On trading
- Why would two parties agree to trade in a market
in uncertainty? - They disagree on expected values (probs)
- They differ in their risk attitude or exposure
they trade to reallocate risk - Both (most likely)
- Aside legality is murky, though generally (2) is
legal in the US while (1) often is not. In
reality, it is nearly impossible to differentiate.
35On computational issues
some
?
- Information aggregation is a form of distributed
computation - Agent level
- nontrivial optimization problem, even in 1
marketultimately a game-theoretic question - probability representation, updating algorithm
(Bayes net) - decision representation, algorithm (POMDP)
- agent problems computational complexity,
algorithms, approximations, incentives
36On computational issues
some
?
- Mechanism level
- Single market
- What can a market compute?
- How fast (time complexity)?
- Do some mechanisms converge faster (e.g.,
subsidy) - Multiple markets
- How many securities to compute a given fn? How
many secs to support sufficient social
welfare?(expressivity and representational
compactness) - Nontrivial combinatorics (auctioneers
computational complexity algorithms
approximations incentives)
37On computational issues
some
?
- Machine learning, data mining
- Beat the market (exploiting combinatorics?)
- Explain the market, information retrieval
- Detect fraud
382. Background
- Single agent perspective
- Subjective probability
- Utility, risk, and risk aversion
- Decision making under uncertainty
- Multiagent perspective
- Trading/allocating risk
- Pareto optimality
- Securities markets in uncertainty
39Decision making under uncertainty
- How should agents behave (make decisions, choose
actions) when faced with uncertainty? - Decision theory Prescribes maximizing expected
utility
40Why reason about uncertainty?
- Propositional logic No uncertaintyCould never
explain seatbelt use - Decisions D - drive car S - wear seatbelt
- Events A - accident occurs
- A ? ?D
- ?A ? ?S
- Cant explain DS
- Key A is uncertain
41Why Bayesian uncertainty?
- E.g. You can buy skis for bOr you can rent for
b/k, kgt1 - Worst-case analysisRent for k days, then
buyYoull spend at most 2b - But what if you strongly believe youll skimore
than k times? ? Buy earlier - That k1st time is your last? ? Dont buy
- Expected (utility) case often more appropriate
42Decision making under uncertainty, an example
ABC TVs Who Wants to be a Millionaire?
43Decision making under uncertainty, an example
- v15 1,000,000 if correct 32,000 if
incorrect500,000 if walk away
44Decision making under uncertainty, an example
- if you answer
- Ev15 1,000,000 Pr(correct)32,000
Pr(incorrect) - if you walk away
- 500,000
45Decision making under uncertainty, an example
- if you answer
- Ev15 1,000,000 0.532,000 0.5
- 516,000
- if you walk away
- 500,000
- you should answer, right?
46Decision making under uncertainty, an example
- Most people wont answer risk averse
- U(x) log(x)
- if you answer
- Eu15 log(1,000,000) 0.5log(32,000)
0.5 - 6/24.5/2 5.25
- if you walk away
- log(500,000) 5.7
47Decision making under uncertainty, an example
- Maximizing Eui for ilt15 more complicated
Q7, L1,3
walk
answer
L1
L3
Q7, L3
? 0.4
X 0.6
log(2k)
walk
answer
L3
log(1k)
Q8, L1,3
? 0.8
X 0.2
log(2k)
log(1k)
Q8, L3
48Decision making under uncertainty, in general
?set of all possible future states of the world
49Decision making under uncertainty, in general
- ? are disjoint exhaustivestates of the world
- ?i rain tomorrow Bush elected Y! stock up
car not stolen ... - ?j rain tomorrow Bush elected Y! stock up
car stolen ...
?1
?2
?3
?i
?
?
??
50Decision making under uncertainty, in general
- Equivalent, more natural
- Ei rain tomorrowEj Bush elected
- Ek Y! stock up
- El car stolen
- ?2n
E1
E2
?
Ei
En
Ej
51Decision making under uncertainty, in general
- Preferences, utility
- ??igt?j ? u(?i) gt u(?j)
- Expected utility
- Eu ?? Pr(?)u(?)
- Decisions (actions) can affect Pr(?)
- What you should dochoose actions to maximize
expected utility - Why? To avoid being a money pump de
Finetti74, among other reasons...
52Preference under uncertainty
- Define a prospect, ? p, ?1 ?2
- Given the following axioms of ?
- orderability (?1 ? ?2) ? (?1 ? ?2) ? (?1 ?2)
- transitivity (?1 ? ?2) ? (?2 ? ?3) ? (?1 ? ?3)
- continuity ?1 ? ?2 ? ?3 ? ? p. ?2 p, ?1 ?3
- substitution ?1 ?2 ? p, ?1 ?3 p, ?2 ?3
- monotonicity ?1 ? ?2 ? pgtq ? p, ?1 ?2 ? q,
?? ?2 - decomposability
- p, ?1 q, ?2 ?3 q, p, ?1 ?2 p, ?1
?3 - Preference can be represented by a real-valued
expected utility function such that - u(p, ?1 ?2) p u(?1) (1p)u(?2)
53Utility functions
- (??? a probability distribution over ?)
- Eu ????represents preferences,
- Eu(?) ? Eu(??) iff ??? ???
- Let ?(?) au(?) b, agt0.
- Then E?(?) Eaub(?) a Eu(?) b.
- Since they represent the same preferences, ? and
u are strategically equivalent (? u).
54Utility of money
- Outcomes are dollars
- Risk attitude
- risk neutral u(x) x
- risk averse (typical) u concave (u??(x) lt 0 for
all x) - risk prone u convex
- Risk aversion function
- r(x) u??(x) / u?(x)
55Risk aversion hedging
- Eu.01 (4).99 (4.3) 4.2980
- Action buy 10,000 of insurance for 125
- Eu4.2983
- Even better, buy 5974.68 of insurance for 74.68
- Eu 4.2984 ? Optimal
?1 car stolenu(?1) log(10,000)
?2 car not stolenu(?2) log(20,000)
u(?1) log(19,875)
u(?2) log(19,875)
u(?1) log(15,900)
u(?2) log(19,925)
56Securities market s
- Note that, in previous example, risk-neutral
insurance company also profitsEv
.01(-5,900) 0.99(74.68) 14.93Both parties
gain from bilateral agreement - Securities market generalizes this to
- arbitrary states
- more than two parties
- Market mechanism to allocate risk among
participants
57Pareto optimality
- An allocation is Pareto optimal iff there does
not exist another solution that is - better for one agent and
- no worse for all the rest.
a minimal (and maximal?) condition for social
optimality, or efficiency.
58What is traded Securities
- Specifies state-contingent returns, r
(r1,,r?) in terms of numeraire (e.g., ) - Examples
- (1,,1) riskless numeraire (1)
- (0,,0,1,0,,0) pays off 1 in designated state
(Arrow security for
that state) - ri 1 if ?i?E1, ri 0 otherwise
-
1 if E1
59Terms of trade Prices
- Price pltEigt associated with security
- Relative prices dictate terms of exchange
- Facilitate multilateral exchange via bilateral
exchange - defines a common scale of resource value
- Can significantly simplify a resource allocation
mechanism - compresses all factors contributing to value into
a single number - A default interface for multiagent systems
1 if Ei
60Equilibrium
- General (competitive, Walrasian) equilibrium
describes a simultaneous equilibrium of
interconnected markets - Definition A price vector and allocation such
that - all agents making optimal demand decisions
(positive demand buy negative demand sell) - all markets have zero aggregate demand(buy
volume equals sell volume)
61Complete securities market
- A set of securities is complete if rank of
returns matrix ? ?1 - For example, set of ? ?1 Arrow securities
Arrow-Debreu securities market - Market with complete set of securities guarantees
a Pareto optimal allocation of risk, under
classical conditions
62Incomplete markets
- Securities do not span states of nature (always
the case in practice) - Equilibria may exist, but may not be Pareto
optimalExample missed insurance opportunity - More Theory of Incomplete Markets, Magill
Quinzii, MIT Press, 1998
63Why trade securities?
- Profit from perceived mispricings
- Price pltE1gt differs significantly enough from
traders belief Pr(E1) - speculation
- Insure against risk
- Traders marginal value for wealth in E1,
relative to pltE1gt, differs from that in other
states - e.g., home fire insurance
- hedging
64Societal roles of security markets
- From speculation
- Aggregate beliefs
- Disseminate information
- From hedging
- Allocate risk
65Summary Background
- General equilibrium framework for market-based
exchange - Incorporate uncertainty through securities
- Agents trade securities in order to optimize
expected utility, thereby - Allocating risk
- Reaching consensus probabilities
663. Mechanisms, examples empirical studies
- What howInstruments mechanisms
- Real-money marketsExamples evaluations
- Iowa Electronic Market
- Options
- TradeSports Effects of war
- Horse racing, sports betting
- Play-money markets
67Building a market in uncertainty
- What is being traded?the good
- Define
- Random variable
- Payoff function
- Payoff output
- How is it traded?the mechanism
- Call market
- Continuous double auction
- Continuous double auction w/ market maker
- Pari-mutuel
- Bookmaker
- Combinatorial (later)
68What is being traded?
- Underlying statistic / random variable
- Binary Discrete
- Continuous interest rate, dividend flow
- Clarity e.g., Saddam out, House burns,Gore
wins, Buchanan wins - Payoff function
- Arrow (0,0,0,1,0) Portfolio (2,4,0,1,0)
- Dividends, options Max0,s-k, arbitrary
(non-linear) fn - Payoff output
- dollars, fake money, commodities
6
69How is it traded?
- Call market
- Orders are collected over a period of time
collected orders are matched at end of period - One-time or repeated
- Pre-defined or randomized stopping time/rule
- Mth price auction
- M1st price auction
- k-double auction
- lim period?0 Continuous double auction
70A note on selling
- In a securities market, you can sell what you
dont have you agree to pay according to terms - Binary case sell 1 if A for 0.3
- Receive 0.3 (now, or contractually later), pay
1 if A - Exactly equivalent to buying 1 if A for 0.7
- sell 1 if A _at_ 0.3
- buy 1 if A _at_ 0.7
- Alternative Market institution always stands
ready to buy/sell exhaustive bundle for 1.00 - Iowa Electronic Market
A occurs A occurs-1.3 -.7 0.3 .3 0 -.7
-.7 1 -.7 .3
71Mth price auction
- N buyers and M sellers
- Mth price auction
- sort all bids from buyers and sellers
- price the Mth highest bid
- let n of buy offers gt price
- let m of sell offers lt price
- let x min(n,m)
- the x highest buy offers and x lowest sell offers
transact
72Call market
0.30
0.15
0.17
0.12
0.13
0.09
0.11
0.05
0.08
73Mth price auction
1
0.30
0.17
2
0.15
3
?
0.13
4
price 0.12
0.12
5
?
0.11
?
0.09
0.08
?
0.05
74M1st price auction
1
0.30
0.17
2
0.15
3
?
0.13
4
0.12
5
?
price 0.11
0.11
?
6
0.09
0.08
?
0.05
75k-double auction
1
0.30
0.17
2
0.15
3
?
0.13
4
price 0.11 0.01k
0.12
5
?
0.11
?
6
0.09
0.08
?
0.05
76Continuous double auctionCDA
- k-double auction repeated continuously
- buyers and sellers continually place offers
- as soon as a buy offer ? a sell offer, a
transaction occurs - At any given time, there is no overlap btw
highest buy offer lowest sell offer
77http//tradesports.com
78http//www.biz.uiowa.edu/iem
http//us.newsfutures.com/
79CDA with market maker
- Same as CDA, but with an extremely active, high
volume trader (often institutionally affiliated)
who is nearly always willing to buy at some price
p and sell at some price q gt p - Market maker essentially sets prices others take
it or leave it - While standard auctioneer takes no risk of its
own, market maker takes on considerable risk, has
potential for considerable reward
80CDA with market maker
- E.g. World Sports Exchange (WSE)
- Maintains 5 differential between bid ask
- Rules Markets are set to have 50 contracts on
the bid and 50 on the offer. This volume is
available first-come, first-served until it is
gone. After that, the markets automatically move
two dollars away from the price that was just
traded. - The depth of markets can vary with the contest.
- Also, WSE pauses market adjusts prices
(subjectively?) after major events (e.g., goals) - http//www.wsex.com/about/interactiverules.html
81CDA with market maker
- E.g. Hollywood Stock Exchange (HSX)
- Virtual Specialist automated market maker
- Always willing to buy sell at a single point
price ? no bid-ask spread - Price moves when buys/sells are imbalanced
- Fake money, so its OK if Virtual Specialist
loses money in fact it does Brian Dearth,
personal communication - http//www.hsx.com/
82http//www.wsex.com/
http//www.hsx.com/
83Bookmaker
- Common in sports betting, e.g. Las Vegas
- Bookmaker is like a market maker in a CDA
- Bookmaker sets money line, or the amount you
have to risk to win 100 (favorites), or the
amount you win by risking 100 (underdogs) - Bookmaker makes adjustments considering amount
bet on each side /or subjective probs - Alternative bookmaker sets game line, or
number of points the favored team has to win the
game by in order for a bet on the favorite to
win line is set such that the bet is roughly a
50/50 proposition
84Pari-mutuel mechanism
- Common at horse races, jai alai games
- n mutually exclusive outcome (e.g., horses)
- M1, M2, , Mn dollars bet on each
- If i wins all bets on 1, 2, , i-1,i1, , n
lose - All lost money is redistributed to those who bet
on i in proportion to amount they bet - That is, every 1 bet on i gets1 1/Mi
(M1, M2, ,Mi-1, Mi1, , Mn) 1/Mi (M1,
M2, , Mn)
85Pari-mutuel market
- E.g. horse racetrack style wagering
- Two outcomes A B
- Wagers
86Pari-mutuel market
- E.g. horse racetrack style wagering
- Two outcomes A B
- Wagers
?
87Pari-mutuel market
- E.g. horse racetrack style wagering
- Two outcomes A B
- Wagers
?
88Pari-mutuel market
- E.g. horse racetrack style wagering
- Two outcomes A B
- 2 equivalentways to considerpayment rule
- refund share of B
- share of total
?
89Pari-mutuel market
- Before race begins, odds are reported, or the
amount you would win per dollar if betting ended
now - Horse A 1.2 for 1 Horse B 25 for 1 etc.
- Normalized odds consensus probabilities
- Actual payoffs depend only on final odds, not
odds at time of bet incentive to wait - In practice track takes 17 first, then
redistributes what remains
90Examples of markets
- Continuous double auction (CDA)
- Iowa Electronic Market (IEM)
- TradeSports, experimental Soccer market
- Financial markets stocks, options, derivatives
- CDA with market maker
- World Sports Exchange (WSE)
- Hollywood Stock Exchange (HSX)
- Pari-mutuel horse racing
- Bookmaker NBA point spread betting
91Example IEMIowa Electronic Market
http//www.biz.uiowa.edu/iem
US Democratic Pres. nominee 2004
1 if other wins
1 if Kerry wins
1 if Lieberman wins
1 if Gephardt wins
1 if H. Clinton wins
priceECPr(C)0.056
as of 4/22/2003
92Example IEMIowa Electronic Market
http//www.biz.uiowa.edu/iem
US Presidential election 2004
1 if Democrat votes gt Repub
1 if Republican votes gt Dem
priceERPr(R)0.494
as of 7/25/2004
93IEM vote share market
US Pres. election vote share 2004
1 ? 2-party vote share of Bush v. other
1 ? 2-party vote share of other Dem
1 ? vote share of Bush v. Kerry
1 ? vote share of Kerry
priceEVS for K0.148
as of 4/22/2003
94IEM vote share market
US Pres. election vote share 2004
1 ? 2-party vote share of Kerry
1 ? vote share of Bush v. Kerry
priceEVS for B v. K0.508
1 ? vote share of Dean
1 ? vote share of Bush v. Dean
as of 7/25/2004
95Example IEM 1992
Source Berg, DARPA Workshop, 2002
96Example IEM
Source Berg, DARPA Workshop, 2002
97Example IEM
Source Berg, DARPA Workshop, 2002
98Example IEM
Source Berg, DARPA Workshop, 2002
99Example IEM
Source Berg, DARPA Workshop, 2002
100Speed TradeSports
Source Wolfers 2004
Contract Pays 100 if Cubs win game 6 (NLCS)
Price of contract (Probability that Cubs win)
Fan reaches over and spoils Alous catch. Still
1 out.
Cubs are winning 3-0 top of the 8th1 out.
The Marlins proceed to hit 8 runs in the 8th
inning
Time (in Ireland)
101The marginal trader Forsythe 1992,1999 Oliven
1995 Rietz 1998
- Individuals in IEM are biased, make mistakes
- Democrats buy too many Democratic stocks
- Arbitrage is left on the table
- When there are multiple equivalent trades, the
cheapest is not always chosen - Yet market as a whole is accurate, efficient
- Why? Prices are set by marginal traders, not
average traders - Marginal traders are active traders, price
setters, unbiased, better performers
102Forecast error bounds Berg 2001
- Single market gives Ex
- IEM winner takes all P(candidate wins) P(C)
- IEM vote share Ecandidate vote share EV
- Can we get error bounds? e.g. Varx?
- Yes combine the two markets
103Evaluating accuracyRecall log scoring rule
- Logarithmic scoring rule(one of several proper
scoring rules) - Pay an expert approach
- Offer to pay the expert
- 100 log r if
- 100 log (1-r) if
- Expert should choose rPr(A), given caveats
104Evaluating accuracy
- Log score gives incentives to be truthful
- But log score is also an appropriate measure of
experts accuracy - Experts who are better probability assessors will
earn a higher avg log score over time - We advocate evaluate the market just as you
would evaluate an individual expert - For a given market (person), compute average log
score over many assessments
105? log score ? information
- Log score dynamics also shows speed of
information incorporation - Expected log score P(A) log P(A) P(A) log
P(A) - entropy - Actual log score at time t
- - amount market is surprised by true outcome
- - of bits of info provided by revelation of
true outcome - As bits of info flow into market, log score ?
106Avg log score dynamics
IEM
FX
WSE bball
HSX
WSE soccer
107Avg log score22 IEM political markets
Average log score ?i log (pi)/N pi ith
winners normalized price
108Example options
- Options prices (partially) encode a probability
distribution over their underlying stocks - Arbitrary derivative ? P(underlying asset)
payoff
10
20
30
40
50
stock price s
109Example options
- Options prices (partially) encode a probability
distribution over their underlying stocks - Arbitrary derivative ? P(underlying asset)
butterfly spread
payoff
10
20
30
40
50
stock price s
- 2call30
110Example options
- Options prices (partially) encode a probability
distribution over their underlying stocks - Arbitrary derivative ? P(underlying asset)
payoff
10
20
30
40
50
stock price s
- 2call40
111Example options
- call10 - 2 call20 call30 2.13
relative - call30 - 2 call30 call40 5.73 likelihood
of falling - call30 - 2 call40 call50 3.54 near
center
payoff
2.13
5.73
3.54
10
20
30
40
50
stock price s
112Example options
- More generally, uses prices as constraintsEMax0
,s-10p10 EMax0,s-20p20 ... etc. - Fit to assumed distribution or maximize
entropy, smothness, etc. subject to constraints
Jackwerth 1996
probability
10
20
30
40
50
stock price s
113Example TradeSports
Source Wolfers 2004
114Source Wolfers 2004
115Source Wolfers 2004
116Source Wolfers 2004
117State Price Distribution
Source Wolfers 2004
118State Price Distribution War and Peace
Source Wolfers 2004
119Example horse racing
- Pari-mutuel mechanism
- Normalized odds match objective frequencies of
winning very closely - 31 horses win about twice as much as 61 horses,
etc. - Slight favorite-longshot bias (favorites are
better bets extremely rarely Ereturn gt 0) - Ali 77 Rosett 65 Snyder 78 Thaler 88
Weitzman 65
120Example horse racing
- Tracks can be biased, e.g., Winning Colors,a S
Californian horse, 1988 Kentucky Derby - 1 paid in MA 10.60, ..., in FL 10.40,
...,KY 8.80,..., MI 7.40, ..., N.CA 5.20,
..., S.CA 4.40 Wong 2001 - Some teams apparently make more than a decent
living beating the track using computer models
e.g., Bill Benters team in Hong Kong - logistic regression standard now SVMs Edelman
2003 - http//www.unr.edu/gaming/confer.asp
- http//www.wired.com/wired/archive/10.03/betting_p
r.html
121Example sports betting
- US NBA Basketball
- Closing lines set by market are unbiased
estimates of game outcomes?better than opening
lines set by experts Gandar 98 - Soccer (European football)Experimental market in
Euro 2000 Championship Schmidt 2002 - Market prediction gt betting odds gt random
- Market confidence statistically meaningful
122World Sports Exchange WSE
- Online in-game sports betting markets
- Trading allowed continuously throughout game as
goals are scored, penalties are called, etc. ?
i.e. as information is revealed! - National Basketball Association (NBA)
- Soccer World Cup
- MLB, NHL, golf, others
- http//wsex.com
- Debnath, EC-2003
Same concept,better site
123Soccer World Cup 2002
- 15 Soccer markets (June 715, 2002)
- Several 1st round and 2nd round games
- All games ended without penalty shoot-out
- Scores recorded from www.LiveScore.com
- Sampled the stream of price and score
information every 10 seconds
124Ex Price reaction to goals
- Sweden vs. Nigeria (Final score 2-1, goals scored
at 31st (0-1), 39th (1-1) and 83rd (2-1)
minutes. Yellow bars indicate goals.
125Ex Price reaction to goals
- Denmark vs. France (Final Score 2-0, goals
scored at the 22nd (1-0) and 85th (2-0) minute of
the game) Yellow bars indicate goals
126Avg log score entropy
127Delay Calculation
Where Timestamp of scoring Timestamp of
price update Delay in updating score
network delay Delay in updating the price
network delay
128Reaction time after goals
129NBA 2002
- 18 basketball markets during 2002 Championships
(May 631, 2002) - Score recorded from www.SportsLine.com
- Sampled the stream of price and score
information every 10 seconds
130Correlation between price and score
- San Antonio vs. LA Lakers (May 07, 2002, Final
Score 88-85, Correlation 0.93).
131Correlation between price and score
132Avg log score entropy
133Soccer vs. NBA
Soccer World Cup 2002
NBA Championship 2002
134Soccer vs. NBA
- Soccer characteristics
- Price does not change very often
- Price change is abrupt immediate after goal
- Average entropy decreases gradually toward 0
- Comebacks less likely?more surprising when they
occur - Basketball characteristics
- Price changes very often by small amounts
- Price is well correlated with scoring
- More uncertainty until late in the games
- entropy gt 0.7 for 77 of game gt0.8 for 55.5 of
game - More exciting late?outcome is unclear until
near end
135Basketball as coin flips
- Model scoring as a series of coin flips
- tails Boston 1
- heads Detroit 1
- Current scores Bt,Dt
- Final scores BT,DT
- ComputeP(BT-DT gt 5.5 Bt,Dt)
- ED B 180
- EB - D 5.5
- EB92.75ED87.25
- p P(tails) P(Boston) 92.75/180
0.515
May10 505 DETROIT o/u 180 0700 506 BOSTON -5.5
180-Bt-Dt
180-Bt-Dtj
? ( )pj (1-p)(180-Bt-Dt-j)
j93-Bt
136Basketball as coin flips
137Explain the marketParallel IR
Pennock 2002
IEM Giuliani NY Senate 2000
Use expected entropy lossto determine the key
wordsand phrases that bestdifferentiate between
textstreams before and after thedate of interest
138Explain the marketParallel IR
us.politics
IEM Gore US Pres 2000
florida, ballots, recount, palm
beach, ballot, beach county, palm beach
county
FX Extraterrestrial Life
sci.space.news
meteorite, life, evidence, martian
meteorite,primitive, gibson, organic, of
possible,martian, life on mars, ...
139Applications future work
- Monitoring dynamics
- Automatic explanations
- Low probability event detection
- Sporting events auto highlights, auto summary,
attention scheduling, finding turning points,
most exciting games/moments, modeling different
sports...
140Play-money market games
http//www.hsx.com/
http//www.newsfutures.com/
http//www.100world.com/
http//www.ideosphere.com/
http//www.ipreo.com/
http//www.incentivemarkets.com/
141Play-money market games
- Many studies show that prices in real-money
markets provide accurate likelihoods - Researchers credit monetary incentives/risk
- Can play money markets provide accurate
forecasts? - Incentives in market games may derive from
entertainment value, educational value,
competitive spirit, bragging rights, prizes
142Market games analyzed
- Hollywood Stock Exchange (HSX)
- Play-money market in movies and stars
- Movie stocks movie options
- Award options (e.g., Oscar options)
- Foresight Exchange (FX)
- Market game to bet on developments in science
technology e.g., Cancer cured by 2010 Higgs
boson verified Water on moon Extraterrestrial
life verified - NewsFutures
- Newsworthy events items of pop interest
143Put-call parity
- stock price s - call price put price strike
price k
payoff
10
20
30
40
50
stock price s
144Internal coherence HSX
- Prices of movie stocks and options adhere to
put-call parity, as in real markets - Arbitrage loopholes disappear over time, as in
real markets
145Internal coherenceHSX vs IEM
- Arbitrage closure for HSX award options
- Arbitrage closure on IEM qualitatively similar to
HSX, though quantitatively more efficient
146Forecast accuracy HSX
- 0.94 correlation
- Comparable to expert forecasts at Box Office Mojo
147Combining forecasts
- HSX Box Office Mojo (expert forecast)
- Correlation of errors 0.818
148Probabilistic forecastsHSX
- Bins of similarly-priced options
- Observed frequency? average price
- Analysis similar for horse racing markets
- Error bars 95 confidence intervals assuming
events are indep Bernoulli trials
149Avg logarithmic score
HSX Oscar options 2000
forecast source avg log score Feb 19 HSX
prices -0.854 DPRoberts -0.874 Fielding -1.04 e
xpert consensus -1.05 Feb 18 HSX
prices -1.08 Tom -1.08 John -1.22 Jen -1.25
150Probabilistic forecastsFX
- Prices 30 days before expiration
- Similar results
- 60 days before
- specific date
- Average logarithmic score
FX
151Real marketsvs. market games
IEM
HSX
averagelog score
arbitrageclosure
152Real marketsvs. market games
HSX
FX, F1P6
forecast source avg log score F1P6 linear
scoring -1.84 F1P6 F1-style scoring -1.82 betting
odds -1.86 F1P6 flat scoring -2.03 F1P6 winner
scoring -2.32
expectedvalueforecasts489 movies
153Does money matter? Play vs real, head to head
- Experiment
- 2003 NFL Season
- Online football forecasting competition
- Contestants assess probabilities for each game
- Quadratic scoring rule
- 2,000 experts, plus
- NewsFutures (play )
- Tradesports (real )
- Used last trade prices
- Results
- Play money and real money performed similarly
- 6th and 8th respectively
- Markets beat most of the 2,000 contestants
- Average of experts came 39th
Forthcoming, Electronic Markets, Emile
Servan-Schreiber, Justin Wolfers, David Pennock
and Brian Galebach
154(No Transcript)
155Does money matter? Play vs real, head to head
StatisticallyTS NFNF gtgt Avg TS gt Avg
156Market games summary
- Online market games can contain a great deal of
information reflecting interactions among
millions of people - Naturally attract well-informed and
well-motivated players - Game players tend to be knowledgeable and
enthusiastic - Internet polls - skewed demographic
- Polls typically ask questions of the form What
do you want? - Games ask questions of the form What do you
think will happen?
157Market games discussion
- Are incentives strong enough?
- Yes (to a degree)
- Manifested as price coherence, information
incorporation, and forecast accuracy - Reduced incentive for information discovery
possibly balanced by better interpersonal
weighting - Statistical validations show HSX, FX, NF are
reliable sources for forecasts - HSX predictions gt expert predictions
- Combining sources can help
158Applications
- Obtain information from existing games
- Build new games in areas of interest
- Alternative to costly market research
- Easy/inexpensive to setup compared to real
markets - Few regulations compared to real markets
- Worldwide audience
159Future work
- Data mining and fusion algorithms can improve
predictions - Weight users by expertise, reliability, etc.
- Controlling for manipulation
- Merging with other sources
- Box office prediction (market chat groups,
query logs, movie reviews, news, experts) - Weather forecasting (futures, derivatives
experts, satellite images) - Privacy issues and incentives
1604. Lab experiments theory
- Laboratory experiments, field tests
- Theoretical underpinnings
- Rational expectations
- Efficient markets hypothesis
- No-Trade Theorems
- Information aggregation
161Laboratory experiments
- Experimental economics
- Plott and decendents Ledyard, Hanson, Fine,
Coughlan, Chen, ... (and others) - Controlled tests of information aggregation
- Participants are given information, asked to
trade in market for real monetary stakes - Equilibrium is examined for signs of information
incorporation
162Plott Sunder 1982
- Three disjoint exhaustive states X,Y,Z
- Three securities
- A few insiders know true state Z
- Market equilibrates according to rational
expectations as if everyone knew Z
163Plott Sunder 1982
- Three disjoint exhaustive states X,Y,Z
- Three securities
- Some see samples of joint can infer P(Zsamples)
- Results less definitive
164Plott Sunder 1988
- Three disjoint exhaustive states X,Y,Z
- Three securities
- A few insiders know true state is not X
- A few insiders know true state is not Y
- Market equilibrates according to rational
expectations Z true
165Plott Sunder 1988
- Three disjoint exhaustive states X,Y,Z
- One security
- A few insiders know true state is not X
- A few insiders know true state is not Y
- Market does not equilibrate according to rational
expectations
166Forsythe and Lundholm 90
- Three disjoint exhaustive states X,Y,Z
- One security
- Some know not X
- Some know not Y
- As long as traders are sufficiently knowledgeable
experienced, market equilibrates according to
rational expectations
1 if Z
not X
not Y
1
price of Z
0
time
167Small groups
- In small, illiquid markets, information
aggregation can fail - Chen, Fine, Huberman EC-2001 propose a two
stage process - Trade in a market to assess participants risk
attitude and predictive ability - Query participants probabilities using the log
score compute a weighted average of
probabilities, with weights derived from step 1
168Small groups
Source Fine DARPA Workshop, 2002
169Field test Hewlett Packard
- Plott Chen 2002 conducted a field test at
Hewlett Packard (HP) - Set up a securities market to predict, e.g. next
months sales (in ) of product X - 1 iff 0 lt sales lt 10K 1 iff
20K lt sales lt 30K - 1 iff 10K lt sales lt 20K 1 iff sales
gt 30K - Employees could trade at lunch, weekends, for
real - Market predictions beat official HP forecasts
170Why does it work?Rational expectations
- Theory Even when agents have asymmetric
information, market equilibrates as if all agents
had all info Grossman 1981 Lucas 1972 - Procedural explanation agents learn from prices
Hanson 98 Mckelvey 86 Mckelvey 90 Nielsen 90 - Agents begin with common priors, differing
information - Observe sufficient summary statistic (e.g.,
price) - Converge to common posteriors
- In compete market, all (private) info is revealed
171Efficient market hypotheses (EMH)
- Internal coherenceprices are self-consistent or
arbitrage-free - Weak form Internal unpredictabilityfuture
prices unpredictable from past prices - Semi-strong form Unpredictabilityfuture prices
unpredictable from all public info - Strong form Expert-level accuracyunpredictable
from all public private infoexperts cannot
outperform naïve traders - More
stronger assumps
http//www.investorhome.com/emh.htm
172How efficient are markets?
- Good question as many opinions as experts
- Cannot prove efficiency can only detect
inefficiency - Generally, it is thought that large public
markets are very efficient, smaller markets
questionable - Still, strong form is sometimes too strong
- There is betting on Oscars until winners are
announced - Prices do not converge completely on eventual
winners - Yet aggregating all private knowledge in the
world (including Academy members votes) would
yield the precise winners with certainty
173No-trade theorems
- Why trade? These markets are zero-sum games
(negative sum w/ transaction fees) - For all money earned, there is an equal (greater)
amount lost am I smarter than average? - Rational risk-neutral traders will never trade
Milgrom Stokey 1982Aumann 1976. Informally - Only those smarter than average should trade
- But once below avg traders leave, avg goes up
- Ad infinitum until no one is left
- Or If a rational trader is willing to trade with
me, he or she must know something I dont know
174But... Trade happens
- Volume in financial markets, gambling is high
- Why do people trade?
- 1. Different risk attitudes (insurance, hedging)
Cant explain all volume - 2. Irrational (boundedly rational) behavior
- Rationality arguments require unrealistic
computational abilities, including infinite
precision Bayesian updating, infinite
game-theoretic recursive reasoning - More than 1/2 of people think theyre smarter
than average - Biased beliefs, differing priors, inexperience,
mistakes, etc. - Note that its rational to trade as long as some
participants are irrational
175A theory of info aggregationNotation
Pennock 2002
- Event A (event negation A)
- Security
- Probability Pr(A)
- Likelihood L(A) Pr(A)/(1-Pr(A))
- Log-likelihood LL(A) ln L(A)
- Price of at time t pt
- Likelihood price lt pt/(1-pt)
- Log-likelihood price llt ln lt
1 if A
1 if A
176Assumptions
- Efficiency assumptionLet pt be the price of
at time tThen - Pr(Apt,pt-1,pt-2,,p0) pt
- (Markov assumpt. accuracy assumpt.)
1 if A
177Consequences
- Eptpt-1 x xexpected price at time t
is price at t-1 - log-likelihood price is e? as likely to go up
by ? in worlds where A is true, as it is to go up
? in worlds where A is false
178Consequences
- Pr(ptyA,pt-1x) Pr(ptypt-1x)price
is y/x times as likely to go from x to y in
worlds where A is true - given A is true, expected price at time t is
greater than price at t-1 by an amount prop. to
the variance of price
EptA,pt-1x x
179Empirical verification
Distribution of changes e in log-likelihood price
over 22 IEM markets, consistent with theory
Distribution of changes e in log-likelihood price
of winning candidates divided by losing
candidates. Line is ee, as predicted by theory
180Avg log score dynamics
IEM
FX
WSE bball
HSX
WSE soccer
181Applications future work
- Better understanding of market dynamics
assumptions required for predictive value - Closeness of fit to theory is a measure of market
forecast accuracy could serve as an evaluation
metric or confidence metric - Explaining symmetry, power-law dist in IEM
182Coin-flip model
- Previous theory minimalist assumptions no
explicit notion of evidence - Coin-flip model of evidence incorporation
- A ? occurrence of n/2 tails out of n flips
- Release