Last time we considered pixel by pixel background models, and looked for independent objects as fail

1
Lecture 14

• Last time we considered pixel by pixel background
  models, and looked for independent objects as
  failures in those background models.
• but what if the background is changing but not
  moving?

2

• The coefficients are specific to each image. If
  the images are in temporal order, so are the
  coefficients. Thus, we can plot the coefficients
  as a function of time.

3
flower video
4
This is, now, a representation of video. What
can you do with it?

• Recreation extending a video
• Recognition
• Maybe compression.

5

• The video can now be considered a time-series
  of coefficients. Lots of tools to predict the
  future values of a time series. (why??).

6
Example time series
7
ARMA

• Auto-regressive moving average

A0 is a matrix, A0 xt is a vector, together this
sum is a prediction of the x(t1) vector. These
A matrices describe the expected changes in the
PCA coefficients. How much data do you need to
solve for these As?
This is an example of a stationary stochastic
process.
8
Tricks

• Assumptions about relations between coefficients.
  (assume other coefficients dont matter).
• Heuristic constraints (make sure no PC gets too
  big).
• Other methods of extending the time sequence are
  fine --- (for instance, frequency based methods).

9
(No Transcript)
10
(No Transcript)
11
(No Transcript)
12
(No Transcript)
13
(No Transcript)
14
What more can you do?

• Editing of dynamic textures.
• Artificially weighting different coefficients
  changes the appearance of the video.
• Can also change the speed and contrast at which
  the new frames are generated

15
Steam Sequence
Input Video
16
Steam Sequence
Principal Components
17
Smoke Sequence
Input Video
18
Smoke Sequence
Principal Components
19
Toilet Sequence
Input Video
20
Toilet Sequence
Principal Components
21
Escalator Sequence
Input Video
22
Escalator Sequence
Principal Components
23
Flag Sequence
Input Video
24
Flag Sequence
Principal Components