Learning Piecewise Linear Maps for Approximation of Invertible Maps

1
Learning Piecewise Linear Maps for Approximation
of Invertible Maps

• Richard Groff, Pramod P. Khargonekar,
• and Daniel Koditschek
• University of Michigan and University of Florida
• Funded in part by NSF GOALI ECS-96322801

2
OUTLINE

• Motivation from Color Xerography
• Why Piecewise Linear Maps?
• GI Algorithm for Training Piecewise Linear Maps
• Numerical Experience
• Conclusions

3
Modeling and Control of Color Xerographic
Processes

• A University of Michigan and Xerox Corporation
  GOALI Collaborative Project

The Document Company XEROX
4
Color Printing Background

• Color printing is a very big business!!
• More than 4 trillion color prints last year
• Electrophotographic printing is advantaged in
  many applications
• Every print can be different from its predecessor
  and successor
• One-to-one marketing, customer catalogs, on
  demand books, etc.
• Customers require stringent performance levels
  for color printing
• Color predictability and stability with tight
  tolerances
• Every print, every job, every printer,
  everywhere, every time
• Its a control problem

5
The Office Environment
6
What Small Color Differences Look Like
7
How to Think About Color

• A three dimensional coordinate space
• Every point has a color
• Lots of different representations
• RGB, HSB, CMY, CIELab
• Documents may be stored/transmitted using any
  representation
• Monitors scanners use RGB, Printers use CMY
• The conversions between color spaces are
  multidimensional, non-linear, time-varying
  functions
• The functions are usually expressed as highly
  parameterized look-up tables.
• Because of the time variation, the look-up tables
  must be refreshed periodically - every day or
  more.

8
Electrophotographic Engine
Electrophotography (xerography) is one means of
arranging 100 million pigmented plastic particles
on a sheet of paper to faithfully replicate an
original.
9
Color Space Transformations

• Lab color space coordinate system based on
  psychophysics
• CMY device dependent color space coordinate
  system
• Xerographic engine
• Approximate , so that the desired color is
  printed

10
Color System Calibration
f-1 (L, a, b) (C,M,Y)
4 Send calibration data to the computer
Paper (L,a,b)
3 Read 1000 patches
Iterate until an error or frustration limit is
achieved.
2 Print the calibration document
1 Send a calibration document to the printer
f (C,M,Y) (L, a, b)
Digital (C,M,Y)
11
Color Space Transformation
Printer Gamut
Monitor Gamut
f (C,M,Y)
f-1 (L,a,b)
Domain CMY Space Device Dependent Specification
Co-Domain Lab space Device Independent
Specification
gamut the volume containing all the colors a
device can achieve
12
Piecewise Linear Function Approximation

• Ingredients
• A triangulation of the domain and codomain
• A map of domain vertices to codomain vertices
• an isomorphic map yields a piecewise linear
  homeomorphism
• Automatic invertibility
• Related Literature
• Pure mathematics (topology)
• Approximation theory (splines)
• MARS and other statistical approaches
• Two Problems
• approximation underlying function known
• estimation underlying function unknown but given
  finite noisy measurements

13
Key Issues

• Given data, how can one compute optimal piecewise
  linear approximation?
• A piecewise linear map can be completely
  specified by defining a partition of the domain
  and the parameters describing the linear
  functions on the individual cells in the
  partition.
• If we fix the partition is fixed, then the
  problem is simple least squares approximation
• But if we want to find optimal partition, then
  the problem is nonlinear in the parameters.

14
Relevant Literature for PL Approximation

• (discontinuous) Piecewise Polynomial
• Algorithm for 1D PP approximation Kioustelidis,
  Computing 81
• Uniqueness results for best PP approximation
  under generalized convexity conditions Gayle,
  Wolf, MathComp 96
• Algorithm for 1,2D PP approximation Baines,
  Tourigny, MathComp 97
• Non-parametric statistical methods
• CART, MARS Friedman, AnStat, 91
• Polynomial splines and their tensor products
  Stone et al., AnStat 97
• Locally linear weighted by confidence Schaal,
  Atkeson, Neur.Comp 98
• Few general results for optimal approximation
  using PL functions
• Highly nonlinear (in knot locations) optimization
  problem

15
The Scalar Case Moving Vertices with the Graph
Intersection Algorithm

• Alternative to nonlinear gradient descent on MSE
  
• Algorithm alternates between
• 1) Adjust Approximation
• Compute the least squares fit in each cell
• 2) Adjust Partition
• Move the knot to the intersection point of the
  least
• squares fits of the knots two neighboring cells
• What happens if lines
• nearly parallel?
• special rules

16
Performance of the GI Algorithm on Families of
Scalar Endomorphisms

• Compare with same parameters
• Taylor Polynomial (TP)
• Neural Network (NN)
• PL w/MSE optimization
• PL using GI algorithm (GI PL)
• GI consistently outperforms
• On most families PL perform similarly to
  NN
• Polynomial family is an exception

• Computational cost
• NN and constr PL have highest cost
• GI PL has low cost, comparable to
  linear-in-parameters TP

17
Analytical Result

• Approximation version of the algorithm.
• function available directly rather than through a
  finite data set
• MSE minimized, though GI does not explicitly aim
  to do so!
• Empirically, GI has good convergence properties

18
Difficulties in higher dimensions

• More than two simplices own each knot
• No unique graph intersection point
• Mesh tangles can occur
• Combinatorial optimization over triangulations
• Visualization/intuition already difficult with 2D
  domain (4D graph)

19
The Minvar Algorithm

• A working generalization of GI to higher
  dimensions
• Calculate least squares linear map
• Move knot points
• Move domain vertices by performing the following
  minimization
• Minimize variance - where maps are closest to
  agreeing, analogous to graph intersection
• Regularization - avoid tangles
• Calculate the corresponding range vertex as a
  weighted average of the LS maps at the new domain
  vertex

20
Constrained Knot Movement

• Largest errors occurred on domain boundaries
• interpolate from fixed knots far from input
• hand-tune boundary point locations ? improvement
• Solution constrained movement
• But now

21
Color Data Preliminary Numerical Results

• With improvements in minvar, LUT performance is
  within reach
• NN performs extremely well,
• but requires separate inverse
• composition of forward and inverse NNs does not
  give the identity map
• NN trains in 2hrs.
• minvar trains in 30sec.

22
Test Data minvar in action
23
Color Data
24
Results of 343 initial conditions
Piecewise linear sampled data Piecewise linear
approximant Noise free data Zero final error for
343 (73) starting points
25
Results of 343 initial conditions
Piecewise linear sampled data Piecewise linear
approximant Noise free data Zero final error for
343 (73) starting points
26
Analytical Result

• We have generalized the one dimensional local
  convergence result to n-dimensions
• If the function to be approximated is piecewise
  linear, and if the initial condition on the
  Minvar algorithm is close enough to the true PL
  function, then the Minvar algorithm converges to
  the true function in L .

27
Future Work

• Online version of minvar (active calibration)
• Estimation version of the problem effect of
  noise on convergence and approximation errors
• Applications to certain problems in robotics
• Connections to information theory