Smoothing Splines

1
Smoothing Splines

• Naresh P. Cuntoor

2
Introduction

• Problem yif(xi) ei ei N(0,1)
• Want to estimate the underlying expectation
  function f from n observations
• f non-linear. Write
• linear basis expansions
• Properties of should be consistent with
  those of

3
Overview

• Piecewise polynomials and splines definition
  and examples
• Cubic splines extremal property
• Smoother matrix
• B-splines
• Control theoretic smoothing splines

4
Piecewise polynomials
5
Piecewise cubic
6
Cubic splines

• Function is a cubic spline on
  if it
• Interpolates
• Is smooth s, s, s are continuous
• Is a cubic polynomial in each interval between
  knots
• Problem with polynomial fit erratic boundaries
• Natural cubic splines linear beyond knots
• K knots gt K basis functions
• Cubic polynomials have extremal property among
  all differentiable functions, the natural cubic
  splines minimize the squared integrated second
  derivative

7
Smoothing splines
8
Computing smoothing splines

• Via regression splines
• Penalty term estimation difficult.
• B-splines
• Use polynomials which are zero outside range
• Linear case

9
B-splines

• Augment knot sequence,
• Set up a recursive relation for calculating the
  (Haar) basis functions Bi,m(x) the ith B-spline
  basis of order m

10
Smoothing

• Rewrite the objective function (RSS) as
• Bayesian interpretation

11
Control theoretic formulation

• Linear SISO system
• Make objective function more general by adding
  weights to the different components
• Issues conditioning, convergence