Applied Symbolic Computation (CS 680/480) Lecture 6: Integer Multiplication, Interpolation, and the Chinese Remainder Theorem

1
Applied Symbolic Computation (CS
680/480)Lecture 6 Integer Multiplication,
Interpolation, and the Chinese Remainder Theorem

• Jeremy R. Johnson
• May 9, 2001

2
Introduction

• Objective To show the relationship between the
  Chinese Remainder Theorem and Interpolation. To
  derive a series of fast integer multiplication
  algorithms using interpolation.
• Karatsubas Algorithm
• Polynomial algebra
• Polynomial version of the Chinese Remainder
  Theorem
• Interpolation
• Vandermonde Matrices
• Toom-Cook algorithm
• Polynomial multiplication using interpolation
• References Lipson, Cormen et al.

3
Karatsubas Algorithm

• Using the classical pen and paper algorithm two n
  digit integers can be multiplied in O(n2)
  operations. Karatsuba came up with a faster
  algorithm.
• Let A and B be two integers with
• A A110k A0, A0 lt 10k
• B B110k B0, B0 lt 10k
• C AB (A110k A0)(B110k B0)
• A1B1102k (A1B0 A0 B1)10k A0B0
• Instead this can be computed with 3
  multiplications
• T0 A0B0
• T1 (A1 A0)(B1 B0)
• T2 A1B1
• C T2102k (T1 - T0 - T2)10k T0

4
Complexity of Karatsubas Algorithm

• Let T(n) be the time to compute the product of
  two n-digit numbers using Karatsubas algorithm.
  Assume n 2k. T(n) ?(nlg(3)), lg(3) ? 1.58
• T(n) ? 3T(n/2) cn
• ? 3(3T(n/4) c(n/2)) cn
  32T(n/22) cn(3/2 1)
• ? 32(3T(n/23) c(n/4)) cn(3/2
  1)
• 33T(n/23) cn(32/22 3/2 1)
• ? 3iT(n/2i) cn(3i-1/2i-1
  3/2 1)
• ...
• ? cn((3/2)k - 1)/(3/2 -1) ---
  Assuming T(1) ? c
• ? 2c(3k - 2k) ? 2c3lg(n) 2cnlg(3)

5
Divide Conquer Recurrence

• Assume T(n) aT(n/b) ?(n)
• T(n) ?(n) a lt b
• T(n) ?(nlog(n)) a b
• T(n) ?(nlogb(a)) a gt b

6
Polynomial Algebra

• Let Fx denote the set of polynomials in the
  variable x whose coefficients are in the field F.
• Fx becomes an algebra where , are defined by
  polynomial addition and multiplication.

7
Polynomial Algebra mod a Polynomial

• A(x) ? B(x) (mod f(x)) ? f(x)(A(x) - B(x))
• This equivalence relation partitions polynomials
  in Fx into equivalence classes where the class
  A(x) consists of the set A(x) k(x)f(x),
  where k(x) and f(x) are in Fx.
• Choose a representative for A(x) with degree lt
  deg(f(x)). Can choose rem(A(x),f(x)).
• Arithmetic
• A(x) B(x) A(x) B(x)
• A(x) B(x) A(x)B(x)
• The set of equivalence classes with arithmetic
  defined like this is denoted by Fx/(f(x))

8
Modular Inverses

• Definition B(x) is the inverse of A(x) mod
  f(x), if A(x)B(x) ? 1 (mod f(x))
• The equation A(x)B(x) ? 1 (mod f(x)) has a
  solution iff gcd(A(x),f(x)) 1.
• In particular, if f(x) is irreducible, then
  Fx/(f(x)) is a field.
• By the Extended Euclidean Algorithm, there exist
  u(x) and v(x) such that A(x)u(x) B(x)v(x)
  gcd(A(x),f(x)). When gcd(A(x),f(x)) 1, we get
  A(x)v(x) f(x)v(x) 1. Taking this equation
  mod f(x), we see that A(x)v(x) ? 1 (mod f(x))

9
Polynomial Version of the Chinese Remainder
Theorem

• Theorem Let f(x) and g(x) be polynomials in
  Fx (coefficients in a field). Assume that
  gcd(f(x),g(x)) 1. For any A1(x) and A2(x)
  there exist a polynomial A(x) with A(x) ? A1(x)
  (mod f(x)) and A(x) ? A2(x) (mod g(x)).
• Theorem Fx/(f(x)g(x)) ? Fx/(f(x)) ?
  Fx/(g(x)). I.E. There is a 1-1 mapping from
  Fx/(f(x)g(x)) onto Fx/(f(x)) ? Fx/(g(x))
  that preserves arithmetic.
• A(x) ? (A(x) mod f(x), A(x) mod g(x))

10
Constructive Chinese Remainder Theorem

• Theorem If gcd(f(x),g(x)) 1, then there exist
  Ef(x) and Eg(x) (orthogonal idempotents)
• Ef(x) ? 1 (mod f(x))
• Ef(x) ? 0 (mod g(x))
• Eg(x) ? 0 (mod f(x))
• Eg(x) ? 1 (mod g(x))
• It follows that A1(x) Ef(x) A2(x) Eg(x) ? A1(x)
  (mod f(x)) and ? A2(x) (mod g(x)).
• Proof.
• Since gcd(f(x),g(x)) 1, by the Extended
  Euclidean Algorithm, there exist u(x) and v(x)
  with f(x)u(x) g(x)v(x) 1. Set Ef(x)
  g(x)v(x) and Eg(x) f(x)u(x)

11
Interpolation

• A polynomial of degree n is uniquely determined
  by its value at (n1) distinct points.
• Theorem Let A(x) and B(x) be polynomials of
  degree m. If A(?i) B(?i) for i 0,,m, then
  A(x) B(x).
• Proof.
• Recall that a polynomial of degree m has m roots.
• A(x) Q(x)(x- ?) A(?), if A(?) 0, A(x)
  Q(x)(x- ?), and deg(Q) m-1
• Consider the polynomial C(x) A(x) - B(x).
  Since C(?i) A(?i) - B(?i) 0, for m1 points,
  C(x) 0, and A(x) must equal B(x).

12
Lagrange Interpolation Formula

• Find a polynomial of degree m given its value at
  (m1) distinct points. Assume A(?i) yi
• Observe that

13
Polynomial Evaluation, Interpolation and the CRT

• Since A(x) Q(x)(x- ?) A(?), A(x) ? A(?) (mod
  (x- ?))
• If ? ? ?, then gcd((x- ?),(x- ?)) 1.
  Therefore, we can apply the CRT to find a
  quadratic polynomial A(x) such that A(?) y and
  A(?) z.
• A(x) A(?) E?(x) A(?) E?(x), where
• E?(x) (x - ?)/(? - ?)
• E?(x) (x - ?)/(? - ?)
• Observe that
• E?(?) 1 and E?(?) 0, so that E?(x) ? 1 (mod
  (x- ?)) and E?(x) ? 0 (mod (x- ?)). The
  equivalent results hold for E?(x)

14
Multifactor CRT

• The CRT can be generalized to the case when we
  have n pairwise relatively prime polynomials. If
  f1(x),,fn(x) are pairwise relatively prime, i.e.
  gcd(fi(x),fj(x)) 1 for i ? j, then given
  A1(x),,An(x) there exists a polynomial A(x) such
  that A ? Ai(x) (mod fi(x)).
• Moreover, there exist a system of orthogonal
  idempotents E1(x),,En(x), such that Ei(x) ? 1
  (mod fi(x)) and Ei(x) ? 0 (mod fj(x)) for i ? j.
• A(x) A1(x)E1(x) An(x)En(x)

15
Lagrange Interpolation and the CRT

• Assume that ?0, ?1,, ?n are distinct and let
  fi(x) (x- ?i). Then gcd(fi(x),fj(x)) 1 for i
  ? j.
• Let
• Then Ei(x) ? 1 (mod fi(x)) and Ei(x) ? 0 (mod
  fj(x)) for i ? j, and Lagranges interpolation
  formula is
• A(x) A (?0)E0(x) A(?n )En(x)

16
Matrix Version of Polynomial Evaluation

• Let A(x) a3x3 a2x2 a1x a0
• Evaluation at the points ?, ?, ?, ? is obtained
  from the following matrix-vector product

17
Vandermonde Matrix
V(?0,, ?n) is non-singular when ?0,, ?n are
distinct.
18
Matrix Interpretation of Interpolation

• Let A(x) anxn a1x a0 be a polynomial of
  degree n. The problem of determining the (n1)
  coefficients an,,a1,a0 from the (n1) values
  A(?0),,A(?n) is equivalent to solving the linear
  system

19
Matrix Interpretation of Interpolation

• The previous system has a solution when the ?0,,
  ?n are distinct. The solution can be obtained
  using Lagrange interpolation. In fact, the
  inverse of V(?0,, ?n) is obtained from the
  idempotents

20
Polynomial Multiplication using Interpolation

• Compute C(x) A(x)B(x), where degree(A(x)) m,
  and degree(B(x)) n. Degree(C(x)) mn, and
  C(x) is uniquely determined by its value at mn1
  distinct points.
• Evaluation Compute A(?i) and B(?i) for distinct
  ?i, i0,,mn.
• Pointwise Product Compute C(?i) A(?i)B(?i)
  for i0,,mn.
• Interpolation Compute the coefficients of C(x)
  cnxmn c1x c0 from the points C(?i)
  A(?i)B(?i) for i0,,mn.

21
Interpolation and Karatsubas Algorithm

• Let A(x) A1x A0, B(x) B1x B, C(x)
  A(x)B(x) C2x2 C1x C0
• Then A(10k) A, B(10k) B, and C C(10k)
  A(10k)B(10k) AB
• Use interpolation based algorithm
• Evaluate A(?), A(?), A(?) and B(?), B(?), B(?)
  for ? 0, ? 1, and ? ?.
• Compute C(?) A(?)B(?), C(?) A(?) B(?), C(?)
  A(?)B(?)
• Interpolate the coefficients C2, C1, and C0
• Compute C C2102k C110k C0

22
Matrix Equation for Karatsubas Algorithm

• Modified Vandermonde Matrix
• Interpolation

23
Integer Multiplication Splitting the Inputs into
3 Parts

• Instead of breaking up the inputs into 2 equal
  parts as is done for Karatsubas algorithm, we
  can split the inputs into three equal parts.
• This algorithm is based on an interpolation based
  polynomial product of two quadratic polynomials.
• Let A(x) A2x2 A1x A0, B(x) B2x2 B1x
  B, C(x) A(x)B(x) C4x4 C3x3 C2x2 C1x
  C0
• Thus there are 5 products. The divide and
  conquer part still takes time O(n). Therefore
  the total computing time T(n) 5T(n/3) O(n)
  ?(nlog3(5)), log3(5) ? 1.46

24
Asymptotically Fast Integer Multiplication

• We can obtain a sequence of asymptotically faster
  multiplication algorithms by splitting the inputs
  into more and more pieces.
• If we split A and B into k equal parts, then the
  corresponding multiplication algorithm is
  obtained from an interpolation based polynomial
  multiplication algorithm of two degree (k-1)
  polynomials.
• Since the product polynomial is of degree 2(k-1),
  we need to evaluate at 2k-1 points. Thus there
  are (2k-1) products. The divide and conquer part
  still takes time O(n). Therefore the total
  computing time T(n) (2k-1)T(n/k) O(n)
  ?(nlogk(2k-1)).

25
Asymptotically Fast Integer Multiplication

• Using the previous construction we can find an
  algorithm to multiply two n digit integers in
  time ?(n1 ?) for any positive ?.
• logk(2k-1) logk(k(2-1/k)) 1 logk(2-1/k)
• logk(2-1/k) ? logk(2) ln(2)/ln(k) ? 0.
• Can we do better?
• The answer is yes. There is a faster algorithm,
  with computing time ?(nlog(n)loglog(n)), based
  on the fast Fourier transform (FFT). This
  algorithm is also based on interpolation and the
  polynomial version of the CRT.