Linear Network Coding

1
Linear Network Coding

• Networking Seminar
• Presented by Zhoujia Mao

Li, S.-Y. R., Yeung, R. W. and Cai, N. Linear
Network Coding. IEEE, 2003, Trans. Information
Theory, Vol. 49, pp. 371-381.
2
Outline

• Introduction
• Basic notations
• Generic LCM
• Transmission scheme for acyclic network
• Construction of generic LCM for acyclic network
• Transmission scheme for cycled network
• Construction of generic LCM for cycled network
• Conclusion

3
I. Introduction

• Model
• Multicast (certain source nodes transmit to a set
  of receivers)
• Multi-hop fashion
• Regard a block of data as a vector over a certain
  base and allow a node to apply linear
  transformation to a vector before passing it on
• Wired network with noiseless channels (links),
  and every single channel has unit capacity
• Assume no processing delay

4

• Questions
• How fast each sink node can receive the complete
  information?
• Whether the linear coding is sufficient to
  achieve the optimum (max-flow from the source to
  each receiving node)

5

• Example1

S
S
b1
b2
b1
b2
U
U
T
T
b1
b2
b1
b2
W
W
b1
b2
b1
b2
b1b2
?
X
b1b2
b1b2
Y
Z
Y
Z
(a)
(b)
6

• Wired network
• No processing delay
• S multicast b1 and b2 to both Y and Z
• (a) can not fulfill at one time
• (b) can fulfill at one time using network coding

7

• From the example, the information rate from the
  source to a sink can potentially become higher
  when the permitted class of coding schemes is
  wider
• Question Does the information rate has an upper
  bound?

8

• The law of commodity/physical flow The total
  volume of the outflow from a nonsource node can
  not exceed the total volume of the inflow
• The law of information flow The content of any
  information flowing out of a set of nonsource
  nodes can be derived from the accumulated
  information that has flown into the set of nodes

9

• Replication of data can be regarded as a special
  case of coding
• Coding is certain transform of data
• Transform of information does not increase the
  information content
• Laws of physical and information flow bound the
  information rate

10

• Main work of the paper
• Prove constructively that by linear coding alone,
  the rate at which a message reaches each node can
  achieve the individual max-flow bound
• Provide realization of transmission scheme and
  practically construct linear coding approaches
  for both acyclic and cycled network

11
II. Basic notations

• Convention for the following discussion
  T,U,W,X,Y,Z stand for the nonsource nodes S
  stands for the source node XY stands for any
  channel from X to Y
• Flow a collection of busy channels from S to T
  (sink)
• Busy channels should satisfy
• Do not form directed cycles
• For nodes except S and T, incoming busy channels
  outgoing busy channels
• outgoing channels of S incoming channels of T
• Volume of the flow outgoing busy channels from S

Law of flows
12

• Cut a collection C of nodes which includes S but
  not T. A channel XY is said to be in cut C only
  if X C and Y C
• Value of a cut channels in a cut
• Max-Flow Min-Cut Theorem For every nonsource
  node T, the minimum value of all cuts between S
  and T is equal to maxflow(T)
• Notation d stands for the maximum of maxflow(T)
  over the set of nonsource nodes stands for
  the d-dimensional vector space

13

• Definition1 A linear-code multicast (LCM) v on a
  communication network (G,S) is an assignment of a
  vector space v(X) to every node X and v(XY) to
  every channel XY s.t.
• v(S)
• v(XY) v(X)
• For any collection of nonsource nodes, linear
  span ltv(T) T gt ltv(XY) X , Y gt
• The vector assigned to any outgoing channel from
  T, i.e., v(TY) should be a linear combination of
  vectors assigned to the incoming channels of T,
  i.e., v(XT) (law of information flow at a node)

If T, then v(T)ltv(XT)gt. This shows that an
LCM is completely determined by the vectors it
assigned to the channels
14

• Example2
• dmax maxflow(T) T 2
• We can let information vector be 2-demensional
  row vector (b1, b2)

S
maxflow(U)1
maxflow(T)1
U
T
maxflow(W)2
W
X
Y
Z
maxflow(X)1
maxflow(Z)2
maxflow(Y)2
15

• Since we know from the definition1(3) that LCM
  is completely determined by the vectors it
  assigned to the channels. Only vectors on the
  channel need to be assigned (check
  definition1(1-4))
• v(ST)v(TW)v(TY)(1,0)
• v(SU)v(UW)v(UZ)(0,1)
• v(WX)v(XY)v(XZ)(1,1)
• The data sent on a channel is the product of the
  information vector with the assigned channel
  vector, e.g., data sent on WX is b1b2

Attention the channel vector is always
1-dimensional from the row standpoint. Actually 1
means one channel and 2 rows due to is
2-dimensional
16

• Proposition1 For every LCM v on a network, for
  all nodes T, dim(v(T))ltmaxflow(T)
• Proof Choose an arbitrary T and any cut C
  between S and T, since T C, so T ZZ C and
  v(T) ltv(Z)Z Cgtltv(YZ)Y C and Z Cgt by
  definition1(3). Hence, dim(v(T))ltdim(ltv(YZ)Y C
  and Z Cgt) which is at most the value of the cut.
  Since the cut should also be arbitrary, then
  dim(v(T)) is upper-bounded by the minimum value
  cuts between S and T, by Max-Flow Min-Cut
  Theorem, the proposition is proved

17

• Explain for proposition1 The dimension of a
  channel vector stands for one channel, if the
  channel vectors in a cut are linearly
  independent, then the dimension of their linear
  span reaches the maximum which is the number of
  channels (value of the cut). The dimension of
  node vector stands for the amount of information
  it receives, since node vector is determined by
  channel vector, in other words, amount of
  information received is determined by value of
  cuts

18
III. Generic LCM

• Intuition From proposition1 and my explanation,
  we can see to achieve the upper bound, the
  channel vectors need to be independent,
  especially in the minimum cut
• Definition2 An LCM is said to be generic if the
  following condition holds for any collection of
  channels X1Y1, X2Y2, , XmYm for 1ltmltd v(Xk)
  ltXjYjj kgt for 1ltkltm if and only if the
  vectors v(X1Y1), v(X2Y2), , v(XmYm) are linearly
  independent

19

• Explain for definition2 If v(X1Y1), v(X2Y2), ,
  v(XmYm) are linearly independent, then v(XkYk)
  ltv(XjYj)j kgt, since v(XkYk) v(Xk), so
  v(Xk) ltv(XjYj)j kgt is always true. A
  generic LCM requires the converse is also true.
  In this sense, generic LCM assigns vectors as
  linearly independent as possible to the channels
• Confusion How if there exist no situation like
  v(Xk) ltv(XjYj)j kgt? the proof of theorem 1
  2 will clear up
• My intuitionistic explanation Since v(S) ,
  is d-dimensional, so d independent channel
  vectors will be assigned to d channels outgoing
  from S. This ensures opportunities for v(Xk)
  ltv(XjYj)j kgt

20

• Example3
• Define LCM as
• v(ST)v(TW)v(TY)(1,0)
• v(SU)v(UW)v(UZ)(0,1)
• v(WX)v(XY)v(XZ)(1,0)

S
U
T
W
X
Y
Z
21

• Not generic. Consider ST, WX, where
  v(S)v(W)lt(1,0), (0,1) gt. Then v(S) ltv(WX)gt
  and v(W) ltu(ST)gt, but v(ST) and v(WX) are not
  linearly independent
• Lemma1 Let v be a generic LCM. Any collection of
  channels XY1, XY2, , XYm from a node X with m lt
  dim(v(X)) must be assigned linearly independent
  vectors by v.
• Explanation Suppose X has ngtdim(v(X)) outgoing
  channels, write the channel vector as a dn
  matrix, the rank of the matrix is dim(v(X)), so
  we can at most simply the matrix to make any
  dim(v(X)) column be non-zero. Since mltdim(v(X)),
  the lemma explained

Welcome for different opinions
22
recall

• LCM
• Linear LCM
• Proposition1

23

• Theorem1 If v is a generic LCM on a
  communication network, then for all nodes T,
  dim(v(T))maxflow(T)
• Proof
• Step1 We only care about the amount of data
  received per time unit, so consider any node T
  not equal to S. For convenience, let f be the
  value of maxflow(T). By Proposition1,
  dim(v(T))ltf, so we only need to show dim(v(T))gtf

24

• Step2 We plan to assume dim(v(T))ltf and prove
  dim(v(T))gtf by contradiction. However, it is
  difficult to compare dim(v(T)) with other
  variables directly
• Idea1 by definition1(3), v(T)ltv(X,T)gt, so
  dim(v(T))dim(ltv(X,T)gt)
• Idea2 if C is any cut between S and T, define
  dim(C)dim(ltv(X,Y) X C and Y Cgt), then
  combining idea1, we can transform the
  contradiction on dim(v(T))ltf to dim(C)ltf
• Idea3 define a set of certain cuts AC
  dim(C)ltf and C is cut between S and T. From
  idea2, if we can find a member in A with its
  dimensiongtf, then we get the contradiction.
  Again, let V be the set of all nodes in network,
  then V\T constructs a cut between S and T, by
  definition of dimension of cut,
  dim(V\T)dim(ltX,Tgt X V\T)dim(v(T))ltf, so A
  is non-empty and it is possible to find a member
  in A

25

• Step3 Now we attend to find a member in A with
  dimensiongtf. From the definition of dimension of
  cut, we can see a cuts dimension is determined
  by the dimension of linear span of its channels,
  i.e., the number of independent channels it has.
  Combining with definition2, for generic LCM, if
  we find certain conditions as in definition2, we
  can find independent channels
• A minimal U in A means for any Z U\S ,
  U\Z A
• The set of boundary nodes B in U means Z B if
  and only if Z U and there is a channel (Z,Y)
  s.t. Y U
• Let K be the set of channels in cut U, and it
  is easy to understand these channels starts from
  nodes in B. We claim for all nodes W B, v(W)
  ltv(X,Y) (X,Y) Kgt, and we can see this condition
  is more strict than the one in definition2

26

• Step4 We need to prove the claim in last step,
  still by contradiction. The set of channels in
  cut U\W but not in K is given by (X,W)X
  U\W. Since v is an LCM, ltv(X,W)X U\Wgt
  ltv(X,W)X V\Wgt v(W), V is set of all nodes.
  If we assume v(W) ltv(X,Y)(X,Y) Kgt for all W,
  then ltv(X,Y) X U\W,Y U\Wgt ltv(W) any
  W in Bgt ltv(X,Y)(X,Y) Kgt. This implies
  dim(v(X,Y) X U\W,Y U\W)ltdim(v(X,Y)(X,
  Y) K), so dim(U\W)ltdim(U)ltf, a contrdiction,
  because U is minimal member, so U\W A.
  Therefore, for all W B, v(W) ltv(X,Y)(X,Y) Kgt

27

• Step5 For any (W,Y) K, since ltv(X,Z)(X,Z)
  K\(W,Y)gt ltv(X,Y)(X,Y) Kgt, so from step4,
  v(W) lt v(X,Y)(X,Y) Kgt implies v(W) lt
  v(X,Z)(X,Z) K\(W,Y)gt. Then, by definition2,
  K channels are independent, dim(U)K.
  Besides, dim(U)ltd, so dim(U)min(K,d). By
  Max-Flow Min-Cut Theorem, Kgtf, also dgtf, then
  dim(U)gtf. Contradiction (proof completed)

28

• Question Theorem1 ensures that for each node
  with generic LCM, the upper bound is reached. How
  to make all nonsource nodes reach their upper
  bound, i.e., receive message at the maximum rate?
• Lemma2 Let X, Y and Z be nodes such that,
  maxflow(X)i, maxflow(Y)j and maxflow(Z)k,
  where iltj and igtk. By removing any edge UX in
  the graph, maxflow(X) and maxflow(Y) are reduced
  by at most 1, and maxflow(Z) remains unchanged

29

• Proof
• Step1 We first consider X and Y. By removing an
  edge UX, the value of a cut C between the source
  S and node X (respectively, node Y) is reduced by
  1 if edge UX is in C, otherwise, the value of is
  unchanged. By the Max-Flow Min-Cut Theorem, if C
  is the minimum cut of X (respectively, Y), then
  the maxflow is reduced by 1, so maxflow(X) and
  maxflow(Y) are reduced by at most 1 when edge UX
  is removed from the graph

30

• Step2 Now consider the value of a cut C between
  the source S and node Z. If C contains node X,
  then edge UX is not in C attention UX is from U
  to X, edges in C should have the direction from S
  side to Z side, and, therefore, the value of C
  remains unchanged upon the removal of edge UX. If
  does not contain node X, then is a cut between
  the source S and node X. By the Max-Flow Min-Cut
  Theorem, the value of C is at least i. Then upon
  the removal of edge UX, the value of C is
  lower-bounded by i-1gtk. Hence, by the Max-Flow
  Min-Cut Theorem, maxflow(Z) remains to be k upon
  the removal of edge UX

31

• Example4
• Consider a communication network for which
  maxflow(T)4,3 or 1 for nodes in the network. The
  source S is to broadcast 12 symbols a1, , a12
  taken from a sufficiently large base field F.
  Define the set SiTmaxflow(T)i, for i4,3,1,
  so there are 3 kinds of nodes. Use second for
  time unit. How to make all nodes receive data at
  their maximum rate?

32

• Let v1 be generic LCM with d4. Let A1(a1 a2 a3
  a4), A2(a5 a6 a7 a8), A3(a9 a10 a11 a12). Then
  after 3s, since using v1, nodes in S4 has rate of
  4 symbols/s, so they receive all symbols nodes
  in S3 receive 9 symbols, since rank of their
  decode matrix is 3 dim(v(T)maxflow(T) under
  generic LCM, each second, only 3 symbols of Ai
  can be recovered, 1symbole lost, and we can
  simplify the matrix to see which symbol is lost
  nodes in S1 receives totally 3 symbols for the
  same reason

33

• Let v2 be generic LCM with d3 used in the 4th
  second, r be independent vector in F4 with other
  3 base vectors in F4 for nodes T in S3, s.t. ltr,
  v1(T)gtF4. Remove incoming edges of nodes in S4
  then by lemma 2, maxflow of nodes in S4 becomes
  3, other nodes do not change, such operation is
  to make v2 valid. Define biAir, B(b1 b2 b3).
  Then nodes in S3 can recover B which contains
  information of lost symbols in first 3 seconds
  and then recover lost symbols with r. Nodes in S1
  receives 1 symbol of B

34

• Let v1 be generic LCM with d1 in the following
  seconds. Define s1, s2 in F3 s.t. lts1, s2,
  v2(T)gtF3. Remove edges to make maxflow of nodes
  in S3, S4 become 1. Then in 5, 6seconds, nodes in
  S1 can recovers lost 2 symbols of B. Then two
  column of matrix A1 A2 A3 are recovered
• Define t1, t2 in F4 s.t. ltt1,t2,r,v1(T)gtF4.
  Similarly, nodes in S1 can recover the remaining
  symbols until 12nd second

35
IV. Transmission scheme for acyclic network

• Question How to physically realize the
  transmission scheme with an LCM
• Definition3 A communication network (G, S), is
  said to be acyclic if the directed graph G does
  not contain a directed cycle
• Lemma3 An LCM on an acyclic network (G, S), is
  an assignment of a vector space v(X) to every
  node and a vector v(XY) to every channel (XY)
  such that
• v(S)
• v(XY) v(X)
• For any collection of nonsource nodes, linear
  span ltv(T) T gt ltv(XY) X , Y gt
• Law of information flow at a single node implies
  Law of information flow at a set of nodes

36

• Proof
• Step1 Let an edge XY be an internal edge of
  when X and Y , an incoming edge of when X
  and Y . We need to show for every internal
  edge UZ of , v(UZ)ltv(XY) XY is an incoming
  edge of gt. We tend to prove by induction, so
  we should divide nodes into two groups. Because
  the network is acyclic, there exists a node T in
  without any edge TX, where X . If such T
  doesnt exist, there will be a cycle. Thus, there
  is no incoming edge to \T one group from
  T another group, so 1) every incoming edge of
  \T is an incoming edge of .

37

• Step2 By induction on , we may assume that
  for every internal edge UZ of \T, 2) v(UZ)
  is generated by v(XY) XY is an incoming edge of
  \T. By 1), v(UZ) is generated by v(XY) XY
  is an incoming edge of . Therefore, we only
  need to consider internal edge of that goes
  to node T.
• Step3 Given an internal edge WT of , we need
  to show that v(WT) is generated by v(XY) XY is
  an incoming edge of . From the law of
  information at a single node, we know v(WT) is
  generated by v(QW) QW is an edge. It suffices
  to show that QW is generated by v(XY) XY is an
  incoming edge of .

38

• Step4 If the edge QW is an incoming edge of
  \T, then it is incoming to by 1). Otherwise,
  v(QW) is generated by v(XY) XY is an incoming
  edge of \T according to 2) and, therefore,
  is also generated by v(XY) XY is an incoming
  edge of because of 1).

39

• Lemma4 The nodes on an acyclic communication
  network can be sequentially indexed such that
  every channel is from a smaller indexed node to a
  larger indexed node
• Lemma5 Assume that nodes in a communication
  network are sequentially indexed as X0S, X1, ,
  Xn such that every channel is from a smaller
  indexed node to a larger indexed node. Then,
  every LCM (attention not generic LCM) on the
  network can be constructed by the following
  procedure
• for (j 0 j lt n j)
• arrange all outgoing channels XjY from Xj
  in an arbitrary order, here Y stands for any
  different end nodes of outgoing channels from Xj
• take one outgoing channel from Xj at a
  time
• let the channel taken be XjY
• assign v(XjY) to be a vector in the
  space v(Xj)
• v(Xj1) linear span by vectors v(XXj1)
  on all incoming channels XXj1 to Xj1, here X
  stands for any different start nodes of incoming
  channels to Xj1

40
V. Construction of generic LCM for acyclic network

• A generic LCM exists on every acyclic
  communication network, provided that the base
  field of is an infinite field or a large
  enough finite field
• Procedure of construction Let the nodes in the
  acyclic network be sequentially indexed as X0S,
  X1, , Xn such that every channel is from a
  smaller indexed node to a larger indexed node.
  The following procedure constructs an LCM by
  assigning a vector v(XY) to each channel XY, one
  channel at a time

41

• for all channels XY
• v(XY) the zero vector //
  initialization
• for (j 0 jltn j)
• arrange all outgoing channels XjY from Xj
  in an arbitrary order
• take one outgoing channel from Xj at a
  time
• let the channel taken be XjY
• choose a vector w in the space v(Xj)
  such that w ltv(UZ) UZ gt for any collection
  of at most d-1 channels with v(Xj) ltv(UZ)UZ
  gt
• v(XjY) w
• v(Xj1) the linear span by vectors
  v(XXj1) on all incoming channels XXj1to Xj1

Greedy algorithm!
42
VIII. Conclusion

• Contribution
• Proof linear network coding can achieve
  upper-bound of transmission rate under single
  session multicast environment
• Construct such coding scheme
• Acyclic
• Cycled
• Memory
• Memoryless

43

• Unsolved
• Simpler coding construction scheme
• Proof of existence of an optimal time-invariant
  code for cyclic network
• Multi-session
• Synchronization is a problem when network coding
  is implemented in computer or satellite networks
  real time application