ECE%20101%20An%20Introduction%20to%20Information%20Technology%20Information%20Coding

1
ECE 101 An Introduction to Information
TechnologyInformation Coding
2
Information Path
Source of Information
Information Display
Digital Sensor
Information Receiver and Processor
Information Processor Transmitter
Transmission Medium
Information Storage
3
Information Coding

• Fixed Length (same bits per word)
• Error detection
• Error correction
• Standard codes
• Bar and credit card codes
• Variable length (frequently used words small
  number of bits) - Data compression
• Huffman code
• Facsimile (Fax) code
• Encryption

4
Error Detection and Correction

• Codes can be written that detect errors
• add redundant bits in the code words that do not
  add information (other than the possible presence
  of an error)
• where probability of errors is small, they detect
  single errors only
• could just repeat (doubles the data size) plus
  the error would be detected but not corrected

5
Parity Bit

• Add a parity bit to each word
• even-parity adds a (redundant) bit to each word
  to form a word that contains an even number of
  1s similarly for odd-parity with an odd
  number of 1s.
• more efficient than bit repetition
• identifies the existence of an error but does not
  correct it
• Error Correction
• addition of redundancy-check code word
• size of data (plus parity bit) increased by one
  word

6
Redundancy Check
Word
Symbol

• Information to be sent 00 01 10 11
• With even parity, the above is converted to 00
  0 01 1 10 1 11 0
• First bits are 0 0 1 1 Odd parity bit 1
• Second bits are 0 1 0 1 Odd parity bit 1
• Odd parity bits 1 1 Even parity bit 0
• Transmitted 000 011 101 110 110

Parity bits
7
Redundancy Check Error

• Transmitted 000 011 101 110 110
• Received 000 111 101 110 110
• Even parity tells us that the second symbol has
  an error
• Comparing Odd parity with the first bit in each
  symbol shows us that the first bit in the second
  symbol should be a 0
• Comparing Odd parity with the second bit in each
  symbol shows us that everything is OK

8
Fixed Length Codes

• Same number of bits in each word
• Use of n bits can create 2n different words
• ASCII Code
• American Standard Code for Information
  Interchange
• computer memories structured with 8 bit (one
  byte) words
• ASCII - conventional code for representing
  alphanumeric symbols as bytes

9
Digital Watermark

• Vertically shifting particular lines by 1/600 of
  an inch and then impose the original on it and
  observe the shading of the output.
• Can shift any number of lines or use symbols to
  quickly create a large number of variations.

10
Variable Length Codes

• Reduce the number of bits by assigning short code
  words to common symbols and longer code words to
  less common symbols
• Huffman Coding Procedure
• uses a code tree, consisting of nodes connected
  by branches that ultimately terminate in leaves
• node at top is root
• branches from a node are 1 or 0
• so only 2 branches from any node

11
Entropy

• Minimum average number of bits to encode a domain
  of probabilities
• H - ?i1n PXi log2 PXi bits/symbol,
  where n is number of possible outcomes, or
• H 3.32 ?i1n PXi log10 1/PXi bits/symbol

12
Huffman Coding Procedure

• Determine the probabilities of occurrence of all
  possible values
• List symbols in order of decreasing probability
• Start at bottom of the list and assign a zero to
  the least probable and 1 to next least probable.
• Combine the two least probable symbols into one
  composite symbol (sum of probabilities)
• Revise list of symbols using the composite symbol
  in order of decreasing probability
• Repeat steps until only two symbols remain and
  assign a 0 to less probable entry and 1 to the
  other(NOTE in your textbook its the other way
  around, ie. 1 to the least probable entry,
  however it does not matter which protocol is
  used)

13
Huffman Coding
The Huffman coding procedure finds the optimum,
uniquely decodable, variable length code
associated with a set of events, given their
probabilities of occurrence.

• Creates a Binary Code Tree
• Nodes connected by branches with leaves
• Top node root
• Two branches from each node

14
Huffman Coding

• A 0
• B 10
• C 110
• D 111
• Given the adjacent Huffman code tree, decode the
  following sequence 11010001110

15
Huffman Code Construction

• First list all events in descending order of
  probability.
• Pair the two events with lowest probabilities and
  add their probabilities.

.3 Event A
.3 Event B
.13 Event C
.12 Event D
.1 Event E
.05 Event F
0.15
.3 Event A
.3 Event B
.13 Event C
.12 Event D
.1 Event E
.05 Event F
16
Huffman Code Construction

• Repeat for the last pair and add 0s to the left
  branches and 1s to the right branches.

1
0
0.4
0.6
0
1
0.15
0.25
0
1
1
0
0
1
.3 Event A
.3 Event B
.13 Event C
.12 Event D
.1 Event E
.05 Event F
00
01
100
101
110
111
17
Data Compression

• Two approaches to Data Compression
• Lossless compression
• retains all information present in the original
  data
• tenfold data compression is typical (WinZip or
  hard drive compression)
• Uniquely decodable
• Huffman codes are often used to compress large
  data files into smaller files without loosing any
  information.

18
Data Compression

• Lossy compression
• further reduction in data by permitting some loss
  of information
• uses close approximations to the data rather than
  actual data
• can be 100 fold compression
• can specify the perceptual quality of the result
• little perceptible distortion

19
Huffman Fax Code

• To further reduce the number of bits to transmit,
  use the Huffman code
• this involves estimating the relative frequency
  of occurrence of different runs of black and
  white
• i.e. common words, short code words
• code the make-up words (64m) and terminating code
  words (r) separately

20
Fax Code Errors

• Fax must transmit alternating black and white
  codes, otherwise and error is detected
• If errors are detected the page must be sent
  again
• Fax machines do not currently use correction
  codes
• trade-off for faster transmission times (no
  redundant bits in the code words) and expense of
  resending a fax when errors occur

21
Encryption

• Encryption is a way to randomly scramble data so
  that only the intended recipient can use the
  information
• easiest way is use an exclusive-or (XOR) gate
  with the data and random binary sequence as
  inputs output is then sent as an encrypted
  message
• random binary sequence can be produced using a
  pseudo-random number generator (PRNG)
• retrieved by applying the encrypted data with the
  same random binary sequence to and XOR gate

22
Encryption using PRNG

• Standard encryption requires a key for a number
  that determines the random binary sequence used
  to both encode and decode the data
• Choose the PRNG that produces the random 8 byte
  (byte sized patterns with the generator equation)
  
• Xn A ? Xn-1 Bmod(256) where
• A is an arbitrary multiplier of Xn-1
• B prevents the sequence from degenerating into a
  set of zeroes
• to get started we need an arbitrary X0, or seed

23
Transmitting the Seed

• Assume that A, B, N256 for the PRNG are known by
  everyone
• Need only to transmit the seed X0, the key
• T (transmitting person) selects x privately and
  transmits to R (receiving person) X axmod(N)
  (note that this N need not be same as the value
  256 given above)
• R selects y privately and transmits to T Y
  aymod(N)
• T computes Yxmod(N) and R computes Xymod(N)
  which equal one another so this becomes the seed
  value which they now both understand but the
  outside world does not.

24
Sample Exercise (1)

• Four groups, 2G and 2Y (with calculators)
• Each group privately selects two (of the 128)
  ASCII symbols page 164-5 of the text
• Arbitrarily well use Xn177Xn-159mod256
• To find X1 the key now is to determine X0. To
  find that, use N11 and a9
• The two G groups select privately an odd number,
  g, from 3 to 9 (arbitrary choice)
• The two Y groups select privately an even number,
  y, from 2 to 10 (arbitrary choice)

25
Sample Exercise (2)

• G groups compute G 9gmod 11 and sends the
  result to corresponding group Y
• Y groups compute Y 9ymod 11 and sends the
  result to corresponding group G
• G groups compute X0 Ygmod 11
• Y groups compute X0 Gymod 11
• Each group computes X1177X059mod256 and
  X2177X159mod256 for the PRNG random binary
  sequence convert to binary

26
Sample Exercise (3)

• Each group does XOR of this PRN (X1 X2) of 16
  bits with the two symbols it selected.
• Each group sends its coded message to the other
  group
• Notice that this message is secure to people who
  do not know the PRN selected.
• Each group uses XOR with the received message and
  PRN to decode the message sent.