Toward Networks with Cognitive Packets

1
Toward Networks with Cognitive Packets

• Erol Gelenbe Ricardo Lent
• Esin Seref Zhiguang Xu
• School of Electrical Engineering Computer
  Science
• University of Central Florida
• Orlando, FL 32816

2
Overview

• Introduction to CPN
• Cognitive Packet Networks
• Packet Routing Using Neural Networks
• Random Neural Networks Learning
• Random Neural Networks with Reinforcement
  Learning
• Simulation (LFRNN, RNN RL, Bang-Bang)
• Cognitive Adaptive Routing
• Analytical Model Predictions using G-Networks
• CPN Test-Beds
• Future Work

3
Problem Definition

• Packet networks have capabilities for flow
  control, error control and routing
• Cognitive Packet Networks (CPN)
• concentrate intelligent capabilities for routing
  in the packets rather than in the nodes and
  protocols
• For the time being, we assume that error and
  flow control in CPN are handled using current
  techniques

4
Introduction to CPN

• A packet switching network where packets route
  themselves
• Packets are assigned goals before entering the
  network
• Packets learn to achieve their goals
• Learning is performed by sharing information
  between packets
• Packets sharing same goals can be grouped into
  classes
• Packets do not rely on nodes for routing

5
OSI Layers

• TCP/IP is a layered protocol stack
• Application handles particular applications,
  Presentation handles compression and encryption
  of data, Session controls establishment,
  management, termination of sessions
• Transport provides flow of data
• Network handles the transmission of packets in
  the network
• Data-link is responsible for the interaction of
  the device driver in the operating system and the
  network card in the machine
• Physical defines electrical and mechanical
  specifications

CPN
6
Transport Layer (TCP)

• Connection-oriented client and server
  applications establish connection before data
  exchange
• Reliable sends checksum on data and header,
  takes care of the order of arriving packets,
  takes care of duplicate packets, sends
  acknowledgments, maintains timer
• Flow-control receiving side allows the sending
  side to send as much as it can buffer

7
Network Layer (IP)

• Unreliable best effort service, no guarantees to
  get to the destination
• Connectionless no state information about
  successive datagrams, each handled independently
• Routing done on a hop-by-hop basis according to
  the routing tables, required information is kept
  in the IP header (Routing table information comes
  from the routing protocols)

8
Current Routing Protocols

• Distance-vector Protocols
• Routing tables updated according to a vector of
  distances (hop counts)
• Regular and triggered updates
• Whole network topology is distributed
• No information about the network topology
• Drawbacks
• Regular updates consume bandwidth
• Takes too long to stabilize after failure
• Using only hop count omits other variables

9
Current Routing Protocols

• Link-state Protocols
• Each router tests the status of its link to its
  neighbors and sends this information to its other
  neighbors
• Converges faster than distance-vector protocols
• Each router can be assigned a cost based on
  throughput,reliability, round-trip time, etc
• Load balancing possible
• Multicasting reduces load on systems not
  participating
• Drawbacks
• No policy based routing
• Needs more memory and CPU power

10
CPN and CP

• CPN Packet switching networks in which
  
  intelligent
  capabilities for routing are concentrated in
  packets rather than in the nodes and protocols
• CP Packets in CPNs which have the route
  themselves

11
Structure of CPs

• Identifier Field The unique identifier of a
  packet
• Data Field Payload
• Cognitive Map S-D information, the packets view
  of the state of the network, etc
• Executable Code Code used to update the CM and
  the decision algorithm

12
Nodes in a CPN

• Storage area for CPs Input and Output Buffers
• Mailboxes are used to exchange data between CPs
• The Node executes the code for each CP in the
  input buffer

13
Update of CM by Node

• Packet arrives at the input buffer of a node
• The code of each CP in the input buffer is
  executed
• The CP retrieves relevant information from the
  mailbox
• The packets CM is updated
• Some information is moved from the CP to the MB
• The CP is moved to an output buffer

14
Cognitive Packet Classes
A Class is a set of packets having the same QoS
requirements, sets of internal states, control
rules, input-output signals
Class (S,D,L)
15
Random Neural Networks

• A spiked RNN with a mathematical structure
  similar to a queuing network (Gelenbe in Neural
  Computation 93, IEEE Trans. on NN 99 etc.)
• Probability that the neuron is excited
• where
• is the rate at which neuron i sends
  excitation spikes to neuron j when i is excited
• is the rate at which neuron i sends
  inhibition spikes to neuron j when i is excited

and
16
Learning Feed-Forward RNN

• Input to each neuron is represented as a pair of
  excitatory and inhibitory signals
• Each connection is represented as a pair of
  weights
• Output of the system is the q of the output layer
  neurons

17
LFF RNN Equations

• Normalized training set
• Network parameters are computed from the training
  set
• Parameters adjusted to minimize the error
  function
• Weight updates

18
LFF RNN Equations

• Where

if ui,vi if ui,v!i if u!i,vi otherwise
if ui,v!i if u!i,vi otherwise
19
RNN with Reinforcement Learning

• Fixed input values
• Each connection is represented as a pair of
  negative and positive weights
• Output of the system is the largest q of all the
  neurons

20
RNN RL Algorithm

• Decision threshold
• Compute
• If

21
RNN RL Algorithm

• Re-normalize all weights by calculating
• and

22
Simulation

• 100 nodes
• Link speeds normalized 1
• Arrivals are Poisson
• Buffer sizes are unlimited
• Packet loss simulated probabilistically,
  intentionally high in some areas
• Goal is to minimize a weighted combination of
  delay W and loss L

23
Bang-Bang to Minimize Average Delay
24
Bang-Bang to Minimize Avg Delay Loss
25
RNN Reinforcement Learning to Minimize Avg Delay
Loss
26
LFF RNN Control to Minimize Avg Delay Loss
27
Bang-Bang Shortest Path in the Network Without
Loss
28
Cognitive Adaptive Routing (CAR)

• There are three types of packets
• The packets sent into the network by the source
  with small rate ? which travel intelligently to
  their destination (SMART or Cognitive Packets)
• A SMART packet which reaches its Destination
  generates an ACK back to the source ACKs travel
  back along the same (reverse) route as the packet
• Reception of an ACK at the Source may trigger the
  transmission of a DUMB packet along the route
  which the older packet had taken
• ACKs may not be necessary for each packet

29
Cognitive Adaptive Routing (CAR)

• The ACK coming back from Destination D deposits
  Delay Information W(n,O,D) when it passes
  through Node n, entering it from Node O
• W(n,O,D) is used to Update the CAR Learning
  Algorithm (eg Reinforcement Learning) either by
  the Smart Packet, or by the Node itself

30
Analyzing CAR with G-Networks

• G-Networks are Queuing Networks with Three Types
  of Customers
• Normal Customers
• Negative Customers They Destroy Normal Customers
• Triggers They Move Normal Customers to Another
  Queue
• Multiple Class Models Allowed
• G-Networks Have Product Form They Can be Solved
  Analytically and then Numerically
• Work of Gelenbe (89, 91, 93, 98) and Many Others
• Key Papers Mainly in Journal of Applied
  Probability
• G-Networks Can Model Worst-Case CAR and
  Best-Case CAR

31
G-Network Equations

• Non-Linear Equations for the Queue Utilizations
• - queue S is Source, queue D is Destination
• - is a route from S to D with
  queue i and queue j as components and j is the
  predecessor of i
• - ACKs will trigger packets from queue 0, which
  constantly feeds queue S with small rate ?0ltS,Dgt

32
Analytical Solution

• Calculate the average No of packets in each
  queue
• Calculate the average transit delay for a packet

33
Cognitive Adaptive Routing Performance
All Packets Are Smart but Not Always
Successful
Fewer Packets Are Smart Dumb Packets Follow
Successful Smart Packets
34
Cognitive Adaptive Routing Performance
35
CAR Performance as a Function of Smart Packet
Injection Rate
36
Test-beds for CPN

• Goals
• Performance evaluation and capability
  enhancements
• Demonstration of applications
• Requirements
• Explicit separation and clear borderline between
  the different protocol layers
• Standardized interfaces both between the
  application layer and the CPN layer, and between
  the CPN layer and the data transport network
• Compatibility with a wide range of computing
  platforms
• Interchangeability of the data transport
  networking layer

37
A CPN-VN (Virtual Network)

• Standard Linux on Intel-PCs
• New CPN Code Instead of IP

38
CPN Test-Bed

• Different Selected Logical Connectivities

Different Selected Physical Connectivities
39
CPN-VP Test-Bed

• Current Test-Bed Connectivity

40
Special Purpose CPN-Router Test-Bed
41
Special-Purpose CPN-Router
42
Smart Dumb Packets

• CPN
• Header

Routing Data Code
Payload
Identifier
SMART or DUMB Packet Indicator
Cognitive Map and Executable Code
43
Packet Contents

• CPN
• Header

44
Mail Boxes in CPN-Routers

• The Mailbox (MB) is a LRU stack where information
  is stored delaysweights for different directions
  associated with certain ltS,Dgt, ...
• Packets (esp. ACKs) deposit information in the MB

45
CPN-Router Software
46
State Diagram of CPN-Router
PDU Arrived (either from higher layer or from
peer entity)
PDU Assembled
Complete Route Available
No Route Available
Incomplete Route Available
Update Route
Consult MB Update CM
Complete / Incomplete Route
No Route Found
PDU Ready
Proceed at Random
Send
47
Work Accomplished

• CPN packet network architecture in which
  routing is carried under Packet Control --
• Cognitive Adaptive Routing Smart or Cognitive
  Packets Control the Routing of the Dumb Packets
• Cognitive Packet Routing is Designed Using
  Random Neural Network based Reinforcement
  Learning, Feed-Forward Learning, Bang-Bang
  Control
• Analytical Modeling and Simulation Results are
  Presented
• Test-Bed architectures are being implemented

48
Work to be Done I

• Run and Measure Significant Applications
• (eg File Transfer, Video-Conferencing, Voice,
• Web Browsing )
• Evaluate Network Adaptation under Time-Varying
  Workloads
• Deploy a Large Test-Bed
• Examine Implications with respect to TCP
• Consider CAR as a Joint Routing Flow Control
  Scheme

49
Work to be Done II

• Can CAR be used to enhance IP ?
• Can CPN run in conjunction with IP ?
• Can Generalized CAR be a way of selecting paths
  in BGP ?