Soft Computing Techniques for High Dimensional Problems: An illustration with Bond rating Prediction

1
Soft Computing Techniques for High Dimensional
Problems An illustration with Bond rating
Prediction

• Sethuraman J
• Indian Institute of Management Calcutta

2
Agenda

• Introduction
• Problem definition
• Techniques
• Experimental Set up Results
• Conclusion

3
Introduction
4
High dimensional problems

• Proposition A problem with n input variables is
  said to be an n-dimensional problem

5
Credit Rating

• According to Standard Poors (SP), The bond
  or credit rating is an opinion of the general
  creditworthiness of an obligor with respect to a
  particular debt security or other financial
  obligation, based on relevant risk factors.

6
Motivation

• Market there exists number of bonds which are not
  rated
• Paper deals with rating long term debt
  instruments
• Credit Rating is Multi-class classification
  problem

7
Approach

• Exact methodology followed by rating agencies is
  unknown
• Credit rating depends upon both Financial and
  non-financial information like market condition
  etc
• We have considered only Financial data

8
Model used for rating

• Data driven models
• 1. MDA (Multi-discriminant analysis). Brian
  S. Everitt et al, Applied Multivariate Data
  Analysis
• 2. Regression GS Maddala, Econometrics .
• Disadvantages
• Assumes a common functional forms
•  
• We have used some NN , Fuzzy model hybrid
  techniques and compared it with Regression
  techniques.

9
Problem Definition
10
Problem Formulation

• 15 types of bond ratings
• The input consisted of 46 financial ratios H. K.
  Manuj, An Evaluation of the Statistical
  Properties of Financial Ratios
• Included also age of the company and advertising
  marketing expenses of the company. Sehgal,
  Ramasubramanian, Rajesh et al, Application of
  fuzzy table look-up method to credit rating

11
Variables
12
Variables
13
Techniques
14
Techniques Used

• 1. Kohonen network or self-organizing map, a
  neural network Kohonen, T, Self-Organization and
  Associative Memory based approach.
• 2. Fuzzy C-means (FCM), a fuzzy based clustering
  technique Bezdek, J.C ,Pal Fuzzy Models for
  Pattern Recognition
• 3. Fuzzy Kohonen, a neuro-fuzzy system Tsao, E.
  C.-K, Bezdek, J.C., Pal , Fuzzy Kohonen
  Clustering Networks .

15
Kohonen network

• The SOM - mapping from the input data space
  spanned by x1..xn onto a one- or two-dimensional
  array of nodes.
• The mapping is performed in a way that the
  topological relationships in the n-dimensional
  input space are maintained when mapped to the
  SOM.
• The local density of data is also reflected by
  the map areas of the input data space which are
  represented by more data are mapped to a larger
  area of the SOM.

16
Kohonen network

• Intialise the neuron weights to random values
• select an object from the training set
• find the node (Winning unit mc which is closest
  to the selected data (i.e. the distance between
  wij and the training data is a minimum)
• adjust the weight vectors of the closest node and
  the nodes around it in a way that the mi move
  towards the training data

17
FCM

• The clustering (or training) algorithm of the
  fuzzy c-means algorithm reads as follows
• 1. Initialize the membership values of the k
  objects to each of the i clusters for and (for
  example randomly) such that

18
FCM

• Calculate Cluster centers using these membership
  values
• Calculate the new membership values using these
  cluster centers

19
FCM
20
Labelling
21
Fuzzy kohonen clustering network (FKCN)

• A fuzzy kohonen clustering network (FKCN) Tsao,
  E. C.-K, Bezdek, J.C., Pal, N.R., Fuzzy Kohonen
  Clustering Networks is a neuro-fuzzy model,
  where self organizing kohonen map Kohonen, T,
  Self-Organization and Associative Memory is
  combined with the fuzzy c-means algorithm
• Bezdek, J.C, Pattern Recognition with Fuzzy
  Objective Function Algorithms

22
Fuzzy kohonen clustering network (FKCN)

• 1. The elements Wij the weights vector Wi are
  initialized using random numbers, m(t0)m0
• 2. Calculate for each input vector xk the
  membership µik to the individual neurons

23
Fuzzy kohonen clustering network (FKCN)

• Calculate the learning rate using these
  membership values
• 3.Adjust the weight vectors wi such that

24
Fuzzy kohonen clustering network (FKCN)

• 4. Let, m(t1) m(t)- ?m

25
Experimental Set up
26
Selection of Data Sets

• we have used CRISIL99 data set
• A)     Training Data Set It contains 170
  records with the output data
• B)      Testing Data Set It contains 34 records
  for testing

27
Implementation

• Rating scales have also been reduced.
• 15 dimension to 6 dimension
• Complexity of the problem
• Important Classification is Sufficient
•     AAA and AA - Highest Safety Bonds.
•     AA and AA- - High Safety Bonds
•     A , A and A- Adequate Safety Bonds
•     BBB, BBB and BBB- Moderate Safety Bonds.
•     BB and BB -Inadequate Safety Bonds.
• B, C and D - High Risk Bonds (Junk Bonds)

28
Procedure

• Pre-processing
• outlier range was discarded
• Each input variable was normalized to
  standard normal form
• Dimensionality Reduction
• PCA (Principal component analysis)
  -Components with a variance of 50 were only
  Considered.

29
Terms used

• Strict Accuracy (SA)
• Accurate prediction
• CLB (conservative lower band)
• Percentage of cases where the output of the
  system falls, at most one level below the actual
  rating

30
Results
31
Dimensionality Reduction Technique

• To reduce the dimension of the problem we have
  used Principal Component Analysis (PCA).
• This was applied on all the 46 Variables. The
  components were chosen based upon their Eigen
  Values (components with Eigen values greater than
  the average were chosen).
• These components almost covered more then 50 of
  the variance in the samples.

32
Dimensionality Reduction Technique
33
Reduced Variable Technique

• In this technique Linear regression was used to
  reduce the variable from 45 to 20.Linear
  regression techniques were used to reduce the
  variables.

34
Reduced Variables
35

Reduced Variables
36
Variable Reduction Technique
37
Conclusion

• unsupervised networks applied to bond rating
  prediction, which is a high dimensional problem.
• We find that Neuro-fuzzy systems outperform other
  types of networks.
• The future scope of work is to go for some sort
  of rule base inferencing models like TSK Takagi
  and M. Sugeno inside FNN networks.

38
Thank You