Software Project: Sparse Matrix Multiplication

1
Software ProjectSparse Matrix Multiplication
2
Sparse Matrices

• A sparse matrix is a matrix populated primarily
  with zeros.
• Manipulating huge sparse matrices with the
  standard algorithms may be impossible due to
  their large size.
• Sparse data is by its nature easily compressed,
  which can yield enormous savings in memory usage.

3
Exercise 2

• Goals
• Implement several data structures for compressed
  representation of sparse matrices.
• Analyze the advantages and the disadvantages of
  each implementation.
• Measure the running times and select the most
  efficient implementation.

4
Exercise 2.1

• Use the linked list representation.
• The naïve implementation
• Replace each row by a linked list.

5
Disadvantages

• Accessing the columns elements is inconvenient.
  We need to go over all the rows to search for a
  certain column.
• It is more convenient to view matrix
  multiplication as multiplication of a row by a
  column.

(,j)
(i,j)

(i,)
A
B
C
Row-wise
6
Doubly Linked List Representation
7
Data Type Definition

• typedef struct cell_t
• struct cell_t row_next / pointer to the next
  element in the row. /
• struct cell_t col_next / pointer to the next
  element in the column. /
• int rowind / index in row /
• int colind / index in column /
• int value / value of the elem
  /
• cell_t / matrix cell data type /
• typedef struct
• int n / size /
• cell_t rows / array of row lists /
• cell_t cols / array of col lists /
• sparse_matrix_lst / sparse matrix
  representation /

8
Question 2.1 sparse_mlpl_lst

• Implement the multiplication of sparse matrices
  represented as doubly linked lists.
• Create sparse matrices of different size and fill
  them with random integers.
• Perform multiplication of matrices of different
  sizes.
• Measure the performance using the clock()
  function.
• Prepare 2 graphs which plot the running times in
  CPU ticks as the function of matrix size and the
  number of its non zero elements.

9
User Interface

• Input
• Case 1 0 or negative
• Case 2 An integer number followed by a matrix
  values
• Output
• Case 1 The running times
• Case 2 A matrix, which is the square of the
  input one.

10
Disadvantages of Linked Lists

• Memory allocation is performed for each non zero
  element of the matrix.
• Not a cache friendly code, i.e. we can not
  optimize it for a better cache memory utilization.

11
Exercise 2.2

• Use a compressed array representation.

values
rowind
colptr
12
Compressed data structure
22
84
61
values
2
1
0
rowind
colptr
13
Data Type Definition

• typedef struct
• int n / size /
• int colptr / pointers to where
  columns begin in rowind and values /
• int rowind / row indexes /
• elem values
• sparse_matrix_arr

14
Question 2.2 sparse_mlpl_arr

• Implement the multiplication of sparse matrices
  represented by the compressed array data
  structure.
• Perform multiplication of matrices of different
  sizes.
• Measure the performance using the clock()
  function.
• Prepare 2 graphs which plot the running times in
  CPU ticks as the function of matrix size and the
  number of its non zero elements.
• Compare the performance of the two
  representations of sparse matrices.
• Discuss the advantages and the disadvantages of
  each.

15
Compressed Array Implementation

• Observations
• The described data type is compressed by
  column.
• Accessing row elements is inconvenient.

22
84
61
values
2
1
0
rowind
colptr
16
Multiplication by column
The inner loop can be viewed as
Remember the jki loop ordering from exercise 1.1
(,j)
(,k)
(k,j)
/ jki / for (j0 jltn j) for (k0 kltn
k) r bkj for (i0 iltn i)
cij aik r
A
B
C
17
Multiplication by column
The second loop can be viewed as
Remember the jki loop ordering from exercise 1.1
(,j)
(,k)
(k,j)
/ jki / for (j0 jltn j) for (k0 kltn
k) r bkj for (i0 iltn i)
cij aik r
A
B
C
The column j in C is obtained by the
multiplication of the matrix A by the column
vector of B.
18
Matrix by vector multiplication
b
A

• Sometimes a better way to look at it
• Ab is a linear combination of As columns!

19
Files and locations

• All of your files should be located under your
  home directory /soft-project/assign2/sparse_mlpl_
  lst
• The source files and the executable should match
  the exercise name (e.g. sparse_mlpl_lst.c)
• Strictly follow the provided prototypes and the
  file framework.

20
Exercise 2 Notes

• Avoid unnecessary memory allocations.
• Read the input directly into the sparse matrix.
• Free all the allocated memory.
• Consult the website for recent updates and
  announcements.
• Use your own judgment!!!

21
The Optional Exercise 3.1

• Computational Goal
• Given a square non-singular matrix A to find its
  first eigenvector and eigenvalue.
• Implement a matrix decomposition method based on
  the Power Method algorithm.
• Remarks
• The exercise is not for submission.
• It is provided for those who would like to get
  prepared for the project.
• Additional details about this exercise will be
  provided after the Passover vacation.

22
The Compilation Environment
23
The Programming Process
Editors - emacs, vim, Microsoft Studio etc.)
Write a program
Compiler - translates to machine code Linker
creates the executable file
Compile the program
Run the program
24
Process Overview
Libraries
File1.c File2.c
File1.o
Link
Exe
Edit
Compile
File2.o
Executable
Object File (Machine Language)
Source Code (High-Level Language)
25
Stage 1 Writing the Code
26
Stage 2 Compiling the code
27
Stage 2 Compiling the code from Emacs
28
Stage 2 Compiling the code from Emacs
29
Stage 3 Running the program
30
Good Luck!!!