Lyric alignment in popular songs

1
Lyric alignment in popular songs

• Luong Minh Thang

2
Outline

• Project description techniques
• Some important knowledge
• Base-line system

3
Outline

• Project description techniques
• Some important knowledge
• Base-line system

4
Project description

• Given
• textual transcription of lyrics
• acoustic musical signal
• of a song
• Purpose find the timestamps for the beginning
  and ending points for each line of the song.

5
Techniques to be investigated

• A repetition based technique for detecting
  self-similarities in both the audio and the text
• Dynamic programming (also known as dynamic time
  warping) will be employed to align the repetition
  analysis of both media together to produce an
  alignment

6
Outline

• Project description techniques
• Some important knowledge
• Base-line system

7
Chroma vectors
0.3
0.5
0.1
0.7
8
Dynamic time warping algorithm

• Actually, dynamic programming

9
Word similarity using phoneme dictionary

• Using CMU phoneme dictionary
• Each word is decomposed into phoneme sequence
• ALIGNMENT AH0 L AY1 N M AH0 N T
• ALGORITHMS AE1 L G ER0 IH2 DH AH0 M Z
• Similarity of 2 words is the similarity of the 2
  phoneme sequences

10
Outline

• Project description techniques
• Some important knowledge
• Base-line system

11
Base-line system overview
Musical signal input
Text input
Music notation sequence
Word sequence
Word self-similarity matrix
Symbol Self-similarity matrix
12
Base-line system overview
Musical signal input
Text input
Music notation sequence
Word sequence
Word self-similarity matrix
Symbol Self-similarity matrix
13
Base-line system overview
Musical signal input
Text input
Music notation sequence
Word sequence
Word self-similarity matrix
Symbol Self-similarity matrix
14
Base-line system overview
Musical signal input
Text input
Music notation sequence
Word sequence
Symbol Self-similarity matrix
Word self-similarity matrix
15
Base-line system overview
Musical signal input
Text input
Music notation sequence
Word sequence
Symbol Self-similarity matrix
Word self-similarity matrix
16

• Thank you !