Automated Transcription of Polyphonic Piano Music A Brief Literature Review - PowerPoint PPT Presentation

About This Presentation
Title:

Automated Transcription of Polyphonic Piano Music A Brief Literature Review

Description:

Automated Transcription of Polyphonic Piano Music A Brief Literature Review Catherine Lai MUMT-611 MIR February 17, 2005 1 Outline of Presentation Introduction ... – PowerPoint PPT presentation

Number of Views:20
Avg rating:3.0/5.0
Slides: 15
Provided by: Tria265
Category:

less

Transcript and Presenter's Notes

Title: Automated Transcription of Polyphonic Piano Music A Brief Literature Review


1
Automated Transcription of Polyphonic Piano
MusicA Brief Literature Review
  • Catherine Lai
  • MUMT-611 MIR
  • February 17, 2005

1
2
Outline of Presentation
  • Introduction
  • transcription of polyphonic music
  • targeted on specific instruments
  • Current state-of-the-art various approaches
  • Recent published piano transcription systems
  • Dixon, 2000
  • Raphael, 2002
  • Monti and Sandler, 2002
  • Marolt, 2004
  • Discussion and Conclusion
  • Links to examples of transcription of piano music
    recordings
  • Bibliography

2
3
Introduction
  • Transcription of polyphonic music
  • acoustical waveform --gt parametric representation
  • extract pitches, starting times, durations
  • First attempt by Moorer, 1975
  • note range limitation
  • two voices constraint
  • Martin, 1996
  • piano transcription system up to four voices
  • chorale style of J.S. Bach (long durations with
    block chords)
  • Future systems tackled limitations
  • targeted system on specific instruments
  • Focus of this literature review
  • automated transcription of polyphonic piano music

3
4
Current State-of-the-Art Various Approaches
  • Automated transcription of polyphonic piano music
  • input audio files containing polyphonic piano
    music
  • output MIDI representing pitch, timing, volume
  • Simon Dixon, 2000. On the Computer Recognition
    of Solo Piano Music
  • standard SP, adaptive peak-picking, pattern
    matching
  • Christopher Raphael, 2002. Automatic
    Transcription of Piano Music
  • HMM
  • Monti and Sandler, 2002. Automatic Polyphonic
    Piano Note Extraction Using Fussy Logic in a
    Blackboard System
  • blackboard algorithm
  • Matija Marolt, 2004 A connectionist approach to
    automatic transcription of polyphonic music
  • neural network models

4
5
Published Piano Transcription SystemSimon
Dixon, 2000. On the computer recognition of solo
piano music standardized SP approach
  • 1st processing stage
  • low-filtering --gt down-sampling signal (12kHz)
  • Time-frequency representation
  • STFT --gt power spectrum --gt spectral peak
    extraction (local maxima gt threshold, adaptive
    peak-picking algorithm)
  • frequency tracks --gt grouping partials --gt
    musical notes
  • Evaluation 13 Mozart piano sonata performed by a
    concert pianist
  • Bösendorfer SE290 computer-monitored piano --gt
    MIDI
  • Results Nno. correctly i.d. notes FPno. note
    reported not played FNno. notes played not
    reported by system incorrectly I.d. note FP
    and FN
  • score N/(FP FN N)
  • recognition accuracy of 70-80
  • Future development
  • accuracy of dynamic and offset times

5
6
Published Piano Transcription SystemChristopher
Raphael, 2002. Automatic transcription of piano
musicHMM
  • HMM- trained likelihood model
  • statistical pattern recognition and machine
    learning for structures
  • Process
  • segment signal to frames extract features
    (vector) from frames assign label for content
    description
  • Precise vector features
  • total energy (play or silent)
  • local burstiness (attack, steady behavior)
  • pitch configuration
  • Label
  • sound pitches collection and re-articulation
    (attack, sustain, rest)
  • Model setup
  • hidden process (label process) observable
    process (feature vector)
  • generate reasonable hypotheses for each frame and
    construct search graph of the hypotheses

6
7
Published Piano Transcription SystemChristopher
Raphael, 2002. Automatic transcription of piano
musicHMM
  • Experiment
  • Mozart piano sonata
  • limitations on range (c two octave below middle c
    to the f to two and a half octave above middle c)
  • number of voices 4 or less
  • Evaluation
  • borrowed from speech evaluation of Word Error
    Recognition Rate
  • Error Rate 100 (Insertions Deletions
    Substitutions) / (Total Words in Truth Sentence)
  • preliminary results have a Note Error Rate of
    39
  • 184 substitutions, 241 deletions, 108 insertions
    out of 1360 notes
  • Future improvement
  • simple additions may yield better results
  • likelihood of chord sequence
  • informative note onsets acoustic cues

7
8
Published Piano Transcription System Monti and
Sandler 2002. Automatic polyphonic piano note
extraction using fussy logic in a blackboard
system Blackboard algorithm
  • Implementation
  • Polyphonic Note Recognition using a Fuzzy
    Inference System (FIS) as part of the Knowledge
    Sources (KSs) in a Blackboard system
  • Blackboard model arrangement
  • hierarchy of data abstraction level
  • KSs dictate advancement and is
  • activated by Scheduler
  • FIS
  • take spectral peaks not selected
  • create new Note Candidates
  • evaluate Candidate by features
  • fundamental of note
  • harmonic rate
  • difference bt max peak in
  • spectrum and Candidates fundamental energy

Blackboard system (Monti and Sandler, 2002)
8
9
Published Piano Transcription System Monti and
Sandler 2002. Automatic polyphonic piano note
extraction using fussy logic in a blackboard
system Blackboard algorithm
  • Evaluation
  • 14 piano pieces by various composer including
    Beethoven, Mozart, Debussy, Ravel, and Scarlatti
  • Results Ncorrectly i.d. notes FPnote not
    played FNnotes not reported by sys
  • score N/(FP FN N) Dixons
  • detection success rate 45 correct
  • 75 correctly detected note / total transcribed
    notes

9
10
Published Piano Transcription System Matija
Marolt, 2004 A connectionist approach to
automatic transcription of polyphonic piano
music. Neural networks approach
  • New model based on networks of adaptive
    oscillators was proposed and implemented in SONIC
    to partial tracking and note recognition

5.adaptive oscillators try to synchronize to
signals in output freq channels of the auditory
model by adjusting its phase and frequency 6.
When synchronized to the output freq indicate the
freq is periodic and a partial with feq sim to
filter present
1. acoustical waveform --gttime-feq space with an
auditory model 2. auditory model output set of
freq channel 3. periodicity in frequency
channels is related to pitch perception 4. use
adaptive oscillators to calculate periodicity in
frequency channels
76 neural networks others tested multilayer
perception, radial basis function, etc.
Marolt, 2004
10
11
Published Piano Transcription System Matija
Marolt, 2004 A connectionist approach to
automatic transcription of polyphonic piano
music. Neural networks approach
  • Evaluation
  • tested on synthesized and real recordings of
    various genre
  • Results
  • synthesized recoding around 90 of all notes
  • real recording results not as good (not
    available)
  • most common error (gt 50) octaves and rapidly
    played notes (e.g.arpeggios, trills)
  • greatest challenge very expressive playing
  • Chopins Nocturnes
  • quiet and almost inaudible left hand
  • Further Development
  • detecting repeated notes

Marolt, 2004
11
12
Discussion and Conclusion
  • Various approaches proposed
  • standard S.P. techniques HMM blackboard
    algorithm neural networks
  • Common mistakes
  • octave, rapid passages, and quiet notes
  • Difficulties
  • lack standard set of test examples
  • evaluation function
  • various constraints and formula -- gt comparison
    difficult

Piano transcription system Performance results
Dixon 70-80 correct
SONIC 80-95 correct
Raphael 39 wrong
Monti and Sandler 74 correct
12
13
Links to examples of transcription of piano music
recordings
  • http//lgm.fri.uni-lj.si/matic/SONIC.html
    (Marolt)
  • http//www.ai.univie.ac.at/simon/ (Dixon)

13
14
Bibliography
  • Dixon, S. 2000. On the Computer Recognition of
    Solo Piano Music. Australasian Computer Music
    Conference. 31-7.
  • Marolt, M. 2004. A connectionist approach to
    automatic transcription of polyphonic piano
    music. IEEE Transactions on Multimedia 6, no. 3
    (June) 439-49.
  • Martin, K. 1996. A blackboard system for
    automatic transcription of simple polyphonic
    music. MIT Media Laboratory Perceptual Computing
    Section Technical Report No. 385.
  • Montipi, G, and M. Sandler. 2002. Automatic
    Polyphonic Piano Note Extraction Using Fuzzy
    Logic in a Blackboard System. Proceedings of the
    International Conference on Digital Audio
    Effects. 39-44.
  • Moorer, J. 1975. On the segmentation and analysis
    of continuous musical sound by digital computer.
    Ph.D. thesis, Stanford University, CCRMA.
  • Raphael, C. 2002. Automatic Transcription of
    Piano Music. Proceedings of the International
    Conference on Music Information Retrieval.

14
Write a Comment
User Comments (0)
About PowerShow.com