11 Applications of Machine Learning to Music Research: Empirical Investigations into the Phenomenon of Musical Expression

1
11 Applications of Machine Learning to Music
Research Empirical Investigations into the
Phenomenon of Musical Expression

• 99419-811
• ? ? ?

2
Introduction

• Expressive music performance
• Why music?
• A set of difficult learning task
• weak(imprecise, incomplete) domain knowledge
• the notion of musical knowledge

3
Expressive music performance

• Shaping a piece of music
• not exactly as given in the written score
• continuously varying certain musical parameters
• dynamics(variations of loudness)
• rubato, expressive timing(variations of local
  tempo)
• Input - melodies (sequences of notes)
• loudness(dynamics dimension), tempo
• Given new pieces, how loud? How fast?

4
(No Transcript)
5
The nature and importance of background knowledge

• The task is to learn to draw correct,
  sensible curves above new melodies.
• One symbol alone does not uniquely determine the
  numeric value.
• It is not at all clear what the relevant context
  is.
• Humans possess additional knowledge about the
  meaning of the symbols.
• Expression is not arbitrary but highly correlated
  with the structure of music.

6
Approach I Learning at the note level

• Learning proceeds at the level of notes.
• The goal is to learn rules that determine the
  precise degrees of loudness and tempo to be
  applied to each note in a piece.
• Distinguish two classes of notes rise and fall
• crescendo, decrescendo
• accelerando, ritardando

7
The Qualitative domain theory

• Knowledge about relevant musical structure is
  needed.
• Two major components
• model of structural hearing
• set of programs that perform a structural
  analysis of a given melody and explicitly
  annotate the melody with various musical
  structures that are perceived by human listeners.
• qualitative dependency network
• intuitions concerning possible relations between
  structural aspects of the music and appropriate
  expressive performance decisions

8
IBL-SMART(1/3)

• Two major component
• symbolic learning component
• learns to distinguish between the symbolic target
  concepts(e.g. crescendo and decrescendo)
• utilize domain knowledge in the form of a
  quantitative model
• instance-based component
• stores the instances with their precise numeric
  attributes
• predict the target value for some new note by
  numeric interpolation over known instances

9
IBL-SMART(2/3)

• Each rule learned by the symbolic component
  describes a subset of instances
• These are assumed to represent a subtype of the
  target concept(e.g. some particular type of
  crescendo situations)
• All the instances covered by a rule are given to
  the instance-based learner to be stored together
  in a separate instance space.

10
IBL-SMART(3/3)

• Predicting the target value for some new note in
  a new piece involves matching the note against
  the symbolic rules.
• Using only those numeric instance
  spaces(interpolation tables) for prediction whose
  associated rules are satisfied by note.

11
Experiment

• J S Bachs Notenbuchlein fur Anna Magdalena Bach
• Played on an electronic piano and recorded
  through a MIDI interface.
• Two part
• learning with the second half
• tested with the first half

12
Learning at the structure level

• The note level is not really appropriate from a
  musical point of view.
• Lacked a certain smoothness
• performers tend to comprehend music in terms of
  higher-level abstract forms like phrase
• Alternative approaches are needed.

13
Learning at the structure level

• Tries to learn expression rules directly at the
  level of musical structures.
• Transforms the training examples and the entire
  learning problem to a musically plausible
  abstraction level.
• Proceeds in two stages.

14
Learning at the structure level

• The system first performs a musical analysis of
  the given melody.
• Analysis routines identify various structures in
  the melody that might be heard as units or
  chunks by a listener or musician.
• In the second step, the abstract target concepts
  for the learner are identified.
• Tries to find prototypical shapes in the given
  expression curves that can be associated with
  these structures.
• Even_level, ascending,descending, asc_desc,
  desc_asc

15
Learning at the structure level

• The results ltmusical structure, expressive shapegt
  are passed on to IBL-SMART.

16
An experiment

• experiments with waltzes by Chopin
• The results look and sound musically convincing.

17
A machine learning analysis of real artistic
performances

• Real data - performances of a complete piece by
  internationally famous pianists.
• tested with Schumanns Traumerei
• by Claudio Arrau, Vladimir Ashkenazy, Alfred
  Brendel
• showed considerable agreement in the overall
• Different results
• Vladimir Horowitzs performance decisions cant
  be so easily related to by obvious structural
  features of the music.

18
Quantitative analysis

• A precise quantitative evaluation of the results
  is not possible.
• Simply counting the number of matching decisions
  is far too simplistic.
• Apply simple weighting scheme

19
Useful qualitative results for musicology

• While abstraction to the structure level
  generally provides better results for various
  types of classical music, for other styles like
  jazz the note level is more adequate.
• Ritardando(Note, X) - interval_prev(Note, I),
  at_least(I, maj6), dir_prev(Note, up).
• Increase the duration(by a certaion amount X) of
  all notes that terminate an upward melodic leap
  of at least a major sixth

20
Conclusion

• Music is in many ways softer
• many aspects ar not quantifiable
• difficult to perform precise experiments
• Machine learning can make useful qualitative
  contributions
• thorough analysis of the application domain