Machine Discoveries: A few Simple, Robust Local Expression Principles

1
Machine Discoveries A few Simple, Robust Local
Expression Principles

• Written by Gerhard Widmer
• presented by Siao Jer, ISE 575b, Spring 2006

2
Presentation Overview

• General Overview
• Introduction
• Training Data
• Target Classes
• Experimental Results
• Quantitative Validation
• Conclusion
• Future Research

3
Gerhard Widmer

• Head of the Department of Computational
  Perception at Johannes Kepler University Linz,
  Austria
• Head of Machine Learning, Data Mining, and
  Intelligent Music Processing Group at the
  Austrian Research Institute for Artificial
  Intelligence
• Numerous publication, awards, projects

4
General Overview

• Discovering rules of expressive music performance
• Inductive machine learning
• Experiments with large data sets
• Simple and general principles
• Robust with surprisingly high level of accuracy

5
Introduction

• What do performers do to make music come alive?
• Studies done through a few classical approaches
• Proposal of inductive machine learning
• No preconceptions and expectations
• Huge data sets allowed for more validity

6
Introduction

• Previous work
• Success in ability of machine learning (Widmer
  2000)
• Extremely complex
• Attempt to find a complete model
• Current goals
• Testing new learning algorithm based on partial
  models
• Learn rules of timing, dynamics, articulation
• Testing degrees of fit over various styles and
  performers

7
Training Data

• 13 complete Mozart piano sonatas
• Performed by Roland Batik
• On computer monitored grand piano
• MIDI format
• Includes hammer speed, impact times, pedal
  movements measured xformed
• Written score coded into computer format
• Timing, dynamics, articulation computed
• 106,000 total notes
• Melody restriction limits us to 41,000 notes

8
Target Classes

• Objective find note-level rules
• Limit predictions to categorical decisions
• Timing Dimension note N is considered lengthened
• If the note is lengthened relative to the
  instantaneous tempo over the previous note
• If lengthened relative to local tempo over the
  last 20 notes
• Analogous to this is a note shortened

9
Target Classes

• Dynamics louder if
• Louder than previous note
• And louder than average level of piece
• Analogous to this is softer
• Articulation
• Staccato if played duration ratio (PDR) is less
  than 0.8
• Legato if greater than 1.0
• Portato otherwise, but study only concerned with
  staccato and legato
• Pedaling not taken into account for articulation
• Notes do not necessarily have to fall into one of
  these classes

10
Learning Partial Rule-based Models

• No expectation to cover and describe all
  instances
• Describe parts and define in meaningful terms
• PLCG algorithm developed with these ideas in mind
• Goal to come up with rules that covered lots of
  cases with good accuracy

11
Learning Partial Rule-based Models

• General Steps
• Separation into subsets
• Learning partial rules within subsets
• Merge all rules
• Clustering of rules
• One generalization per cluster
• Optimize trade-offs (coverage vs. accuracy)
• Result 383 specialized rules narrowed to 17
  general rules

12
Experimental ResultsTiming Lengthening Notes

• "Lengthen the middle note in a cummulative
  3-note rhythm situation (ie, given 2 notes of
  equal duration followed by a longer note,
  lengthen the note that precedes the final, longer
  one).
• Most important one as it has highest prediction
  value
• Lengthen a note if it is followed by
  substantially longer note (ie the ratio between
  its duration and the duration of the next note is
  lt 13)
• Lengthen a note if it preceds an upward melodic
  leap of more than a perfect forth, if it is in a
  metrically weak position, and if it is preceded
  by (at most) stepwise motion
• 2 cases above have atleast 70 prediction rate

13
Experimental ResultsTiming Lengthening Notes

• Lengthen a note if it preceds an upward melodic
  leap of more than a perfect forth, if it is in a
  metrically weak position, and if it is preceded
  by (at most) stepwise motion
• More of a tendancy than a rule
• Interesting Note
• previously observed
• But not over such a large data set

14
Experimental ResultsTiming Shortening Notes

• Difficult, but understandable
• No strong rules, but a few tendencies
• Shorten a note in a sequence PN-N-NN if it is
  longer than its predecessor and longer than its
  successor.
• Shorten a note in fast pieces in 3/8 time if the
  duratio ratio between previous note and current
  note is larger than 21, the current note is at
  most a sixteenth, and is again followed by a
  longer note.
• Example of a specialized rule
• Correlation with Gabrielsson 1987

15
Experimental ResultsDynamics Stressing Notes

• Clear rules emerge, low coverage
• Interesting note relating stress to
• melodic contour
• Upward melodic movement
• Observation by previous research as well
• Stress a note by playing it louder if it is
  preceded by an upward melodic leap larger than a
  perfect fourth.

16
Experimental ResultsDynamics Stressing Notes

• Stress a note by playing it louder if it forms
  the apex of an up-down melodic contour and is
  preceded by an (upward) leap larger than a minor
  third.
• Stress a note by playing it louder if it at
  least twice as long as its predecessor, is
  reached by upward motion, and is in a quite
  strong metrical position.

17
Experimental ResultsDynamics Attenuating Notes

• Difficult to predict
• Attenuate a note by playing it softer if it is
  less than 1/5 the duration of its predecessor.
• Attenuate a note by playing it softer if it is
  preceded by a downward leap larger than a major
  third, is metrically weak, and is preceded by a
  note at least 1/3 of a beat long.
• Attenuate a note by playing it softer if it is
  preceded by a downward leap larger than a perfect
  fifth and is metrically weak.
• Observation linking metrically weak notes
  reached by downward leaps

18
Experimental ResultsArticulation Staccato

• Most easily predictable, 4 strong rules
• Play a note staccato if the note is marked with
  a staccato dot in the score.
• Play a note staccato if it is followed by a note
  of the same pitch (ie the interval between the
  note and its successor is a unison).
• Observations
• Combine for 90 accuracy 6,000 cases
• Previously observed in KTH Rules (Friberg 1995)
• Physical reasons and explanations

19
Experimental ResultsArticulation Staccato

• Insert a micropause after a note if it precedes
  an upward leap larger than a perfect fourth and
  is metrically weak.
• Insert a micropuase after a note of it is
  reached by downward motion and is followed by a
  note more than twice as long (ie the ratio
  between its duration and duration of the next
  note is lt 0.4).
• Observations
• Correlation to lengthening rules
• Supported by Cumulative Rhythm (Nramour 1977)
• Articulation Staccato ? 30 of expression observed

20
Experimental ResultsArticulation Legato

• Most difficult to predict
• A LOT fewer instances vs. staccato
• No markings on score
• Low prediction rate (53.7)
• Play a note legato if it is not marked staccato
  in the score, if it forms the apex of an up-down
  melodic contour, if it is quite short (lt1/3 of a
  beat), and is metrically quite strong.
• Observations
• Melodic peak ? legato?

21
Quantitative ValidationGenerality I

• Different Performer (Philippe Entremont)
• Same pieces
• No significant degradation in coverage and
  accuracy
• Exception of softer
• Higher coverage in
• lengthen
• louder
• staccato

22
Quantitative ValidationGenerality II

• Testing on Different Styles Artists
• 2 Chopin pieces
• 22 skilled pianist from Univ. of Music in Vienna
• Surprising Results
• softer and legato ? unpredictable
• louder ? high of positive examples, but high
  level of false predictions too
• lengthen, shorten, staccato ? extremely
  good
• Need more diversity of pieces

23
Conclusion

• Small Step
• Basic simple rules
• Robust model of local expression principles
• Observations from other researchers
• Autonomous discovery
• Large data sets
• Possible foundation

24
Further Research

• Further evaluation of rules
• different performers
• Different types of music
• Extension to other dimensions
• (e.g. Harmony)
• Going beyond note level
• (e.g. phrase structure)
• Comprehensive multi-level model