Semantic Web Application: Music Retrieval - PowerPoint PPT Presentation

Loading...

PPT – Semantic Web Application: Music Retrieval PowerPoint presentation | free to download - id: 26d7c-M2Q5M



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Semantic Web Application: Music Retrieval

Description:

'An extension of the current Web in which information is given well-defined ... MusicalManifestation (Record, Track, Playlist, etc.), MusicalItem (Stream, ... – PowerPoint PPT presentation

Number of Views:87
Avg rating:3.0/5.0
Slides: 49
Provided by: nor302
Learn more at: http://vw.indiana.edu
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Semantic Web Application: Music Retrieval


1
Semantic Web Application Music Retrieval
  • Ying Ding
  • SLIS, IU

2
What is the Semantic Web?
  • An extension of the current Web in which
    information is given well-defined meaning, better
    enabling computers and people to work in
    cooperation.
  • Sir Tim Berners-Lee et al., Scientific American,
    2001 tinyurl.com/i59p

3
Semantic Web -- Web 3.0
  • How to realize that
  • machine-understandable semantics of information,
    and
  • millions of small specialized reasoning services
    that provide support in automated task
    achievement based on the accessible information

4
The current (syntactic / structural) Web
5
Was the Web meant to be more?
Hyperlinks typed hyperlinks Document - data
6
Ontology
  • The semantic Web is essentially based on
    ontologies
  • ontologies are formal and consensual
    specifications of conceptualizations
  • providing a shared and common understanding of a
    domain that can be communicated across people and
    application systems

7
Metadata and Semantics
8
Semantic Web - Language tower
9
What is Semantic Web for?
  • Integrating - trying to solve the problem of data
    and service integration
  • Searching - Providing better communication
    between human and computers by adding
    machine-processable semantics to data.
  • Form keyword search ? data search ? query answer

10
What is current Semantic Web effort?
  • Lifting document web to data web
  • Weaving the data web through semantic links
    (types hyperlinks)

11
Bubbles in April 2008
2B RDF triples Around 3M RDF links
12
  • http//www.elec.qmul.ac.uk/easaier/

Enabling Access to Sound Archives through
Integration, Enrichment and Retrieval
13
The EASAIER Project
  • EASAIER - Enabling Access to Sound Archives
    through Integration, Enrichment and Retrieval
  • EU funded project, 30month duration (started May
    2006)?
  • Partners

14
EASAIER - Goals
  • Overcome problems for many digital sound archives
    concerning online access
  • sound materials and related media often separate
  • searching audio content limited
  • EASAIER Framework
  • Integration of Sound Archives
  • Low level audio feature extraction
    (speech/music)?
  • Intelligent User Interface
  • Enhanced Access Tools
  • looping, marking of audio
  • sound source separation
  • time and pitch scale modification
  • Semantic Search
  • Evaluation

15
Semantics in EASAIER
  • Description of metadata using an ontology
  • High-level metadata
  • e.g. title, author of an audio asset
  • sources are databases, files in e.g. DC, MARC
  • Low-level metadata
  • e.g. speech event occurs at timestamp xyz
  • feature extractor tools
  • Semantic Search
  • Search across variety of metadata
  • Search across multiple archives
  • Similarity Search
  • Related content acquisition from the Web

16
The EASAIER System
17
Music Ontology
  • Overview
  • Merging existing related ontologies
  • Developed by QMUL
  • Cover the major requirements
  • Widely-adopted
  • Four core MO components
  • FRBR
  • FOAF
  • Event
  • Timeline

http//musicontology.com/
18
The Music Ontology Timeline Ontology
  • Expressing temporal information, e.g.
  • This performance happened the 9th of March, 1984
  • This beat is occurring around sample 32480
  • The second verse is just before the second chorus

19
The Music Ontology Event Ontology
  • Event An arbitrary classification of a
    space/time region
  • This performance involved Glenn Gould playing the
    piano
  • This signal was recorded using a XXX microphone
    located at that particular place
  • This beat is occurring around sample 32480

20
The Music Ontology FRBR FOAF
  • FRBR Functional Requirements for Bibliographic
    Records
  • Work e.g. Franz Schubert's Trout Quintet
  • Manifestation e.g. the "Nevermind" album
  • Item e.g. my "Nevermind" copy
  • FOAF Friend of a Friend
  • Person
  • Group
  • Organization

21
The Music Ontology Music Production Concepts
  • On top of FRBR
  • MusicalWork, MusicalManifestation (Record, Track,
    Playlist, etc.), MusicalItem (Stream, AudioFile,
    Vinyl, etc.)?
  • On top of FOAF
  • MusicArtist, MusicGroup, Arranger, Engineer,
    Performer, Composer, etc. all these are defined
    classes every person involved in a performance
    is a a performer...
  • On top of the Event Ontology
  • Composition, Arrangement, Performance, Recording
  • Others
  • Signal, Score, Genre, Instrument, ReleaseStatus,
    Lyrics, Libretto, etc.

22
The Music Ontology Music Production Workflow
23
Metadata in RDF
  • Low-level metadata is output in RDF using Music
    Ontology
  • Audio Feature extractor
  • Speech recognition service
  • Emotion detection service
  • High-level metadata import
  • DB Schema Mapping
  • e.g. D2R, Virtuoso RDF Views
  • Standardized Metadata import
  • DC, MARC, METS, ...
  • Linked Data ?
  • DBPedia, Geonames, ...

24
Use Case Archive Publication - HOTBED
Publishing
Extending
Hotbed Database
Music Ontology
Instruments Taxonomy
Querying
Query Interface
the Semantic Archivist
Sound Accesstools
FeaturesExtraction,Visualization,...
Hotbed RDF
25
1) editing the ontology
  • using WSMT editor to extend the ontology

Graphical Edit
Music Ontology
Text Edit
26
2) performing tests on the new extension
  • What are the instruments in my taxonomy ?
  • Did i forget any kind of pipe ?

27
3)mapping Scottish Instruments to a general
Instruments taxonomy
28
4) relating and publishing Hotbed
  • Relate tables from hotbed to concepts from the MO
  • Publish on the semantic web via the D2R tool

Mapping
Music Ontology
Hotbed Database
RDF Publicationvia D2R tool
  • The server offers a SPARQL end-point for external
    apps

29
Mapping Metadata to the Music Ontologies
music a moSignal dctitle "File 2"
dcauthor "Oliver Iredale Searle"
music-performance a moPerformance
morecorded_as music mocomposer
OliverIredaleSearle moinstrument moflute
moperformer KatiePunter mobpm 50
mometer "4/4" mokey BFlatMajor. KatiePunt
er a foafPerson . ss1 a afPersonPlaying afpe
rson KatiePunter eventtime tlonTimeLine
tl1234 tlbeginsAt "PT0S"
tlduration "PT16S" .
Title File 2 Author Oliver Iredale
Searle Perfomers Katie Punter Source Type
Audio Source File 2 Instrument
Flute Instrument occurrence timings 0"-16" Time
Signature 4/4 Beats per minute 50 Tonality Bb
major
Searle Testbed
30
Mapping Metadata to the Music Ontologies
ALL web service output
eveResult audio_material"c/hotbed/performance/1004.wav"
position_sec"10" duration_sec"5"
confidence"89" /
T10S a afText aftext "power" afconfidence
"89" eventtime a timetimeInterval tlon
Timeline /1234 tlbeginsAtDuration "PT10S" tldurati
onXSD "PT5S" .
31
Mapping Metadata to the Music Ontologies
Vamp Output

description"Detected Beats" unit"N/A"
0.0928" duration"0" label"224.69 bpm"/

eventtime a timeInstant tlonTimeLine
tl898 tlat "PT0.0928S"
mobpm "224.69"
32
RDF Storage and Retrieval Component
  • Built on top of OpenRDF Sesame 2.0
  • Query interfaces
  • Web Service (Servlet)?
  • HTTP SPARQL Endpoint
  • Web Service provides predefined SPARQL query
    templates
  • Themes
  • Music, Speech, Timeline, Related media,
    Similarity
  • Dynamic FILTER constructs
  • Results in SPARQL Query ? Results XML Format
  • Interface for RDF metadata import using the
    Archiver application

33
Enhanced Client
34
Web client
35
Related media
36
Related media on the web (1)?
37
Related media on the web (2)?
38
Demo
  • http//www.elec.qmul.ac.uk/easaier/index-3.html
  • http//easaier.deri.at/demo/

39
Demo
  • Time and Pitch Scale Modification (demo)
  • Sound source separation (demixing/remixing, Noice
    reduction, etc.) (demo)
  • Video time stretching (to slow down or speed up
    images while retaining optimal sound) (demo)

40
Scenario 1 Artist Search
  • Aggregation of music artist information from
    multiple web sources
  • Ontology based search
  • MusicBrainz data mapped to the MusicOntology
  • MusicBrainz Web Service
  • allows to retrieve artist URI by literal based
    search
  • MusicBrainz RDF Dump
  • retrieve RDF
  • use SPARQL to perform queries (e.g. resolve
    relationships)
  • Web2.0 Mashups
  • Retrieve data (videos, images) from external
    sources
  • utilize RSS Feeds, APIs etc. from Youtube,
    LyricWiki, Google
  • more accurate results using references from
    MusicBrainz RDF data

41
Scenario 1 Artist Search
WS Interface
Beatles


process data...
RDF Dump
42
Scenario 1 Artist Search
43
Scenario 1 Artist Search
44
Scenario 2 Instrument Reasoning
  • Reasoning over HOTBED instrument scheme
  • Ontologize data from HOTBED (Scottish Music
    Archive)
  • Usage of D2R to lift data from legacy DBs to RDF
  • Ontologies
  • MusicOntology
  • Instrument Ontology (domain related taxonomy)
  • Subsumption reasoning
  • Retrieve instrument tree
  • Search for persons that play an instrument
  • Subclass relations resolve persons playing more
    specific instruments
  • Example Wind-Instrument

45
Scenario 2 Instrument Reasoning
  • Example
  • Search for people playing instrument of type
    Woodwind

46
Demo 3 Rules
  • Infer new knowledge with rules
  • Domain Rule
  • Sophisticated Query
  • Albums based on certain Band/Artist/Instrument
  • UseCase The Velvet Underground discography
  • Available information
  • Membership durations
  • Album release dates
  • Founders of the band ?
  • exist _artist, ,
  • forall ?x, , onDuration, ?time
  • ?
  • Albums corresponding members

47
Demo 3 Rules
Basic Information
Band Founder
Band Duration (Members Albums)
Album Tracks
48
Thanks
  • Contact
  • Ying Ding
  • LI029
  • (812) 855 5388
  • dingying_at_indiana.edu
About PowerShow.com