The Problem of Concept Drift: Definitions and Related Work - PowerPoint PPT Presentation

About This Presentation
Title:

The Problem of Concept Drift: Definitions and Related Work

Description:

The Problem of Concept Drift: Definitions and Related Work Alexev Tsymbalo paper. (April 29, 2004) Abstract A. Tsymbal, The problem of concept drift: definitions ... – PowerPoint PPT presentation

Number of Views:180
Avg rating:3.0/5.0
Slides: 24
Provided by: JasonDP
Learn more at: https://s2.smu.edu
Category:

less

Transcript and Presenter's Notes

Title: The Problem of Concept Drift: Definitions and Related Work


1
The Problem of Concept Drift Definitions and
Related Work
  • Alexev Tsymbalo paper. (April 29, 2004)

2
Abstract
  • A. Tsymbal, The problem of concept drift
    definitions and related work, Available here.
  • Real World Problem
  • Concepts are often not stable but change with
    time.
  • Weather Prediction
  • Customers Preference
  • The underlying data distribution may change with
    time.

3
Definitions and Peculiarities
  • Concept Drift
  • Changes in the hidden context that can induce
    more or less radical changes in the target
    concept.
  • The cause of the change is hidden and not known a
    priori.
  • Such as an effect of a car accident on a yearly
    budget.
  • Often Reoccur
  • Weather patterns such as El Nino and La Nina.
  • Hidden Context
  • A dependency not given explicitly in the form of
    predictive features.

4
  • An Ideal Concept Drift Handling System
  • Quickly adapts to concept drift.
  • Is robust to noise and distinguishes it from
    concept drift.
  • Recognizes and reacts to reoccurring contexts.
  • Such as seasonal differences.

5
Types of Concept Drift
  • There are two kinds of concept drift
  • Sudden (abrupt, instantaneous)
  • Gradual
  • Moderate
  • Slow
  • Hidden changes can change the target concept, but
    may also cause a change of the underlying data
    distribution.
  • Such as a week of record warm temperatures.

6
  • Virtual Concept Drift
  • The necessity in the change of current model due
    to the change of data distribution.
  • Sampling Shift
  • Real Concept Drift
  • Concept Shift
  • Virtual concept drift often occurs with real
    concept drift.

7
Systems for Handling Concept Drift
  • Three main approaches
  • Instance Selection
  • Instance Weighting
  • Ensemble Learning (learning with multiple concept
    descriptions)

8
Systems for Handling Concept Drift (Instance
Selection)
  • The goal is to select instances relevant to the
    current concept.
  • Usually generalized via a window that moves over
    recently arrived instances and uses the learnt
    concepts for prediction only in the immediate
    future.
  • The window size can be fixed or heuristically
    determined (Adaptive).

9
Systems for Handling Concept Drift (Instance
Selection)
  • Case-based editing strategies in case-based
    reasoning that delete noise, irrelevant cases,
    and redundant cases are also considered instance
    selection.

10
Systems for Handling Concept Drift (Instance
Weighting)
  • Uses the ability of some learning algorithms such
    to process weighted instances
  • Support Vector Machines
  • Weighting by
  • Age
  • Relevance to the current concept.
  • Instance weighting handles concept drift worse
    than analogous instance selection techniques.
  • Likely due to data overfitting.

11
Systems for Handling Concept Drift (Ensemble
Learning)
  • Maintains a set of
  • concept descriptions
  • predictions of which are combined using voting or
    weighted voting
  • most relevant description
  • Complicated concept descriptions are produced
    iteratively using feature construction (according
    to relevance).

12
  • All incremental ensemble approaches use some
    criteria to dynamically delete, reactivate, or
    create new ensemble members, which are normally
    based on the base models consistency with the
    current data.

13
Base Learning Algorithms
  • Rule-Based Learning
  • Decision trees
  • Including incremental decision trees
  • Naïve Bayes
  • SVMs
  • Radial Basis Functions networks
  • Instance-Based Learning

14
  • Global Eager Learners
  • Unable to adapt to local concept drift
  • Concept drift is often local
  • Record highs temps in a part of the world doesnt
    necessarily mean that temps around the globe are
    higher.
  • Local Lazy Learning
  • able to adapt well to local concept drift due to
    its nature.
  • Performs well with disjoint concepts.
  • Easy to update (Case-Based Learners).
  • Allows easy sharing of knowledge for some
    problems.
  • Easier to maintain multiple distributed
    case-bases.

15
Common Testing Datasets
  • STAGGER Moving Hyper-plane
  • Allow
  • controlling the type and rate of concept drift
  • context recurrence
  • presence of noise
  • irrelevant attributes
  • Disallow
  • Checking Scalability

16
  • Real-World Test Problems
  • Flight simulator data
  • Web page access data
  • Text Retrieval Conference (TREC)
  • Credit card fraud data
  • Breast cancer
  • Anonymous web browsing
  • US Census Bureau data
  • Email data
  • Unfortunately most real-world data sets contain
    little concept drift.

17
Theoretical Results
  • A maximal frequency of concept changes (rate of
    drift) that is acceptable by any learner, implies
    a lower bound for the size of a window of
    drifting concepts to be learnable.
  • It is sufficient for a learner to see a fixed
    number of the most recent instance.
  • Large window sizes in the theoretical bounds
    would be impractical to employ.

18
Incremental (Online) Learning vs. Batch Learning
  • Most of the algorithms for handling concept drift
    consider incremental (online) learning
    environments as opposed to batch learning.
  • Because real life data often needs to be
    processed in an online manner.
  • Data Streams incremental learning
  • Databases batch learning

19
Criteria for Updating the Current Model
  • Many algorithms for handling concept drift employ
    regular model updates while new data arrive.
  • Can be very costly
  • An alternative is to detect changes and adapt the
    model only if inevitable.
  • Based on the average confidence in correct
    prediction of the model on new instances
  • Observes the fraction of instances for which the
    confidence is below a given threshold.

20
  • Cased-Based Criteria
  • Problem-solution regularity
  • Problem-distribution regularity
  • May be good measures of quality of a case-base
  • Real-World Not easy to apply these measures as
    triggers for model updating because the drift
    rate and the level of noise may vary drastically
    with time.

21
Conclusions
  • Two kinds of concept drift
  • Real
  • Hidden Contexts
  • Virtual
  • Data Distribution
  • Three Basic approaches
  • Instance Selection
  • Instance Weighting
  • Ensemble learning

22
  • There are problems with most of the real-world
    datasets.
  • These data sets contain little concept drift or
    contain concept drift that is introduced
    artificially.
  • Criteria needs to be developed for detecting
    crucial changes that allow adapting the model
    only if inevitable.
  • Triggers are not robust enough to differentiate
    types of concept drift and different levels of
    noise.

23
Thank You
Write a Comment
User Comments (0)
About PowerShow.com