An introduction to Apache Falcon

1
Apache Falcon

• What is it ?
• Benefits
• Architecture
• Example

www.semtech-solutions.co.nz info_at_semtech-solutions
.co.nz
2
Apache Falcon What is it ?

• A data life cycle management framework
• Created for Hadoop
• Logic based in Falcon rather than apps
• Simplifies data management
• Developed by InMobi and HortonWorks
• Falcon can manage
• Work flows
• Replication
• Provides data abstraction

www.semtech-solutions.co.nz info_at_semtech-solutions
.co.nz
3
Apache Falcon What is it ?

• Falcon provides services
• Data import / replication
• Scheduling / coordination
• Lifecycle policies
• Cluster management
• SLA Management
• An enterprise solution for data lifecycle
  management
• Currently an Apache incubator project

www.semtech-solutions.co.nz info_at_semtech-solutions
.co.nz
4
Apache Falcon Benefits

• Reduce workflow / ETL development time
• Reduce costs
• No need to re implement functionality
• Already in Falcon
• Already tested
• Use a single Falcon configuration file to
• Define replication points
• Define data processing pipeline

www.semtech-solutions.co.nz info_at_semtech-solutions
.co.nz
5
Apache Falcon Architecture
www.semtech-solutions.co.nz info_at_semtech-solutions
.co.nz
6
Apache Falcon BI Example

• Falcon used to manage work flow
• Falcon used to manage Cluster data replication
• BI example
• Staged and presented data replicated
• Presented data visible for
• Reporting
• Analytics
• See next slide .....

www.semtech-solutions.co.nz info_at_semtech-solutions
.co.nz
7
Apache Falcon BI Example
www.semtech-solutions.co.nz info_at_semtech-solutions
.co.nz
8
Contact Us

• Feel free to contact us at
• www.semtech-solutions.co.nz
• info_at_semtech-solutions.co.nz
• We offer IT project consultancy
• We are happy to hear about your problems
• You can just pay for those hours that you need
• To solve your problems