An Introduction to Apache Hama

1
Apache Hama

• What is it ?
• What is it used for ?
• How does it work ?
• Architecture

www.semtech-solutions.co.nz info_at_semtech-solutio
ns.co.nz
2
Hama What is it ?

• A Distributed computing framework
• For massive scientific calculations
• For distributed computing
• Uses Bulk Synchronous Parallel Computing ( BSP )
  
• Written in Java
• Licensed via Apache
• Available for Hadoop greater than 1.0.x
• Requires Java 1.6.x or higher

www.semtech-solutions.co.nz info_at_semtech-solutio
ns.co.nz
3
Hama What is it used for ?

• large scale computations that
• exceed the abilities of Map reduce
• Machine learning
• Matrix
• Graph
• Network Algorithms

www.semtech-solutions.co.nz info_at_semtech-solutio
ns.co.nz
4
Hama How does it work ?

• A series of super steps
• Each of which is
• Concurrent computation on local data
• Point to point communication
• Synchronise all processors
• Uses Hadoop RPC for communication

www.semtech-solutions.co.nz info_at_semtech-solutio
ns.co.nz
5
Hama Architecture

• Hama consists of BSP Master, GroomServers,
  ZooKeeper

www.semtech-solutions.co.nz info_at_semtech-solutio
ns.co.nz
6
Hama Architecture

• Role of BSPMaster
• Maintain groom server statuses
• Controlling cluster super steps
• Maintaining job progress information
• Scheduling Jobs
• Assign tasks to groom servers
• Fault control
• Provide cluster control to users

www.semtech-solutions.co.nz info_at_semtech-solutio
ns.co.nz
7
Hama Architecture

• Role of GroomServer
• Performs BSP tasks
• Reports status to master
• Runs on a single HDFS data node
• Starts with a BSP Peer instance

www.semtech-solutions.co.nz info_at_semtech-solutio
ns.co.nz
8
Hama Architecture

• Role of ZooKeeper
• Manages barrier synchronisation of BSP Peers

www.semtech-solutions.co.nz info_at_semtech-solutio
ns.co.nz
9
Contact Us

• Feel free to contact us at
• www.semtech-solutions.co.nz
• info_at_semtech-solutions.co.nz
• We offer IT project consultancy
• We are happy to hear about your problems
• You can just pay for those hours that you need
• To solve your problems