Data Staging in SAP BW

1
Data Staging in SAP BW
2
Teemad

• Architecture of a Data Warehouse
• Building and Running a Data Warehouse 

3
Architecture of a Data Warehouse

• Layer architecture
• ?     Persistent staging area
• ?     Data warehouse
• ?     Architected data marts
• ?     Operational data store

4
Building and Running a Data Warehouse 

• Data Staging
• Extraction, transformation, loading (ETL) of
  data All data sources can be accessed by means
  of extraction in the background (via JDBC, file,
  XMLA, ODBO, ..). Extractors are delivered for
  SAP applications or can be generated. The
  standard applications of other providers can be
  accessed by integrating the ETL tools of non-SAP
  providers.
• Real-time data warehousing Event-near
  availability of data in the operational data
  store can be realized using real-time data
  acquisition technology.
• Remote data access Data can be accessed without
  being saved in the BI system using
  VirtualProviders
• Modeling a layer architecture
• Transformation
• Modeling the data flow
• Staging data for analysis

5
Data Acquisition 
6
Source System 

• SAP systems
• BI systems
• Flat files for which metadata is maintained
  manually and transferred to BW using a file
  interface
• Database management systems into which data is
  loaded from a database supported by SAP using DB
  Connect, without using an external extraction
  program
• Relational or multidimensional sources that are
  connected to BI using UD Connect
• Web Services that transfer data to BI by means of
  a push
• Non-SAP systems for which data and metadata is
  transferred using staging BAPIs.

7
Data Extraction from SAP Source Systems 

• Application-specific extractors - hard-coded for
  the DataSource that was delivered with BI
  Content, and which fill the extraction structure
  of this DataSource.
• Generic extractors - to extract more data from
  the SAP source system and transfer it into BI.
• Regardless of application, you can generically
  extract master data attributes or texts, or
  transaction data from all transparent tables,
  database views or SAP query functional areas or
  using the function module.
• PlugIn for SAP Systems
• BI-specific source system functions, extractors
  and DataSources are delivered for specific SAP
  systems by plug-ins.
• Communication between the SAP source system and
  BI is only possible if the appropriate plug-in is
  installed in the source system.

8
Extraction Structure 

• In the extraction structure, data from a
  DataSource is staged in the source system. It
  contains the amount of fields that are offered by
  an extractor in the source system for the data
  loading process.
• You can edit DataSource extraction structures in
  the source system. In particular, you can
  determine the DataSource fields in which you hide
  extraction structure fields from the transfer.
  This means filtering the extraction structure
  and/or enhancing the DataSource for fields,
  meaning completing the extraction structure.

9
Data Sources

• BI Content DataSources
• Generic DataSource for Database View or InfoSet
• Generic DataSource for Function Module
• Productive DataSource with Direct Access

10
Delta Process

• The delta process is a feature of the extractor
  and specifies how data is to be transferred. As a
  DataSource attribute, it specifies how the
  DataSource data is passed on to the data target.
  The type of delta process affects the update into
  a data target. When you update data in an ODS
  object, you need to serialize it so that you can
  also overwrite it. According to the delta
  process, the system decides whether it is
  necessary to serialize by request or by data
  packet.
• Deltas with after, before and reverse images that
  are updated directly in the delta queue an after
  image shows the status after the change, a before
  image the status before the change with a
  negative sign and the reverse image also shows
  the negative sign next to the record while
  indicating it for deletion. This serializes the
  delta packets. The delta process controls whether
  adding or overwriting is permitted. In this case,
  adding and overwriting are permitted. This
  process supports an update in an ODS object as
  well as in an InfoCube. (technical name of the
  delta process in the system) ABR)
• The extractor delivers additive deltas that are
  serialized by request. This serialization is
  necessary since the extractor within a request
  delivers each key once, and otherwise changes in
  the non-key fields are not copied over correctly.
  It only supports the addition of fields. It
  supports an update in an ODS object as well as in
  an InfoCube. This delta process is used by LIS
  DataSources. (technical name of the delta process
  in the system) ADD)
• Deltas with after image, which are updated
  directly in the delta queue. This serializes data
  by packet since the same key can be copied more
  than once within a request. It does not support
  the direct update of data in an InfoCube. An ODS
  object must always be in operation when you
  update data in an InfoCube. For numeric key
  figures, for example, this process only supports
  overwriting and not adding, otherwise incorrect
  results would come about. It is used in FI-AP/AR
  for transferring line items, while the variation
  of the process, where the extractor can also send
  records with the deletion flag, is used in this
  capacity in BBP. (technical name of the delta
  process in the system) AIM/AIMD)

11
Transferring Data

• by data request
• using web services
• using UD Connect
• using DB Connect
• from flat files

12
Data Transformation
A transformation consists of at least one
transformation rule. Various rule types,
transformation types, and routine types are
available. Transformation rules Transformation
rules map any number of source fields to at least
one target field. You can use different rules
types for this. Rule type A rule type is a
specific operation that is applied to the
relevant fields using a transformation
rule. Transformation type The transformation
type determines how data is written into the
fields of the target. Rule group A rule group is
a group of transformation rules. Rule groups
allow you to combine various rules. Routine You
use routines to implement complex transformation
rules yourself. Routines are available as a rule
type. There are also routine types that you can
use to implement additional transformations.
13
Data Loading