Informatica CDQ | How to Create & Execute a Data Profile + Metrics Explained

1
Day5-Informatica Cloud Data Quality(CDQ)
Agenda

• Secure agent Installation and configuration
• How to Start Secure agent
• Create connections
• Data profiling service
• Create Data profile and execute it
• Explain data profiling metrics

InventModel Technology Solution
91-98219-31210
support_at_inventmodel.com
2
Secure agent Installation and configuration

• Secure Agent - a lightweight program that
  facilitates secure data integration between cloud
  and on-premises environments, enabling secure
  communication across firewalls
• Login to Administrator Service in IDMC
• Download Secure agent
• Install Secure agent
• Configure secure agent
• Enable services and connectors
• Create Flat file connection and test

InventModel Technology Solution
91-98219-31210
support_at_inventmodel.com
InventModel Technology Solution
91-98219-31210
support_at_inventmodel.com
3
How to Start Secure agent

• To start the Informatica Secure Agent in
  Informatica Cloud, follow these steps
• 1. Verify Prerequisites
• Ensure that the Secure Agent is installed on your
  machine.
• Check that your system meets the hardware and
  software requirements.
• 2. Starting the Secure Agent (Windows)
• Go to the Start Menu ? Informatica Secure Agent ?
  Start Secure Agent.
• Alternatively, press Win R, type services.msc,
  and hit Enter.
• Locate Informatica Secure Agent in the list of
  services.
• Right-click on it and select Start.

InventModel Technology Solution
91-98219-31210
support_at_inventmodel.com
InventModel Technology Solution
91-98219-31210
support_at_inventmodel.com
4
How to Start Secure agent

• 2. Starting the Secure Agent (Linux)
• Open a terminal window.
• Navigate to the Secure Agent directory
• bash
• cd /opt/Informatica/secureagent
• Start the agent with
• bash
• ./agent.sh start

InventModel Technology Solution
91-98219-31210
support_at_inventmodel.com
InventModel Technology Solution
91-98219-31210
support_at_inventmodel.com
5
How to Start Secure agent

• 3. Check Agent Status
• WindowsOpen the Secure Agent application, and
  the status will show as "Running".
• Linux
• bash
• ./agent.sh status
• 4. Verify in Informatica Cloud
• Log in to Informatica Cloud.
• Go to Administrator ? Secure Agents.
• The agent should show as "Online".

InventModel Technology Solution
91-98219-31210
support_at_inventmodel.com
InventModel Technology Solution
91-98219-31210
support_at_inventmodel.com
6
Create connections

• To create connections in Informatica Data
  Management Cloud (IDMC) as an administrator,
  follow these steps
• ?? Steps to Create Connections in IDMC
• Log in to IDMC
• Go to Informatica Cloud and log in with your
  admin credentials.
• Navigate to the Administrator Console
• In the top navigation bar, click on
  "Administrator".
• Select "Connections" from the drop-down menu.
• Create a New Connection
• Click on "New Connection".
• Choose the connection type (e.g., Salesforce,
  Oracle, SQL Server, FTP, etc.).

InventModel Technology Solution
91-98219-31210
support_at_inventmodel.com
InventModel Technology Solution
91-98219-31210
support_at_inventmodel.com
7
Create connections

• Configure Connection Details
• Name Enter a descriptive name for the
  connection.
• Description Optional, but helpful for
  documentation.
• Connection Properties Fill in the required
  details like
• Host, Port, Username, Password (for databases)
• API Keys, Tokens, or OAuth Details (for cloud
  apps)
• Database Name, Schema, etc.
• Advanced Settings (Optional)
• Configure additional settings like SSL
  certificates, timeout values, etc., if needed.

InventModel Technology Solution
91-98219-31210
support_at_inventmodel.com
InventModel Technology Solution
91-98219-31210
support_at_inventmodel.com
8
Create connections

• Test the Connection
• Click "Test Connection" to ensure everything is
  set up correctly.
• If successful, you'll see a confirmation message.
  If not, check the error details to troubleshoot.
• Save the Connection
• Click "Save" to finalize the connection.
• Assign to Secure Agent (if needed)
• If the connection requires on-premises data
  access, ensure the connection is assigned to the
  appropriate Secure Agent.

InventModel Technology Solution
91-98219-31210
support_at_inventmodel.com
InventModel Technology Solution
91-98219-31210
support_at_inventmodel.com
9
Data profiling service

• Data Profiling Service in Informatica Data
  Management Cloud (IDMC) is a powerful feature
  that helps you understand the structure, quality,
  and content of your data. It provides insights
  into data patterns, inconsistencies, and
  anomalies, which are crucial for data governance,
  data quality, and analytics.
• ?? Key Features of Data Profiling Service
• Column Profiling
• Analyzes data at the column level to identify
  data types, null values, unique counts, patterns,
  and more.
• Data Quality Metrics
• Measures data accuracy, completeness,
  consistency, and validity.
• Pattern Recognition
• Detects patterns like phone numbers, email
  formats, ZIP codes, etc.
• Anomaly Detection
• Identifies outliers or unusual data points.
• Data Relationship Discovery
• Discovers relationships between tables or
  datasets.

InventModel Technology Solution
91-98219-31210
support_at_inventmodel.com
InventModel Technology Solution
91-98219-31210
support_at_inventmodel.com
10
Data profiling service

• ?? How to Use Data Profiling Service in IDMC
• Log in to IDMC.
• Go to "Data" ? "Data Profiling".
• Click "Create New Profile".
• Select the connection (e.g., database, flat file,
  etc.) and the specific data objects to analyze.
• Choose the profiling rules or configurations.
• Click "Run" to start profiling.
• Review the profiling reports for insights on data
  quality, patterns, and anomalies.

InventModel Technology Solution
91-98219-31210
support_at_inventmodel.com
InventModel Technology Solution
91-98219-31210
support_at_inventmodel.com
11
Data profiling service

• Common Use Cases
• Data Quality Assessment Identifying missing
  values, duplicates, or inconsistent data.
• Data Migration Validation Ensuring accuracy
  during system migrations.
• Regulatory Compliance Verifying data integrity
  for GDPR, HIPAA, etc.
• ETL Optimization Understanding data
  characteristics to improve data pipelines.

InventModel Technology Solution
91-98219-31210
support_at_inventmodel.com
InventModel Technology Solution
91-98219-31210
support_at_inventmodel.com
12
Create Data profile and execute it

• To create and execute a Data Profile in
  Informatica Data Management Cloud (IDMC), follow
  these steps. I'll guide you through the process
  step-by-step.
• ?? Step 1 Log In to IDMC
• Go to Informatica Cloud and log in with your
  credentials.
• ?? Step 2 Access Data Profiling
• Click on the "Data" tab in the top navigation
  bar.
• Select "Data Profiling" from the dropdown.
• ??? Step 3 Create a New Data Profile
• Click "New Profile" or "Create".
• Provide a name for the data profile (e.g.,
  "Customer Data Profile").
• Add an optional description for future reference.

InventModel Technology Solution
91-98219-31210
support_at_inventmodel.com
InventModel Technology Solution
91-98219-31210
support_at_inventmodel.com
13
Create Data profile and execute it

• ?? Step 4 Select Data Source
• Choose the connection to your data source (e.g.,
  SQL Server, Oracle, Flat File, etc.).
• Select the specific table or file you want to
  profile.
• ?? Step 5 Configure Profiling Settings
• Choose the profiling rules you want to apply
• Column Profiling Analyzes data types, null
  values, unique counts.
• Data Quality Checks Looks for duplicates,
  patterns, etc.
• Relationship Discovery Identifies data
  relationships.
• You can customize the settings as needed.
• ?? Step 6 Run the Data Profile
• Click "Run" or "Execute" to start the profiling
  process.
• The system will analyze the data and generate
  reports.

InventModel Technology Solution
91-98219-31210
support_at_inventmodel.com
InventModel Technology Solution
91-98219-31210
support_at_inventmodel.com
14
Create Data profile and execute it

• ?? Step 7 Review the Results
• Once the profile is complete, go to the "Results"
  tab.
• Review metrics like
• Data distribution
• Data quality scores
• Anomalies or outliers detected
• You can export the report if needed.

InventModel Technology Solution
91-98219-31210
support_at_inventmodel.com
InventModel Technology Solution
91-98219-31210
support_at_inventmodel.com
15
Explain data profiling metrics

• Data Profiling Metrics are key indicators that
  help you assess the quality, structure, and
  content of your data. They provide insights into
  data consistency, completeness, uniqueness, and
  more, enabling informed decisions for data
  governance, analytics, and quality management.
• Heres a breakdown of the common data profiling
  metrics

InventModel Technology Solution
91-98219-31210
support_at_inventmodel.com
InventModel Technology Solution
91-98219-31210
support_at_inventmodel.com
16
Explain data profiling metrics

• ?? 1. Column-Level Metrics
• These metrics provide insights into individual
  columns within your dataset
• Data Type Identifies the type of data (e.g.,
  integer, string, date).
• Distinct Count Number of unique values in the
  column.
• Null Count Number of null or missing values.
• Null Percentage Percentage of nulls compared to
  the total number of rows.
• Max/Min Values Identifies the highest and lowest
  values.
• Data Length Length of text data or number of
  characters.
• Data Patterns Detects patterns like email
  formats, phone numbers, ZIP codes.
• Value Distribution Shows the frequency
  distribution of values.

InventModel Technology Solution
91-98219-31210
support_at_inventmodel.com
InventModel Technology Solution
91-98219-31210
support_at_inventmodel.com
17
Explain data profiling metrics

• ?? 2. Data Quality Metrics
• These metrics assess how well the data meets
  business requirements
• Completeness Measures if all required data
  fields are populated.
• Uniqueness Identifies duplicate records or
  values.
• Consistency Checks if data values are consistent
  across different datasets.
• Validity Ensures data adheres to defined
  formats, rules, and standards.
• Accuracy Compares data to a trusted source to
  verify correctness.

InventModel Technology Solution
91-98219-31210
support_at_inventmodel.com
InventModel Technology Solution
91-98219-31210
support_at_inventmodel.com
18
Thank You !
References https//informatica.com
InventModel Technology Solution
91-98219-31210
support_at_inventmodel.com