Cloud Application Integration Process - Databricks
Posted by: Informatica Cloud
CAI bundle for Databricks comes with preconfigured processes which integrate with Databricks to trigger jobs and get status of these jobs using Databricks REST API.
These processes can be called from task flows in Cloud Data Integration platforms using its ‘IntegrationOps Task’. This task opens up the orchestration capabilities of Cloud Data Integration to integrate with other ETL tools.
Overview
CAI Bundle for DBT provide following functionalities:
- Authentication: Databricks API supports token-based authentication. The service token can be generated from the Darabricks admin console.
- Pipeline execution: Using Databricks API, the pipelines can be triggered. As a response, runID will be returned.
- Pipeline Status: Status of a pipeline can be verified by passing the runID to the API. A detailed response with start time, end time, pipeline name, status etc. will be returned.
CAI Bundle Assets:
Sr. No |
Asset Name |
Asset Type |
1 |
svc_Databricks |
Service Connector |
2 |
con-Databricks |
Application connection |
3 |
Databricks-TriggerPipeline |
Process |
4 |
Databricks-MonitorPipelineStatus |
Process |
Resources
Support
Please Email:
- vpothuganti@informatica.com
- vvemuganti@informatica.com
- mkarthikeyan@informatica.com