Denodo Scanner for EDC

Denodo Scanner for EDC

Posted by: Darren Wrigley

The Denodo Scanner for EDC and CDGC extracts field-level metadata and internal/external lineage from Denodo.

Overview

The Denodo custom scanner extracts metadata and Lineage directly from Denodo (via jdbc connection) virtual tables & creates lineage via connection assignments to relational and local flat file sources.  Field level lineage is extracted for all denodo virtual tables including expression details.
Tested with Denodo v7 and v8 and EDC v10.2.2hf1+, fully tested with EDC v10.5 and 10.4.1
Version 1.1.000 or later also supports CDGC

Supports the following features:-

  • uses Denodo jdbc driver to scan Denodo virtual tables/views, including lineage inside and outside of Denodo
  • can scan multiple Denodo databases (these are actually more like schemas for other database types)
  • supports lineage across Denodo databases
  • extracts lineage between views within Denodo, at both the view & column level
  • expression fields are processed - lineage should be created to all referenced source fields
    • the expression logic is stored in the "View Statement" system attribute com.infa.ldm.relational.ViewStatement
  • for relational sources, and local delimited files, custom lineage is generated to link source tables/views to the actual dbms tables/views via connection assignment
  • supports both include and exclude filtering using wildcards for <databsae>.<table|view>
    e.g. exclude.datasets=*.*test*
  • works with Denodo 7 or 8.  To ensure that it works for your version, replace lib/denodo-vdp-jdbcdriver.jar provided with the scanner, with your Denodo jdbc driver jar file
    (note: you cannot use a Denodo 7 jdbc driver with v8, or a v8 driver with v7).
For more information – see latest documents and readme on the custom scanner github page

 

Features

  • Denodo field level lineage
Specifications:
  • Tested with all EDC versions v10.2.2hf1 and later, including 10.4.1.3 and 10.5
  • Works with Denodo versions 7.x and 8.x

Resources

Support

This is a field-developed solution created by Darren Wrigley, a leading expert in creating metadata solutions.   Developed in conjunction with existing denodo users & tested against large denodo installations (with thousands of virtual tables).
Issues can raised via github  

For fastest support – raise an issue on github

Email: dwrigley@informatica.com

Website URL

Community URL

Comments (1) Comment can only be posted by Signed/Logged in user

Sort: Newest | Oldest
  • Thank you Darren for creating this custom scanner for Denodo. I have requested your product team to include this scanner as part of EDC road map. 

    I submitted the feature request "MDX-35037" to track this enhancement for future releases. You can check the status of the FR by logging into https://network.informatica.com/ and selecting "Change Request Tracking" under "Resources". Since I just opened the FR, it may take some time to show up in that report. 
     
     This feature request will go through the standard process of assessment and approval by product management and when approved, it will be addressed in future releases of the product. Product Management will come up with an ETA and update the CR. We suggest you to please vote and track the CR on https://network.informatica.com/community/informatica-network/cr. You can also contact your Customer Success Manager to follow up on the CR.