Denodo Scanner for EDC

Denodo Scanner for EDC

Posted by: Darren Wrigley

The Denodo Scanner for EDC extracts field-level metadata and internal/external lineage from Denodo.

Overview

The Denodo custom scanner extracts metadata and Lineage directly from Denodo (via jdbc connection) virtual tables & creates lineage via connection assignments to relational and local flat file sources.  Field level lineage is extracted for all denodo virtual tables including expression details.
Tested with Denodo v7 and v8 and EDC v10.2.2hf1+, fully tested with EDC v10.5 and 10.4.1
Supports the following features:-

  • uses Denodo jdbc driver to scan Denodo virtual tables/views, including lineage inside and outside of Denodo
  • can scan multiple Denodo databases (these are actually more like schemas for other database types)
  • supports lineage across Denodo databases
  • extracts lineage between views within Denodo, at both the view & column level
  • expression fields are processed - lineage should be created to all referenced source fields
    • the expression logic is stored in the "View Statement" system attribute com.infa.ldm.relational.ViewStatement
  • for relational sources, and local delimited files, custom lineage is generated to link source tables/views to the actual dbms tables/views via connection assignment
  • supports both include and exclude filtering using wildcards for <databsae>.<table|view>
    e.g. exclude.datasets=*.*test*
  • works with Denodo 7 or 8.  To ensure that it works for your version, replace lib/denodo-vdp-jdbcdriver.jar provided with the scanner, with your Denodo jdbc driver jar file
    (note: you cannot use a Denodo 7 jdbc driver with v8, or a v8 driver with v7).
For more information – see latest documents and readme on the custom scanner github page

 

Features

  • Denodo field level lineage
Specifications:
  • Tested with all EDC versions v10.2.2hf1 and later, including 10.4.1.3 and 10.5
  • Works with Denodo versions 7.x and 8.x

Resources

Support

This is a field-developed solution created by Darren Wrigley, a leading expert in creating metadata solutions.   Developed in conjunction with existing denodo users & tested against large denodo installations (with thousands of virtual tables).
Issues can raised via github issues

Website URL

Community URL

Comments (0) Comment can only be posted by Signed/Logged in user