Collibra Data Lineage Automation Engineer

• Jobseeker Video Testimonials
  
• Employee Glassdoor Reviews
  

We are an IT Solutions Integrator/Consulting Firm helping our clients hire the right professional for an exciting long term project. Here are a few details.

We are seeking a highly experienced Collibra Data Lineage Automation Engineer to lead the design, implementation, and scaling of automated end-to-end data lineage solutions across a complex enterprise ecosystem. The ideal candidate will have deep expertise in lineage frameworks (e.g., Spline, OpenLineage), a strong foundation in AI/ML-driven metadata intelligence, and proven experience delivering lineage automation in hybrid cloud and legacy environments.

This is a hands-on engineering role with significant influence on architecture, governance, and enterprise data strategy.

• Lead the implementation of automated lineage across diverse systems:
  

  • Cloud platforms (e.g., Snowflake, AWS)
    

  • Legacy RDBMS and ETLs (e.g., SQL Server, Oracle)
    

  • NoSQL databases (e.g., MongoDB)
    

  • BI/reporting tools (e.g., Tableau, Power BI)
    

• Design and extend lineage capture frameworks (Spline, OpenLineage, Marquez) for enterprise-wide adoption.
  

• Build custom connectors, extractors, and agents to capture lineage in systems not natively supported.
  

• Integrate lineage with metadata/governance platforms (Collibra, Alation) to ensure usability and traceability.
  

• Apply AI/ML techniques to infer lineage from logs, query patterns, or unstructured code (e.g., Java-based ETLs).
  

• Develop reusable lineage components and promote automation-first, reusable design principles.
  

• Collaborate with architecture, governance, and engineering teams to define lineage standards, storage models, and best practices.
  

• Proven success implementing automated lineage solutions in hybrid environments (cloud + legacy).
  

• Hands-on experience with Spline, OpenLineage, Marquez, or similar open-source lineage frameworks.
  

• Deep knowledge of metadata capture, ETL tracing, and query execution mapping.
  

• Strong AI/ML background, with experience in:
  

  • Metadata intelligence
    

  • NLP-based code parsing
    

  • Pattern detection in lineage gaps
    

• Programming expertise in Python, Scala, or Java.
  

• Strong SQL skills with hands-on exposure to logs/queries from Snowflake, SQL Server, Oracle, MongoDB, etc.
  

• Experience integrating lineage into data governance platforms (e.g., Collibra, Alation).
  

• Hands-on experience with commercial lineage tools (evaluation or implementation).
  

• Background in regulated industries (finance, healthcare, insurance).
  

• Familiarity with event-driven architectures for real-time lineage propagation.
  

• Exposure to data mesh and domain-driven lineage approaches.
  

• Has successfully delivered automated lineage at enterprise scale.
  

• Operates at the intersection of data engineering, metadata management, and AI innovation.
  

• Acts as a strategic thought partner to architects, governance leads, and business stakeholders.
  

• Brings an automation-first, reusable component mindset to all lineage solutions.