Job Description:
- As a Data Engineer you will design and implement data ingestion canonical mapping validation lineage and data processing capabilities for an enterprise platform supporting regulated workflows
- You will work with architects backend engineers AI engineers QA and domain specialists to ensure data is reliable traceable governed and reusable across modules
Key Responsibilities:
- Design and build data ingestion pipelines for files APIs SFTP sources databases and enterprise data extracts
- Implement canonical data models source to target mapping transformation rules validation checks rejected record handling and lineage capture
- Develop data quality checks reconciliation routines control result tables exception datasets and metadata structures for auditable data processing
- Work with PostgreSQL SQL object storage data lake patterns ETL ELT tools and batch processing frameworks as needed
- Partner with AI engineers to provide clean contextual and grounded data for AI summaries narratives retrieval and explanations
- Create data dictionaries mapping specifications interface contracts test datasets and data quality documentation
- Support performance tuning partitioning indexing data retention archival and privacy aware data handling
- Work with QA teams to defi
Technical Requirements:
- Minimum 7 years of experience in data engineering ETL ELT development data warehousing data platform engineering or related roles
- Strong SQL skills and hands on experience with relational databases such as PostgreSQL SQL Server Oracle or MySQL
- Experience designing ingestion pipelines transformations validation rules data quality checks and source to target mapping specifications
- Understanding of data modeling canonical models metadata lineage audit fields reconciliation and data quality concepts
- Experience with Python Spark Databricks Azure Data Factory dbt Airflow or equivalent data engineering tools is preferred
- Ability to work with structured semi structured and file based data formats such as CSV JSON Excel XML and Parquet
- Strong documentation discipline and ability to collaborate with functional SMEs and engineers on data requirements
Preferred Skills:
Technology->Analytics - Packages->Python - Big Data,Technology->Data Engineering->Databricks,Technology->Data Services Testing->Data Warehouse Testing,Technology->ETL & Data Quality->ETL - Others