Senior Data Engineer

Function

We’re looking for an experienced Data Engineer to join our client’s team and contribute to the development of a secure, on-premise data platform in a highly regulated environment. You’ll work as part of a multidisciplinary data team alongside architecture and BI specialists, helping build robust data solutions where security, governance, traceability and data sovereignty are key priorities.

Role & Responsibilities

• Design and deploy reliable data ingestion pipelines from heterogeneous sources, including files, APIs and relational databases.
• Implement incremental and streaming ingestion, ensuring reliability, idempotency and end-to-end traceability.
• Develop transformation models following a Medallion Architecture (Bronze, Silver and Gold).
• Implement data historization patterns such as SCD Type 2, upserts and append-only models.
• Develop and maintain Delta Lake tables and exports, leveraging ACID transactions, time travel and optimization capabilities.
• Build and maintain orchestration assets, monitor pipeline runs, handle failures and automate alerts.
• Manage technical and business metadata, contribute to end-to-end data lineage and support the production of governance documentation.
• Develop endpoints to expose curated Gold-layer data to BI dashboards and collaborate closely with reporting teams.

Technical Profile

The ideal candidate has strong hands-on experience with Python and advanced SQL, as well as proven expertise with dbt-core, including models, snapshots, macros, tests and profiles. You should be comfortable working with DuckDB, Delta Lake, MSSQL and PostgreSQL, including advanced SQL and CDC concepts, and have experience with an orchestration framework such as Dagster, Airflow or an equivalent solution. Experience with Linux/OpenShift and PowerShell is expected, together with familiarity with data governance and metadata platforms such as DataHub or OpenMetadata. A solid understanding of data engineering, data architecture, Data Lakes, DWH and Lakehouse/Medallion concepts, combined with BI development experience, is highly valued.

Nice to Have

• Experience with Kafka and Debezium for real-time data capture.
• Knowledge of Apache Spark, including Spark SQL, Thrift Server and Beeline.
• Experience with Power BI Report Server (PBIRS) or Apache Superset.
• Understanding of security requirements in restricted-access environments.
• Practical knowledge of data governance, metadata management and end-to-end lineage.
• Previous experience working in complex, highly governed or security-sensitive environments.

Contactperson & Reference

• Reference #: INW27553
• Pieter Messely
• pieter.messely@i4m.be