We are looking for an experienced Data Engineer to support a data migration and integration programme within a relatively low maturity Azure environment.
The role will focus on designing and building robust data pipelines, migrating data from a wide range of legacy and enterprise source systems, and establishing scalable data flows into Azure Data Lake and Databricks. A key part of the role will also involve integrating data into a proprietary data transformation solution where applicable.
Key Skills & Experience
-
Azure Data Engineering
-
Azure Data Factory (ADF)
-
Azure Data Lake
-
Databricks
-
Building ETL / ELT pipelines
-
Large-scale data migration and integration
-
Oracle and/or Oracle Fusion data sources
-
SharePoint data extraction and integration
-
Working with multiple and varied source systems
-
Data transformation, mapping and validation
-
Designing data pipelines in environments where architecture and processes are still developing
Key Responsibilities
-
Design, build and maintain data migration and ingestion pipelines within Azure.
-
Support the setup and development of the Azure Data Lake environment.
-
Build integrations between source systems, Azure Data Lake and Databricks.
-
Develop and manage pipelines using Azure Data Factory (ADF).
-
Extract and migrate data from a variety of source systems, including:
-
Oracle
-
Oracle Fusion
-
Informatica
-
SharePoint, which will be an initial priority
-
Additional enterprise and legacy systems as the programme develops
-
Help define scalable data ingestion, transformation and migration patterns for future source systems.
-
Support data quality, validation and troubleshooting throughout the migration process.
-
Work closely with technical and business stakeholders to understand source data, target structures and transformation requirements.
-
Help improve the overall maturity of the Azure data environment, introducing appropriate engineering standards and best practices
