Hybrid role based in JHB
This role would suit a professional who is keen to grow their career in a busy team that values cognitive diversity and diversity of lived experience.
You will be part of a multi-disciplinary technology team and work closely with our customers (business, software vendors, and partners).
Within the team, you will be responsible for a suite of data processes and will participate in all aspects from design through to testing and implementation.
You will be surrounded by data professionals that strive for excellence and data best practices to realize business value.
As a Data Engineer, you will work in the data engineering team to build and maintain data pipelines to ingest data into the warehouse and support integration into other systems.
You will apply your knowledge of good data engineering practices and standards, and data technologies to deliver target state design and implementation.
The role is challenging, and you must be adept at problem-solving and able to respond to changing priorities and rapidly evolving requirements that may have a direct impact on services to users.
Overview of duties & responsibilities
- Reporting to the Data Engineer Practice Manager
- Key focus areas will be to design and support delivery of engineering solutions in the data warehouse, balancing enhancing ETL processes and modelling while supporting short-term data and reporting requirements from across the business.
- Work collaboratively with cross-functional teams within the business, including the Infrastructure DevOps team, solution architects, subject matter experts, data modelers, finance, underwriting, operations, etc.
- Collaboration aims to ensure the business data platform ecosystem is optimal in supporting business needs.
- Additionally, you will work collaboratively with vendors and partners to ensure that data engineering delivery and practices.
More detailed duties and responsibilities include:
- Maintain, support, and monitor existing production SSIS packages and SQL Queries and Stored Procedures, CICD pipelines to ensure all data loads on the data warehouse meet data quality standards and business SLA requirements.
- Build, maintain, support, and monitor Synapse data engineering pipelines on the data platform if required.
- Participate and contribute to data architecture design, data modelling, and analysis of data requirements; understand, document, communicate, and build appropriate solutions.
- Participate, design, build, deliver, and document data-related projects with various environment-specific data analytics technologies.
- Promote data engineering best practices with CICD pipelines and automation.
- Collaborate and work closely with team members and contribute significantly to building a high-performing, collaborative, transparent, and result-driven data engineering team.
- Support the Data Engineering Practice Team Manager with best fit-to-purpose data engineering solutions, quality engineering artifacts, and high-standard documentation.
- Follow Data Ethics standards to protect personal information and meet our customers', partners', and community's expectations.
Minimum Requirements:
- 8+ years demonstrable experience in design, build, and support of data engineering pipelines in data warehousing, data ingestion, cleansing, manipulation, modelling, and reporting.
- Experience in ETL using Microsoft technologies.
- Strong experience in writing MS SQL server queries, stored procedures, and SSIS Packages. Experience with SSRS would be an advantage.
- Experience of manipulating semi-structured data (XML, JSON).
- Strong knowledge and extensive experience in working in an Agile framework with CI/CD using modern DevOps / Data Ops integrated processes with YAML pipelines.
- Bachelor's degree in computer/data science technical or related field is a must. Post-graduate is highly regarded.
- Knowledge of Azure Synapse data engineering pipelines, PySpark notebooks, data platform lake house architecture, and Azure SQL ODS storage is desirable.
Start a new chapter of your career with one click today.
#J-18808-Ljbffr