Data Engineer

Network IT • Other • Cape Town, Western Cape, South Africa • $200 - $300 / year • 3h ago

Reference: BIT002679-CdP-1

We are looking for a skilled Data Engineer to join our clients’ investment team to oversee their database architecture, data flow, and data processes. The role involves efficiently managing the reception, transfer, processing, analysis, and integration of third-party systems for investment research and business operational processes. The ideal candidate will possess expertise in version control and database solutions, while also adhering to the companies’ collaborative team approach in line with their value framework.

Duties & Responsibilities

Design and develop scalable data pipelines and ETL processes to collect, process, and integrate data from various sources into a centralized data warehouse or data lake.
Implement data models, schemas, and database structures to organize and optimize data storage for efficient data retrieval, writing, and analysis.
Collaborate with quantitative analysts to understand their data requirements and provide necessary infrastructure and tools for data exploration, analysis, and visualization.
Ensure data quality and consistency by implementing data validation, cleaning, and transformation procedures.
Monitor and maintain the performance, availability, and security of data systems, including database servers, data warehouses, and data processing frameworks.
Troubleshoot and resolve data-related issues and performance bottlenecks in collaboration with the investment teams.
Evaluate and integrate new data technologies, tools, and frameworks to improve data infrastructure and processing capabilities.
Document data pipelines, workflows, and technical specifications to facilitate knowledge sharing and maintain systems.
Stay up to date with industry trends, best practices, and emerging technologies in the field of data engineering.
Assist and collaborate on team-wide projects.
Contribute to the analytics platform from a Data Engineering perspective.
Assist and investigate ad-hoc data-related queries.

Skills & Experience Required

Up to 3 years' experience with a specific focus on database management, data analysis, and producing ETL processes at scale is essential.
Basic understanding of financial instruments and markets.
Exceptional skill and experience in Python and SQL.
Linux server administration.
Git version control.
Solid experience in managing and analysing large datasets.