Senior Data Engineer (Databricks)
Application Deadline: 13 December 2024
Department: Delivery
Employment Type: Contract / Temp
Location: South Africa
Reporting To: Tom Bruce
Compensation: R700,000 - R1,200,000 / year
Description
A bit about us…
Snap Analytics is a high-growth data analytics consultancy with offices in the UK and India, we work with enterprise clients to simplify complex data and drive business value. We’re customer-centric in our approach and dedicated to helping organisations achieve their strategic goals through innovative cloud analytics solutions.
We pride ourselves on using our innovative Snap 360 delivery framework, combined with a strong culture of teamwork and knowledge-sharing, to consistently deliver exceptional results and ensure 100% customer satisfaction.
Joining Snap at this exciting stage of growth offers a unique opportunity to make a significant impact. As a rapidly expanding consultancy, you'll have the chance to shape our direction and play a key role in driving our success and strategy forward.
A bit about the role...
As a Senior Data Engineer at Snap, building on your wealth of experience, you’ll have the opportunity to explore the latest technologies within analytics; from ETL and Modern Cloud Data Platforms, to BI and ML/AI tools.
We’ll also look to you to lead in client interaction and engagement, support and mentor more junior members of the team and assist in building internal capabilities for use with clients.
Here’s a breakdown of what you’ll be doing…
Data Migration & Integration:
- Lead and manage complex data migration projects, ensuring smooth transitions from multiple data sources to cloud environments.
- Design and implement efficient ETL/ELT processes for seamless integration between systems.
Data Warehousing:
- Design, build, and optimize cloud-based data warehouses (e.g. Databricks (must have), Snowflake, AWS Redshift, Google BigQuery, Azure Synapse) to support advanced analytics and reporting needs. Ensure data accuracy, security, and compliance.
Data Modelling:
- Develop and maintain robust data models (star, snowflake, etc.) that ensure high-quality, scalable, and efficient data storage and retrieval.
- Implement best practices for data modelling to support analytics and reporting requirements.
Cloud Infrastructure:
- Develop and maintain scalable, robust, and cost-effective cloud-based data architectures on platforms such as AWS, Azure, and GCP.
- Provide best practices for data storage, governance, and access control.
Pipeline Development:
- Build, automate, and monitor data pipelines that handle large volumes of structured and unstructured data.
- Implement best practices for data cleansing, transformation, and enrichment.
Collaboration & Stakeholder Management:
- Collaborate with data architects, data scientists, and business teams to understand requirements and translate them into effective data engineering solutions.
- Communicate technical challenges and provide insights on data-driven solutions.
Performance Optimization:
- Analyse and improve the performance of data systems and pipelines, ensuring low-latency data access, high availability, and optimized cloud resource usage.
Mentorship & Leadership:
- Mentor junior engineers, fostering a culture of collaboration and continuous learning.
- Provide technical guidance, code reviews, and best practices to elevate the overall team performance.
Innovation & Continuous Improvement:
- Stay updated with the latest trends and technologies in data engineering, cloud platforms, and big data to propose innovative solutions for clients' evolving data needs.
This role is for you if you have...
- Proven experience (5 years+) as a data engineer with a focus on data migration, integration and cloud-based data warehousing working with big data for enterprise organisations.
- Expertise in building data architecture on cloud platforms such as AWS, GCP or Microsoft Azure.
- 5+ years of experience of ETL/ELT design and development using tools like Matillion, Informatica, SAP Data Services, Talend and Cloud Data Platforms including Databricks (must have), Snowflake, Redshift or BigQuery.
- A deep understanding of data modelling principles and techniques (e.g., star schema, snowflake schema), and experience in developing optimized models to support analytics and reporting.
- Experience with version control, CI/CD pipelines, and containerization tools (e.g., Git, Jenkins, Docker, Kubernetes).
- Strong knowledge of relational and non-relational databases and hands-on experience with cloud data warehouse including AWS Redshift, Google BigQuery, Azure Synapse.
- A 2:1 University degree or above (or equivalent other).
- Excellent communication skills, with the ability to work collaboratively with both technical and non-technical stakeholders.
- Strong problem-solving skills, with the ability to troubleshoot complex data issues and deliver optimized solutions.
It’s useful (not a must) if you have exposure to…
- Data Science and Machine Learning.
- Tools like dbt and Dataiku.
Something to note…
Here at Snap, our values drive us. So, aligning with them is incredibly important. Our values are – be S mart, be N ice, be A ccountable, be P assionate. Be SNAP.
So, what does being SNAP look like?
- Bringing positivity and enthusiasm to the workplace
- Being dynamic and proactive, always ready to tackle new challenges
- Having a strong attention to detail, ensuring excellence in everything you do
- Having a strong desire to learn and share knowledge
All applicants are welcome!
Research shows that some may hesitate to apply unless they meet every requirement. However, your unique experience, skills, and passion are what set you apart, so, we’d love to hear from you even if you don’t hit all the skills above.
So, what’s next?
If you’ve got this far and you’re feeling excited, apply! Our hiring process is as follows…
- Initial screening with a member of our Talent Acquisition Team.
- A technical interview with one of our Principal Consultants.
- A technical task and final interview with our Delivery Director.
If you need any adjustments made to our process, let us know so we can provide the right support and ensure you can perform at your best!
#J-18808-Ljbffr
