Data Engineer

JOB SUMMARY

As a Data Engineer, you will design, develop, and maintain the data architecture that enables data management and integration across cloud and on-prem environments using Microsoft technologies. You will work closely with other data engineers, analysts, and stakeholders to build and optimise data pipelines and workflows to support business objectives, ensuring that data flows seamlessly across systems, applications, and services.

KEY RESPONSIBILITES 

• Microsoft Data Fabric Implementation:

Design, implement, and manage Microsoft Data Fabric architecture (leveraging Azure Data Services, Power BI, and other MS tools) to integrate and manage data across hybrid cloud and on-prem environments. Create, manage, and optimise data pipelines to ensure the smooth flow of data from source to destination systems.

• Data Integration & Automation:

Develop and implement data pipelines and ETL/ELT processes, ensuring automation and scalability. Work on integrating data from multiple sources (including SQL, NoSQL, APIs, flat files) into the unified Microsoft Data Fabric.

• Data Management & Quality:

Work on implementing data governance practices across the data fabric to ensure data consistency, quality, and security. Monitor and optimise data flows to ensure the high availability and reliability of the data infrastructure.

• Collaboration & Support:

Collaborate with data scientists, analysts, and business intelligence teams to ensure data is optimised and easily accessible for analytical and reporting purposes. Assist in troubleshooting data issues, supporting downstream applications, and improving data usability for business users.

• Performance Monitoring & Optimisation:

Continuously monitor and optimise data pipelines, ensuring they meet performance and scalability requirements. Troubleshoot and resolve issues related to data processing, ingestion, and transformation.

KEY COMPETENCIES & SKILLS

• 3 years’ experience in data engineering.
• Experience in implementing and managing data integration, ETL/ELT pipelines.
• Hands-on experience in building data architectures with Microsoft SQL Server, Azure SQL Database, and other related MS tools.
• Strong proficiency in SQL and PYTHON.
• Experience in building, managing, and optimising cloud-based data pipelines.
• Experience with Power BI for data visualisation and reporting is a plus.
• Familiarity with data security practices and data privacy regulations in the context of data management.

KEY BUSINESS PARTNERS

      Internal directors.

      Group directors.

      Management Team.

      Operational teams including; regulated business, accounting, company secretarial and compliance teams.