Top Data Engineer Jobs in Edinburgh

Designs, builds, and maintains production data pipelines using Microsoft Fabric, Python, and PySpark. Owns architecture, data modeling, CI/CD, reliability, troubleshooting, and technical decision-making. Collaborates with architects, product teams, and clients to translate requirements into robust solutions while mentoring junior engineers and improving engineering practices.

Designs and manages research data pipelines and scalable infrastructure across clinical studies, sensor platforms, and HPC environments. Owns the data lifecycle, develops ETL/ELT and data-quality tools, maintains storage and security systems, and establishes data architecture standards. Collaborates with AI engineers and researchers to support machine learning, signal processing, and clinical validation applications. The role requires strong Python, SQL, database, automation, and CI/CD expertise, with medical device or algorithm experience preferred.

Build large-scale data pipelines and infrastructure for frontier AI model training. Develop data-processing models such as classifiers, quality filters, and labeling systems; design deduplication, quality scoring, labeling, and augmentation strategies; and create reliable tooling for researchers to explore and train on massive datasets. The role also evaluates how data quality and composition affect model outcomes, with web crawler experience as a bonus.

Build, maintain, and release reliable cloud software for the Customer Data Platform. Lead complex feature delivery, make architectural decisions, improve code quality, testing, observability, security, and operational reliability. Collaborate with Product, Design, and adjacent engineering teams, participate in on-call support, contribute to planning and OKRs, and mentor engineers through pairing, reviews, and technical guidance.

Design, build, deploy, and optimize scalable AI and machine learning systems using large datasets. Develop predictive models, recommendation systems, personalization engines, forecasting solutions, generative AI applications, and production ML pipelines. Collaborate with data, backend, and product teams to integrate models into applications, while monitoring performance, data quality, drift, reliability, and cost. Establish best practices for evaluation, experimentation, reproducibility, and deployment, and research emerging AI technologies.

Own production Python data pipelines, ETL/ELT workflows, and the ontology powering an AI platform. Integrate warehouses and business applications with graph databases and vector stores, while managing deployment tooling, cloud infrastructure, monitoring, and security. The role begins as hands-on individual contribution and develops toward technical leadership through mentoring and architecture ownership.

Build and operate scalable cloud data pipelines, warehouses, transformation layers, and ML production systems. Lead data infrastructure architecture, warehouse performance optimization, DataOps monitoring, CI/CD improvements, data quality testing, and large-scale data product delivery. Collaborate across teams, promote data stewardship, and support batch and streaming pipelines while deploying, serving, and monitoring machine learning models.

Build end-to-end data applications, APIs, internal tools, and user interfaces using AI-assisted development. Work across Python backends, React frontends, data models, Databricks, dbt, pipelines, cloud infrastructure, testing, security, and performance. Review AI-generated code, establish guardrails, improve engineering workflows, and help shape application architecture and technical direction.

Build, scale, and operate Databricks and AWS infrastructure supporting automated, reproducible machine learning workflows. The role includes infrastructure-as-code with Terraform or Pulumi, containerized platforms using Docker and Kubernetes, system architecture, reliability and cost optimization, and cross-functional collaboration with data and ML teams. The engineer will lead complex projects, improve platform scalability and usability, document systems, and mentor peers while operating production-scale data and ML platforms.

Design, build, and maintain scalable data pipelines and platform services to support analytics, ML, and product teams. Implement infrastructure-as-code (Terraform) and Kubernetes-based deployments, develop data services/APIs, and ensure data quality, observability, security, and compliance. Partner with cross-functional teams, document systems, and continuously improve performance and reliability.

Design, develop, and support scalable cloud-based data solutions across AWS and Azure. Build ETL/ELT pipelines, data integrations, migration processes, warehouses, data lakes, and lakehouses. Perform data profiling, quality assessments, validation, reconciliation, troubleshooting, and root-cause analysis. Create source-to-target mappings and technical documentation while supporting data governance, security, and compliance. Collaborate with technical and business stakeholders to deliver secure, high-quality data solutions.

Lead the design, deployment, and operation of scalable data pipelines and cloud infrastructure for Kpler’s marine insurance initiative. Build containerized AWS services using Python, Docker, Kubernetes, Airflow, and Terraform; manage PostgreSQL and Kafka/Flink streaming systems; establish observability standards; provide technical direction; collaborate cross-functionally; and mentor engineers.

Build and launch scalable data pipelines, products, APIs, and tools for proprietary market research. Develop big data processing systems, reusable software components, AI-ready data assets, cloud infrastructure, and governance controls. Partner with investment professionals, researchers, data scientists, and Compliance to deliver reliable, auditable, compliant solutions while optimizing system performance, cost, and reliability.

Designs, develops, and supports secure cloud-based data integration solutions using Microsoft Fabric and Azure. Responsibilities include client discovery, requirements gathering, ETL and ELT development, REST API and SFTP integrations, data pipeline implementation, production monitoring, incident investigation, and reliability improvements. The role collaborates with architects, analysts, project managers, and delivery teams across multiple client engagements.

Leads the design, development, testing, integration, and optimization of scalable data-processing and persistence components. Establishes engineering and operational standards, ensures pipeline performance and compliance, troubleshoots systems, supports proposals, and collaborates with architects, operations, security teams, customers, and managers. Coaches and develops engineers while overseeing code quality, data models, testing, deployment readiness, and technical delivery.

Leads electrical design for hyperscale and mission-critical data centers across Europe. Responsibilities include managing HV/MV/LV systems, substations, utilities, consultants, permitting, project delivery, construction support, commissioning, and integrated systems testing. The role provides client-side technical leadership, reviews designs and project documents, coordinates multidisciplinary teams, manages risks and milestones, and ensures compliance with standards and operational requirements.

Build and evolve CluePoints’ enterprise data platform using Microsoft Fabric and Azure. Responsibilities include migrating Azure SQL and ADF workloads, integrating SaaS and web data, developing scalable ingestion and transformation pipelines, maintaining Kimball dimensional models, and establishing CI/CD, data quality, lineage, security, and monitoring. The role partners with analytics leadership and supports trusted Power BI reporting across multiple business functions.

Designs and delivers scalable data pipelines and Lakehouse architectures using Databricks, Spark, PySpark, and Delta Lake. The role covers ETL/ELT, data governance, security, quality, performance optimization, cloud integration, and CI/CD. It involves technical leadership across customer programmes, architectural decision-making, stakeholder collaboration, mentoring, and supporting reliable production data solutions.

Design, build, and optimize cloud-based data platform components: ingest and transform varied datasets, create metadata-driven pipelines and data APIs, support AI/ML infrastructure, implement monitoring and metrics, and improve data processes to enable analytics and operational systems.

Own the end-to-end design, reliability, resilience, and recovery of production data ingestion systems and pipelines. Build batch and streaming ETL/ELT workflows, establish data quality checks and SLAs, improve monitoring and alerting, resolve platform issues, reduce technical debt, and optimize performance and cost. Collaborate with stakeholders, AI teams, and international expansion initiatives while supporting time-series, textual, and event-stream data workloads.

Own the design, delivery, and operation of complex data pipelines, datasets, and platform components. Translate business needs into scalable data models and reliable data products, improve data quality and observability, resolve production issues, and establish reusable engineering tools and standards. Collaborate with product and business stakeholders, contribute to architecture decisions, mentor engineers, and participate in on-call support.

Operate and architect large-scale distributed data infrastructure across Kafka, Hadoop HDFS, Ceph, SQL Server, bare-metal systems, and hybrid cloud environments. Responsibilities include capacity planning, reliability engineering, automation, observability, incident response, backup and recovery, platform tooling, and architecture decisions. The role requires expertise with petabyte-scale systems or billions of daily events, mentoring engineers, and shaping infrastructure standards across engineering organizations.

As a Data Engineer, you will design, build, and maintain data pipelines and infrastructure to support data-driven decision-making in a collaborative environment.

The Data Engineer will design and support data warehouses, create analytics dashboards, perform data quality analysis, and identify process improvements. Responsibilities include collaborating with stakeholders to develop insights and ensure data governance.

Support ledger migration and data lake cleanup by extracting, transforming, reconciling and validating large datasets. Develop and validate SQL queries, batch jobs and processing logic, perform regression testing, document mappings and validation rules, and collaborate with engineering, product and business teams to ensure production readiness and post-migration validation.