Top Data Engineer Jobs in Edinburgh

Own the design, delivery, and operation of complex data pipelines, datasets, and platform components. Translate business needs into scalable data models and reliable data products, improve data quality and observability, resolve production issues, and establish reusable engineering tools and standards. Collaborate with product and business stakeholders, contribute to architecture decisions, mentor engineers, and participate in on-call support.

Operate and architect large-scale distributed data infrastructure across Kafka, Hadoop HDFS, Ceph, SQL Server, bare-metal systems, and hybrid cloud environments. Responsibilities include capacity planning, reliability engineering, automation, observability, incident response, backup and recovery, platform tooling, and architecture decisions. The role requires expertise with petabyte-scale systems or billions of daily events, mentoring engineers, and shaping infrastructure standards across engineering organizations.

Serve as the technical authority for enterprise customers in data security, owning the technical sales process. Deliver technical presentations, demos, and whiteboard sessions, advise on AI/ML-driven data protection, and drive outcomes through value-based selling with partners and GTM teams.

As a Data Engineer, you will design, build, and maintain data pipelines and infrastructure to support data-driven decision-making in a collaborative environment.

The Data Engineer will design and support data warehouses, create analytics dashboards, perform data quality analysis, and identify process improvements. Responsibilities include collaborating with stakeholders to develop insights and ensure data governance.

The Data Engineer will build and manage scalable data pipelines and databases, collaborate on architecture, and drive data systems towards real-time capabilities in a fast-paced, innovative environment.

Support ledger migration and data lake cleanup by extracting, transforming, reconciling and validating large datasets. Develop and validate SQL queries, batch jobs and processing logic, perform regression testing, document mappings and validation rules, and collaborate with engineering, product and business teams to ensure production readiness and post-migration validation.

Design, build, and maintain scalable batch and streaming data pipelines, internal tools, and APIs to support analytics, product features, and ML. Collaborate with product and engineering to improve data infrastructure, ensure reliability, testing, and observability in a cloud-native environment.

The Data Engineer will build and scale data pipelines, ensure data quality, implement dbt best practices, and empower cross-functional teams with self-service data access.

Build large-scale data pipelines and infrastructure for frontier AI model training. Develop data-processing models such as classifiers, quality filters, and labeling systems; design deduplication, quality scoring, labeling, and augmentation strategies; and create reliable tooling for researchers to explore and train on massive datasets. The role also evaluates how data quality and composition affect model outcomes, with web crawler experience as a bonus.

Migrate SQL Server data products and analytical workloads to a modern cloud platform. Redesign legacy SQL, ETL processes, and business logic; build scalable pipelines and analytical models; validate data quality; collaborate with stakeholders; and improve platform reliability, performance, cost efficiency, and AI-assisted data engineering automation.

Design, build and maintain scalable, automated cloud data solutions and pipelines. Provide technical leadership to a small engineering squad, own complex backlog tasks, develop data frameworks and standards, and ensure code quality via reviews and pair programming using Databricks, Python, SQL and Spark.

Design and maintain reliable data solutions, lead a squad of engineers, develop scalable data pipelines, and ensure adherence to data standards.

Design, deploy, and maintain AWS and Databricks data infrastructure, data lakes, and scalable ETL pipelines. Build custom ingestion services and connectors using Airbyte, Go, and Python; optimize performance, reliability, monitoring, and data quality. Implement cloud data governance, security, access controls, encryption, and compliance practices while collaborating with platform engineers, analysts, and stakeholders.

Build, optimise and stabilise AWS-based data pipelines (S3, Glue, Redshift, Lake Formation). Resolve technical debt, support the Architect on target-state design, collaborate with client engineering teams to transfer knowledge, and deliver production-ready platforms in regulated, high-volume gaming environments.

Build and operate Python-based data ingestion, ETL, detection, and forensic artifact extraction pipelines for a mobile EDR platform. Productionize threat detection prototypes, optimize large-scale data storage and queries, deploy services on AWS, and establish monitoring, validation, fault tolerance, and data quality controls. Collaborate closely with security researchers to develop reliable tooling and scalable systems for identifying mobile spyware and other threats.

Design, build, and scale cloud-native data warehouses and robust ETL/ELT pipelines (primarily BigQuery). Optimize performance, implement partitioning and cost controls, ensure data governance, observability, and regulatory reporting support, and collaborate with analytics and engineering stakeholders.

Own and operate Databricks production data pipelines for a predictive CRM platform: monitoring, incident response, cost and performance optimization, migrating legacy ETL, onboarding customers, enforcing Delta Lake architecture and multi-tenant isolation, and applying CI/CD, security, and observability best practices to ensure reliable, scalable data delivery.

Triage and resolve Master Data Management (Party, Risk, Reference) issues in Stibo STEP, run targeted data-cleansing sprints, monitor data quality dashboards, support releases/UAT, and maintain documentation and playbooks to prevent recurring data issues.

As a Python and Kubernetes Software Engineer, you will develop data analytics solutions using open source tools, collaborating with a distributed team.

Build and maintain production data pipelines and features in Python; own project lifecycle from design to reliable production; set technical direction for the Insights platform; improve engineering standards (testing, observability, data quality); apply AI across products and pipelines; help establish semantic/AI-ready data layers.

Design, build, and maintain scalable batch and streaming data pipelines. Write and review code (Python, SQL, Spark), implement data quality checks, monitor pipelines, and collaborate with analysts, data scientists, and product managers to deliver reliable, documented data solutions.

Design, build, test and document secure batch and real-time data pipelines, perform logical/physical database design and data modelling, expand platform capabilities, analyse and resolve data processing issues, and ensure alignment with data architecture for downstream consumers.

Build, test, and document secure batch and real-time data pipelines for a government client; design logical and physical databases; extend platform capabilities; ensure data models align with architecture; perform data analysis and root-cause investigations to improve processing and availability for downstream consumers.

Build and operate shared AI platform infrastructure for LLM applications, agents, RAG, tool calling, and AI workflows. Develop data and knowledge pipelines, secure enterprise integrations, reusable frameworks, deployment and observability standards, and developer tooling. Take AI systems from experimentation to reliable production while ensuring security, privacy, governance, scalability, quality, and cost efficiency. Collaborate across Engineering, Product, and IT to establish practical AI platform patterns and guardrails.