As a Senior Data Engineer, you will develop and optimize data pipelines for a new Consumer Data Lake, ensuring data accuracy and compliance. You will collaborate with architects and product owners to translate design into functionality while monitoring data quality and implementing security measures.
Join an exciting journey to create a greenfield, cutting-edge Consumer Data Lake for a leading global organization based in Europe. This platform will unify, process, and leverage consumer data from various systems, unlocking advanced analytics, insights, and personalization opportunities. As a Senior Data Engineer, you will play a pivotal role in shaping and implementing the platform's architecture, focusing on hands-on technical execution and collaboration with cross-functional teams.
Your work will transform consumer data into actionable insights and personalization on a global scale. Using advanced tools to tackle complex challenges, you’ll innovate within a collaborative environment alongside skilled architects, engineers, and leaders.
Key Responsibilities:
- Hands-On Development: Build, maintain, and optimize data pipelines for ingestion, transformation, and activation.
- Create and implement scalable solutions to handle diverse data sources and high volumes of information
- Data Modeling & Warehousing: Design and maintain efficient data models and schemas for a cloud-based data platform.
- Develop pipelines to ensure data accuracy, integrity, and accessibility for downstream analytics.
- Collaboration: Partner with Solution Architects to translate high-level designs into detailed implementation plans.
- Work closely with Technical Product Owners to align data solutions with business needs.
- Collaborate with global teams to integrate data from diverse platforms, ensuring scalability, security, and accuracy.
- Platform Development: Enable data readiness for advanced analytics, reporting, and segmentation.
- Implement robust frameworks to monitor data quality, accuracy, and performance.
- Testing & Quality Assurance: Implement robust security measures to protect sensitive consumer data at every stage of the pipeline
- Ensure compliance with data privacy regulations (e.g., GDPR, CCPA ..) and internal policies.
- Monitor and address potential vulnerabilities, ensuring the platform adheres to security best practices.
Requirements:
- Over 4+ years of experience showcasing technical expertise and critical thinking in data engineering.
- Hands-on experience with DBT and strong Python programming skills.
- Proficiency in Snowflake and expertise in data modeling are essential.
- Demonstrated experience in building consumer data lakes and developing consumer analytics capabilities is required.
- In-depth understanding of privacy and security engineering within Snowflake, including concepts like RBAC, dynamic/tag-based data masking, row-level security/access policies, and secure views.
- Ability to design, implement, and promote advanced solution patterns and standards for solving complex challenges.
- Familiarity with multiple cloud platforms (Azure or GCP preferred, with a focus on Azure).
- Practical experience with Big Data batch and streaming tools.
- Competence in SQL, NoSQL, relational database design (SAP HANA experience is a bonus), and efficient methods for data retrieval and preparation at scale.
- Proven ability to collect and process raw data at scale, including scripting, web scraping, API integration, and SQL querying.
- Experience working in global environments and collaborating with virtual teams.
- A Bachelor’s or Master’s degree in Data Science, Computer Science, Economics, or a related discipline.
We offer:
- Flexible working format - remote, office-based or flexible
- A competitive salary and good compensation package
- Personalized career growth
- Professional development tools (mentorship program, tech talks and trainings, centers of excellence, and more)
- Active tech communities with regular knowledge sharing
- Education reimbursement
- Memorable anniversary presents
- Corporate events and team buildings
- Other location-specific benefits
Top Skills
Azure
Dbt
GCP
NoSQL
Python
Sap Hana
Snowflake
SQL
Similar Jobs
Healthtech • Information Technology • Software • Analytics • Consulting • Cybersecurity • App development
As a Senior Data Engineer, you will design and implement databases and pipelines, develop end-to-end solutions for data processing, and work with stakeholders to ensure data security and usability. You'll handle scalable solutions for data and machine learning throughout the data lifecycle.
Top Skills:
AWSData EngineeringDatabricksEnergy IndustryETLGeospatial DataPythonSQL
Artificial Intelligence
As a Sr. Data Engineer, you will optimize data processing pipelines for audio initiatives, design and maintain data workflows, and collaborate closely with the TTS team to enhance data quality for AI model training within the Research Data Team.
Top Skills:
AWSAzureCi/CdData PipelinesGCPPythonWorkflow Orchestrators
Artificial Intelligence
As a Data Platform Engineer, you will support the ML Platform Team in developing a high-scale compute and data platform, provision and expose data for researchers, implement workflow orchestrators, and ensure ML engineering best practices are followed.
Top Skills:
AirflowArgoAWSAzureCi/CdDagsterData PipelinesGCPKubeflowKubernetesPython
What you need to know about the Edinburgh Tech Scene
From traditional pubs and centuries-old universities to sleek shopping malls and glass-paneled office buildings, Edinburgh's architecture reflects its unique blend of history and modernity. But the fusion of past and future isn't just visible in its buildings; it's also shaping the city's economy. Named the United Kingdom's leading technology ecosystem outside of London, Edinburgh plays host to major global companies like Apple and Adobe, as well as a growing number of innovative startups in fields like cybersecurity, finance and healthcare.
