The Senior Data Engineer will create and maintain data solutions, model data for stakeholders, enhance data quality, and ensure reliable data practices.
Working at Atlassian
Atlassians can choose where they work - whether in an office, from home, or a combination of the two. That way, Atlassians have more control over supporting their family, personal goals, and other priorities. We can hire people in any country where we have a legal entity. Interviews and onboarding are conducted virtually, a part of being a distributed-first company.
Team: Core Engineering Reliability Team
- Collaborate with engineering and TPM leaders, developers, and process engineers to create data solutions that extract actionable insights from incident and post-incident management data, supporting objectives of incident prevention and reducing detection, mitigation, and communication times.
- Work with diverse stakeholders to understand their needs and design data models, acquisition processes, and applications that meet those requirements.
- Add new sources, implement business rules, and generate metrics to empower product analysts and data scientists.
- Serve as the data domain expert, mastering the details of our incident management infrastructure.
- Take full ownership of problems from ambiguous requirements through rapid iterations.
- Enhance data quality by leveraging and refining internal tools and frameworks to automatically detect issues.
- Cultivate strong relationships between teams that produce data and those that build insights.
Minimum Qualifications / Your background:
- BS in Computer Science or equivalent experience with 8+ years as a Senior Data Engineer or similar role
- 10+ Years of progressive experience in building scalable datasets and reliable data engineering practices.
- Proficiency in Python, SQL, and data platforms like DataBricks
- Proficiency in relational databases and query authoring (SQL).
- Demonstrable expertise designing data models for optimal storage and retrieval to meet product and business requirements.
- Experience building and scaling experimentation practices, statistical methods, and tools in a large scale organization
- Excellence in building scalable data pipelines using Spark (SparkSQL) with Airflow scheduler/executor framework or similar scheduling tools.
- Expert experience working with AWS data services or similar Apache projects (Spark, Flink, Hive, and Kafka).
- Understanding of Data Engineering tools/frameworks and standards to improve the productivity and quality of output for Data Engineers across the team.
- Well versed in modern software development practices (Agile, TDD, CICD)
Desirable Qualifications
- Demonstrated ability to design and operate data infrastructure that deliver high reliability for our customers.
- Familiarity working with datasets like Monitoring, Observability, Performance, etc..
Top Skills
Airflow
AWS
Databricks
Flink
Hive
Kafka
Python
Spark
SQL
Similar Jobs at Atlassian
Cloud • Information Technology • Productivity • Security • Software • App development • Automation
As a Senior Data Engineer, you'll build and maintain data pipelines, work on data lakes, and enhance self-service capabilities while transforming vague requirements into actionable solutions.
Top Skills:
AirflowAWSEmrHiveJavaKinesisPythonRdsS3ScalaSparkSQLSqs
Cloud • Information Technology • Productivity • Security • Software • App development • Automation
As a Senior Data Engineer at Atlassian, you'll build and optimize data pipelines and micro-services, working with AWS-based data lakes and enhancing data accessibility for stakeholders while ensuring data quality.
Top Skills:
AirflowAWSDatabricksEmrHiveJavaKinesisPythonRdsS3ScalaSparkSQLSqs
Cloud • Information Technology • Productivity • Security • Software • App development • Automation
Join the GTM-DE team to build data lakes and optimize data pipelines, enabling efficient data ingestion and effective use of streaming technologies.
Top Skills:
AirflowAWSEmrHiveJavaKinesisPythonRdsS3ScalaSparkSQLSqs
What you need to know about the Edinburgh Tech Scene
From traditional pubs and centuries-old universities to sleek shopping malls and glass-paneled office buildings, Edinburgh's architecture reflects its unique blend of history and modernity. But the fusion of past and future isn't just visible in its buildings; it's also shaping the city's economy. Named the United Kingdom's leading technology ecosystem outside of London, Edinburgh plays host to major global companies like Apple and Adobe, as well as a growing number of innovative startups in fields like cybersecurity, finance and healthcare.
