Sigma Software Group
Sigma Software Group
1,001 – 5,000 Employees
AutomotiveConsultingHealthcare
Sigma Software Group is a multinational technology consulting company founded in 2002. Its teams provide software development, graphic design, testing, and technical support for clients in sectors including automotive, healthcare, telecommunications, aviation, banking, gaming, real estate, and advertising. The company hires IT specialists to work on complex engineering projects and supports professional development through mentoring and continuous learning. Remote work opportunities are part of its working model, with projects serving organizations such as AstraZeneca, Scania, and SAS.

Senior Data Engineer, Databricks Migration

Migrate a large-scale retail analytics platform from BigQuery to Databricks, building Lakehouse architectures, Spark pipelines, and governed data solutions at Sigma Software Group.

Description

  • Help migrate a large-scale analytics platform from BigQuery to Databricks.
  • Design and build scalable Lakehouse architectures with Databricks and Delta Lake.
  • Assess existing ETL and ELT workloads and define migration plans.
  • Develop and tune pipelines for high volumes of retail and analytical data.
  • Create incremental processing strategies and scalable transformation frameworks.
  • Build and maintain Spark data processing solutions with PySpark.
  • Design and maintain Bronze, Silver, and Gold layers in the medallion architecture.
  • Apply data governance and security practices with Unity Catalog.
  • Work with Data Science, Analytics, Product, and Customer Engineering teams.
  • Contribute to architecture discussions and technical solution design.
  • Create reusable data platform components and engineering standards.
  • Review code and improve platform reliability and maintainability.
  • Diagnose and optimize complex SQL and Spark workloads.
  • Support production releases and platform modernization.

Requirements

  • At least five years of professional data engineering experience.
  • Strong Python skills and advanced SQL proficiency.
  • Commercial, hands-on experience with Databricks.
  • Strong Apache Spark knowledge, especially PySpark.
  • Experience designing and building modern cloud data platforms.
  • Experience developing ETL or ELT pipelines and large-scale data processing solutions.
  • Hands-on experience with Delta Lake.
  • Experience with Spark Declarative Pipelines.
  • Experience monitoring clusters, analyzing metrics, and optimizing performance.
  • Strong understanding of distributed data processing architectures.
  • Sound knowledge of data warehousing and dimensional modeling.
  • Experience with Airflow or a similar orchestration tool.
  • Experience tuning complex analytical SQL workloads.
  • Experience implementing CI/CD practices for data engineering platforms.
  • Strong troubleshooting and performance optimization skills.
  • Ability to collaborate with cross-functional teams across international settings.
  • Upper-intermediate English or higher.

Benefits

  • Ongoing opportunities to learn.
  • Opportunities to grow your technology expertise.
  • Meaningful impact through engineering work.
  • Work on complex international projects.

Related Jobs

Jerry

Reddit Social Media Manager at Jerry

Jerry

Build authentic Reddit communities for Jerry’s AI-powered car ownership platform through useful content, reputation management, and community engagement. Turn conversations and car-owner insights into guidance for product and content teams.

Open