The Codest
The Codest
The Codest is an international software development company with its headquarters in the United Kingdom and technology hubs in Poland. It builds digital products and extends client engineering teams with experienced developers across backend and frontend development, cloud engineering, and data engineering. The company works with enterprises, scale-ups, and software vendors through project development and staff augmentation, including clients in fintech, e-commerce, and adtech.

Senior Databricks Data Engineer

Build Databricks platforms and data pipelines remotely from Poland. Contribute to a customer-facing banking app for a major Japanese financial group.

विवरण

  • Build and maintain scalable Databricks data platforms and pipelines.
  • Partner with cross-functional teams to support reliable, accessible data for analytics and business intelligence.
  • Use Bronze, Silver, and Gold lakehouse layers to implement Medallion Architecture.
  • Create data quality frameworks with automated tests and monitoring.
  • Use Delta Lake time travel, vacuum operations, and Z-ordering.
  • Develop and maintain complex ETL/ELT pipelines for large daily datasets.
  • Set up CI/CD workflows for data pipelines with Databricks Asset Bundles or comparable tools.
  • Build batch and real-time processing solutions with Structured Streaming and Delta Live Tables.
  • Improve Spark job performance and cost efficiency through cluster auto-scaling and resource management.
  • Integrate Databricks APIs with external systems.
  • Design scalable data architectures with Unity Catalog, Delta Lake, and Apache Spark.
  • Build data mesh architectures with governance and lineage tracking.
  • Contribute to a customer-facing banking application for a leading financial group in Japan.

आवश्यकताएँ

  • Bring at least six years of data engineering experience.
  • Have a strong record of designing and deploying production-grade data pipelines and ETL/ELT workflows.
  • Be proficient in Python, PySpark, and SQL.
  • Have extensive hands-on experience with AWS Glue, Lambda, Redshift, and S3.
  • Know data governance and management tools, including Unity Catalog and AWS SageMaker Unified Studio.
  • Have experience with data migrations and modernizing legacy systems.
  • Communicate effectively in English.
  • Databricks Certified Professional Data Engineer certification or an equivalent is a plus.
  • Experience with containerization and orchestration tools such as Docker and Kubernetes is an advantage.

लाभ

  • Work in a collaborative, engaging environment.
  • Access opportunities for professional growth.
  • Contribute to an international banking app project.
  • Work remotely.

संबंधित नौकरियाँ

Recom

Remote Creative Director at Recom

Recom

Lead Recom’s B2B brand across campaigns, sales materials, events, and digital channels. Guide designers and creative partners as the Amazon accelerator grows.

खोलें
GoLean Health

Medical Virtual Assistant for a Remote Chiropractic Clinic

GoLean Health

Support a remote chiropractic clinic with insurance verification, claims, billing, and clinical documentation. Track operating metrics and coordinate priorities across the provider and care team.

खोलें
GoLean Health

Medical Virtual Assistant for a Remote Chiropractic Clinic

GoLean Health

Support a remote chiropractic clinic with insurance, claims, billing, and patient documentation. Track operational metrics and coordinate work across the provider, care team, and administrative staff.

खोलें
GoLean Health

Remote Medical Virtual Assistant (MVA)

GoLean Health

Support a Caracas chiropractic clinic remotely with insurance verification, claims, EOBs, and billing workflows. Monitor documentation, appointments, payments, and operational reporting.

खोलें
GoLean Health

Medical Virtual Assistant

GoLean Health

Support a remote chiropractic clinic with insurance verification, claims, billing, and clinical documentation. Prepare operational reports and coordinate work across the provider and care team.

खोलें
GoLean Health

Remote Medical Virtual Assistant (Philippines)

GoLean Health

Support a U.S. chiropractic clinic with insurance, claims, billing, and documentation operations. Track clinic metrics and coordinate priorities across the care team.

खोलें
GoLean Health

Medical Virtual Assistant (Peru, Remote)

GoLean Health

Support a remote U.S. chiropractic clinic by managing insurance verification, claims, EOBs, patient balances, documentation, and operational reporting.

खोलें
GoLean Health

Medical Virtual Assistant for a U.S. Chiropractic Clinic

GoLean Health

Support a U.S. chiropractic clinic remotely with insurance verification, claims, billing, and clinical documentation. Track operational metrics and coordinate work across the provider and care team.

खोलें
GoLean Health

Medical Virtual Assistant (Colombia, Remote)

GoLean Health

Support a US chiropractic clinic remotely with insurance verification, claims, EOBs, patient accounts, documentation, and operational reporting.

खोलें
GoLean Health

Remote Medical Virtual Assistant (Peru)

GoLean Health

Support a chiropractic clinic remotely with insurance verification, claims, billing, and documentation. Prepare operational reports and coordinate work across the provider, care team, and administrative staff.

खोलें