Allata
Allata
Allata este o firmă globală de consultanță care ajută organizațiile să transforme strategia digitală în îmbunătățiri concrete ale activității. Echipele sale lucrează în domenii precum strategia, experiența clienților, cloud și tehnologie, date și inteligență artificială, sprijinind clienții în modernizare, personalizare și eficiență operațională. Allata deservește sectoare precum industria auto, sănătatea, logistica, serviciile financiare și tehnologia avansată, valorificând experiența foștilor lideri IT și a profesioniștilor în consultanță pentru a ghida inițiative digitale complexe.

Databricks Data Platform Manager (Remote, Texas)

Lead architecture and delivery of governed Lakehouse and enterprise data warehouse platforms for Allata. Guide data pipelines, security, scalability, and data product development.

Descriere

  • Set architecture, technology choices, reference patterns, and integration blueprints for Lakehouse and enterprise data warehouse platforms.
  • Create conceptual, logical, and physical models for multi-tenant and industry-specific data products.
  • Standardize data layers for ingestion, raw storage, curation, and serving.
  • Establish governance practices for metadata, catalogs, lineage, data contracts, and classification.
  • Define security and compliance controls for access, masking, encryption, network segmentation, and auditing.
  • Plan for platform scalability, high availability, disaster recovery, capacity, and cost management.
  • Lead selection and integration of Databricks, Delta Lake, Delta Live Tables, Fivetran, Azure Data Factory or Data Fabric, orchestration, and monitoring tools.
  • Set and enforce CI/CD practices for data assets, including tests, automated releases, and rollback plans.
  • Define batch and streaming ingestion, compaction, partitioning, and storage approaches.
  • Set observability standards for metrics, SLAs, dashboards, logs, traces, and alerts.
  • Provide technical leadership and mentorship to Data Engineering teams.
  • Review architecture and code for critical components.
  • Work with Data Product Owners and Security, Infrastructure, BI, and ML stakeholders to turn requirements into solutions and roadmaps.
  • Build, test, and deploy processing modules with Spark, PySpark or Scala, Spark SQL, and stored procedures.
  • Develop and tune data pipelines on Databricks and database platforms.
  • Apply DevOps practices, infrastructure as code, CI/CD, automated testing, and version control.
  • Resolve complex data quality, performance, and availability problems, and drive ongoing improvements.

Cerințe

  • Prior experience as an architect or technical lead for enterprise data platforms.
  • Hands-on knowledge of Databricks technologies, including Delta Lake, Unity Catalog, Delta Live Tables, Auto Loader, and Structured Streaming.
  • Strong skills in Spark, PySpark and/or Scala, Spark SQL, and distributed job optimization.
  • Background in data warehouse and Lakehouse design.
  • Practical knowledge of Medallion, Lambda, and Kappa patterns.
  • Experience integrating SaaS, ETL, or connector tools such as Fivetran.
  • Experience with orchestration tools such as Airflow, Azure Data Factory, and Data Fabric.
  • Experience with relational and hybrid databases, including MS SQL Server, PostgreSQL, Oracle, Azure SQL, and AWS RDS/Aurora or equivalents.
  • Proficiency in data pipeline CI/CD using Azure DevOps, GitHub Actions, Jenkins, or similar tools.
  • Experience packaging and deploying artifacts such as .whl files and containers.
  • Experience with batch and streaming processing, file compaction, partitioning, and storage tuning.
  • Understanding of cloud security, IAM/RBAC, encryption, VPC/VNet concepts, and cloud networking.
  • Familiarity with observability and monitoring tools such as Prometheus, Grafana, Datadog, or native cloud services.
  • Automation experience with CI/CD pipelines and trunk-based development is a plus.
  • Advanced PySpark skills are a plus.
  • Advanced Spark workflow optimization and orchestration skills are a plus.
  • Databricks, cloud architecture, or data certifications are a plus.
  • Ability to identify, troubleshoot, and resolve complex data problems.
  • Strong collaboration and communication skills.
  • Curiosity and openness to new tools, technologies, and methods.

Beneficii

  • Full-time role.
  • Equal opportunity and inclusive work environment.
  • Commitment to diversity.

Locuri de muncă similare

Lunar Solar Group

Marketing Measurement Data Scientist (Remote, US)

Lunar Solar Group

Build attribution, incrementality, and media mix measurement for Lunar Solar Group, a growth consultancy and digital marketing agency. Turn analysis into practical client budget recommendations.

Deschide
Airbnb

Senior Data Scientist, People Analytics Research

Airbnb

Lead people analytics research at Airbnb, using causal inference and statistical modeling to inform workforce, compensation, recruiting, and retention decisions. Work with stakeholders on sensitive analyses and reporting.

Deschide
The Cigna Group

Senior Lead Data Builder, Remote (US)

The Cigna Group

Build reusable Silver and Gold data products for Cigna healthcare services, using Databricks and governed MVP and CI/CD pipelines. Turn business needs into production-ready data solutions and establish standards across Data Operations.

Deschide