EXL
EXL
EXL je podjetje za poslovno svetovanje in storitve, ki organizacijam pomaga uporabljati podatke, analitiko in tehnologijo za izboljšanje delovanja ter sprejemanje boljših odločitev. Njegovo delo vključuje upravljanje poslovanja, analitiko za sprejemanje odločitev in digitalno preobrazbo, pri čemer ima izkušnje s podporo strankam v zdravstvu, zavarovalništvu, financah in logistiki. EXL združuje podatkovno znanost s praktičnim poslovnim strokovnim znanjem za razvoj rešitev, prilagojenih ciljem, operativnemu modelu in panožnemu kontekstu posamezne organizacije.

Databricks Tech Lead at EXL — Chennai, Hybrid

Lead Databricks lakehouse engineering for financial crime platforms. Build Delta Lake pipelines and manage Unity Catalog across development, test, and production.

Opis

  • Build and maintain Databricks workspaces, clusters, and compute pools across development, test, and production.
  • Set up Unity Catalog governance, access controls, fine-grained permissions, and data lineage.
  • Tune cluster types, autoscaling, and spot or preemptible nodes to balance cost and performance.
  • Establish workspace structures, access controls, and secret management using Databricks Secrets, Azure Key Vault, or AWS Secrets Manager.
  • Manage Databricks jobs, workflows, and multi-task orchestration, including dependencies.
  • Design Delta Lake tables with partitioning and Z-ordering, and compact files with OPTIMIZE and VACUUM.
  • Organize structured data lakes into Bronze, Silver, and Gold layers using the Medallion Architecture.
  • Develop Delta Live Tables pipelines with declarative ETL/ELT and built-in data quality expectations.
  • Handle schema changes, table versions, time travel, and Change Data Feed for incremental processing.
  • Create lakehouse patterns that connect Delta Lake with Kafka, ADLS, S3, and GCS.
  • Build Databricks pipelines for financial crime platforms covering AML, KYC, CRA, sanctions screening, transaction monitoring, fraud detection, and regulatory reporting.

Zahteve

  • Bachelor’s or master’s degree in computer science, information technology, data engineering, or a related discipline.
  • At least 8 years of experience in data engineering or software engineering.
  • At least 3 years of hands-on Databricks experience in production.
  • Strong grounding in big data engineering, cloud data platforms, and distributed computing.
  • In-depth experience with Databricks Workspaces, Clusters, Jobs, Workflows, and Repos.
  • Proficiency in Unity Catalog, including metastore setup, catalog, schema and table management, access controls, and data lineage.
  • Practical experience developing, monitoring, and setting expectations for Delta Live Tables pipelines.
  • Strong understanding of Delta Lake internals, including its transaction log, ACID guarantees, file layout, and optimization methods.
  • Experience with Databricks SQL Warehouses, SQL Analytics, and dashboard development.
  • Knowledge of the Databricks Photon engine, serverless compute, and cost optimization.

Sorodna delovna mesta

Weekday

Senior Full-Stack Java Engineering Manager

Weekday
501 – 1000 Zaposleni
E-trgovinaMaloprodajaModa

Lead engineering teams building scalable Java and Spring Boot services with modern frontend applications. Guide architecture, reliability, mentorship, and delivery for a software development client.

Odpri
Nokia

Senior Software Engineer, Cloud-Native Platforms

Nokia

Build Nokia’s MantaRay NM cloud-native network observability platform, working across distributed systems, Kubernetes, and CI/CD automation. Apply AI-assisted engineering tools and automation to improve software development.

Odpri
Nokia

Senior Software Engineer, Cloud-Native Platforms

Nokia

Build Nokia’s MantaRay network management platform for telecommunications providers. Lead cloud-native distributed systems, Kubernetes, CI/CD automation, and AI-assisted engineering.

Odpri