Keep IT Simple
Keep IT Simple
Keep IT Simple (KIS) este un furnizor de servicii și soluții IT cu sediul în Silicon Valley, care deservește organizații din California și nu numai. Înființată în 1988, compania își ajută clienții să răspundă unor nevoi tehnologice complexe prin servicii de securitate cibernetică, virtualizare, soluții cloud și consultanță pentru infrastructura de rețea. Activitatea sa acoperă domenii precum logistica, sănătatea și securitatea cibernetică, cu accent pe îndrumare IT practică, soluții eficiente din punct de vedere al costurilor și servicii prompte pentru clienți.

Senior Databricks Data Engineer, AI Platforms

Build Databricks and cloud data platforms for a global insurance and asset management company. Support governed analytics, machine learning, and generative AI solutions.

Descriere

  • Build and maintain scalable data pipelines and products with Databricks, Spark, Python, and SQL.
  • Create and improve batch, streaming, and real-time integrations across enterprise and third-party sources.
  • Implement Databricks Lakehouse architectures with Delta Lake and Medallion patterns.
  • Deliver data products for analytics, predictive modeling, machine learning, and generative AI.
  • Work with data scientists and AI engineers to prepare, transform, and govern data for AI and ML use cases.
  • Create feature engineering pipelines and support machine learning lifecycle processes.
  • Build data solutions for RAG, vector and semantic search, and LLM applications.
  • Improve Spark jobs, SQL workloads, and data frameworks for performance, scale, and cost efficiency.
  • Establish data quality, observability, lineage, governance, and monitoring capabilities.
  • Meet data privacy, security, and responsible AI standards.
  • Support CI/CD, Infrastructure as Code, and DataOps across the data platform.
  • Guide junior engineers and reinforce sound engineering practices.
  • Partner with data scientists, AI engineers, architects, and business stakeholders to deliver trusted, governed data products.
  • Support enterprise AI and generative AI initiatives with robust, secure, governed data platforms.

Cerințe

  • Bachelor’s degree in computer science, information systems, engineering, mathematics, or a related discipline.
  • At least 7 years in data engineering, ETL development, or large-scale data platforms.
  • At least 3 years of practical experience with Databricks and Apache Spark.
  • Strong Python and SQL skills, with experience in distributed data processing.
  • Background building cloud data lakes, warehouses, and Lakehouse architectures.
  • Experience supporting AI, machine learning, or advanced analytics initiatives.
  • Knowledge of data modeling, governance, and enterprise data management.
  • Experience building and tuning large-scale pipelines on AWS, Azure, or Google Cloud.
  • Experience with Databricks Delta Lake, Unity Catalog, Delta Live Tables, MLflow, Mosaic AI, and Databricks Workflows.
  • Practical experience with generative AI, large language models, RAG, vector databases, or AI-powered applications.
  • Familiarity with LangChain, Semantic Kernel, OpenAI APIs, Hugging Face, or comparable tools.
  • Experience with feature stores, model training pipelines, and MLOps.
  • Experience with Kafka, Event Hub, or other streaming technologies.
  • Databricks Certified Data Engineer Professional or an equivalent cloud certification is preferred.
  • Expertise in Databricks platform engineering.
  • Apache Spark development experience.
  • AI and machine learning data engineering expertise.
  • Generative AI data solutions experience.
  • Lakehouse architecture experience.
  • Data modeling and data warehousing expertise.
  • Data governance and security expertise.
  • Data observability and reliability engineering experience.
  • Cloud data platform experience with AWS or Azure.
  • DataOps, CI/CD, and automation experience.
  • Technical leadership and mentoring experience.

Beneficii

  • Engagement as an independent contractor (PJ).
  • Hybrid schedule with three days per week in person at the Pinheiros, São Paulo office.

Locuri de muncă similare

Kreato Global | BPO and Language Solutions

Remote English-Spanish OPI/VRI Interpreter

Kreato Global | BPO and Language Solutions

Interpret remotely between English- and Spanish-speaking people in medical, financial, social service, and customer care settings. Provide language support for Kreato Global across Latin America.

Deschide