Keep IT Simple
Keep IT Simple
Keep IT Simple (KIS) is a Silicon Valley-based IT services and solutions provider serving organizations across California and beyond. Established in 1988, the company helps clients address complex technology needs through cybersecurity, virtualization, cloud solutions, and network infrastructure consulting. Its work spans industries including logistics, healthcare, and cybersecurity, with an emphasis on practical IT guidance, cost-conscious solutions, and responsive customer service.

Senior Databricks Data Engineer, AI Platforms

Build Databricks and cloud data platforms for a global insurance and asset management company. Support governed analytics, machine learning, and generative AI solutions.

Description

  • Build and maintain scalable data pipelines and products with Databricks, Spark, Python, and SQL.
  • Create and improve batch, streaming, and real-time integrations across enterprise and third-party sources.
  • Implement Databricks Lakehouse architectures with Delta Lake and Medallion patterns.
  • Deliver data products for analytics, predictive modeling, machine learning, and generative AI.
  • Work with data scientists and AI engineers to prepare, transform, and govern data for AI and ML use cases.
  • Create feature engineering pipelines and support machine learning lifecycle processes.
  • Build data solutions for RAG, vector and semantic search, and LLM applications.
  • Improve Spark jobs, SQL workloads, and data frameworks for performance, scale, and cost efficiency.
  • Establish data quality, observability, lineage, governance, and monitoring capabilities.
  • Meet data privacy, security, and responsible AI standards.
  • Support CI/CD, Infrastructure as Code, and DataOps across the data platform.
  • Guide junior engineers and reinforce sound engineering practices.
  • Partner with data scientists, AI engineers, architects, and business stakeholders to deliver trusted, governed data products.
  • Support enterprise AI and generative AI initiatives with robust, secure, governed data platforms.

Requirements

  • Bachelor’s degree in computer science, information systems, engineering, mathematics, or a related discipline.
  • At least 7 years in data engineering, ETL development, or large-scale data platforms.
  • At least 3 years of practical experience with Databricks and Apache Spark.
  • Strong Python and SQL skills, with experience in distributed data processing.
  • Background building cloud data lakes, warehouses, and Lakehouse architectures.
  • Experience supporting AI, machine learning, or advanced analytics initiatives.
  • Knowledge of data modeling, governance, and enterprise data management.
  • Experience building and tuning large-scale pipelines on AWS, Azure, or Google Cloud.
  • Experience with Databricks Delta Lake, Unity Catalog, Delta Live Tables, MLflow, Mosaic AI, and Databricks Workflows.
  • Practical experience with generative AI, large language models, RAG, vector databases, or AI-powered applications.
  • Familiarity with LangChain, Semantic Kernel, OpenAI APIs, Hugging Face, or comparable tools.
  • Experience with feature stores, model training pipelines, and MLOps.
  • Experience with Kafka, Event Hub, or other streaming technologies.
  • Databricks Certified Data Engineer Professional or an equivalent cloud certification is preferred.
  • Expertise in Databricks platform engineering.
  • Apache Spark development experience.
  • AI and machine learning data engineering expertise.
  • Generative AI data solutions experience.
  • Lakehouse architecture experience.
  • Data modeling and data warehousing expertise.
  • Data governance and security expertise.
  • Data observability and reliability engineering experience.
  • Cloud data platform experience with AWS or Azure.
  • DataOps, CI/CD, and automation experience.
  • Technical leadership and mentoring experience.

Benefits

  • Engagement as an independent contractor (PJ).
  • Hybrid schedule with three days per week in person at the Pinheiros, São Paulo office.

Related Jobs

Kreato Global | BPO and Language Solutions

Remote English-Spanish OPI/VRI Interpreter

Kreato Global | BPO and Language Solutions
201 – 500 Employees
HealthcareHospitalityLogistics

Interpret remotely between English- and Spanish-speaking people in medical, financial, social service, and customer care settings. Provide language support for Kreato Global across Latin America.

Open
SPERTON - Where Great People Meet

Sales Executive, Elevators and Car Parking Systems

SPERTON - Where Great People Meet
51 – 200 Employees

Drive elevator and car parking system sales across Mumbai’s Western Region. Build client and dealer relationships, and manage deals from initial enquiry through project execution.

Open