Visa
Visa
Visa is a global payments technology company serving consumers, merchants, financial institutions, and governments through its electronic payments network. Its platform supports credit, debit, and prepaid transactions, alongside payment processing, fraud prevention, security, analytics, and other services for consumers and businesses. Visa also develops emerging payment solutions, including AI-enabled commerce initiatives, and supports financial inclusion through the Visa Foundation.

Senior Data Engineer, Data Lake (Brazil Remote)

Build and maintain Spark, Databricks, and Airflow pipelines for Visa’s corporate data lake. Improve data quality, orchestration, and dependable self-service access to data.

Description

  • Develop, test, and maintain Databricks ingestion, transformation, and data-quality pipelines with PySpark and SparkSQL.
  • Create and update MWAA Airflow DAGs to orchestrate data workflows.
  • Write and optimize SQL for transformation and validation tasks.
  • Implement and monitor data-quality checks with Great Expectations or an equivalent tool.
  • Contribute to peer code reviews.
  • Troubleshoot pipeline failures and data-quality issues with support from senior engineers.
  • Create and update documentation for datasets and pipelines.
  • Take part in sprint planning, estimation, and retrospectives.
  • Apply the team’s CI/CD, testing, and governance standards.
  • Help strengthen the reliability and quality of Visa’s corporate data lake.

Requirements

  • Be based in Brazil.
  • Communicate in English at B2 level or higher (Upper-Intermediate).
  • Have at least 3 years of relevant experience with a Bachelor’s or Associate’s degree, or at least 5 years of relevant experience.
  • Use Python for automation and data processing.
  • Understand batch data pipelines and ETL/ELT patterns.
  • Know the fundamentals of Apache Spark, including PySpark or SparkSQL.
  • Work with Databricks jobs, workflows, cluster management, and performance tuning.
  • Use intermediate to advanced SQL, including joins, window functions, and CTEs.
  • Understand Amazon S3 data-lake design, including partitioning, layout, and lifecycle management.
  • Know core AWS concepts, including S3, IAM, and CloudWatch.
  • Understand dimensional or Kimball modeling and medallion-layer architectures.
  • Use Git/GitHub workflows and participate in code reviews.
  • Preferred: experience with Delta Lake, Apache Airflow/MWAA, CDC patterns, data-quality concepts, Terraform fundamentals, or BI integration such as Superset and dashboarding.

Benefits

  • Remote work arrangement.
  • Opportunity to make an impact at scale.
  • Professional development and skill growth through meaningful work.
  • Remote employees may have scheduled, advance-notice opportunities to work from a Visa office.

Related Jobs

Jeeves

Forward Deployed Engineer, Mexico (Remote)

Jeeves

Integrate Jeeves’ stablecoin banking platform with enterprise finance systems and lead customer implementations across Latin America. Own payment workflows and technical delivery from discovery through go-live and adoption.

Open