LogoFresh Jobs Ahead
Circana
Circana
5,001 – 10,000 Employees
B2BConsulting
Circana is a market research and consulting company that helps brands and retailers understand demand and make better decisions. Its work combines large-scale purchasing data with AI-powered visualization to analyze consumer behavior across online and physical stores, supporting challenges such as innovation, growth, and meeting changing market needs. Built from the legacy organizations IRI and NPD, Circana brings extensive industry experience to more than 7,000 brands and retailers worldwide. The company’s research and insights have been featured by outlets including Bloomberg, CNBC, and The New York Times.
LinkedIn
Company Jobs

Senior Director of Software Engineering, Data Engineering

Lead Azure data engineering teams at Circana, a global market intelligence company. Build scalable Spark pipelines and guide AI-assisted engineering practices.

USUnited StatesILIllinois
$145,000 – $190,000 / year
HybridFull-timeSeniorEnglish RequiredFull-stack EngineerSoftware Engineer
AirflowApacheAWSAzureCloudDockerKubernetesPySparkPythonSparkSQLTerraform

Description

  • Create and refine scalable data workflows with Python, PySpark, and Airflow
  • Process data in real time and in batches with Spark
  • Apply data quality, governance, and security practices across the data lifecycle
  • Maintain data availability, reliability, and performance through monitoring and automation
  • Manage cloud infrastructure and control costs for data processing workloads
  • Build CI/CD pipelines for data workflows
  • Develop and improve large-scale processing pipelines with Apache Spark and PySpark
  • Tune Spark workloads through partitioning, caching, and performance optimization
  • Use structured and unstructured data for analytics and machine learning initiatives
  • Create and maintain Airflow DAGs to automate complex workflows
  • Monitor, troubleshoot, and improve job execution and dependencies
  • Lead and mentor data engineers, providing technical direction
  • Promote coding standards, version control, and sound documentation practices
  • Work directly with clients to understand business needs and explain technical concepts
  • Partner with engineering, product, and architecture teams to build scalable, maintainable products
  • Coordinate with Program Management on delivery timelines and risk mitigation

Requirements

  • At least 8 years of data engineering experience
  • Expert knowledge of Python, PySpark, Apache Spark, and Apache Airflow
  • Strong Python and SQL programming skills
  • Ability to write efficient, maintainable code
  • In-depth knowledge of Spark internals, including RDDs, DataFrames, DAG execution, and partitioning
  • Experience creating Airflow DAGs and managing scheduling and dependencies
  • Knowledge of Git, Docker, Kubernetes, Terraform, and DevOps practices for CI/CD workflows
  • Experience optimizing large-scale data processing
  • Experience leading teams and working in Agile or Scrum environments
  • Experience with common data warehouse or data lake platforms
  • Knowledge of cloud technologies such as Azure, AWS, or Google Cloud
  • Demonstrated use of generative AI tools for software development, data engineering, system design, testing, and troubleshooting
  • Strong grasp of AI-assisted engineering, including prompt engineering, code generation, automated testing, and AI-driven productivity workflows
  • Ability to identify ways AI can improve team productivity, code quality, and operational efficiency
  • Machine learning engineering experience is a plus
  • Strong communication and collaboration skills
  • Ability to work effectively with remote teams
  • Candidates may be asked to consent to background checks

Benefits

  • Flexible and accommodating work environment
  • Paid time off
  • Medical, dental, and vision insurance
  • 401(k) plan
  • Eligibility for bonus pay
  • Commitment to diversity, equity, and inclusion
  • Accessibility accommodations for applicants with disabilities
  • Background checks conducted in accordance with local law
  • Current employer will not be contacted without permission

Related Jobs

Zendesk

Senior Partner Platform Architect

Zendesk
5,001 – 10,000 Employees
H1B Visa SponsorConsultingLogisticsMarketing
USUnited StatesALAlabamaAZArizonaCACaliforniaCOColoradoFLFloridaILIllinoisMIMichiganNJNew JerseyNYNew YorkNCNorth CarolinaTXTexasVTVermontVAVirginiaWIWisconsin
$200,000 – $300,000 / year
HybridFull-timeSeniorEnglish RequiredBackend EngineerSoftware Engineer
AWSDynamoDBNode.jsPython

Lead partner enablement and technical architecture for Zendesk contact center solutions built on Amazon Connect and AWS. The role spans onboarding, co-selling support, solution design, and partner training.

Open
PwC

Senior Forward Deployed Software Engineer, Palantir Foundry

PwC
H1B Visa SponsorConsultingFinanceSaaS
USUnited StatesALAlabamaAZArizonaCACaliforniaCOColoradoCTConnecticutFLFloridaILIllinoisIAIowaLALouisianaMDMarylandMAMassachusettsMIMichiganMNMinnesotaMOMissouriNVNevadaNJNew JerseyNYNew YorkNCNorth CarolinaOHOhioOKOklahomaPAPennsylvaniaSCSouth CarolinaTNTennesseeTXTexasUTUtahVTVermontVAVirginiaWAWashingtonWIWisconsin
$77,000 – $202,000 / year
HybridFull-timeSeniorEnglish RequiredFull-stack EngineerSoftware Engineer
PythonTypeScript

As a Senior Associate at PwC, you’ll build and deploy AI and machine learning data solutions in Palantir Foundry. You’ll lead technical teams, client relationships, and project delivery.

Open
PwC

Forward Deployed Software Engineer Manager, Palantir Foundry

PwC
H1B Visa SponsorConsultingFinanceSaaS
USUnited StatesALAlabamaAZArizonaCACaliforniaCOColoradoCTConnecticutFLFloridaILIllinoisIAIowaLALouisianaMDMarylandMAMassachusettsMIMichiganMNMinnesotaMOMissouriNVNevadaNJNew JerseyNYNew YorkNCNorth CarolinaOHOhioOKOklahomaPAPennsylvaniaSCSouth CarolinaTNTennesseeTXTexasUTUtahVTVermontVAVirginiaWAWashingtonWIWisconsin
$99,000 – $232,000 / year
HybridFull-timeMiddleSeniorEnglish RequiredFull-stack EngineerSoftware Engineer
PythonTypeScript

Lead client delivery of Palantir Foundry data engineering and AI solutions at PwC. Manage and coach teams, oversee project execution, and guide technology transformation.

Open