State Street
State Street
State Street is a global financial services company serving institutional investors through investment management and investment servicing solutions. Its offerings include asset management, custodial services, and investment research, supported by expertise across finance, fintech, and banking. The company focuses on helping clients manage complex investment activities with data, insights, and operational capabilities designed to improve efficiency and support financial decision-making.

Data Engineer, Assistant Vice President - Princeton, NJ

Build AWS and Databricks lakehouse platforms supporting State Street’s legal and compliance operations. Develop governed data pipelines for analytics, reporting, and AI/ML applications.

Description

  • Build and maintain scalable data pipelines with PySpark, Python, and Spark SQL
  • Develop and optimize Databricks ETL/ELT workflows with Delta Lake
  • Implement Bronze, Silver, and Gold lakehouse architecture for enterprise data platforms
  • Manage Databricks Jobs, Workflows, and Notebooks for batch and streaming workloads
  • Create reusable frameworks for data ingestion, processing, and orchestration
  • Containerize data workloads with Docker and automate processes through scripting
  • Connect Databricks pipelines with Power Apps and Power Automate
  • Expose data to business users through APIs, connectors, and curated datasets
  • Design and optimize Databricks lakehouse architectures
  • Integrate SQL Server, Oracle, and other enterprise data sources
  • Apply dimensional modeling, partitioning, and optimization methods
  • Process structured and semi-structured formats including JSON and Parquet
  • Improve performance through caching, indexing, and Spark optimization
  • Publish curated datasets for Power BI, Power Apps, and Power Automate workflows
  • Maintain data quality with unit tests, validation frameworks, and automated checks
  • Monitor and troubleshoot distributed Spark workloads and pipelines
  • Investigate logs and resolve production issues across Databricks and cloud environments
  • Maintain data lineage, consistency, and audit readiness
  • Partner with Legal, Security, Compliance, and Enterprise Data teams
  • Convert business requirements into data engineering solutions
  • Serve as a subject matter expert in Databricks and lakehouse architecture
  • Lead initiatives independently and take ownership of deliverables
  • Support governance with Databricks Unity Catalog and AWS IAM and KMS controls
  • Meet data privacy, regulatory, security, and audit requirements
  • Maintain access controls and data classification and handling standards
  • Work with IAM and security teams to protect data access
  • Design and maintain CI/CD pipelines with Harness, Azure DevOps, or GitHub
  • Automate Databricks asset deployments with Databricks Repos and CLI
  • Monitor, schedule, and optimize workflows with Databricks orchestration tools
  • Maintain architecture, data flow, and runbook documentation
  • Improve platform performance, scalability, and cost efficiency

Requirements

  • Bachelor’s or master’s degree in computer science, data engineering, information systems, or a related technical field
  • At least 8 years of experience in data engineering or data platform development
  • Hands-on expertise with Databricks and Apache Spark
  • Proficiency in PySpark, Python, and SQL
  • Experience with AWS data services including S3, Glue, Lambda, and IAM
  • Experience with Delta Lake and lakehouse architecture
  • Strong knowledge of distributed data processing
  • Strong knowledge of ETL and ELT frameworks
  • Strong knowledge of data modeling techniques
  • Hands-on experience with Databricks components such as Delta Lake, Workflows, and Unity Catalog
  • Extensive experience building end-to-end batch and streaming pipelines with AWS and Databricks
  • Familiarity with Spark and Delta Lake performance optimization
  • Experience supporting analytics, reporting, or AI/ML workloads on a lakehouse platform
  • Knowledge of data governance, metadata management, and security controls
  • Experience in Legal, Compliance, Financial Services, or other regulated industries
  • Understanding of legal data concepts including contracts, clauses, obligations, and matters
  • Exposure to unstructured data processing or document and NLP pipelines
  • Experience with Power BI, Power Apps, or Power Platform
  • Experience managing sensitive data in audit-driven environments

Benefits

  • 401(k) retirement savings plan with company matching
  • Insurance options including basic life, medical, dental, vision, long-term disability, and other optional coverage
  • Paid leave covering vacation, sick time, short-term disability, and family care responsibilities
  • Employee Assistance Program
  • Incentive compensation with eligibility for annual performance-based awards
  • Eligibility for select tax-advantaged savings plans
  • Inclusive professional development opportunities
  • Flexible work-life support
  • Paid volunteer days
  • Employee networks

Related Jobs

General Motors

Principal Software Engineer, Developer Experience - General Motors

General Motors

Lead General Motors’ developer experience strategy by building golden paths, automation, measurable engineering improvements, and AI-enabled development workflows. Guide teams across Core IT to improve software delivery, developer productivity, and engineering quality.

Open
Rackspace Technology

Senior Technical Evangelist, AI Infrastructure and GPU Technologies

Rackspace Technology

Lead technical evangelism for Rackspace Technology’s GPUaaS business through partner enablement, technical education, and industry thought leadership. Represent AI infrastructure and GPU technologies in events, customer forums, and technical discussions.

Open
Benco Dental

Regional Sales Manager, Customer Advocate - Benco Dental (Michigan Remote)

Benco Dental
1,001 – 5,000 Employees
ConsultingHealthcareLogistics

Lead regional sales, branch operations, and customer advocacy for Benco Dental, the largest privately owned full-service dental distributor in the United States. Manage sales teams, service quality, logistics, and customer relationships across an assigned Michigan territory.

Open
Motorola Solutions

Customer Success Advocate at Motorola Solutions — United States Remote

Motorola Solutions

Help public-safety software customers adopt Motorola Solutions products and achieve measurable operational results. Manage customer health, workflows, renewals, and expansion opportunities across assigned accounts.

Open