Simple Machines is an Australian consulting firm working across data architecture, data engineering, data science and software development. Its consultants design and build distributed data platforms and machine learning applications for enterprises, government agencies and global technology companies. The team also provides Kafka consulting and training as a Preferred Confluent partner, alongside Spark and Scala consulting and training through its partnerships with Databricks and Lightbend. For people exploring roles in data engineering, data science or software development, Simple Machines offers a specialist consulting environment focused on putting data to work in real-world systems.
Principal Data Engineer (Remote, Poland)
Lead architecture and delivery of cloud-native data platforms, pipelines, and data products for Simple Machines, a global technology consultancy. Set technical direction and partner with clients while working remotely from Poland.
Aprašymas
- Own the architecture of modern cloud-native data platforms from design through delivery.
- Create scalable data ecosystems with data mesh, data products, and data contracts.
- Set architecture across data ingestion, storage, processing, and access.
- Ensure platforms meet security, compliance, and production-readiness standards.
- Deliver cloud-native platforms with Databricks, Snowflake, AWS, and GCP.
- Connect client systems to provide scalable data access for consumers.
- Build and tune batch and real-time data pipelines.
- Use Kafka, Flink, Kinesis, and Pub/Sub for streaming and messaging.
- Coordinate workflows with Airflow, Dataflow, and Glue.
- Process and transform large datasets with Spark and Flink.
- Work with relational, NoSQL, and analytical databases, including Postgres, BigQuery, Snowflake, Cassandra, and MongoDB.
- Improve storage formats and access patterns with Parquet, Delta, ORC, and Avro.
- Build secure, compliant solutions with security considered from the outset.
- Integrate data governance without slowing developer workflows.
- Collaborate directly with clients to understand challenges and shape solutions.
- Turn business requirements into practical engineering choices.
- Advise clients as a trusted technical partner.
- Define engineering standards, patterns, and practices.
- Review designs and code, offering technical guidance and mentorship.
- Strengthen data quality, testing, observability, and operational practices.
Reikalavimai
- Bring strong Python and SQL skills.
- Have deep experience with Spark and modern data platforms such as Databricks or Snowflake.
- Understand cloud data services on AWS or GCP.
- Have owned architecture for large-scale data platforms.
- Apply strong data modelling skills and make sound architectural decisions.
- Balance performance, cost, and complexity when choosing solutions.
- Have built and operated large-scale production data pipelines.
- Have experience with multiple storage technologies and data formats.
- Use infrastructure-as-code tools such as Terraform or Pulumi.
- Have built CI/CD pipelines with tools such as GitHub Actions or ArgoCD.
- Have used data testing and quality frameworks such as dbt, Great Expectations, or Soda.
- Have experience in consulting or professional services.
- Be able to question assumptions and steer clients toward better outcomes.
- Be comfortable mentoring senior engineers and shaping technical culture.
Privalumai
- Work on challenging problems with meaningful impact.
- Build modern platforms instead of maintaining legacy systems.
- Collaborate with experienced engineers.
- Have autonomy and influence in your work.
- Have opportunities to develop professionally.