Instituto de Pesquisas Eldorado is a Brazilian research and development institute established in 1999, helping organizations turn technology initiatives into tested products and practical innovation. Its work includes telecommunications product certification, laboratory testing, and innovation consulting for sectors such as automotive, healthcare, logistics, agriculture, energy, and Oil & Gas. The institute also develops training and education programs and collaborates with industry and research partners to expand technological capabilities.
Data Analyst – PySpark, SQL and Power BI
Build data pipelines, models and dashboards for Instituto de Pesquisas Eldorado’s Brazilian R&D center. Work with PySpark, SQL, Databricks and Power BI.
Description
- Create and deploy ingestion pipelines using REST APIs, web scraping, legacy SQL procedures and files
- Help implement read-only replication for systems without direct database access, accounting for performance impact
- Transform data with PySpark and SQL
- Assist with processing cost and performance benchmarks and efficient resource sizing
- Help build Bronze, Silver and Gold data layers using the medallion architecture
- Build resilient scraping pipelines with layout-change detection and blocking mitigation
- Deduplicate and normalize data, and test its quality
- Help implement data contracts and schema-drift alerts for third-party integrations
- Create Power BI dashboards and reports
- Turn business needs into data models and clear visualizations
- Apply Row-Level Security to reports in line with RBAC policies
- Create data quality tests and declarative transformations with dbt
- Help configure self-service analytics features, including Fabric Copilot
- Work with data governance teams to maintain lineage and classification
- Find and fix performance issues in pipelines and queries
- Run user acceptance tests and improve dashboards based on feedback
Requirements
- Bachelor’s degree in Computer Science, Computer Engineering, Systems Analysis or a related field
- Mid-level experience in data engineering, Python or PySpark, and SQL
- Familiarity with Databricks or Spark
- Familiarity with orchestration tools such as Azure Data Factory
- Familiarity with analytical data formats such as Parquet and Delta Lake
- dbt knowledge is an advantage
- Familiarity with Git and CI/CD for data pipelines
- Experience integrating data through REST APIs
- Knowledge of data visualization practices and data storytelling
- Experience working with varied data sources, including SQL, APIs, files, Lakehouse and Warehouse
- Experience implementing Row-Level Security
- Willingness to travel to São Paulo is an advantage
- Advanced English is an advantage
Benefits
- Equal opportunity for all candidates
- A diverse and inclusive workplace
- Open to candidates with disabilities
- Hybrid work arrangement
- Opportunities to build experience and contribute to relevant projects
- Access to the organization’s laboratory infrastructure and R&D projects