Data Engineer -Databricks - Tieto Tech Consulting (m/f/d)

Databricks Data Engineer

Role Overview

We are looking for an experienced Databricks Data Engineer with 5 - 10 years of overall data engineering experience, including strong hands-on expertise in Azure Databricks, PySpark, SQL and Delta Lake.

The candidate capable of designing and developing scalable data pipelines and Lakehouse solutions. The role requires close collaboration with Data Architects, Business Analysts, Analytics teams and business stakeholders to transform enterprise data into reliable and business-ready data products.

Key Responsibilities

• Design, develop and maintain scalable data pipelines using Azure Databricks, PySpark, Python, SQL and Delta Lake.
• Implement Bronze, Silver and Gold Lakehouse data layers and reusable data-engineering frameworks.
• Develop complex ETL/ELT solutions involving SAP and other enterprise source systems.
• Ingest and transform data originating from SAP environments, applying appropriate business and transformation logic.
• Build data solutions supporting manufacturing business processes, reporting and analytics.
• Implement batch and incremental data-loading patterns, schema evolution, historical data handling and pipeline recovery.
• Develop and optimize complex Spark/PySpark workloads and SQL transformations for scalability and performance.
• Implement data-quality checks, validation, reconciliation, monitoring and exception handling.
• Develop analytical and dimensional data models, including Fact/Dimension and Star Schema models.
• Follow engineering best practices for Git, CI/CD, automated deployment, coding standards and documentation.
• Work with Data Architects and business stakeholders to translate technical and business requirements into scalable data solutions.
• Participate in design reviews, code reviews and continuous improvement of Databricks engineering practices.

Technical Skills & Certifications Essential Skills

• 5 - 10 years of experience in Data Engineering, Data Warehousing or Data Platform development.
• Strong hands-on experience with Azure Databricks.
• Strong development skills in PySpark, Apache Spark, Python and SQL.
• Good understanding of Delta Lake and Lakehouse architecture.
• Experience implementing Bronze, Silver and Gold / Medallion architecture.
• Strong experience developing enterprise-scale ETL/ELT and data pipelines.
• Experience with data modelling, including Fact/Dimension models and Star Schemas.
• Understanding of data quality, governance, metadata, lineage and security concepts.
• Experience with Git, CI/CD and DevOps practices for data-engineering solutions.
• Strong troubleshooting, performance optimization and problem-solving skills.
• Good communication skills and ability to work with globally distributed technical and business teams.

Good to have :

• Experience working with SAP as a source system and good understanding of SAP data extraction/integration scenarios.
• Manufacturing industry/customer experience is required, with an understanding of data associated with manufacturing business processes.

Certifications

• Databricks Certified Data Engineer Associate preferred.
• Databricks Certified Data Engineer Professional would be a strong advantage.
• Microsoft Azure/Data Engineering certifications are an added advantage.
• Databricks AI Certification - Good to have.