Data Engineer

Responsibilities:

• Design, build, and maintain scalable ETL/ELT pipelines for batch and real-time data.
• Develop parameterized and reusable pipelines using Microsoft Fabric, Azure Data Factory, or Synapse Pipelines.
• Optimize large-scale data ingestion and transformation processes using Databricks, PySpark, SQL, and Dataflows
• Work on data migration projects, moving workloads from on-premises to Azure Cloud or Microsoft Fabric.
• Integrate data from multiple sources (databases, APIs, Event Hubs, third-party systems, etc.).
• Implement data partitioning, chunking, and metadata-driven approaches for performance optimization.
• Design and maintain Lakehouse, Data Warehouse, and Delta Lake structures.
• Build and manage semantic models for reporting with Power BI.
• Collaborate with analysts and business stakeholders to create efficient data models supporting analytics use cases.
• Ensure proper data governance, lineage, and cataloging using Microsoft Purview.
• Manage access and security via Microsoft Entra ID (Azure AD), roles, and service principals.
• Implement best practices for data quality, integrity, and compliance with organizational and industry standards.
• Work closely with data analysts, BI developers, and business stakeholders to understand requirements and deliver solutions.
• Document technical designs, processes, and best practices for maintainability.
• Contribute to knowledge sharing and mentoring within the data team.

Requirements:

• Strong experience with Microsoft Fabric, Azure Data Factory, Synapse Analytics, and Power BI.
• Proficiency in SQL (T-SQL, query optimization, stored procedures).
• Experience with PySpark, Python, or Databricks-like environments for big data processing.
• Knowledge of data modeling (star schema, data vault, dimensional modeling).
• Familiarity with real-time streaming (Event Hubs, Kafka, Service Bus, or similar).
• Experience with data governance tools (e.g., Microsoft Purview).
• Solid understanding of CI/CD pipelines, Git, and DevOps practices.
• 4+ years of proven experience in a Data Engineering role (or similar).
• Strong problem-solving skills and ability to optimize data processes.
• Excellent communication and ability to translate technical solutions into business language.
• Experience in client-facing or cross-functional roles is an advantage.
• Relevant Microsoft certifications is a plus.