Lead Data Engineer

Job Description

REQUIREMENTS

• Bachelor’s degree in computer science, Information Technology, Engineering, or a related discipline.
• 8+ years of experience in Data Engineering with at least 3–4 years of hands-on experience in Microsoft Fabric or similar cloud-based data platforms.
• Proven experience in designing and implementing enterprise ETL/ELT solutions.
• Strong understanding of modern data architecture and cloud-based analytics platforms.
• Hands-on experience with Microsoft Purview for data governance, data cataloging, metadata management, and data lineage.
• Experience provisioning and administering Microsoft Fabric workspaces.
• Good understanding of metadata management concepts and their implementation in relational database systems.
• Familiarity with DevOps methodologies, CI/CD processes, Git-based source control, and deployment automation.
• Strong analytical, problem-solving, leadership, and communication skills.

Technical Skills

• Microsoft Fabric
• Microsoft Purview
• Fabric Workspace Provisioning and Administration
• Fabric Data Pipelines
• PySpark Notebooks
• Metadata-Driven Design
• Metadata Management in Relational Databases
• OneLake
• Fabric Data Warehouse
• Data Engineering and ETL/ELT Architecture
• Data Modeling
• SQL
• Python
• Power BI
• PII Handling and Data Governance
• Azure DevOps
• Git
• CI/CD Pipeline Implementation
• Performance Optimization and Data Pipeline Monitoring

RESPONSIBILITIES

• Lead the design and implementation of enterprise-scale data engineering solutions using Microsoft Fabric.
• Design, develop, and optimize ETL/ELT pipelines for processing.
• Architect and implement metadata-driven data ingestion and transformation frameworks.
• Develop and maintain scalable PySpark notebooks for data processing.
• Design and implement robust data warehousing solutions using Microsoft Fabric.
• Provision and manage Microsoft Fabric workspaces, ensuring appropriate security, governance, and environment configuration.
• Implement and manage data governance, cataloging, metadata management, and data lineage using Microsoft Purview.
• Ensure data quality, security, governance, and compliance, including handling Personally Identifiable Information (PII).
• Collaborate with business stakeholders, solution architects, and reporting teams to understand business requirements and translate them into technical solutions.
• Optimize data pipelines for performance, scalability, reliability, and maintainability.
• Support Power BI reporting by developing optimized data models and curated datasets.
• Work closely with DevOps teams to establish CI/CD pipelines, deployment strategies, and release management processes.

Are you interested in this position?

Apply by clicking on the “Apply Now” button below!

#CrossChannelJobs #JobSearch
#CareerOpportunities #HiringNow
#Employment #JobOpenings
#JobSeekers
#FacebookLinkedIn