Job Overview
Technical skills:
• Should have 6+ years of hands-on experience in Data engineering
• Excellent command over ETL tools covering Snowflake, Azure Data Lake, Databricks and Azure Data Factory
• Good command over writing Python Scripts for custom build data ingestion
• Proficient in SQL scripts (Stored procedure, Views, Functions, Joins)
• Experience with Snowflake
• Experience with DBT (for ETL transformation logic)
• Experience with FIVETRAN (for data ingestion)
• Good understanding of data warehouse fundamentals and principles, database modelling knowledge and experience • Experience utilizing data pipelines and connectors across different source systems
• Experience with cloud data integration tools and services related to Fivetran, DBT or similar data ingestion and data transformation tools
• Experience working Git version control, preferably GitHub
• Professional IT related education or equivalent (university) education (e.g., B.E/BTech/MCA/MSC) in a relevant subject, preferably in Computer Science is a preferred
• Experience working as ETL/ELT developer or a Data engineer
Requirements for the role:
• Analyse and organize raw data, providing datasets and sources for business reporting requirements
• Implement data quality rules to enhance data reliability • Understand business needs and objectives
• Conduct complex data analysis and data modelling, interpreting trends and patterns
• Collaborate with data scientists and Reporting engineers to help resolve the reporting and Analytics needs
• Provide expertise and leadership in the development of new products, services, and processes from a Data Engineering technical perspective.
• Work independently adhering to high technical standards, taking ownership of projects ensuring they conform to departmental standards before going to QA.
• Work effectively within an agile, deadline-driven production process to complete project development with rapid, iterative cycles.
• Liaise with Team Leaders, Product and Project Managers to ensure projects are taken from creation to completion – on time and on budget.
• Provide accurate estimates for existing and new projects alike.
• Be an integral part of the code review process, providing constructive feedback against code produced by other team members.
• Ensure good communication is maintained with other team members/departments throughout project life cycles.
• Write clear, clean, and concise scripts, SQL and transformation logic
• Fix bugs and transformation problems rapidly
• Use lateral thinking and investigative techniques to solve challenges across various technical streams.
• Participate in the ongoing development, adoption and promotion of Data Engineering standards
• Investigate and participate in discussions with team members about best practices and help identify optimal technical solutions
• Actively contribute to departmental learning and standards within the team environment through involvement in presentations/demonstrations and participate in the continual drive for collaboration and knowledge sharing amongst the team
• Keep abreast of the latest trends and emerging technologies within the Data Engineering community and continually develop skill
