Senior Data Engineer - Azure Data Factory

Job Title : Senior Data Engineer Location : Gurgaon, India Experience : 4 to 8 Years Work Mode : Hybrid (3 days Work From Office) Key Responsibilities : - Design, develop, and maintain robust data pipelines using Azure Data Factory (ADF) to ingest, transform, and load data into data warehouses and data lakes. - Optimize data pipelines for performance and scalability. - Implement data quality checks and monitoring mechanisms. - Leverage Azure Data Bricks for data processing and analysis. - Utilize Azure Synapse Analytics for data warehousing and business intelligence. - Effectively use Azure Data Lake Storage (ADL) for data storage and retrieval. - Design and implement efficient ETL processes to extract, transform, and load data from various sources (e.g, databases, APIs, files). - Optimize ETL processes for performance and accuracy. - Write clean, efficient, and well-documented Python code to automate tasks, process data, and build data pipelines. - Utilize Python libraries like Pandas, NumPy, and Scikit-learn for data analysis and machine learning. - Identify and resolve data quality issues and performance bottlenecks. - Continuously improve data pipelines and processes to enhance efficiency and accuracy. - Collaborate with data analysts, data scientists, and business stakeholders to understand data requirements and deliver data-driven insights. - Work effectively with cross-functional teams to achieve project goals. Required Skills and Experience : - 4-8 years of experience in data engineering or a related field. - Proficiency in Azure Data Factory (ADF), Azure Data Bricks, Azure Synapse Analytics, and Azure Data Lake Storage (ADL). - Strong Python programming skills, including knowledge of data manipulation and analysis libraries. - Experience with SQL and data modeling techniques. - Knowledge of data warehousing and data lake concepts. - Understanding of cloud computing principles and Azure services. - Strong problem-solving and analytical skills. - Excellent communication and teamwork skills. Preferred Skills : - Experience with data visualization tools (Power BI, Tableau). - Knowledge of machine learning and AI techniques. - Experience with data security and privacy best practices (ref:hirist.tech)

Location: in, IN

Posted Date: 11/20/2024