Data Engineer

Job Description : - Design, build, and maintain data pipelines in the Azure environment, ensuring smooth operation in both batch and real-time processing. - Optimize Azure data infrastructure to facilitate accurate data extraction, transformation, and loading from a variety of data sources. - Develop and automate ETL processes using tools like Pyspark, Azure Data Factory, and Databricks to facilitate data extraction and manipulation from diverse sources. - Transform raw data stored in Azure Data Warehouses into accessible datasets suitable for both technical and non-technical stakeholders. - Demonstrate expertise in Azure services and big data technologies, including but not limited to Azure Data Factory, Databricks, and other relevant tools. - Proficiency in SQL and Python is also crucial. - Engage in business discussions, collaborate with stakeholders, and take ownership of deliverables, ensuring that the data engineering solutions align with business objectives. - Lead development teams or project teams, providing guidance and expertise to ensure the successful execution of data engineering projects. - Function as an independent developer who can guide design discussions and make informed decisions to drive project success. - Leverage extensive experience to handle larger projects and manage multiple teams effectively. - Exhibit excellent communication skills, translating functional and technical requirements into actionable solutions that meet business needs. - The Person Minimum of 4 years of hands-on experience in Azure data engineering. - Proficiency in a range of key technologies is essential, including Pyspark, Azure Data Factory (ADF), Databricks, big data tools, SQL, and Python. - Should be adept at handling business discussions and be capable of taking ownership of deliverables. - Effective stakeholder management is crucial, including leading development teams or project teams to ensure successful project outcomes. - Self-sufficient developer who can provide guidance in design discussions and make informed decisions. - Demonstrated experience in successfully managing larger projects and multiple teams is a key requirement. - Excellent communication skills are necessary to understand and bridge the gap between functional and technical requirements, providing practical and effective solutions (ref:hirist.tech)

Location: hyderabad, IN

Posted Date: 10/9/2024