Staff Software Engineer

Designing and implementing end-to-end ETL processes using Azure Data Factory to extract data from multiple sources, transform it to meet business requirements, and load it into a data warehouse. Collaborating with cross-functional teams to understand data requirements, business rules, and data quality standards to ensure accurate and consistent data delivery. Building and maintaining scalable and efficient ETL pipelines that handle large volumes of data while adhering to best practices for data integration and data governance. Developing and implementing data validation and reconciliation processes to ensure data accuracy and consistency across all data sets. Troubleshooting and resolving issues related to data transformation, data loading, and data quality, while proactively identifying opportunities for process optimization and performance tuning. Collaborating with data architects, data scientists, and other stakeholders to understand data models and data requirements to support data exploration, analytics, and reporting initiatives. Documenting and maintaining technical specifications, data pipeline documentation, and data lineage to ensure data governance and regulatory compliance. Staying up-to-date with the latest industry trends and technologies related to Azure Data Factory and ETL to continuously improve data engineering practices and solutions. Qualifications: To perform this job successfully, an individual must be able to perform each essential duty satisfactorily. The requirements listed below are representative of the knowledge, skill, and / or ability required. Knowledge / Skills/Abilities Education: Bachelor's Degree in an Information Technology related field. Overall Experience: 7 years Preferred: 5 years in Data Architecture with experience in designing and implementing end-to-end ETL process Strong expertise in SQL and experience with data integration, data transformation, and data loading techniques using ETL tools and frameworks. Hands-on experience with cloud platforms, preferably Microsoft Azure, including familiarity with Azure Data Lake Storage, Azure SQL Data Warehouse, and Azure Databricks. Preferred in programming languages such as Python, R, Powershell, for data processing, data manipulation, and data transformation. Knowledge of data modeling concepts and experience with data modeling tools is a plus. Excellent problem-solving skills, with the ability to analyze complex data scenarios and design efficient and scalable solutions. Strong communication and collaborative skills, with the ability to work effectively in cross-functional teams and communicate technical concepts to both technical and non-technical stakeholders.

Location: bangalore, IN

Posted Date: 11/22/2024