Data Engineer - ETL/Python

Role Description : About the Role : We are seeking a talented Data Engineer to join our team and play a crucial role in designing, developing, and maintaining robust data pipelines and data engineering solutions. You will work with cutting-edge technologies like Snowflake, AWS, and Python to transform raw data into valuable : Data Pipeline Development : - Design and implement efficient ETL/ELT pipelines to ingest, transform, and load data from various sources into Snowflake. - Ensure data quality, consistency, and accuracy throughout the pipeline. - Optimize data pipelines for performance and scalability. Cloud Data Architecture : - Manage data storage, security, and performance optimization in AWS (S3, etc.). - Implement data governance and security best practices. - Design and implement data lake and data warehouse architectures. Data Processing : - Utilize PySpark to process large-scale datasets efficiently. - Perform complex data transformations and aggregations. - Optimize data processing jobs for performance and Integration : - Build and consume APIs to integrate data sources and systems. - Ensure seamless data flow between systems. - Implement API security and authentication mechanisms. Automation : - Automate data pipelines and processes using tools like Apache Airflow. - Develop scripts and tools to streamline data operations. - Monitor and troubleshoot data pipelines to ensure reliability and : - 4 years of experience in data engineering and data warehousing. - Strong proficiency in Python and PySpark. - In-depth knowledge of Snowflake and AWS. - Experience with data modeling, ETL/ELT processes, and data quality assurance. - Understanding of SQL and database design principles. - Experience with API development and integration. - Familiarity with cloud-native technologies and data engineering best practices. - Strong problem-solving and analytical skills. - Excellent communication and collaboration skills. (ref:hirist.tech)

Location: in, IN

Posted Date: 11/8/2024