SPA Enterprise Services - Data Engineer - SQL/Python

Job Description : - Experienced Data Engineer (5-10 years). - Advanced SQL and Python programming skills. - Expertise in Data Engineering - AWS. - AWS Redshift - good experience in data models, query performance tuning, partitions etc. - AWS skills (Glue, EMR, Athena, Step Function, Lambda, Redshift). - Good experience in designing and developing scalable and reliable data pipelines. - Experience in Documentation - create technical specifications and able to translate existing technical designs into data transformation. - Development/Delivery methodologies - Agile Scrum DevOps. Responsibilities : - Design, develop, and maintain efficient and scalable data pipelines using AWS services like Glue, EMR, and Step Functions. - Extract, transform, and load (ETL) data from various sources (e.g., databases, APIs, files) into the data warehouse. - Implement data quality checks and monitoring mechanisms to ensure data accuracy and integrity. - Optimize Redshift performance through query tuning, partitioning, and indexing techniques. - Design and implement efficient data models to support complex analytical queries. - Troubleshoot and resolve performance issues within the Redshift environment. - Adhere to industry best practices for data engineering, including data security, privacy, and governance. - Stay up-to-date with the latest technologies and trends in the data engineering field. - Collaborate with cross-functional teams to understand business requirements and translate them into technical solutions. - Create clear and concise technical documentation, including data flow diagrams, data models, and operational procedures. - Actively participate in Agile Scrum and DevOps methodologies to ensure efficient project delivery. - Collaborate with data scientists and analysts to understand their data needs and provide effective solutions. Qualifications : - 5-10 years of experience in data engineering roles. - Strong proficiency in SQL and Python programming languages. - In-depth knowledge of AWS services, particularly Glue, EMR, Athena, Step Functions, Lambda, and Redshift. - Experience with data modeling, query optimization, and performance tuning. - Strong understanding of data warehousing concepts and ETL processes. - Experience with Agile Scrum and DevOps methodologies. - Excellent problem-solving and analytical skills. - Strong communication and collaboration skills. Preferred Qualifications : - Experience with data visualization tools (e.g., Tableau, Power BI). - Knowledge of machine learning and AI concepts. - Experience with cloud-native technologies and serverless architectures. (ref:hirist.tech)

Location: hyderabad, IN

Posted Date: 11/13/2024