Data Engineer - Scala/Spark

Job Title : Data Engineer Location : Noida/Gurugram Experience : 8 Years Job Overview : We are seeking an experienced Data Engineer with strong expertise in data technologies and AWS Cloud services. The ideal candidate will have a deep understanding of OLAP systems like ClickHouse and experience with distributed data processing frameworks. If you are skilled in Java/Scala and Python and thrive in a collaborative environment, this is an excellent opportunity for you. Key Responsibilities : Data Engineering and Development : - Design, develop, and optimize scalable data pipelines and data warehouses. - Implement OLAP systems, focusing on ClickHouse or similar technologies. AWS Cloud Services : - Leverage AWS Cloud services for data storage, processing, and analytics. - Ensure secure and efficient data management using AWS best practices. Distributed Data Processing Frameworks : - Work with frameworks like Spark (Core, Streaming, SQL), Storm, or Flink for processing large-scale datasets. - Optimize frameworks for high performance and low latency. Programming and Scripting : - Develop robust applications using Java/Scala. - Write scripts and utilities using Python for data transformations and automation. Collaboration and Optimization : - Collaborate with cross-functional teams to understand and fulfill data requirements. - Optimize data workflows and systems for scalability and performance. Skill Set Requirements : Technical Expertise : - 8 years of IT experience with 5 years in data-related technologies. - At least 3 years of expertise in AWS Cloud services related to data engineering. - 3 years of experience with OLAP systems, specifically ClickHouse or similar. Programming Languages : - Proficiency in Java/Scala for building data-intensive applications. - Strong skills in Python for data manipulation and scripting. Distributed Data Processing Frameworks : - Hands-on experience with frameworks like Spark, Storm, or Flink. - Proficiency in Spark modules: Core, Streaming, and SQL. AWS Cloud Expertise : - Experience with services such as AWS S3, Redshift, Glue, Lambda, EMR, and Kinesis. Qualifications : - Bachelor's/Master's degree in Computer Science, Information Technology, or a related field. - Proven experience in designing and optimizing scalable data solutions. - Strong analytical and problem-solving skills. - Excellent communication and collaboration skills. (ref:hirist.tech)

Location: in, IN

Posted Date: 2/14/2025