Cloud Big Data Engineer - Python/Hadoop/Hive

We are hiring Cloud Big Data Engineer. Location : Experience level : 5 years. Required Skills : GCP, BigData, Pyspark, Python, Spark, Kafka, Hadoop, Hive, Scala/Django, DFS Elastic search. Key Responsibilities : - Design, develop, and implement big data solutions on the Google Cloud Platform (GCP). - Develop and maintain data pipelines for data ingestion, transformation, and loading into data warehouses and data lakes. - Work with Apache Spark, Hadoop (HDFS, Hive), and other big data technologies to process and analyze large datasets. - Develop and deploy data pipelines using tools like Apache Kafka. - Utilize NoSQL databases such as ElasticSearch for data storage and retrieval. - Develop and maintain data quality checks and ensure data integrity. - Collaborate with data analysts, data scientists, and other stakeholders to understand business requirements and translate them into technical solutions. - Optimize data pipelines for performance and scalability. - Troubleshoot and resolve data-related issues. - Stay up-to-date with the latest advancements in big data technologies and cloud : Education : Bachelor's degree in Computer Science, Engineering, or a related field. Experience : 5 years of experience in big data engineering, data warehousing, or a related field. Technical Skills : - Strong proficiency in Python and experience with frameworks like Django. - In-depth knowledge of Apache Spark, Hadoop (HDFS, Hive), and other big data technologies. - Experience with data ingestion, transformation, and loading tools. - Experience with NoSQL databases, particularly ElasticSearch. - Experience with cloud platforms, preferably GCP. - Proficiency in Linux/Unix and shell scripting. Preferred Skills : - Experience with data streaming platforms like Kafka. - Experience with data visualization and reporting tools (e.g., Tableau, Power BI). - Experience with containerization technologies (Docker, Kubernetes). - Knowledge of distributed systems concepts (load balancing, sharding, data distribution). - Cloud certifications (e.g., Google Cloud Certified Professional Data Engineer) (ref:hirist.tech)

Location: pune, IN

Posted Date: 2/14/2025