Data Engineer - Spark/Python

Job Title : Data Engineer. Location : Bangalore. Experience Level : 6- 8 years. Employment Type : Full-time with NAM INFO PVT LTD. Shift timings : Rotational Shift (should be able to support 24 x 7). Workdays : 5 days a week. Job Description : We are looking for experienced and talented Senior Data Engineers to join our team in Bangalore. As a Senior Data Engineer, you will be responsible for designing, developing, and optimizing data pipelines, working with both cloud platforms and modern data technologies. You will collaborate with cross-functional teams to ensure data solutions are efficient, scalable, and aligned with business requirements. Key Responsibilities : - Cloud Integration : Design and manage data solutions on AWS or Azure platforms, ensuring smooth integration with data systems. - Data Warehousing : Develop and manage Data Warehouse solutions, ensuring data storage, transformation, and retrieval are efficient and scalable. - Data Pipeline Development : Build and maintain ETL (Extract, Transform, Load) processes using modern ETL tools, with a focus on automation and scalability. - Programming & Scripting : Write efficient and maintainable code using Python, Spark, and PySpark to process large datasets. - Data Platform Management : Work with data platforms like Snowflake or Databricks to optimize data storage and processing, ensuring performance and cost-effectiveness. - Database Management : Use any RDBMS (Relational Database Management System) to manage structured data, ensure data quality, and optimize database performance. - SQL Expertise : Write complex SQL queries for data retrieval, analysis, and reporting, ensuring efficiency and accuracy. - Collaboration : Work closely with data scientists, analysts, and business stakeholders to understand data needs and deliver solutions that support decision-making. Required Skills & Qualifications : - 6 - 8 years of experience in data engineering or a similar role. - Hands-on experience with cloud platforms such as AWS or Azure. - Strong understanding of Data Warehousing concepts and architecture. - Proficiency in Python, Spark, and PySpark for data processing and automation. - Experience with Snowflake or Databricks for cloud-based data warehousing and analytics. - Strong SQL skills and experience working with any RDBMS. - Hands-on experience with any ETL tool to manage data pipelines. - Familiarity with data integration, transformation, and migration strategies. - Strong problem-solving skills and the ability to troubleshoot data-related issues. - Excellent communication skills and the ability to work collaboratively with cross-functional teams. Preferred Qualifications : - Experience in working with large-scale data architectures and cloud data ecosystems. - Familiarity with performance tuning and optimization techniques for ETL pipelines and databases. - Certification in AWS or Azure. - Experience with modern data tools and frameworks for real-time data processing. (ref:hirist.tech)

Location: bangalore, IN

Posted Date: 10/18/2024