Data Engineer - ETL

Role Description : This is a full-time hybrid role for a Data Engineer at DIATOZ. The Data Engineer will be responsible for tasks such as data engineering, data modeling, ETL processes, data warehousing, and data analytics. The role is primarily located in Bengaluru with flexibility for remote work. Experience : 2.5 to 5 years Key Responsibilities : Data Pipeline Development : - Design, develop, and optimize scalable ETL (Extract, Transform, Load) pipelines to ingest and process large volumes of data from diverse sources. - Automate data pipelines for both real-time and batch data processing to ensure efficient data flow and accessibility. Data Modeling & Architecture : - Create and maintain complex data models that support analytics, business intelligence, and data warehousing. - Develop and optimize data schemas and structures to ensure data consistency, accuracy, and scalability. - Work closely with stakeholders to understand business requirements and translate them into efficient and effective data models. Data Warehousing : - Design, implement, and maintain data warehousing solutions to support large-scale analytics and reporting requirements. - Use industry best practices to manage data storage, access, and performance within the data warehouse environment. - Conduct regular audits and optimize data storage strategies to ensure cost-efficiency and scalability. Data Quality Assurance : - Ensure data integrity, quality, and accuracy throughout all data processing and storage systems. - Implement data validation, cleansing, and reconciliation processes to detect and resolve data discrepancies. - Monitor data pipelines and warehouse processes for anomalies or issues, resolving them proactively. Collaboration & Communication : - Work closely with cross-functional teams, including data scientists, analysts, and business stakeholders, to gather requirements and provide data insights. - Communicate technical solutions and progress effectively to both technical and non-technical team members. - Contribute to documentation, training, and sharing knowledge to support a collaborative, transparent team culture. Performance Optimization : - Identify and implement strategies for performance tuning of data processing and ETL workflows. - Leverage best practices for distributed computing and optimize resource utilization on big data platforms like Apache Spark. Data Analytics Support : - Provide analytical support for data-driven decisions by enabling data accessibility, querying, and visualization. - Build ad-hoc queries and support data scientists and analysts with data extractions, transformations, and data set preparation. Required Skills and Qualifications : - Technical Proficiency : Strong expertise in ETL processes, data modeling, and data warehousing. - Programming & Tools : Proficient in Apache Spark, Scala, Python, SQL, and related big data tools. - Data Engineering Experience : Proven experience in designing and implementing complex data solutions. - Analytical Skills : Strong problem-solving skills and analytical capabilities to support effective data-driven solutions. - Educational Background : Bachelor's or Master's degree in Computer Science, Information Technology, Data Engineering, or a related field. - Communication Skills : Excellent verbal and written communication skills, with the ability to explain technical concepts to diverse audiences. - Team & Independent Work : Ability to work effectively both independently and collaboratively in a team environment. Preferred Qualifications : - Experience with cloud platforms like AWS, Google Cloud, or Azure. - Familiarity with CI/CD pipelines, version control, and data security best practices. - Understanding of machine learning concepts and the ability to work with data scientists on model deployment. (ref:hirist.tech)

Location: bangalore, IN

Posted Date: 11/3/2024