AWS Data Engineer

About the Role : We are seeking a highly skilled and motivated Data Engineer/Machine Learning Engineer with expertise in AWS cloud services to join our growing data team. The ideal candidate will be responsible for designing, developing, and deploying robust data pipelines, machine learning models, and scalable applications. You will leverage AWS serverless computing, managed clusters, and database services to build and maintain our data infrastructure. This role requires a strong understanding of data engineering principles, machine learning techniques, and proficiency in relevant programming languages and tools. Responsibilities : Data Pipeline Development and Automation : - Design and implement automated data pipelines using web services that facilitate data movement and transformation. - Utilize AWS services to build scalable and reliable data ingestion, processing, and storage solutions. - Develop and maintain ETL/ELT processes to ensure data quality and consistency. Serverless Application Development : - Leverage serverless computing (e.g., AWS Lambda) to build and deploy application logic. - Design and implement microservices architecture for scalable and maintainable applications. Big Data Processing and Analytics : - Utilize managed cluster platforms (e.g., AWS EMR) to run Apache Spark for large-scale data processing. - Develop and optimize Spark applications for performance and efficiency. - Use Pyspark to enable large-scale data processing and analysis. Cloud Database Management : - Design and manage managed cloud databases (e.g., Amazon RDS, Amazon DynamoDB). - Optimize database performance and ensure data security. Container Orchestration : - Deploy and manage containerized applications using fully managed container orchestration services (e.g., Amazon ECS, Amazon EKS). - Implement containerization best practices for scalability and reliability. - Machine Learning Model Development and Deployment : - Develop and deploy machine learning models using tools like scikit-learn. - Enhance model performance and efficiency using optimized frameworks. - Use tools that explain machine model predictions. - Use tools for machine learning and data mining to facilitate building and evaluating models. Time Series Analysis and Forecasting : - Extract seasonality and trend features for time series data forecasting. Natural Language Processing (NLP) : - Use tools for natural language processing to facilitate text analysis and linguistic data processing. Mathematical and Scientific Computing : - Use technical computing tools to support advanced mathematical functions and algorithms. Programming and Scripting : - Develop and maintain code using key programming languages (e.g., Python, Node.js). - Write efficient and maintainable code for data pipelines, applications, and machine learning models. Collaboration and Communication : - Collaborate with data scientists, analysts, and other engineers to deliver data-driven solutions. - Communicate technical concepts and solutions effectively. Qualifications : - Education : Bachelor's or Master's degree in Computer Science, Data Science, or a related field. - Experience : 5-8 years of experience in data engineering or machine learning engineering. - AWS Expertise : Proven experience with AWS cloud services, including Lambda, EMR, RDS, DynamoDB, ECS, and EKS. Required Skills : Programming Languages : - Proficiency in Python and Node.js. - Experience with Pyspark. Big Data Technologies : - Strong understanding of Apache Spark and related big data technologies. Cloud Computing : - Extensive experience with AWS cloud services. - Understanding of serverless computing and containerization. Databases : - Experience with relational and NoSQL databases. Machine Learning : - Knowledge of machine learning algorithms and techniques. - Experience with machine learning libraries and frameworks (e.g., scikit-learn). Data Engineering : - Strong understanding of ETL/ELT processes and data warehousing concepts. NLP : - Experience with Natural Language Processing. Time series analysis : - Experience with time series analysis. Mathematical and scientific computing : - Experience with tools for mathematical and scientific computing. Analytical and Problem-Solving Skills : - Excellent analytical and problem-solving skills. Communication Skills : - Strong written and verbal communication skills. (ref:hirist.tech)

Location: bangalore, IN

Posted Date: 3/26/2025