Data Architect - AWS Cloud

Job Description : - Design and implement AWS-based Lakehouse architectures using services like S3, Glue, and EMR, Spark Scala to enable modern, scalable data lakes and data warehouses. - Architect scalable, high-performance data pipelines for ingestion, transformation, and processing of structured and unstructured data. - Develop and manage ETL processes using AWS Glue/EMR for data integration, transformation, and loading. - Optimize and manage AWS EMR clusters for big data processing, including Apache Hadoop, Spark, and other distributed computing tools. - Leverage S3 for data storage with a focus on performance, security, and data lifecycle management. - Collaborate with data engineers and scientists to support analytics, machine learning models, and reporting tools using the AWS data ecosystem. - Integrate data sources across various platforms (on-premises and cloud) and ensure smooth data movement and processing. - Perform performance tuning and optimization of data pipelines and distributed processing systems for large datasets. - Provide ongoing support and monitoring for data architectures, including troubleshooting, error resolution, and system enhancements. Required Qualifications : - Bachelor's or master's degree in computer science, Information Technology, Data Science, or related field. - 5 years of experience in data architecture with a strong focus on AWS services (EMR, Glue, S3, Lambda, etc.). - Hands-on experience in designing and deploying data lakes and Lakehouse architectures. - Proficiency in big data processing tools such as Apache Spark, Hadoop, Hive, and Presto. - Expertise in building and managing ETL pipelines using AWS Glue/EMR, AWS Step Functions, - Experience with SQL and Scala for data processing and scripting. - Knowledge of data governance, data quality, and security practices on AWS. - Strong experience with data modeling and best practices in schema design (e.g., partitioning, compaction, etc.). - Familiarity with AWS Redshift, Athena, and other data query services. - Strong problem-solving skills, with the ability to troubleshoot complex data environments. - Excellent communication and collaboration skills to work across teams and stakeholders. (ref:hirist.tech)

Location: bangalore, IN

Posted Date: 10/31/2024