Senior Data Engineer - ETL/Python

Acquia empowers the world's most ambitious brands to create digital customer experiences that matter. With open source Drupal at its core, the Acquia Digital Experience Platform (DXP) enables marketers, developers, and IT operations teams at thousands of global organizations to rapidly compose and deploy digital products and services that engage customers, enhance conversions, and help businesses stand out. Headquartered in the U., Acquia is a Great Place to Work-CertifiedTM company in India, is listed as one of the world's top software companies by The Software Report, and is positioned as a market leader by the analyst community. We are Acquia We are building for the future and we want you to be a part of it. About The Role : As a Senior Data Engineer with expertise in building Enterprise Data Platforms (EDP), you will play a pivotal role in creating a robust and scalable data infrastructure. You will be responsible for designing, building, and maintaining data pipelines that will power various analytics and AI-driven products across the company. Job Responsibilities : - Data Pipeline Design & Development : Lead the design, development, and optimization of data pipelines that enable the collection, transformation, and storage of data from multiple sources like OLTP databases, operational applications (Salesforce, Jira), event collectors, logs, 3rd-party APIs, and file storage (e., S3). - Ingestion & Integration : Build and manage data ingestion processes using tools like Meltano and orchestrate workflows with Dagster, ensuring scalability and real-time data processing. - Data Storage & Management : Implement data storage solutions leveraging Snowflake for data warehousing and S3 for data lakes, optimizing for cost, performance, and security. - Data Transformation : Work with the data transformation layer using dbt and ensure seamless integration with orchestration layers to maintain real-time and batch processing efficiency. - Data Governance : Establish data governance frameworks and data catalogs (e., DataHub) to ensure data quality, discoverability, and security across the platform. - Collaboration : Collaborate with stakeholders across the organization, including data scientists, product managers, and business analysts, to translate business needs into technical solutions. - Real-Time Analytics : Implement real-time analytics solutions by building and maintaining a vector database and real-time analytics database to support machine learning models and predictive analytics. - Semantic Layer & AI Integration : Design and implement a semantic layer (e., cube.js) to facilitate user-facing analytics while integrating AI services and vector databases for advanced analytics. Skills : - 5 years of experience in data engineering or data platform development. - Experience with data engineering tools and frameworks (e., Meltano, dbt, Dagster). - Strong hands-on experience with data lakes (e., S3), and data warehouse tool Snowflake. - Strong hands-on experience with cloud platforms (AWS, GCP, or Azure). - Proficiency in SQL, Python, and ETL/ELT design patterns. - Understanding of real-time analytics and vector databases. - Strong knowledge of data governance practices and cataloging tools like DataHub. - Experience working with cross-functional teams and translating business needs into technical requirements. - Proven expertise in building enterprise-level data platforms. - Experience with real-time data processing and analytics tools. Preferred Qualifications : - Excellent communication skills with the ability to explain technical concepts to non-technical stakeholders. - Familiarity with dashboarding and reporting tools like Domo. - Experience with AI/ML-driven platforms and integrating AI services for advanced analytics. - Acquia is proud to provide best-in-class benefits to help our employees and their families maintain a healthy body and mind. - Core Benefits include: competitive healthcare coverage, wellness programs, take it when you need it time off, parental leave, recognition programs, and much more. - Individuals seeking employment at Acquia are considered without regard to race, color, religion, caste, creed, national origin, age, sex, marital status, ancestry, physical or mental disability, veteran status, gender identity, or sexual orientation. (ref:hirist.tech)

Location: in, IN

Posted Date: 10/27/2024