OSB Group - Azure Data Engineering Manager - Python/PySpark

Job Description : - Overall 12-15 years of IT experience preferably in Data and Analytics implementation. - 7-8 years of Data engineering (Both On-prem and Azure) delivery experience including Architecture / analysis / development / configuration/ deployment. - 4-5 years of implementation experience in Azure Data engineering projects - Mandatory to have knowledge of Big Data Architecture Patterns and experience in the delivery of end-to-end Big Data solutions on Azure Cloud. - Expert in Python/SQL/SparkSQL/PySpark - Expert level understanding on Azure Data Factory, Azure Synapse, Azure SQL, Azure Data Lake, Data bricks, purview and Azure App Service is required. - Extensive hands-on experience implementing data migration and data processing using Azure services : Serverless Architecture, Azure Storage, Azure SQL DB/DW, Data Factory, Azure Stream Analytics, Azure Analysis Service, Databricks Azure Data Catalog, Cosmos Db, Azure Functions, ARM Templates, Azure DevOps, CI/CD etc. - Expert in at least one distributed data processing framework : Spark (Core, Streaming, SQL), Storm or Flink, etc. - Expert in distributed messaging and ingestion frameworks (Kafka, Pulsar, Pub/Sub, etc) and good to know traditional ETL tools like Informatica, SSIS, etc - Should have worked on any NoSQL solutions like Mongo DB, Cassandra, HBase, Cosmos DB etc, or Cloud-based NoSQL offerings like DynamoDB, Big Table, etc. - Good Exposure in development with CI / CD pipelines. Knowledge of containerization, orchestration, and Kubernetes engine would be an added advantage. - Designing and building of data pipelines using API ingestion and Streaming ingestion methods. - Knowledge in Azure Databricks, Azure IoT, Azure HDInsight Spark, Azure Stream Analytics, Power BI is desirable. - Strong problem solving and troubleshooting skills. - Experience in working in a fast-paced agile environment. - Proven experience in People management, hiring and capability building. - Strong experience in common data warehouse modelling principles including Kimball, Inmon. - Experience with data visualization tools such as Power BI. - Experience in Streaming and batch architecture (e.g., Kafka, Kafka streams, Spark, Flink) (ref:hirist.tech)

Location: bangalore, IN

Posted Date: 11/23/2024