Harnham
Databricks Data Engineer (Contract)
Job Location
London, United Kingdom
Job Description
Harnham has partnered with a large retail business who are developing their Databricks platform.
YOUR SKILLS AND EXPERIENCE
- Extensive experience in Spark, Databricks, and Delta Lake, particularly regarding real-time issues and partitioning strategies.
- Experience in is desired dbt (especially regarding versioning) and real-time pipelines, including formats like Avro and ORC
- Experience in streaming data and related to Azure Data Factory (ADF).
- Handling large data sizes 10 GB data with 4 GB Driver memory and understanding the impact on performance.
- Configuration with spark with shuffle partitions.
TECH REQUIRED
- Databricks, Spark, and Deltalake
- DBT
- ADF
- Large data sets
If you have experience in the above requirements and tech stack, please apply using the link and we can have a chat tomorrow morning!
Location: London, GB
Posted Date: 10/18/2024
Contact Information
Contact | Human Resources Harnham |
---|