Job Reference: 250702001
• Job Title: Databricks Platform Lead
• Type : Hybrid / Contract
• Location: Coventry
• Salary: £300-350/day (Outside IR35)
• Experience : 5+ years
About the Role
We’re looking for an experienced and motivated Databricks Platform Engineer to provide critical support and optimization for Databricks environments on AWS. If you’re passionate about cloud infrastructure, big data, and automation, this role is an excellent opportunity to work on cutting-edge technology alongside dynamic engineering and DevOps teams.
Key Responsibilities
Platform Support
• Provide daily operational support for Databricks on AWS
• Monitor and troubleshoot clusters, jobs, workspaces, and other Databricks re-sources
• Manage access controls and data security policies
• Implement and maintain Unity Catalog and Medallion architecture
• Handle incidents, requests, and environment changes.
• Collaborate with cross-functional teams to resolve complex issues
• Monitor and support Databricks ML flows
Performance & Optimization
• Analyze workloads and optimize cluster performance and cost-efficiency
• Recommend best practices for Apache Spark job execution and data processing pipelines
AWS Infrastructure Management
• Configure and manage AWS services including S3, EC2, IAM, Lambda, and VPC
• Ensure robust security and compliance within AWS-Databricks integrations
Automation & DevOps
• Create and maintain automation scripts for monitoring and operational maintenance
• Work with DevOps to automate infrastructure provisioning, scaling, and CI/CD pipelines using tools like Jenkins, Git, and Terraform
Qualifications
• Bachelor’s degree in Computer Science, IT, or related field
• 5+ years of experience supporting Databricks or similar big data platforms
• Expertise in Databricks on AWS (cluster tuning, Spark execution, job management)
• Strong working knowledge of AWS services (EC2, S3, IAM, Lambda, RDS, VPC)
• Proficient in Python, Scala, or SQL for data engineering tasks
• Deep understanding of Apache Spark, ETL pipelines, and data architectures
• Familiar with DevOps practices and tools (Terraform, Ansible, Jenkins)
• Experience with monitoring platforms such as CloudWatch or Datadog
• Excellent analytical, problem-solving, and communication skills.
Please provide your recent CV with the below details
Notice Period –
Residency Status in UK –
Visa Expiry Date (if applicable) –
Consent to share your data with the client – Yes/No
Agreed to work on the given rate – Yes/No