About the Company
My Company is based in London and appears to operate at the intersection of data and artificial intelligence, likely serving organisations that rely on sophisticated analytics and machine learning to drive decisions. The team works with modern cloud and ML infrastructure, suggesting a technically mature environment where data products are built and scaled with care. This is likely a role within a dedicated AI function, indicating that machine learning and data engineering are central to the business rather than peripheral initiatives.
About the Role
This is a senior individual contributor role sitting within the AI department, responsible for shaping how analytical and machine learning systems are built, optimised, and maintained at scale. You will work across the full lifecycle — from refining model performance and hardening LLM pipelines to establishing MLOps practices on Databricks and AWS. The position carries real architectural influence, with the opportunity to set standards that the wider team builds on.
Responsibilities
- Design, build, and maintain robust data and ML pipelines on Databricks, ensuring reliability, observability, and scalability across production workflows.
- Lead model optimisation efforts — identifying performance bottlenecks, tuning hyperparameters, and improving inference efficiency across both classical and large language models.
- Implement LLM hardening practices including prompt engineering guardrails, output validation, safety layers, and evaluation frameworks to ensure production-grade reliability.
- Own and evolve the MLOps practice: manage experiment tracking, model versioning, and deployment workflows using ML Flow and associated tooling.
- Architect and maintain cloud infrastructure on AWS to support data ingestion, model serving, and pipeline orchestration.
- Partner with data scientists, ML engineers, and product stakeholders to translate analytical requirements into scalable, production-ready solutions.
- Define and enforce engineering standards, documentation, and best practices across the AI team to raise the quality bar for all data and model assets.
What we're looking for
- 10+ years of experience in analytics engineering, data engineering, or machine learning engineering, with a consistent track record in senior or principal-level roles.
- Deep hands-on expertise with Databricks, including Delta Lake, Spark-based workflows, and cluster optimisation.
- Proven experience with MLOps tooling — specifically ML Flow for experiment tracking, model registry, and deployment pipelines.
- Strong working knowledge of AWS services relevant to ML workloads (e.g. S3, SageMaker, EC2, Lambda, IAM, and related services).
- Demonstrated experience optimising ML models for performance, cost, and latency in production environments.
- Practical experience working with large language models — including fine-tuning, evaluation, and production hardening techniques.
- Ability to communicate complex technical decisions clearly to non-technical stakeholders and influence architectural direction across teams.
Nice to have
- Familiarity with LLM evaluation frameworks such as RAGAS, LangSmith, or similar tools used to assess output quality and safety.
- Experience with infrastructure-as-code tools (Terraform, CDK) for provisioning and managing AWS environments.
- Exposure to feature store implementations or real-time model serving architectures.
- Background in dbt or similar transformation frameworks used alongside analytics engineering workflows.
Skills
- SQL
- dbt
- Python
- Data Modeling
- ETL/ELT Pipelines
- Apache Spark
- Data Warehouse Platforms
- Version Control (Git)
- Business Intelligence Tools
- Data Quality & Testing
- databricks
- langchain
Benefits
- Competitive salary benchmarked to principal/staff-level compensation in the London market.
- Hybrid working model with flexibility to balance time in the London office with remote work.
- Direct influence over technical direction within a dedicated AI function — not just executing, but shaping how things are built.
- Access to modern tooling and cloud infrastructure with the autonomy to drive improvements in engineering practice.
- Collaborative environment with experienced engineers and data scientists working on substantive AI problems.
- Opportunities for continued professional development, including conference attendance, certifications, and internal knowledge-sharing.