Location: Hybrid (London Office)
Salary: Competitive
About Us
We are an AI-powered consumer intelligence company transforming qualitative research through cutting-edge automated moderation and real-time insight generation. Our platform is powered by advanced Large Language Models (LLMs), GenAI-driven analytics, and a modern cloud data ecosystem across AWS, Azure, and MongoDB.
As we expand our GenAI capabilities, we are seeking an AI Engineer focused on building, refining, and productionising LLM-driven product experiences.
The Role
You will design, refine, and productionise the LLM systems that power our platform — from AI moderators and synthetic respondents to automated insight-generation models, conversation flows, and evaluation frameworks.
This role is ideal for someone who loves shaping model behaviour, running structured experiments, and shipping GenAI features into production with measurable quality.
What You’ll Do
LLM & GenAI Behaviour
Design prompts, reasoning flows, system messages, tool-use patterns, and intelligent behaviours for AI moderators and synthetic respondents.
Build multi-turn conversational experiences that feel natural, consistent, and aligned with qualitative research standards.
Prototype new GenAI features, including automated insight extraction, generative UX testing, and improved moderation capabilities.
Run structured experiments to optimise model performance, prompt strategies, and inference settings.
Fine-tuning & Model Iteration
Fine-tune LLMs using curated corpora (LoRA/QLoRA, supervised fine-tuning, preference/RLHF-style workflows).
Curate and maintain fine-tuning and evaluation corpora for behaviour shaping, tone control, safety, and domain adaptation.
Evaluate tradeoffs between prompting, retrieval, and fine-tuning for different product needs.
RAG, Embeddings & Retrieval
Build and iterate retrieval-augmented generation (RAG) pipelines for accurate, context-rich outputs.
Develop embedding and retrieval workflows, relevance tuning, and grounding strategies to reduce hallucinations and improve reliability.
Work with vector stores / search layers to optimise latency and output quality.
Evaluation & Quality Systems
Develop automated evaluation frameworks to measure LLM accuracy, hallucinations, tone, safety, and behavioural consistency.
Build benchmark datasets and test suites for conversation quality, insight accuracy, and model reliability.
Diagnose model failure modes such as drift, repetition loops, or behaviour degradation.
Implement human-in-the-loop evaluation workflows for calibration and continuous improvement.
Production & Collaboration
Work with engineering to deploy LLM-driven features and integrate evaluation into CI/CD pipelines.
Partner with product and research teams to ensure AI behaviours align with qualitative research needs.
Monitor model quality in production and drive iterative improvement.
Requirements
3+ years of experience in AI/ML engineering, LLM development, or applied NLP.
Hands-on experience fine-tuning LLMs (HuggingFace, LoRA/QLoRA, custom corpora; preference/RLHF-style training a plus).
Strong Python skills and comfort working in modern ML stacks.
Practical experience with embeddings, vector retrieval, and RAG systems.
Experience designing experiments and evaluating model behaviour rigorously.
Ability to communicate clearly with both technical and non-technical stakeholders.
Bonus Points
Experience with multi-agent orchestration, tool-using agents, or complex conversation frameworks.
Familiarity with MLOps tooling such as SageMaker, MLflow, Databricks, or Azure ML.
Experience with vector databases or retrieval platforms (Pinecone, Weaviate, OpenSearch).
Knowledge of alignment techniques, safety evaluation, or interpretability.
Interest in consumer behaviour, research methodologies, or product decision-making.
Startup experience or comfort with high-velocity environments.
Why Work for Bolt Insight?
Hybrid working — home or London office (UK work rights required).
Private Medical Insurance via Vitality.
Nest Pension contributions.
Employee referral bonuses.
Virtual and in-person team gatherings.
Recognition awards for outstanding contributions.
Annual bonus — up to two months’ salary based on company performance.
A unique opportunity to shape the future of AI-driven qualitative research through advanced LLM and GenAI systems.