Tested before it reaches you: inside our merge queue

How every change to Thrive gets tested before it reaches your platform

Written by
Patrick Bontoft
Lead Automation Engineer
November 17, 2026
•
3 mins to read
Glowing green icons for code review, security checks, queueing, testing and merging, linked in a row across a grid of dark tiles

Quality you don't have to think about

When you roll out learning to thousands of people, you need the platform to work, every time. So since 11th September, every change our engineers make to the Thrive web app is tested in our dedicated test environment (a full working version of Thrive) before it's accepted for release.

Pass, and it moves forward. Fail, and it goes back for a fix, long before it could reach you.

It's a simple rule with a big payoff: The version we release from is always ready to go.

Here's how it works.

Why we built a merge queue

Testing each change on its own leaves a gap. While one change is being reviewed, others are being accepted, and two changes can each pass on their own and still clash once they meet.

Picture one change that renames a button from 'Start' to 'Begin', and another, written the same week, that adds a step clicking 'Start'. Each passes its own tests, but put them together and the second one breaks.

A merge queue solves this. Approved changes line up, and each one is tested together with everything already accepted, plus anything queued ahead of it. Only changes that pass as a whole move forward.

How it works

Every change to the Thrive web app now passes three layers of checks:

  1. Peer review. Two engineers approve it, including an expert in that part of the product, alongside an AI code reviewer.
  2. Automated checks. Code quality, security scanning and automated tests run on every change, before it can join the queue.
  3. The merge queue. The change is combined with the latest version of Thrive and loaded into our test environment. Automated tests then click through it the way a real learner or admin would. Pass, and it's merged automatically, ready for release. Fail, and it goes straight back to the engineer.

Urgent fixes still have a fast, controlled route, so we can act quickly when it matters.

flowchart TD
  A["Peer review<br/>Two engineers approve, plus an AI code reviewer"] --> B["Automated checks<br/>Code quality, security and automated tests"]
  B --> C["Joins the merge queue<br/>Lined up with the latest version and changes ahead"]
  C --> D["Live test run<br/>In our test environment, scoped to what it affects"]
  D --> E{"Tests pass?"}
  E -- yes --> F["Merged automatically"]
  E -- no --> G["Back to the engineer<br/>Fix, then review again"]
  G -- fix and try again --> A
Flowchart of the merge queue: peer review, automated checks, merge queue and live test run, then merged or back to the engineer
Every approved change is tested with the latest version of Thrive before it's accepted. A failure goes back to the engineer and never reaches you.

‍

And after the queue

Passing the queue is only the first gate. Twice a day, we bundle accepted changes into a release, and each one goes through another round of checks:

  1. Full test suites run again, this time against our real back-end services.
  2. Our product squads explore it by hand, looking for anything automation might miss.
  3. Our quality assurance (QA) team signs it off.
  4. It reaches staging and sandbox first, then production.

If a release fails any of these checks, it's held back. We either fix it quickly or take the change out, so the problem never reaches you.

Rolled out with care

We introduced the queue in stages. From 25th August, every change went through it in a trial mode, so the team could get used to the new flow with nothing at risk. On 11th September, the live tests became the gate.

Every part of the queue has its own on/off switch, so we can fine-tune it without ever lowering the bar.

Fast as well as thorough

A quality gate only works if it keeps pace with the team, so we built ours for speed:

  • Full test runs around 40% faster than when we started, and still improving
  • Only the tests that matter. Each change runs the tests for the parts of Thrive it could affect, while changes to shared building blocks run everything.
  • 27 machines working at once, sharing the tests between them
  • Flaky results re-checked automatically, so engineers only spend time on real issues

None of this arrived in one go. A small, focused group tuned the queue in short cycles, learning from every run and acting on feedback from our engineering guilds along the way.

As a result, our engineers spend less time chasing bugs late in a release, and more time building the things you've asked for.

What it means for you

  • Fewer bugs reach your platform. Problems are caught before a change is accepted, long before release.
  • Improvements reach you sooner. Because the version we release from is always ready, new features and fixes don't wait on a long testing phase.
  • Confidence in every release. Automated tests guard every change, with manual testing as an extra check on top.

And it's working. In the five weeks since the queue went live, our web app team released 63% more often, in smaller, lower-risk batches. Changes are also reaching release faster, as part of wider improvements across our engineering teams.

Thrive web app Since the queue went live
How often we release 63% more often
Size of each release 14% smaller
Time from first code to release 18% faster

‍

Eight weeks before go-live compared with five weeks after (go-live: 11th September). Source: Swarmia production data.

What's next

We're already working on the next steps:

  • Even broader coverage, with the last group of content tests joining the queue in November
  • Testing related changes together, so improvements reach you even sooner
  • More testing capacity, so the queue keeps pace as we grow

As with our work on accessibility, the aim is simple: build quality in from the very start.

Thanks to Nathan Atherton, Alexander Young, Leigh Darlow, Ben Robinson-McBride, Bobby Ross and the wider front-end team for making it happen.

The latest news from Thrive

Double win for Thrive customers Nando’s and Yopa at the Learning Technologies Awards 2026
Read more

Why teams love Thrive

Learning, AI and mentoring in one platform, built to make a real difference to business performance.
See what makes us different