Today I published version 2.4.0. This one is mostly about reports: what you get to see after the agent tests your app.

It is live on GitHub and on npm as swipium@2.4.0. If you are new to the project, the 2.3.5 post covers the previous release, and the docs cover the setup.

Until now, when the agent finished a run, you mostly had to trust what it told you in the terminal. With this version, Swipium writes a report you can open in the browser, with the steps the agent followed and screenshots of what it saw.

report.html

Every session now writes a report.html file. When the run ends, the agent gives you the path, and you open it in any browser. It works offline, it doesn't need a server or an account, and it has no JavaScript.

report.html for a test session: release gate BLOCK, and the failing checkout step with its screenshot of a React Native red screen and the failure code REDBOX
report.html starts with what failed: the step, its screenshot and the error code.

The first thing in the report is what failed: the step, the screenshot, the error code and the last lines of the log. After that you have every result with its evidence, the full list of steps, the device, and the files from the run. If the run couldn't test the app at all, for example because the build failed or Metro wasn't running, the report starts with that, how to fix it, and the end of the related log.

Screenshots are embedded as small thumbnails, up to 8 MB in total. If there are more, they are listed as file paths.

The steps behind each result

Swipium now keeps a timeline of what happened in the session: taps, typed fields (never the typed text), flow steps, smoke checks and notes, in order, with how long each one took and whether it worked. When a step fails, it keeps a screenshot and the screen's UI tree, so you can see what the agent was looking at.

When the agent records a pass or a fail with qa_note, Swipium also takes its own screenshot at that moment when it can, and links it to the steps since the previous check. If a result has no evidence at all, the report says so.

I also added two flags. A pass is marked contradicted if there was a crash or a failed flow step in its steps, and a test that failed and then passed is marked flaky. Swipium doesn't decide if the agent is right or wrong, it just shows what happened so you can check it.

Reports in CI

In CI, swipium report --format html renders the same report as one file, and you upload it as an artifact. In GitHub Actions it's two steps:

- name: Swipium HTML report
  if: ${{ !cancelled() }}
  run: npx -y swipium report --latest \
    --format html --out swipium-report.html

- name: Upload the HTML report
  if: ${{ !cancelled() }}
  uses: actions/upload-artifact@v7
  with:
    path: swipium-report.html
    archive: false

Before, the recipe uploaded the whole ~/.swipium/runs folder, which has raw screenshots and logs. Now it's just this one file. The GitHub job summary also starts with what failed, and JUnit is cleaner: real error codes, tool or environment problems reported as errors instead of app failures, and one test case per workflow. The CI reports guide has the full recipe.

Other changes

The summary the agent reads at the end of a run is now 2 KB at most: the verdict, what failed with the step and screenshot, and what to do next. This saves tokens and makes it easier for the agent to fix the right thing.

Every screenshot Swipium takes now follows the same rule: if a password or OTP field is on screen, the screenshot is not saved. Log lines in the HTML report are also cleaned of secrets and tokens.

Retries are handled better. If a flow fails and then passes when you run it again, the release gate passes and marks it as flaky, but a crash still blocks. A later skipped or blocked result doesn't hide an earlier failure, and the agent's own note can't clear a failure Swipium found.

Upgrading from 2.3.5

Restart your MCP client after upgrading. Some things change:

CI recipe: render swipium report --format html --out swipium-report.html and upload that file, instead of the whole ~/.swipium/runs folder.

JUnit: tool and environment problems now show as errors, not app failures, and only high-severity findings are test cases, so your counts will change.

Release gate: a flow that fails and then passes on a retry now passes, marked as flaky.

SARIF is now experimental and opt-in, and the Playwright export is deprecated. Still 55 tools, and Node.js 22.12 or newer.

The full upgrade checklist is in the changelog. To try it, upgrade, run one test and open the report:

npx -y swipium@2.4.0

Thoughts

These past few weeks I came across several open-source projects, not QA ones, that have a strong UI and UX side. Until now Swipium has been pretty agent-oriented, and that was (and still is) the intention. My idea was always that you connect it to Claude and it works on its own for you, but I like seeing evidence too: screenshots, what was failing and how it got fixed, etc.

On top of that, the reporting side needed work, and that's what this release focused on. Now the work is more shareable and more tangible than just a check in the terminal. I'm thinking about further improvements here, but this version is already a big improvement.