Test automation · portfolio project

Toolshop: a quality cycle, end to end

32 automated checks against an e-commerce storefront, covering the REST API, the browser and the full purchase flow, driven by a written risk analysis rather than by whatever was easy to automate. Every push runs all of them against a copy of the application that the pipeline starts itself.

Python · pytest · Playwright · httpx · Allure · Docker · GitHub Actions

The run this page was built from

  • 32checks of the application
  • 32passed
  • 0failed
  • 9in the smoke set

They divide into 11 against the REST API, 19 through a browser and 2 end-to-end through the guest purchase flow.

The same run also exercised 80 checks of the framework itself: its configuration, and the build of this page. All of them passed. They are counted apart from the figures above because they prove nothing about the storefront; they are counted at all because the tooling that produced this page is tested too.

Finished 06 October 2026, 15:14 UTC against revision 4510b4d, from a run of 112 results in all. Every number on this page is read out of that run's Allure results when the page is generated; none of them is typed in by hand. The pipeline run that produced them is public.

A test, as it runs

The guest checkout case, recorded by the browser during the run above: catalog, cart, delivery details, payment method. No narration and no cuts.

How it is put together

Three bands beside a shared column: the test modules tests/api, tests/e2e and tests/ui with their case counts; the code they reuse, src/toolshop/api and the page objects in src/toolshop/ui/pages; and the application under test in Docker: the storefront on port 4200, the REST API on port 8091 and MariaDB behind it. The API tests reach the API through the client, the browser tests reach the storefront through Playwright driving Chromium and the end-to-end tests use both; the column on the right holds what every layer shares, config.py, the data factory in data/factories.py and the fixtures in tests/conftest.py, together with the evidence those fixtures leave behind, allure-results and the video.
Tests hold the assertions; everything reusable sits below them, so a change in the application lands in one file.
Three lanes. On every push to main and every pull request: start the stand from four pinned images and seed it, run every layer with writes enabled and video recording on, and collect allure-results and the videos; on main those same artefacts go on to restore the Allure history from the published site, generate the report, build this page and deploy the site to GitHub Pages, which serves it. A third lane is a separate workflow, started by hand and never on a schedule, against the public site: the API suite if its API answers, the browser suite only if a probe answers HTTP 200, and otherwise a skip carrying its reason, not a failure.
The same pipeline that gates a push also publishes this page, so what you are reading came out of a run that had to pass first.

How the coverage was chosen

The suite starts from a ranked list of what costs the business money if it breaks, scored by impact and by how likely the feature is to break: eleven risks, each one traceable to the test that covers it. Six of them are rated high or critical: a wrong total at checkout, add-to-cart failing silently, a guest who cannot finish an order, a catalog that renders empty, search that answers with nothing useful, and a contact form that accepts invalid data or drops valid data. The smoke set is those six and nothing else: 9 checks, each at the layer where the failure shows soonest. If one of them fails, running the rest wastes the pipeline's time. The five risks below them are scored medium or low and stay out, because a smoke set that covers everything gates nothing.

A check then belongs at the lowest layer that can still prove the thing that matters. Pagination, price bounds, response schema and error codes are data rules, so they are asserted against the REST API, where a failure names its own cause and needs no browser: 11 cases. Only behaviour a person can actually see is driven through one, which is the 19 browser cases: rendering, sorting, search, cart arithmetic, form validation. The 2 end-to-end cases exist because the guest purchase path is the one flow whose whole value is that the separate parts hold together.

What this suite deliberately does not cover

  • Accessibility and performance. Both need their own tooling and their own time budget. A green functional suite says nothing about either, and implying otherwise would be the dishonest part.
  • Security. There is one check that user input is not reflected back as markup. That is a sanity check, not an assessment.
  • Cross-browser rendering. The suite runs on Firefox and WebKit on request, but rendering differences are not asserted anywhere.
  • Payment processing. The end-to-end flow stops at choosing a payment method: the demo application has no processor behind it, so going further would assert a fixture, not the product.

The reasoning itself is written down and kept in the repository: scope and requirements, the risk analysis, the test strategy, a worked test design, defect reporting and environments and CI.

Three decisions that keep it maintainable

Locators are the application's own data-test attributes, wherever it provides one

They are part of the markup contract rather than of its presentation, so rewritten copy or a restyle does not turn into a suite-wide failure that has to be triaged before anyone can tell whether the product broke. A few controls carry no such attribute: the checkout's guest tab is one, and it is addressed by its routing fragment, a[href="#guest-tab"], which is structural for the same reason. Visible copy is never a locator.

There are no fixed sleeps

Sorting, searching and filtering refetch the product grid. The suite waits for that response and then for the grid to actually re-render, comparing a fingerprint of the rendered names. Waiting on the response alone still races the render, and that race is what a sleep(2) hides on a fast machine and fails on a slow one.

One suite, two environments with different write permissions

The public demo site is shared with everyone else practising against it, so there writes are intercepted in the browser and the payload is asserted instead of being sent, which also catches the bug class where a field is renamed or dropped on its way out. Against the stand the pipeline starts from images pinned by digest, the same tests run with writes enabled and the order-placing case included. One suite, two contracts about what it is allowed to touch.

The full report

Every case, its steps, its duration and the trend across runs. Failures carry a full-page screenshot, the page HTML, the URL and the browser console, and are grouped by cause — product defect, timeout, selector drift, infrastructure — because that is what decides who picks the failure up.

Open the Allure report Repository and documentation