QA The Other WayQuality assurance for the age of AI-written tests. QA The Other Way.
Practical QA

A soft check still needs a hard stop.

Collect independent failures without letting a failed prerequisite authorize the next action.

5 min readQA The Other Way
An authored observation checklist and hard decision gate before a dependent action.
Authored editorial illustration.
Illustrative video for this article. English on-screen text; select subtitles for this language.
Collect independent failures without letting a failed prerequisite authorize the next action.
Illustrative local test-runner example with invented inputs, not a customer or vendor interface.

A review page has the wrong title and the wrong draft status. A normal assertion stops at the first mismatch. A soft assertion can collect both. That is useful diagnostic evidence, but the test is still failed. Continuing execution is not permission to continue the product workflow.

This guide uses Playwright's assertions documentation with an invented local review page. There is no real approval, submission or account. We want several independent observations, followed by a clear decision gate before a button that represents the next step.

Collect observations that remain meaningful

Playwright documents that failed soft assertions do not terminate execution but mark the test as failed. They work with the Playwright test runner. A browser script importing a different assertion library does not automatically gain this behavior.

Use soft checks when the later observation still makes sense after an earlier mismatch. A wrong page title need not prevent inspection of an independently visible status. A missing prerequisite record is different: trying to read its fields may only produce confusing secondary errors.

await expect.soft(page.getByRole('heading')).toHaveText('Review demo draft');
await expect.soft(page.getByRole('status')).toHaveText('Ready for review');
expect(test.info().errors).toHaveLength(0);
await page.getByRole('button', { name: 'Continue demo' }).click();

The example assumes the controlled fixture and imported test and expect from the runner. The hard assertion on recorded errors is intentional. If either soft check failed, the Continue action is not reached. Its label refers to a local specimen, not a safe default for a real approval button.

Put the gate before dependent actions

Place the hard stop before an action whose meaning depends on the checked state. Putting it at the end of the test is too late: the test might already have acted on the wrong record. Choose the gate from the workflow's actual dependencies, not from a desire to collect the largest possible error list.

The documentation exposes recorded failures through test.info().errors. Do not clear that array or edit the test status to make a diagnostic run appear successful. The collected failures are the result. A later useful screenshot does not erase them.

Keep subsequent read-only observations only while they are meaningful and permitted. In a real authenticated test, even a read may reveal private data in traces. Use approved staging records and review artifacts before sharing them.

Verify the negative case, not only the green one

Our local specimen includes a failing heading and status. The runner records both mismatches, then the hard gate stops execution. A separate after-test assertion checks that the local Continue button's state was never changed. The expected failing run is inspected as failure; it is not relabelled a production pass.

A second controlled case supplies the correct heading and status. It crosses the gate and changes a local marker. Together these cases demonstrate control flow. They do not prove that a server prevented an unauthorized operation or that every generated test uses the same guard.

If a prerequisite is itself safety-critical, a normal hard assertion may be the simpler choice. Softness is a reporting tool, not a requirement to delay every failure. A readable suite uses both forms where their behavior fits.

Leave a decision rule for review

Record which checks are independent, which action depends on them and where execution must stop. That note helps a reviewer spot a generated journey that collects failures but still advances the workflow.

AnyTest describes application exploration and end-to-end tests for human review. Ask whether a proposed test preserves its failed state and guards dependent actions. No specific AnyTest soft-assertion implementation is claimed. More diagnostics are useful only when they do not weaken the decision boundary.

Common questions

Does a failed soft assertion keep the test green?

No. It lets execution continue while marking the test failed.

Where should a hard gate go?

Before the dependent action, not only at the end of the workflow.

Sources