Engineering note · Validation practice
Make a validation finding reviewable: context, expectation, evidence
A short verdict can hide a long chain of assumptions. A useful validation finding gives another engineer enough context to understand what was tested and why the result follows.
Start with a question that can be answered
Consider an illustrative test: after a user changes a setting, a connected-service interface should display the new value. Before testing, specify the environment, software version, account state, transport availability and expected response. Define any timing limit from the requirement before execution.
- Prerequisites: target, version, initial state and required connectivity
- Expected behaviour: the observable state or response that satisfies the requirement
- Evidence: the recording, log interval or screenshot supporting the observation
- Decision: pass, fail or blocked, with the reason
Keep observation separate from interpretation
A request accepted by an API establishes an acknowledgement. A refreshed value in the interface establishes a displayed state. Neither alone establishes every intermediate system action. Describe the observation first, then explain which part of the requirement it supports.
If transport is unavailable, the test may be blocked rather than failed. Record the missing prerequisite so that a rerun answers the same question under the required conditions.
Preserve the route back to the evidence
Use stable names for the reference, input log and exported finding. Include enough timing and version context for another reviewer to find the relevant event. When a result is shared outside the test environment, replace customer or vehicle identifiers with synthetic or redacted examples.
NeonObservatory addresses the comparison and reporting part of this workflow. ADB Validation Console addresses recurring diagnostic actions and evidence capture. They illustrate why the review path is a product requirement in its own right.
Review before declaring success
Ask whether the evidence proves the requested outcome, whether a prerequisite was missing and whether a different explanation remains plausible. A good report makes these limits visible without burying the reviewer in unrelated logs.
- Can another engineer locate the expected behaviour?
- Can they find the observation in the attached evidence?
- Does the verdict stay within what was actually observed?
- Is the next action clear when the test is blocked or fails?