Reconstructing Test, QA and Release Evidence in IT Projects

How test and release evidence is tied to concrete source revisions, builds, environments and deployment states so that the limits of each result remain visible.

By Mathias Ellmann English version: 22 October 2026

IT Projects & Technical Due Diligence

Test and release records are technically useful only when they are tied to a concrete source, build and deployment state. The examination therefore reconstructs what was actually tested, what was not tested, which QA findings existed, which artefact was built, and whether the delivered release can be traced back to the examined source state.

Short answer

Green tests for commit A do not automatically establish the quality of production artefact B. The evidence chain must connect source revision, pipeline, test environment, build artefact, approval and deployment.

QA findings and release decisions should remain separate. A defect can be known and nevertheless accepted for release; conversely, an approval record does not establish that every technical requirement was tested.

Examination chain From a test record to a traceable released artefact
1 Source state Fix the revision and relevant configuration under test.
2 Test execution Record suite, environment, data, result and omissions.
3 Build provenance Link build inputs and artefact identity to the source state.
4 Release state Determine which artefact was approved and actually deployed.

1. Treat test evidence as context-bound evidence

A test result needs version, environment and conditions

Unit, integration, system, acceptance, performance and security tests answer different questions. The examination records which test type ran, against which revision, configuration, data set and environment.

A result such as 'passed' is not self-sufficient. The relevant test definitions, expected results, execution date and any skipped or quarantined cases determine its technical meaning.

Manual testing can be valid evidence when its procedure and observed result are documented. It should not be treated as equivalent to an automated test solely because both use the same status label.

2. Make test scope and untested areas visible

Absence of failure is not evidence of complete coverage

The examination maps tests to requirements, components or risk areas where that mapping exists. Missing tests, disabled suites, flaky tests or environment exclusions are recorded rather than hidden by an overall pipeline status.

Coverage metrics can help describe exercised code or scenarios, but they do not by themselves prove requirement completeness or defect absence.

Productive configuration, external dependencies and large-scale data can differ from the test environment. The transferability of a test result should therefore be bounded.

3. Separate QA findings from release and approval decisions

A known issue and a release decision are different events

QA systems can record defects, severity, retest status and known limitations. Release boards or product owners may make a separate decision to proceed despite unresolved findings.

The technical report reconstructs both layers without inferring that an approval means the finding was invalid or that a finding automatically made the release impermissible.

Where acceptance or sign-off documents exist, they should be tied to the exact build and scope they refer to.

4. Check build and release provenance

The tested source state must connect to the delivered artefact

Build records, artefact hashes, image digests, tags and provenance metadata can establish which inputs produced a release artefact. SLSA provides a model for structuring provenance information around build inputs and process.

If such links are absent, the examination should not assume that a deployed binary came from the tested revision merely because the dates are close.

Manual rebuilds, post-build configuration changes and locally replaced artefacts can break the automated evidence chain even when the pipeline itself was green.

5. Interpret security and quality tests appropriately

A tool result is not a universal quality conclusion

Static analysis, dependency scanning, dynamic testing and penetration testing each observe different properties. A clean result within one tool or rule set does not establish absence of all vulnerabilities.

Likewise, quality gates can encode thresholds selected by the project. Passing a gate establishes that the configured rule passed; it does not independently establish that the rule was sufficient for every requirement.

The report records the test or tool version where relevant, because rule sets and detection capabilities can change over time.

6. Preserve temporal consistency of the evidence chain

Evidence must refer to the same release lineage

Tests, build, approval and deployment should be ordered and connected to the same lineage. If a later configuration package, secret or manual patch changes production, the earlier test result may no longer describe the complete productive state.

Release evidence should therefore identify the artefact and, where possible, the source revision and build process from which it was produced.

The conclusion states where the chain remains complete and where manual or undocumented steps limit traceability.

Examination questions

Does the test and release evidence describe the artefact that was actually delivered?

  • Are test type, source revision, environment and data documented?
  • Are skipped, missing or failed tests visible in the evidence set?
  • Are QA findings distinguished from approval or release decisions?
  • Can the build artefact be traced to concrete source inputs?
  • Is the approved artefact the same artefact that was deployed?
  • Are manual rebuilds or post-build changes documented?
  • Are security and quality tool results interpreted within their configured scope?
  • Are gaps in the evidence chain reported as uncertainty rather than as a passed test?

Constructed example

A pipeline shows complete green tests for commit A. Production, however, contains artefact B, and the available build record for B does not identify a source revision. The technical finding is therefore not 'the production release passed all tests' but that the evidence chain between the tested source state and the deployed artefact is interrupted.

If a later provenance record connects artefact B to commit A and no post-build changes are documented, that additional evidence can strengthen the assignment.

The example is entirely constructed and does not describe a real engagement.

Sources and technical basis

  • International standard ISO/IEC/IEEE 29119-1:2022, used for general software-testing concepts. Open source.
  • NIST guidance NIST SP 800-218, Secure Software Development Framework Version 1.1, used for secure software-development practices across the SDLC. Open source.
  • Technical specification SLSA v1.2, used as a model for software-supply-chain provenance. Open source.
  • Author methodology Sources and Methodology of the German knowledge base. Open source.

Further reading

Critical review and methodological limitation

Test evidence proves that defined checks were performed under defined conditions. It does not prove software is defect-free, that every requirement was tested or that an approval has a particular legal effect.

Release traceability is therefore a chain problem: source, test, build, artefact and deployment must remain connected.

  • Coverage: passed tests do not establish untested behaviour.
  • Provenance: a build result needs a technical link to the delivered artefact.
  • Approval: QA findings and organisational release decisions are distinct evidence classes.
Examination rule: State test and release conclusions only for the documented revision, environment, procedure and artefact. Broken provenance, untested areas and manual release steps remain explicit limitations.

Translation and methodological scope

This English article is based on the reviewed German V1 source article. The translation preserves the technical meaning, methodological distinctions, source status and stated limitations. It does not strengthen technical findings into legal conclusions and does not treat technical references as proof of compliance or suitability for a specific system.

Source relationship: The corresponding German article remains the reference baseline during English V1 development. Open the German source article.

Author's technical literature and publications

The following works provide additional technical and methodological context. They do not replace required primary sources, standards, manufacturer documentation or other independent specialist sources. Unpublished manuscripts, drafts and teaching materials are deliberately not presented here as public sources.

Mathias Ellmann, IT expert

IT Project Assessments & Project Reconstruction

Traceable reconstruction of requirements, change states, repositories, releases, technical decisions and project risks.

Explore this assessment service