Test Traceability Matrix

From Webhuis wiki
Jump to navigation Jump to search

Test Traceability Matrix[edit]

Field Value
Document ID PGCBL-TTM-001
Document Type Test Traceability Matrix
System pgcobol — Prime Numbers Application v1.0
Version 1.0
Status Draft — test cases not yet written; matrix is a planning skeleton
Owner QA Lead / Lead Developer
Author [derived from PGCBL-BRD-001 and PGCBL-NFR-001]
Created 2026-03-17
Last modified 2026-03-17
Classification Internal
Parent document PGCBL-BRD-001, PGCBL-NFR-001

Version History[edit]

Version Date Author Status Change Summary
0.1 2026-03-17 — Draft Planning skeleton


Table of Contents[edit]

  1. Purpose and How to Use This Matrix
  2. Test Levels and Types
  3. Requirements Coverage Matrix
  4. Test Case Catalogue
  5. Defect Traceability
  6. Coverage Summary
  7. Open Issues


1. Purpose and How to Use This Matrix[edit]

A Test Traceability Matrix (TTM) provides bidirectional traceability between: 

Business Requirement  ←→  Design Element  ←→  Test Case  ←→  Test Result

It answers two critical questions:

  1. Completeness: Is every requirement covered by at least one test? (forward trace)
  2. Necessity: Does every test correspond to a requirement? (backward trace — prevents test bloat)

Status values used in this document[edit]

Status Meaning
Planned Test case defined; not yet implemented
Implemented Test case exists in the test suite
Passed Test executed and passed in the latest run
Failed Test executed and failed in the latest run
Blocked Cannot be run due to an open defect or missing dependency
N/A Not applicable to this version

Test case ID scheme[edit]

TC-<LEVEL>-<NNN>

LEVEL:
  UT   Unit test  (single program in isolation)
  IT   Integration test (two or more programs together)
  ST   System test (full end-to-end)
  NFT  Non-functional test (performance, security, etc.)

NNN: zero-padded sequence

Examples:
  TC-UT-001   Unit test #1
  TC-ST-001   System test #1
  TC-NFT-001  Non-functional test #1

2. Test Levels and Types[edit]

2.1 Unit tests (UT)[edit]

Test a single COBOL program in isolation using a synthetic control block. No real database or print file is required; stubs or mocks may be used.

Target programs: primesgen (sieve logic), primes (SQL operations), primesui (print formatting)

2.2 Integration tests (IT)[edit]

Test two or more programs working together. Typically: primesgen + primes (with a real database), or primesgen + primesui (with a real print file).

2.3 System tests (ST)[edit]

Test the complete stack end-to-end: compile all programs, set up the database, run primesmain with a command-line argument, inspect all outputs.

2.4 Non-functional tests (NFT)[edit]

Test quality attributes: performance, security (credential exposure), operability (return code, log completeness).


3. Requirements Coverage Matrix[edit]

Each row is a requirement. Each column is a test case. A cell contains ✓ if the test covers that requirement.

3.1 Functional Requirements (from PGCBL-BRD-001)[edit]

Req ID Requirement summary TC-UT-001 TC-UT-002 TC-UT-003 TC-UT-004 TC-UT-005 TC-IT-001 TC-IT-002 TC-ST-001 TC-ST-002 TC-ST-003 Coverage
FR-01 Prime generation (generate arg) ✓ ✓ Planned
FR-02 Trial-division algorithm correctness ✓ ✓ ✓ ✓ Planned
FR-03 Prime reporting (report arg) ✓ ✓ Planned
FR-04 Print report format (6/line, heading, footing) ✓ ✓ ✓ ✓ Planned
FR-05 Console logging ✓ ✓ Planned
FR-06 Error handling — bad arg ✓ Planned
FR-06 Error handling — DB connect failure ✓ Planned
FR-06 Error handling — print file cannot open ✓ Planned

3.2 Non-Functional Requirements (from PGCBL-NFR-001)[edit]

Req ID Requirement summary TC-NFT-001 TC-NFT-002 TC-NFT-003 TC-NFT-004 TC-NFT-005 Coverage
NFR-PERF-01 Generation throughput ≥ 1,000/sec ✓ Planned
NFR-PERF-03 Divider lookup < 50 ms ✓ Planned
NFR-REL-01 Batch completes without intervention TC-ST-001/002 cover this
NFR-REL-02 Graceful DB failure ✓ Planned
NFR-SEC-01 No hard-coded credentials ✓ Planned
NFR-OPER-02 Non-zero return code on failure ✓ Planned


4. Test Case Catalogue[edit]

Each test case below is a planning entry. Fields marked [TBD] must be completed when the test is implemented.


TC-UT-001 — Sieve: known prime confirmation[edit]

Field Value
ID TC-UT-001
Level Unit
Program under test primesgen (r80-test-number logic)
Requirements covered FR-02
Description Verify that known prime numbers are correctly identified.
Pre-conditions A test harness that can call primesgen with a synthetic primes-session and a stub primes DAL returning pre-set dividers. Prime 2 pre-seeded in stub.
Test data Candidates: 3, 5, 7, 11, 13, 17, 19 (all prime). Stub DAL returns pre-calculated divisors.
Expected result Each candidate is written (r85-write-prime called) exactly once for each prime. No composite candidate is written.
Status Planned
Defects linked —


TC-UT-002 — Sieve: known composite rejection[edit]

Field Value
ID TC-UT-002
Level Unit
Program under test primesgen
Requirements covered FR-02
Description Verify that known composite numbers are not written as primes.
Pre-conditions Same harness as TC-UT-001.
Test data Candidates: 4, 6, 8, 9, 10, 15, 21, 25, 49.
Expected result r85-write-prime is never called for any of these candidates.
Status Planned
Defects linked —


TC-UT-003 — UI: six-per-line accumulation[edit]

Field Value
ID TC-UT-003
Level Unit
Program under test primesui
Requirements covered FR-04
Description Verify that exactly 6 prime entries are accumulated before a line is written to the print file.
Pre-conditions Call primesui with ui-methods=“start”; then call with ui-methods=“write” seven times. Inspect file after calls 6 and 7.
Test data u-sequence = 1–7; u-number = 2, 3, 5, 7, 11, 13, 17.
Expected result One line written after 6th entry. File buffer reset. 7th entry starts a new accumulation cycle.
Status Planned
Defects linked —


TC-UT-004 — UI: print file cannot be opened[edit]

Field Value
ID TC-UT-004
Level Unit
Program under test primesui
Requirements covered FR-06
Description Verify that primesui returns ui-method-result = 1 when the print file cannot be created (e.g. directory is read-only).
Pre-conditions Set up a read-only target directory so primes.prt cannot be created.
Test data Call primesui with ui-methods=“start”.
Expected result ui-method-result = 1; error status displayed on console; no abort/crash.
Status Planned
Defects linked —


TC-UT-005 — DAL: database connection failure[edit]

Field Value
ID TC-UT-005
Level Unit
Program under test primes (DAL)
Requirements covered FR-06
Description Verify that primes returns dal-result = 1 and logs a message when the database is unreachable.
Pre-conditions PostgreSQL not running, or DATASRC points to an invalid host.
Test data CALL primes USING primes-dal with dal-methods=“connect”.
Expected result dal-result = 1; SQLCODE ≠ 0; error message on console; no crash.
Status Planned
Defects linked —


TC-IT-001 — Generation integration: first 100 primes[edit]

Field Value
ID TC-IT-001
Level Integration
Programs under test primesgen + primes (DAL)
Requirements covered FR-01, FR-02
Description Run the full generate path for the first 100 primes (up to 541). Verify correctness of results in the database.
Pre-conditions Database reset (reset-primes.sql). Prime 2 manually inserted. PostgreSQL running. primesui replaced with a stub that accepts “write” calls silently.
Test data Generate until test-number > 541.
Expected result SELECT prime FROM primes.primes ORDER BY ident returns exactly the first 100 primes: 2, 3, 5, 7, 11, … 541.
Pass criteria 100 rows; each row is a known prime; no duplicates; sequence matches insertion order.
Status Planned
Defects linked D-01 (generate loop commented out in primesmain — this test bypasses primesmain and calls primesgen directly)


TC-IT-002 — Report integration: all primes from database[edit]

Field Value
ID TC-IT-002
Level Integration
Programs under test primesgen + primes (DAL) + primesui
Requirements covered FR-03, FR-04
Description Seed 12 known primes in the database; run the report path; inspect primes.prt.
Pre-conditions Database reset. Insert the first 12 primes (2 through 37) manually.
Test data 12 primes → 2 data lines (6 per line); should produce one page with heading, 2 data lines, footing.
Expected result primes.prt contains: heading line, column header line, data line 1 (primes 2–13), data line 2 (primes 17–37), footing line with page 1.
Status Planned
Defects linked —


TC-ST-001 — System test: full generate run (first 1,000 primes)[edit]

Field Value
ID TC-ST-001
Level System
Programs under test All (primesmain → primesgen → primes + primesui)
Requirements covered FR-01, FR-02, FR-05
Description Full end-to-end generate run via primesmain generate.
Pre-conditions All programs compiled and linked. Database reset. Prime 2 seeded. Fix for D-01 (generate loop) applied or tested separately.
Expected result Database contains the first 1,000 primes (2 through 7,919). Console log shows all lifecycle messages. Process exits without crash.
Status Blocked — D-01 (generate loop commented out)
Defects linked D-01


TC-ST-002 — System test: full report run[edit]

Field Value
ID TC-ST-002
Level System
Programs under test All (primesmain → primesgen → primes + primesui)
Requirements covered FR-03, FR-04, FR-05
Description Full end-to-end report run via primesmain report after TC-ST-001 has populated the database.
Pre-conditions TC-ST-001 has run successfully; database contains at least 1,000 primes.
Expected result primes.prt contains all primes in order; correct page headings and footings; last page flushed correctly.
Status Planned
Defects linked —


TC-ST-003 — System test: bad command-line argument[edit]

Field Value
ID TC-ST-003
Level System
Programs under test primesmain
Requirements covered FR-06
Description Run primesmain badarg; verify graceful exit.
Pre-conditions All programs compiled.
Expected result Console message “Bad parameter, program initialisation failed.” Process exits. No database connection attempted.
Status Planned
Defects linked —


TC-NFT-001 — Performance: generation throughput[edit]

Field Value
ID TC-NFT-001
Level Non-functional
Requirements covered NFR-PERF-01
Description Time the generation of the first 10,000 primes; calculate primes-per-second.
Pass criteria ≥ 1,000 primes/second.
Status Planned — requires D-01 fix and index on ident (NFR-PERF-03)


TC-NFT-002 — Performance: divider lookup query plan[edit]

Field Value
ID TC-NFT-002
Level Non-functional
Requirements covered NFR-PERF-03
Description Run EXPLAIN ANALYZE on SELECT prime FROM primes.primes WHERE ident = 500 with 1,000 rows in the table.
Pass criteria Query plan shows Index Scan; execution time < 50 ms.
Status Planned — likely to fail until index on ident is added


TC-NFT-003 — Reliability: graceful DB failure mid-run[edit]

Field Value
ID TC-NFT-003
Level Non-functional
Requirements covered NFR-REL-02
Description Kill the PostgreSQL process during a report run; verify clean exit and no zombie connections.
Pass criteria Program logs an error and exits; primes.prt is closed; no orphan DB connections.
Status Planned


TC-NFT-004 — Security: no credentials in binary[edit]

Field Value
ID TC-NFT-004
Level Non-functional
Requirements covered NFR-SEC-01
Description Run strings primes (the compiled binary) and grep for the password string pr1mes_user.
Pass criteria String not found in binary.
Status Planned — expected to FAIL until ADR-006 remediation is applied


TC-NFT-005 — Operability: non-zero return code on error[edit]

Field Value
ID TC-NFT-005
Level Non-functional
Requirements covered NFR-OPER-02
Description Run primesmain badarg; check echo $?.
Pass criteria Exit code ≠ 0.
Status Planned — expected to FAIL until MOVE 1 TO RETURN-CODE is added


5. Defect Traceability[edit]

Known defects identified during reverse engineering, linked to the test cases that expose them:

Defect ID Description Test cases affected Status
D-01 Generate loop commented out in primesmain TC-ST-001, TC-NFT-001 Open — blocks system-level generation tests
D-02 Prime 2 not seeded by algorithm TC-IT-001, TC-ST-001 Open — manual pre-step required
D-03 Connection alias mismatch (primes vs primesdb) TC-ST-001, TC-ST-002 Open — may cause runtime failure
D-04 Hard-coded credentials TC-NFT-004 Open — expected test failure
D-05 No index on primes.ident TC-NFT-001, TC-NFT-002 Open — performance defect
D-06 No process return code on error TC-NFT-005 Open — expected test failure


6. Coverage Summary[edit]

Category Total requirements Tests planned Tests passing Coverage %
Functional (FR) 8 8 0 0% (tests not yet run)
Non-functional (NFR) 10 6 0 0%
Total 18 14 0 —

Target before release: 100% functional coverage; 80% non-functional coverage.

7. Open Issues[edit]

ID Issue Owner Target Status
OI-01 D-01 (generate loop) blocks TC-ST-001, TC-NFT-001 — must be fixed before system testing Developer — Open
OI-02 No test harness / stub exists for any program; unit tests cannot run until stubs are written QA / Developer — Open
OI-03 TTM coverage for NFR-MAIN (maintainability) NFRs is not yet defined QA — Open

Terug naar: Design standards | Cobol and PostgreSQL

Retrieved from "https://wiki.webhuis.nl:443/index.php?title=Test_Traceability_Matrix&oldid=4754"