Test Traceability Matrix[edit]

Field	Value
Document ID	PGCBL-TTM-001
Document Type	Test Traceability Matrix
System	pgcobol — Prime Numbers Application v1.0
Version	1.0
Status	Draft — test cases not yet written; matrix is a planning skeleton
Owner	QA Lead / Lead Developer
Author	[derived from PGCBL-BRD-001 and PGCBL-NFR-001]
Created	2026-03-17
Last modified	2026-03-17
Classification	Internal
Parent document	PGCBL-BRD-001, PGCBL-NFR-001

Version History[edit]

Version	Date	Author	Status	Change Summary
0.1	2026-03-17	—	Draft	Planning skeleton

1. Purpose and How to Use This Matrix[edit]

A Test Traceability Matrix (TTM) provides bidirectional traceability between:

Business Requirement  ←→  Design Element  ←→  Test Case  ←→  Test Result

It answers two critical questions:

Completeness: Is every requirement covered by at least one test? (forward trace)
Necessity: Does every test correspond to a requirement? (backward trace — prevents test bloat)

Status values used in this document[edit]

Status	Meaning
`Planned`	Test case defined; not yet implemented
`Implemented`	Test case exists in the test suite
`Passed`	Test executed and passed in the latest run
`Failed`	Test executed and failed in the latest run
`Blocked`	Cannot be run due to an open defect or missing dependency
`N/A`	Not applicable to this version

Test case ID scheme[edit]

TC-<LEVEL>-<NNN>

LEVEL:
  UT   Unit test  (single program in isolation)
  IT   Integration test (two or more programs together)
  ST   System test (full end-to-end)
  NFT  Non-functional test (performance, security, etc.)

NNN: zero-padded sequence

Examples:
  TC-UT-001   Unit test #1
  TC-ST-001   System test #1
  TC-NFT-001  Non-functional test #1

2. Test Levels and Types[edit]

2.1 Unit tests (UT)[edit]

Test a single COBOL program in isolation using a synthetic control block. No real database or print file is required; stubs or mocks may be used.

Target programs: primesgen (sieve logic), primes (SQL operations), primesui (print formatting)

2.2 Integration tests (IT)[edit]

Test two or more programs working together. Typically: primesgen + primes (with a real database), or primesgen + primesui (with a real print file).

2.3 System tests (ST)[edit]

Test the complete stack end-to-end: compile all programs, set up the database, run primesmain with a command-line argument, inspect all outputs.

2.4 Non-functional tests (NFT)[edit]

Test quality attributes: performance, security (credential exposure), operability (return code, log completeness).

3. Requirements Coverage Matrix[edit]

Each row is a requirement. Each column is a test case. A cell contains ✓ if the test covers that requirement.

3.1 Functional Requirements (from PGCBL-BRD-001)[edit]

Req ID	Requirement summary	TC-UT-001	TC-UT-002	TC-UT-003	TC-UT-004	TC-UT-005	TC-IT-001	TC-IT-002	TC-ST-001	TC-ST-002	TC-ST-003	Coverage
FR-01	Prime generation (generate arg)						✓		✓			Planned
FR-02	Trial-division algorithm correctness	✓	✓				✓		✓			Planned
FR-03	Prime reporting (report arg)							✓		✓		Planned
FR-04	Print report format (6/line, heading, footing)			✓	✓			✓		✓		Planned
FR-05	Console logging								✓	✓		Planned
FR-06	Error handling — bad arg										✓	Planned
FR-06	Error handling — DB connect failure					✓						Planned
FR-06	Error handling — print file cannot open				✓							Planned

3.2 Non-Functional Requirements (from PGCBL-NFR-001)[edit]

Req ID	Requirement summary	TC-NFT-001	TC-NFT-002	TC-NFT-003	TC-NFT-004	TC-NFT-005	Coverage
NFR-PERF-01	Generation throughput ≥ 1,000/sec	✓					Planned
NFR-PERF-03	Divider lookup < 50 ms		✓				Planned
NFR-REL-01	Batch completes without intervention						TC-ST-001/002 cover this
NFR-REL-02	Graceful DB failure			✓			Planned
NFR-SEC-01	No hard-coded credentials				✓		Planned
NFR-OPER-02	Non-zero return code on failure					✓	Planned

4. Test Case Catalogue[edit]

Each test case below is a planning entry. Fields marked [TBD] must be completed when the test is implemented.

TC-UT-001 — Sieve: known prime confirmation[edit]

Field	Value
ID	TC-UT-001
Level	Unit
Program under test	primesgen (r80-test-number logic)
Requirements covered	FR-02
Description	Verify that known prime numbers are correctly identified.
Pre-conditions	A test harness that can call primesgen with a synthetic primes-session and a stub primes DAL returning pre-set dividers. Prime 2 pre-seeded in stub.
Test data	Candidates: 3, 5, 7, 11, 13, 17, 19 (all prime). Stub DAL returns pre-calculated divisors.
Expected result	Each candidate is written (r85-write-prime called) exactly once for each prime. No composite candidate is written.
Status	Planned
Defects linked	—

TC-UT-002 — Sieve: known composite rejection[edit]

Field	Value
ID	TC-UT-002
Level	Unit
Program under test	primesgen
Requirements covered	FR-02
Description	Verify that known composite numbers are not written as primes.
Pre-conditions	Same harness as TC-UT-001.
Test data	Candidates: 4, 6, 8, 9, 10, 15, 21, 25, 49.
Expected result	r85-write-prime is never called for any of these candidates.
Status	Planned
Defects linked	—

TC-UT-003 — UI: six-per-line accumulation[edit]

Field	Value
ID	TC-UT-003
Level	Unit
Program under test	primesui
Requirements covered	FR-04
Description	Verify that exactly 6 prime entries are accumulated before a line is written to the print file.
Pre-conditions	Call primesui with ui-methods=“start”; then call with ui-methods=“write” seven times. Inspect file after calls 6 and 7.
Test data	u-sequence = 1–7; u-number = 2, 3, 5, 7, 11, 13, 17.
Expected result	One line written after 6th entry. File buffer reset. 7th entry starts a new accumulation cycle.
Status	Planned
Defects linked	—

TC-UT-004 — UI: print file cannot be opened[edit]

Field	Value
ID	TC-UT-004
Level	Unit
Program under test	primesui
Requirements covered	FR-06
Description	Verify that primesui returns ui-method-result = 1 when the print file cannot be created (e.g. directory is read-only).
Pre-conditions	Set up a read-only target directory so `primes.prt` cannot be created.
Test data	Call primesui with ui-methods=“start”.
Expected result	ui-method-result = 1; error status displayed on console; no abort/crash.
Status	Planned
Defects linked	—

TC-UT-005 — DAL: database connection failure[edit]

Field	Value
ID	TC-UT-005
Level	Unit
Program under test	primes (DAL)
Requirements covered	FR-06
Description	Verify that primes returns dal-result = 1 and logs a message when the database is unreachable.
Pre-conditions	PostgreSQL not running, or DATASRC points to an invalid host.
Test data	CALL primes USING primes-dal with dal-methods=“connect”.
Expected result	dal-result = 1; SQLCODE ≠ 0; error message on console; no crash.
Status	Planned
Defects linked	—

TC-IT-001 — Generation integration: first 100 primes[edit]

Field	Value
ID	TC-IT-001
Level	Integration
Programs under test	primesgen + primes (DAL)
Requirements covered	FR-01, FR-02
Description	Run the full generate path for the first 100 primes (up to 541). Verify correctness of results in the database.
Pre-conditions	Database reset (`reset-primes.sql`). Prime 2 manually inserted. PostgreSQL running. primesui replaced with a stub that accepts “write” calls silently.
Test data	Generate until test-number > 541.
Expected result	`SELECT prime FROM primes.primes ORDER BY ident` returns exactly the first 100 primes: 2, 3, 5, 7, 11, … 541.
Pass criteria	100 rows; each row is a known prime; no duplicates; sequence matches insertion order.
Status	Planned
Defects linked	D-01 (generate loop commented out in primesmain — this test bypasses primesmain and calls primesgen directly)

TC-IT-002 — Report integration: all primes from database[edit]

Field	Value
ID	TC-IT-002
Level	Integration
Programs under test	primesgen + primes (DAL) + primesui
Requirements covered	FR-03, FR-04
Description	Seed 12 known primes in the database; run the report path; inspect `primes.prt`.
Pre-conditions	Database reset. Insert the first 12 primes (2 through 37) manually.
Test data	12 primes → 2 data lines (6 per line); should produce one page with heading, 2 data lines, footing.
Expected result	`primes.prt` contains: heading line, column header line, data line 1 (primes 2–13), data line 2 (primes 17–37), footing line with page 1.
Status	Planned
Defects linked	—

TC-ST-001 — System test: full generate run (first 1,000 primes)[edit]

Field	Value
ID	TC-ST-001
Level	System
Programs under test	All (primesmain → primesgen → primes + primesui)
Requirements covered	FR-01, FR-02, FR-05
Description	Full end-to-end generate run via `primesmain generate`.
Pre-conditions	All programs compiled and linked. Database reset. Prime 2 seeded. Fix for D-01 (generate loop) applied or tested separately.
Expected result	Database contains the first 1,000 primes (2 through 7,919). Console log shows all lifecycle messages. Process exits without crash.
Status	Blocked — D-01 (generate loop commented out)
Defects linked	D-01

TC-ST-002 — System test: full report run[edit]

Field	Value
ID	TC-ST-002
Level	System
Programs under test	All (primesmain → primesgen → primes + primesui)
Requirements covered	FR-03, FR-04, FR-05
Description	Full end-to-end report run via `primesmain report` after TC-ST-001 has populated the database.
Pre-conditions	TC-ST-001 has run successfully; database contains at least 1,000 primes.
Expected result	`primes.prt` contains all primes in order; correct page headings and footings; last page flushed correctly.
Status	Planned
Defects linked	—

TC-ST-003 — System test: bad command-line argument[edit]

Field	Value
ID	TC-ST-003
Level	System
Programs under test	primesmain
Requirements covered	FR-06
Description	Run `primesmain badarg`; verify graceful exit.
Pre-conditions	All programs compiled.
Expected result	Console message “Bad parameter, program initialisation failed.” Process exits. No database connection attempted.
Status	Planned
Defects linked	—

TC-NFT-001 — Performance: generation throughput[edit]

Field	Value
ID	TC-NFT-001
Level	Non-functional
Requirements covered	NFR-PERF-01
Description	Time the generation of the first 10,000 primes; calculate primes-per-second.
Pass criteria	≥ 1,000 primes/second.
Status	Planned — requires D-01 fix and index on `ident` (NFR-PERF-03)

TC-NFT-002 — Performance: divider lookup query plan[edit]

Field	Value
ID	TC-NFT-002
Level	Non-functional
Requirements covered	NFR-PERF-03
Description	Run EXPLAIN ANALYZE on `SELECT prime FROM primes.primes WHERE ident = 500` with 1,000 rows in the table.
Pass criteria	Query plan shows Index Scan; execution time < 50 ms.
Status	Planned — likely to fail until index on `ident` is added

TC-NFT-003 — Reliability: graceful DB failure mid-run[edit]

Field	Value
ID	TC-NFT-003
Level	Non-functional
Requirements covered	NFR-REL-02
Description	Kill the PostgreSQL process during a report run; verify clean exit and no zombie connections.
Pass criteria	Program logs an error and exits; `primes.prt` is closed; no orphan DB connections.
Status	Planned

TC-NFT-004 — Security: no credentials in binary[edit]

Field	Value
ID	TC-NFT-004
Level	Non-functional
Requirements covered	NFR-SEC-01
Description	Run `strings primes` (the compiled binary) and grep for the password string `pr1mes_user`.
Pass criteria	String not found in binary.
Status	Planned — expected to FAIL until ADR-006 remediation is applied

TC-NFT-005 — Operability: non-zero return code on error[edit]

Field	Value
ID	TC-NFT-005
Level	Non-functional
Requirements covered	NFR-OPER-02
Description	Run `primesmain badarg`; check `echo $?`.
Pass criteria	Exit code ≠ 0.
Status	Planned — expected to FAIL until `MOVE 1 TO RETURN-CODE` is added

5. Defect Traceability[edit]

Known defects identified during reverse engineering, linked to the test cases that expose them:

Defect ID	Description	Test cases affected	Status
D-01	Generate loop commented out in primesmain	TC-ST-001, TC-NFT-001	Open — blocks system-level generation tests
D-02	Prime 2 not seeded by algorithm	TC-IT-001, TC-ST-001	Open — manual pre-step required
D-03	Connection alias mismatch (`primes` vs `primesdb`)	TC-ST-001, TC-ST-002	Open — may cause runtime failure
D-04	Hard-coded credentials	TC-NFT-004	Open — expected test failure
D-05	No index on `primes.ident`	TC-NFT-001, TC-NFT-002	Open — performance defect
D-06	No process return code on error	TC-NFT-005	Open — expected test failure

6. Coverage Summary[edit]

Category	Total requirements	Tests planned	Coverage %
Functional (FR)	8	8	0% (tests not yet run)
Non-functional (NFR)	10	6	0%
Total	18	14	—

Target before release: 100% functional coverage; 80% non-functional coverage.

7. Open Issues[edit]

ID	Issue	Owner	Target	Status
OI-01	D-01 (generate loop) blocks TC-ST-001, TC-NFT-001 — must be fixed before system testing	Developer	—	Open
OI-02	No test harness / stub exists for any program; unit tests cannot run until stubs are written	QA / Developer	—	Open
OI-03	TTM coverage for NFR-MAIN (maintainability) NFRs is not yet defined	QA	—	Open

Terug naar: Design standards | Cobol and PostgreSQL

Test Traceability Matrix