Editing Test Traceability Matrix (section)

= Test Traceability Matrix =

{|
!width="23%"| Field
!width="76%"| Value
|-
| Document ID
| PGCBL-TTM-001
|-
| Document Type
| Test Traceability Matrix
|-
| System
| pgcobol — Prime Numbers Application v1.0
|-
| Version
| 1.0
|-
| Status
| Draft — test cases not yet written; matrix is a planning skeleton
|-
| Owner
| QA Lead / Lead Developer
|-
| Author
| [derived from PGCBL-BRD-001 and PGCBL-NFR-001]
|-
| Created
| 2026-03-17
|-
| Last modified
| 2026-03-17
|-
| Classification
| Internal
|-
| Parent document
| PGCBL-BRD-001, PGCBL-NFR-001
|}

<span id="version-history"></span>
== Version History ==

{|
! Version
! Date
! Author
! Status
! Change Summary
|-
| 0.1
| 2026-03-17
| —
| Draft
| Planning skeleton
|}


-----

<span id="table-of-contents"></span>
== Table of Contents ==

# [[#1-purpose-and-how-to-use-this-matrix|Purpose and How to Use This Matrix]]
# [[#2-test-levels-and-types|Test Levels and Types]]
# [[#3-requirements-coverage-matrix|Requirements Coverage Matrix]]
# [[#4-test-case-catalogue|Test Case Catalogue]]
# [[#5-defect-traceability|Defect Traceability]]
# [[#6-coverage-summary|Coverage Summary]]
# [[#7-open-issues|Open Issues]]


-----

<span id="purpose-and-how-to-use-this-matrix"></span>
== 1. Purpose and How to Use This Matrix ==

A Test Traceability Matrix (TTM) provides bidirectional traceability between:

<pre>Business Requirement  ←→  Design Element  ←→  Test Case  ←→  Test Result</pre>
It answers two critical questions:

# '''Completeness:''' Is every requirement covered by at least one test? (forward trace)
# '''Necessity:''' Does every test correspond to a requirement? (backward trace — prevents test bloat)

<span id="status-values-used-in-this-document"></span>
=== Status values used in this document ===

{|
!width="50%"| Status
!width="50%"| Meaning
|-
| <code>Planned</code>
| Test case defined; not yet implemented
|-
| <code>Implemented</code>
| Test case exists in the test suite
|-
| <code>Passed</code>
| Test executed and passed in the latest run
|-
| <code>Failed</code>
| Test executed and failed in the latest run
|-
| <code>Blocked</code>
| Cannot be run due to an open defect or missing dependency
|-
| <code>N/A</code>
| Not applicable to this version
|}

<span id="test-case-id-scheme"></span>
=== Test case ID scheme ===

<pre>TC-&lt;LEVEL&gt;-&lt;NNN&gt;

LEVEL:
  UT   Unit test  (single program in isolation)
  IT   Integration test (two or more programs together)
  ST   System test (full end-to-end)
  NFT  Non-functional test (performance, security, etc.)

NNN: zero-padded sequence

Examples:
  TC-UT-001   Unit test #1
  TC-ST-001   System test #1
  TC-NFT-001  Non-functional test #1</pre>

-----

<span id="test-levels-and-types"></span>
== 2. Test Levels and Types ==

<span id="unit-tests-ut"></span>
=== 2.1 Unit tests (UT) ===

Test a single COBOL program in isolation using a synthetic control block. No real database or print file is required; stubs or mocks may be used.

'''Target programs:''' primesgen (sieve logic), primes (SQL operations), primesui (print formatting)

<span id="integration-tests-it"></span>
=== 2.2 Integration tests (IT) ===

Test two or more programs working together. Typically: primesgen + primes (with a real database), or primesgen + primesui (with a real print file).

<span id="system-tests-st"></span>
=== 2.3 System tests (ST) ===

Test the complete stack end-to-end: compile all programs, set up the database, run primesmain with a command-line argument, inspect all outputs.

<span id="non-functional-tests-nft"></span>
=== 2.4 Non-functional tests (NFT) ===

Test quality attributes: performance, security (credential exposure), operability (return code, log completeness).


-----

<span id="requirements-coverage-matrix"></span>
== 3. Requirements Coverage Matrix ==

Each row is a requirement. Each column is a test case. A cell contains <code>✓</code> if the test covers that requirement.

<span id="functional-requirements-from-pgcbl-brd-001"></span>
=== 3.1 Functional Requirements (from PGCBL-BRD-001) ===

{|
!width="5%"| Req ID
!width="14%"| Requirement summary
!width="7%"| TC-UT-001
!width="7%"| TC-UT-002
!width="7%"| TC-UT-003
!width="7%"| TC-UT-004
!width="7%"| TC-UT-005
!width="7%"| TC-IT-001
!width="7%"| TC-IT-002
!width="7%"| TC-ST-001
!width="7%"| TC-ST-002
!width="7%"| TC-ST-003
!width="6%"| Coverage
|-
| FR-01
| Prime generation (generate arg)
|
|
|
|
|
| ✓
|
| ✓
|
|
| Planned
|-
| FR-02
| Trial-division algorithm correctness
| ✓
| ✓
|
|
|
| ✓
|
| ✓
|
|
| Planned
|-
| FR-03
| Prime reporting (report arg)
|
|
|
|
|
|
| ✓
|
| ✓
|
| Planned
|-
| FR-04
| Print report format (6/line, heading, footing)
|
|
| ✓
| ✓
|
|
| ✓
|
| ✓
|
| Planned
|-
| FR-05
| Console logging
|
|
|
|
|
|
|
| ✓
| ✓
|
| Planned
|-
| FR-06
| Error handling — bad arg
|
|
|
|
|
|
|
|
|
| ✓
| Planned
|-
| FR-06
| Error handling — DB connect failure
|
|
|
|
| ✓
|
|
|
|
|
| Planned
|-
| FR-06
| Error handling — print file cannot open
|
|
|
| ✓
|
|
|
|
|
|
| Planned
|}

<span id="non-functional-requirements-from-pgcbl-nfr-001"></span>
=== 3.2 Non-Functional Requirements (from PGCBL-NFR-001) ===

{|
!width="8%"| Req ID
!width="21%"| Requirement summary
!width="12%"| TC-NFT-001
!width="12%"| TC-NFT-002
!width="12%"| TC-NFT-003
!width="12%"| TC-NFT-004
!width="12%"| TC-NFT-005
!width="10%"| Coverage
|-
| NFR-PERF-01
| Generation throughput ≥ 1,000/sec
| ✓
|
|
|
|
| Planned
|-
| NFR-PERF-03
| Divider lookup &lt; 50 ms
|
| ✓
|
|
|
| Planned
|-
| NFR-REL-01
| Batch completes without intervention
|
|
|
|
|
| TC-ST-001/002 cover this
|-
| NFR-REL-02
| Graceful DB failure
|
|
| ✓
|
|
| Planned
|-
| NFR-SEC-01
| No hard-coded credentials
|
|
|
| ✓
|
| Planned
|-
| NFR-OPER-02
| Non-zero return code on failure
|
|
|
|
| ✓
| Planned
|}


-----

<span id="test-case-catalogue"></span>
== 4. Test Case Catalogue ==

Each test case below is a planning entry. Fields marked <code>[TBD]</code> must be completed when the test is implemented.


-----

<span id="tc-ut-001-sieve-known-prime-confirmation"></span>
=== TC-UT-001 — Sieve: known prime confirmation ===

{|
!width="50%"| Field
!width="50%"| Value
|-
| ID
| TC-UT-001
|-
| Level
| Unit
|-
| Program under test
| primesgen (r80-test-number logic)
|-
| Requirements covered
| FR-02
|-
| Description
| Verify that known prime numbers are correctly identified.
|-
| Pre-conditions
| A test harness that can call primesgen with a synthetic primes-session and a stub primes DAL returning pre-set dividers. Prime 2 pre-seeded in stub.
|-
| Test data
| Candidates: 3, 5, 7, 11, 13, 17, 19 (all prime). Stub DAL returns pre-calculated divisors.
|-
| Expected result
| Each candidate is written (r85-write-prime called) exactly once for each prime. No composite candidate is written.
|-
| Status
| Planned
|-
| Defects linked
| —
|}


-----

<span id="tc-ut-002-sieve-known-composite-rejection"></span>
=== TC-UT-002 — Sieve: known composite rejection ===

{|
!width="50%"| Field
!width="50%"| Value
|-
| ID
| TC-UT-002
|-
| Level
| Unit
|-
| Program under test
| primesgen
|-
| Requirements covered
| FR-02
|-
| Description
| Verify that known composite numbers are not written as primes.
|-
| Pre-conditions
| Same harness as TC-UT-001.
|-
| Test data
| Candidates: 4, 6, 8, 9, 10, 15, 21, 25, 49.
|-
| Expected result
| r85-write-prime is never called for any of these candidates.
|-
| Status
| Planned
|-
| Defects linked
| —
|}


-----

<span id="tc-ut-003-ui-six-per-line-accumulation"></span>
=== TC-UT-003 — UI: six-per-line accumulation ===

{|
!width="50%"| Field
!width="50%"| Value
|-
| ID
| TC-UT-003
|-
| Level
| Unit
|-
| Program under test
| primesui
|-
| Requirements covered
| FR-04
|-
| Description
| Verify that exactly 6 prime entries are accumulated before a line is written to the print file.
|-
| Pre-conditions
| Call primesui with ui-methods=“start”; then call with ui-methods=“write” seven times. Inspect file after calls 6 and 7.
|-
| Test data
| u-sequence = 1–7; u-number = 2, 3, 5, 7, 11, 13, 17.
|-
| Expected result
| One line written after 6th entry. File buffer reset. 7th entry starts a new accumulation cycle.
|-
| Status
| Planned
|-
| Defects linked
| —
|}


-----

<span id="tc-ut-004-ui-print-file-cannot-be-opened"></span>
=== TC-UT-004 — UI: print file cannot be opened ===

{|
!width="50%"| Field
!width="50%"| Value
|-
| ID
| TC-UT-004
|-
| Level
| Unit
|-
| Program under test
| primesui
|-
| Requirements covered
| FR-06
|-
| Description
| Verify that primesui returns ui-method-result = 1 when the print file cannot be created (e.g. directory is read-only).
|-
| Pre-conditions
| Set up a read-only target directory so <code>primes.prt</code> cannot be created.
|-
| Test data
| Call primesui with ui-methods=“start”.
|-
| Expected result
| ui-method-result = 1; error status displayed on console; no abort/crash.
|-
| Status
| Planned
|-
| Defects linked
| —
|}


-----

<span id="tc-ut-005-dal-database-connection-failure"></span>
=== TC-UT-005 — DAL: database connection failure ===

{|
!width="50%"| Field
!width="50%"| Value
|-
| ID
| TC-UT-005
|-
| Level
| Unit
|-
| Program under test
| primes (DAL)
|-
| Requirements covered
| FR-06
|-
| Description
| Verify that primes returns dal-result = 1 and logs a message when the database is unreachable.
|-
| Pre-conditions
| PostgreSQL not running, or DATASRC points to an invalid host.
|-
| Test data
| CALL primes USING primes-dal with dal-methods=“connect”.
|-
| Expected result
| dal-result = 1; SQLCODE ≠ 0; error message on console; no crash.
|-
| Status
| Planned
|-
| Defects linked
| —
|}


-----

<span id="tc-it-001-generation-integration-first-100-primes"></span>
=== TC-IT-001 — Generation integration: first 100 primes ===

{|
!width="50%"| Field
!width="50%"| Value
|-
| ID
| TC-IT-001
|-
| Level
| Integration
|-
| Programs under test
| primesgen + primes (DAL)
|-
| Requirements covered
| FR-01, FR-02
|-
| Description
| Run the full generate path for the first 100 primes (up to 541). Verify correctness of results in the database.
|-
| Pre-conditions
| Database reset (<code>reset-primes.sql</code>). Prime 2 manually inserted. PostgreSQL running. primesui replaced with a stub that accepts “write” calls silently.
|-
| Test data
| Generate until test-number &gt; 541.
|-
| Expected result
| <code>SELECT prime FROM primes.primes ORDER BY ident</code> returns exactly the first 100 primes: 2, 3, 5, 7, 11, … 541.
|-
| Pass criteria
| 100 rows; each row is a known prime; no duplicates; sequence matches insertion order.
|-
| Status
| Planned
|-
| Defects linked
| D-01 (generate loop commented out in primesmain — this test bypasses primesmain and calls primesgen directly)
|}


-----

<span id="tc-it-002-report-integration-all-primes-from-database"></span>
=== TC-IT-002 — Report integration: all primes from database ===

{|
!width="50%"| Field
!width="50%"| Value
|-
| ID
| TC-IT-002
|-
| Level
| Integration
|-
| Programs under test
| primesgen + primes (DAL) + primesui
|-
| Requirements covered
| FR-03, FR-04
|-
| Description
| Seed 12 known primes in the database; run the report path; inspect <code>primes.prt</code>.
|-
| Pre-conditions
| Database reset. Insert the first 12 primes (2 through 37) manually.
|-
| Test data
| 12 primes → 2 data lines (6 per line); should produce one page with heading, 2 data lines, footing.
|-
| Expected result
| <code>primes.prt</code> contains: heading line, column header line, data line 1 (primes 2–13), data line 2 (primes 17–37), footing line with page 1.
|-
| Status
| Planned
|-
| Defects linked
| —
|}


-----

<span id="tc-st-001-system-test-full-generate-run-first-1000-primes"></span>
=== TC-ST-001 — System test: full generate run (first 1,000 primes) ===

{|
!width="50%"| Field
!width="50%"| Value
|-
| ID
| TC-ST-001
|-
| Level
| System
|-
| Programs under test
| All (primesmain → primesgen → primes + primesui)
|-
| Requirements covered
| FR-01, FR-02, FR-05
|-
| Description
| Full end-to-end generate run via <code>primesmain generate</code>.
|-
| Pre-conditions
| All programs compiled and linked. Database reset. Prime 2 seeded. Fix for D-01 (generate loop) applied or tested separately.
|-
| Expected result
| Database contains the first 1,000 primes (2 through 7,919). Console log shows all lifecycle messages. Process exits without crash.
|-
| Status
| Blocked — D-01 (generate loop commented out)
|-
| Defects linked
| D-01
|}


-----

<span id="tc-st-002-system-test-full-report-run"></span>
=== TC-ST-002 — System test: full report run ===

{|
!width="50%"| Field
!width="50%"| Value
|-
| ID
| TC-ST-002
|-
| Level
| System
|-
| Programs under test
| All (primesmain → primesgen → primes + primesui)
|-
| Requirements covered
| FR-03, FR-04, FR-05
|-
| Description
| Full end-to-end report run via <code>primesmain report</code> after TC-ST-001 has populated the database.
|-
| Pre-conditions
| TC-ST-001 has run successfully; database contains at least 1,000 primes.
|-
| Expected result
| <code>primes.prt</code> contains all primes in order; correct page headings and footings; last page flushed correctly.
|-
| Status
| Planned
|-
| Defects linked
| —
|}


-----

<span id="tc-st-003-system-test-bad-command-line-argument"></span>
=== TC-ST-003 — System test: bad command-line argument ===

{|
!width="50%"| Field
!width="50%"| Value
|-
| ID
| TC-ST-003
|-
| Level
| System
|-
| Programs under test
| primesmain
|-
| Requirements covered
| FR-06
|-
| Description
| Run <code>primesmain badarg</code>; verify graceful exit.
|-
| Pre-conditions
| All programs compiled.
|-
| Expected result
| Console message “Bad parameter, program initialisation failed.” Process exits. No database connection attempted.
|-
| Status
| Planned
|-
| Defects linked
| —
|}


-----

<span id="tc-nft-001-performance-generation-throughput"></span>
=== TC-NFT-001 — Performance: generation throughput ===

{|
!width="50%"| Field
!width="50%"| Value
|-
| ID
| TC-NFT-001
|-
| Level
| Non-functional
|-
| Requirements covered
| NFR-PERF-01
|-
| Description
| Time the generation of the first 10,000 primes; calculate primes-per-second.
|-
| Pass criteria
| ≥ 1,000 primes/second.
|-
| Status
| Planned — requires D-01 fix and index on <code>ident</code> (NFR-PERF-03)
|}


-----

<span id="tc-nft-002-performance-divider-lookup-query-plan"></span>
=== TC-NFT-002 — Performance: divider lookup query plan ===

{|
!width="50%"| Field
!width="50%"| Value
|-
| ID
| TC-NFT-002
|-
| Level
| Non-functional
|-
| Requirements covered
| NFR-PERF-03
|-
| Description
| Run EXPLAIN ANALYZE on <code>SELECT prime FROM primes.primes WHERE ident = 500</code> with 1,000 rows in the table.
|-
| Pass criteria
| Query plan shows Index Scan; execution time &lt; 50 ms.
|-
| Status
| Planned — likely to fail until index on <code>ident</code> is added
|}


-----

<span id="tc-nft-003-reliability-graceful-db-failure-mid-run"></span>
=== TC-NFT-003 — Reliability: graceful DB failure mid-run ===

{|
!width="50%"| Field
!width="50%"| Value
|-
| ID
| TC-NFT-003
|-
| Level
| Non-functional
|-
| Requirements covered
| NFR-REL-02
|-
| Description
| Kill the PostgreSQL process during a report run; verify clean exit and no zombie connections.
|-
| Pass criteria
| Program logs an error and exits; <code>primes.prt</code> is closed; no orphan DB connections.
|-
| Status
| Planned
|}


-----

<span id="tc-nft-004-security-no-credentials-in-binary"></span>
=== TC-NFT-004 — Security: no credentials in binary ===

{|
!width="50%"| Field
!width="50%"| Value
|-
| ID
| TC-NFT-004
|-
| Level
| Non-functional
|-
| Requirements covered
| NFR-SEC-01
|-
| Description
| Run <code>strings primes</code> (the compiled binary) and grep for the password string <code>pr1mes_user</code>.
|-
| Pass criteria
| String not found in binary.
|-
| Status
| Planned — expected to FAIL until ADR-006 remediation is applied
|}


-----

<span id="tc-nft-005-operability-non-zero-return-code-on-error"></span>
=== TC-NFT-005 — Operability: non-zero return code on error ===

{|
!width="50%"| Field
!width="50%"| Value
|-
| ID
| TC-NFT-005
|-
| Level
| Non-functional
|-
| Requirements covered
| NFR-OPER-02
|-
| Description
| Run <code>primesmain badarg</code>; check <code>echo $?</code>.
|-
| Pass criteria
| Exit code ≠ 0.
|-
| Status
| Planned — expected to FAIL until <code>MOVE 1 TO RETURN-CODE</code> is added
|}


-----

<span id="defect-traceability"></span>
== 5. Defect Traceability ==

Known defects identified during reverse engineering, linked to the test cases that expose them:

{|
!width="20%"| Defect ID
!width="24%"| Description
!width="39%"| Test cases affected
!width="15%"| Status
|-
| D-01
| Generate loop commented out in primesmain
| TC-ST-001, TC-NFT-001
| Open — blocks system-level generation tests
|-
| D-02
| Prime 2 not seeded by algorithm
| TC-IT-001, TC-ST-001
| Open — manual pre-step required
|-
| D-03
| Connection alias mismatch (<code>primes</code> vs <code>primesdb</code>)
| TC-ST-001, TC-ST-002
| Open — may cause runtime failure
|-
| D-04
| Hard-coded credentials
| TC-NFT-004
| Open — expected test failure
|-
| D-05
| No index on <code>primes.ident</code>
| TC-NFT-001, TC-NFT-002
| Open — performance defect
|-
| D-06
| No process return code on error
| TC-NFT-005
| Open — expected test failure
|}


-----

<span id="coverage-summary"></span>
== 6. Coverage Summary ==

{|
!width="20%"| Category
!width="20%"| Total requirements
!width="20%"| Tests planned
!width="20%"| Tests passing
!width="20%"| Coverage %
|-
| Functional (FR)
| 8
| 8
| 0
| 0% (tests not yet run)
|-
| Non-functional (NFR)
| 10
| 6
| 0
| 0%
|-
| '''Total'''
| '''18'''
| '''14'''
| '''0'''
| —
|}

<blockquote>Target before release: 100% functional coverage; 80% non-functional coverage.
</blockquote>

-----

<span id="open-issues"></span>
== 7. Open Issues ==

{|
!width="18%"| ID
!width="18%"| Issue
!width="18%"| Owner
!width="21%"| Target
!width="21%"| Status
|-
| OI-01
| D-01 (generate loop) blocks TC-ST-001, TC-NFT-001 — must be fixed before system testing
| Developer
| —
| Open
|-
| OI-02
| No test harness / stub exists for any program; unit tests cannot run until stubs are written
| QA / Developer
| —
| Open
|-
| OI-03
| TTM coverage for NFR-MAIN (maintainability) NFRs is not yet defined
| QA
| —
| Open
|}
<hr/>
Terug naar: [[Design standards]] | [[Cobol and PostgreSQL]]