WOMMWorks On My Machine · onmymachineitworks.comSample · fictional data

Test execution report · Release 4.2

ACME-T-327: Invoice list respects data retention

Failed → passed on retest
Test case
ACME-T-327 (Xray) · data-driven over all invoices of 2 tenants
Requirement
ACME-430 Data retention per tenant · ACME-432 Hide expired invoices
Environment
Staging · https://api.staging.acme-ledger.example
Covers
GET /invoices, GET /invoices/{id}, dashboard, revenue report, search
Executed
20 Oct 2026 (rc1) · 21 Oct (rc2) · retest 23 Oct 2026 (rc3)
Executed by
Tomasz Kordiak (QA lead)
Defect
ACME-512 Invoices past retention are listed but cannot be retrieved

Summary

On rc1, 41 invoices whose retention had expired were still listed, counted and searchable, but could not be opened (404). Invoices within retention behaved correctly.

rc2 fixed the list but not the dashboard, report or search. rc3 fixed all four; the retest on 23 Oct with the same data passes every step.

Overview

1,284Invoices checked
3,852Data checks (3 per invoice)
1,243Correct
41Listed but 404
5 / 6Steps failed (rc1)
6 / 6Steps passed (rc3)

Test data

TenantRetentionInvoicesPast retentionShould be visible
t-demo-017 years1,019–1,019
t-demo-02365 days26541224
Cut-off dateTest run 2026-10-20 minus 365 days = 2025-10-20
Data sourceSeed script, no production data
Scriptcheck_retention.py (in acme/qa-api-tests)

Step results (rc1)

#ActionExpected resultActual resultResult
1Record the invoice count and retention setting of each test tenant.Counts known; at least one tenant has invoices past retention.t-demo-01: 1,019 invoices, 7 years. t-demo-02: 265 invoices, 365 days, 41 created before 2025-10-20.Pass
2List all invoices with GET /invoices (all pages) for both tenants.Only invoices within retention are listed; totalItems matches.1,284 listed and totalItems = 1284, including the 41 past retention.Fail
3Fetch each listed invoice with GET /invoices/{id}.Every listed invoice returns 200.1243 return 200; the 41 past retention return 404 (3 attempts each).Fail
4Compare the dashboard count for t-demo-02 with the API.Dashboard shows the number of retrievable invoices.Dashboard shows 265, API detail works for 224.Fail
5Check the revenue report for September 2025 (t-demo-02).Expired invoices are not in the report.Report includes them (€18,940 too high).Fail
6Search for an expired invoice number in the UI.Not found.Found in search results; opening it shows 'Invoice not found'.Fail

Affected invoices 41 invoices, tenant t-demo-02

#Invoice IDNumberCreatedRetention endedAmountListedDetailCounted
1inv_Q9SBVD2025/08/1002025-08-042026-08-04€180Yes404Yes
2inv_YH4N8S2025/08/1032025-08-052026-08-05€277Yes404Yes
3inv_7SDZK72025/08/1062025-08-072026-08-07€374Yes404Yes
4inv_E3PBYK2025/08/1092025-08-092026-08-09€471Yes404Yes
5inv_NBZNBZ2025/08/1122025-08-112026-08-11€568Yes404Yes
6inv_WKAZPD2025/08/1152025-08-132026-08-13€665Yes404Yes
7inv_5VKB3S2025/08/1182025-08-142026-08-14€762Yes404Yes
8inv_C5WNE72025/08/1212025-08-162026-08-16€859Yes404Yes
9inv_KD7ZSK2025/08/1242025-08-182026-08-18€956Yes404Yes
10inv_TNGB6Z2025/08/1272025-08-202026-08-20€1,053Yes404Yes
11inv_3XSNHD2025/08/1302025-08-222026-08-22€1,150Yes404Yes
12inv_A74ZWS2025/08/1332025-08-232026-08-23€1,247Yes404Yes
13inv_HFDB972025/08/1362025-08-252026-08-25€1,344Yes404Yes
14inv_RQPNMK2025/08/1392025-08-272026-08-27€1,441Yes404Yes
15inv_ZZZZZZ2025/08/1422025-08-292026-08-29€1,538Yes404Yes
16inv_89ABCD2025/08/1452025-08-312026-08-31€1,635Yes404Yes
17inv_FHKNQS2025/09/1482025-09-012026-09-01€1,732Yes404Yes
18inv_PSWZ472025/09/1512025-09-032026-09-03€1,829Yes404Yes
19inv_X37BFK2025/09/1542025-09-052026-09-05€1,926Yes404Yes
20inv_6BGNTZ2025/09/1572025-09-072026-09-07€2,023Yes404Yes
21inv_DKSZ7D2025/09/1602025-09-092026-09-09€2,120Yes404Yes
22inv_MV4BJS2025/09/1632025-09-102026-09-10€2,217Yes404Yes
23inv_V5DNX72025/09/1662025-09-122026-09-12€2,314Yes404Yes
24inv_4DPZAK2025/09/1692025-09-142026-09-14€2,411Yes404Yes
25inv_BNZBNZ2025/09/1722025-09-162026-09-16€2,508Yes404Yes
26inv_JXAN2D2025/09/1752025-09-182026-09-18€205Yes404Yes
27inv_S7KZDS2025/09/1782025-09-192026-09-19€302Yes404Yes
28inv_2FWBR72025/09/1812025-09-212026-09-21€399Yes404Yes
29inv_9Q7N5K2025/09/1842025-09-232026-09-23€496Yes404Yes
30inv_GZGZGZ2025/09/1872025-09-252026-09-25€593Yes404Yes
31inv_Q9SBVD2025/09/1902025-09-272026-09-27€690Yes404Yes
32inv_YH4N8S2025/09/1932025-09-282026-09-28€787Yes404Yes
33inv_7SDZK72025/09/1962025-09-302026-09-30€884Yes404Yes
34inv_E3PBYK2025/10/1992025-10-022026-10-02€981Yes404Yes
35inv_NBZNBZ2025/10/2022025-10-042026-10-04€1,078Yes404Yes
36inv_WKAZPD2025/10/2052025-10-062026-10-06€1,175Yes404Yes
37inv_5VKB3S2025/10/2082025-10-072026-10-07€1,272Yes404Yes
38inv_C5WNE72025/10/2112025-10-092026-10-09€1,369Yes404Yes
39inv_KD7ZSK2025/10/2142025-10-112026-10-11€1,466Yes404Yes
40inv_TNGB6Z2025/10/2172025-10-132026-10-13€1,563Yes404Yes
41inv_3XSNHD2025/10/2202025-10-152026-10-15€1,660Yes404Yes

Evidence

Request and response for the first affected invoice (tokens and personal data removed):

GET /invoices?limit=50&cursor=eyJwIjo0fQ  →  200
{ "items": [ …, { "id": "inv_Q9SBVD", "number": "2025/08/100", "createdAt": "2025-08-04T10:12:44Z", … } ],
  "totalItems": 1284 }

GET /invoices/inv_Q9SBVD  →  404
{ "error": "not_found", "message": "Invoice not found" }

The full CSV of all checks and the script are attached to ACME-512 in Jira (acme-t-retention-evidence.csv, check_retention.py).

Defect

KeySummarySeverityStatusComponentFound byOwnerCreated
ACME-512Invoices past data retention are listed but cannot be retrieved
Retested and closed on 23 Oct.		HighFixedInvoices APIACME-T-327Backend20 Oct 2026

Observations

IDAreaObservationImpactSuggested action
OBS-01Root causeThe list query filtered by deleted_at but the retention job only sets expired_at.InfoShared repository method for 'visible invoices' (done in rc3).
OBS-02Search indexSearch is served from a separate index that was not updated by the retention job.MediumReindex after each retention run (done in rc3).
OBS-03Test dataOnly one tenant had a short retention period; the bug would not have shown with default seed data.CoverageKeep t-demo-02 with 365-day retention in the seed script.

Execution history

WhenBuildByResultNotes
20 Oct 09:304.2.0-rc1Tomasz KordiakFail41 expired invoices listed; ACME-512 raised
21 Oct 10:154.2.0-rc2Tomasz KordiakFailList fixed, dashboard and report still count expired invoices
23 Oct 10:054.2.0-rc3Tomasz KordiakPassAll steps pass; regression suites green

Retest (rc3)

Passed

GET /invoices lists 1243 invoices, totalItems = 1243; all return 200. Dashboard for t-demo-02 shows 224, the September 2025 report excludes expired invoices and search no longer finds them.

Regression: the API smoke suite and the invoice E2E specs pass on rc3.

#StepResult
1Record the invoice count and retention setting of each test tenant.Pass
2List all invoices with GET /invoices (all pages) for both tenants.Pass
3Fetch each listed invoice with GET /invoices/{id}.Pass
4Compare the dashboard count for t-demo-02 with the API.Pass
5Check the revenue report for September 2025 (t-demo-02).Pass
6Search for an expired invoice number in the UI.Pass

Sample report. Acme Ledger, its data, people and defects are fictional. Your report shows your product, your tickets and your tools.
More sample reports · Book a 15-minute call