Test report · UX / usability
Everyday work is easy: creating and sending an invoice and paying in the portal were completed by all 8 participants and rated easiest. The trouble is in tasks that matter at month end: recurring invoices (3 of 8 unassisted) and credit notes (2 of 8).
SUS is 71 (the commonly cited average is 68). The four high-priority issues are mostly about where actions live and what errors say, not missing features.
Put Repeat this invoice and Credit this invoice on the invoice page. 7 of 8 participants looked there first.
| Key | Summary | Severity | Status | Component | Found by | Owner | Created |
|---|---|---|---|---|---|---|---|
ACME-535 | SVG logo upload fails silently | Medium | Open | Settings | T10 | Frontend | 16 Oct |
ACME-536 | Date picker shows MM/DD while lists use DD/MM | Medium | Open | UI | Heuristic | Frontend | 16 Oct |
ACME-537 | Expired invite link shows 'Something went wrong' | Low | Open | Auth | T9 | Frontend | 16 Oct |
ACME-538 | Report filters reset when switching tabs | Low | Open | Reports | T8 | Frontend | 16 Oct |
ACME-539 | Bulk send limited to one page of results without warning | Low | Open | Invoices | Heuristic | Frontend | 17 Oct |
| ID | Area | Observation | Impact | Suggested action |
|---|---|---|---|---|
OBS-01 | Strength | Creating and sending an invoice (T1) and paying in the portal (T6): 8/8 success, rated easiest. | Positive | Don't change these flows. |
OBS-02 | Segments | Trial users succeeded as often as long-time customers; existing customers were faster but relied on habits. | Info | Onboarding isn't the main problem; task design is. |
OBS-03 | Mental model | 7 of 8 started secondary actions (credit, repeat, refund) from the invoice page. | High | Put actions on the object they act on (UX-01, UX-02). |
OBS-04 | Mobile | Portal on mobile worked for all 8; one needed to scroll to find 'Pay now'. | Low | UX-13. |
OBS-05 | Trust | 3 participants double-clicked Send because nothing happened at first. | Medium | UX-09; also check idempotency on the API. |
OBS-06 | Help | No participant opened the help centre; 2 searched Google instead. | Info | Contextual help links where people get stuck. |
| Task | Success | Assisted | Failed | Median time | Ease (1–7) | Main problem |
|---|---|---|---|---|---|---|
| T1 · Create and send an invoice to an existing customer | 8/8 | – | – | 2 min 10 s | 6.2 | – |
| T2 · Set up a monthly recurring invoice | 3/8 | 2 | 3 | 5 min 40 s | 3.4 | Schedule options hidden behind 'More' |
| T3 · Record a partial payment | 5/8 | 1 | 2 | 1 min 50 s | 4.8 | 'Mark as paid' assumes the full amount |
| T4 · Issue a credit note for one line | 2/8 | 2 | 4 | 4 min 05 s | 2.9 | Started from the Credit notes page |
| T5 · Find last month's unpaid invoices | 7/8 | 1 | – | 1 min 15 s | 5.7 | Filter label 'Period' unclear |
| T6 · Pay an invoice in the customer portal (as a customer) | 8/8 | – | – | 1 min 30 s | 6.5 | – |
| T7 · Turn on payment reminders | 6/8 | 1 | 1 | 2 min 20 s | 5.1 | Settings vs per-customer toggle |
| T8 · Export last quarter's revenue to Excel | 7/8 | 1 | – | 1 min 05 s | 5.9 | Export button only in the report view |
| T9 · Invite a colleague as Accountant | 6/8 | 1 | 1 | 1 min 40 s | 5.2 | Role descriptions missing |
| T10 · Put your logo on invoices | 7/8 | 1 | – | 1 min 25 s | 5.8 | SVG upload fails silently |
Success = completed without help. Ease = Single Ease Question after each task (7 = very easy).
| Task | P1 | P2 | P3 | P4 | P5 | P6 | P7 | P8 |
|---|---|---|---|---|---|---|---|---|
| T1 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| T2 | ✓ | ✕ | A | ✕ | ✓ | ✕ | ✓ | A |
| T3 | ✓ | ✓ | ✕ | ✓ | A | ✓ | ✕ | ✓ |
| T4 | ✕ | A | ✓ | ✕ | ✕ | ✓ | ✕ | A |
| T5 | ✓ | ✓ | ✓ | A | ✓ | ✓ | ✓ | ✓ |
| T6 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| T7 | ✓ | ✓ | A | ✓ | ✓ | ✕ | ✓ | ✓ |
| T8 | ✓ | ✓ | ✓ | ✓ | A | ✓ | ✓ | ✓ |
| T9 | ✓ | ✓ | ✓ | A | ✓ | ✓ | ✕ | ✓ |
| T10 | ✓ | A | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
80 task attempts. ✓ success · A assisted · ✕ not completed.
| ID | Role | Type | Using Acme | Tasks alone | SUS |
|---|---|---|---|---|---|
| P1 | Finance manager | Existing customer | 4 years | 9/10 | 72.5 |
| P2 | Bookkeeper | Existing customer | 2 years | 7/10 | 65 |
| P3 | Office manager | Existing customer | 1 year | 7/10 | 80 |
| P4 | Finance manager | Existing customer | 3 years | 6/10 | 57.5 |
| P5 | Founder (5 people) | Trial user | New | 7/10 | 77.5 |
| P6 | Freelance accountant | Trial user | New | 8/10 | 70 |
| P7 | Operations lead | Trial user | New | 7/10 | 62.5 |
| P8 | Finance assistant | Trial user | New | 8/10 | 85 |
Average SUS 71.3; existing customers 68.8, trial users 73.8.
| ID | Issue | Tasks | Affected | Severity | Recommendation | Effort |
|---|---|---|---|---|---|---|
UX-01 | Recurring options hidden behind 'More' | T2 | 6/8 | High | Add 'Repeat this invoice' next to Send | S |
UX-02 | Credit note only reachable from the Credit notes page | T4 | 6/8 | High | 'Credit this invoice' on the invoice page | M |
UX-03 | 'Mark as paid' assumes the full amount | T3 | 3/8 | High | Ask for the amount, show remaining balance | S |
UX-04 | Error messages don't name the field or the fix | T1, T3, T9 | 5/8 | High | Field-level errors, focus first error | M |
UX-05 | Filter label 'Period' is ambiguous | T5 | 4/8 | Medium | 'Issue date' and 'Due date' filters, presets | S |
UX-06 | Reminders: global setting and per-customer toggle conflict | T7 | 3/8 | Medium | One place, show effective state per customer | M |
UX-07 | Role names without descriptions | T9 | 4/8 | Medium | Short description and permission list per role | S |
UX-08 | SVG logo upload fails without a message | T10 | 2/8 | Medium | Accept SVG or explain allowed formats | S |
UX-09 | No confirmation after sending (2–3 s delay) | T1 | 5/8 | Medium | Immediate toast, disable button while sending | S |
UX-10 | Export button only visible inside the report | T8 | 2/8 | Low | Export also from the reports list | S |
UX-11 | Draft invoices can't be deleted with undo | T1 | 2/8 | Low | Undo toast for 10 s | S |
UX-12 | 'Document' and 'invoice' used for the same thing | T1, T4 | 3/8 | Low | Use 'invoice' everywhere | S |
UX-13 | Portal 'Pay now' below the fold on small phones | T6 | 2/8 | Low | Sticky payment bar on mobile | S |
UX-14 | Dialog buttons ordered differently than page buttons | T4, T9 | 3/8 | Low | Primary action always on the right | S |
UX-15 | Customer search ignores VAT number | T1 | 2/8 | Low | Search by VAT number too | S |
UX-16 | Help article for recurring invoices not linked | T2 | 3/8 | Low | Link help from the schedule screen | S |
UX-01 Recurring options are hidden behind 'More'UX-02 Credit notes start from the wrong placeUX-03 'Mark as paid' assumes the full amountUX-04 Error messages don't say what to fix| Heuristic | Screen | Issue | Severity |
|---|---|---|---|
| Visibility of system status | Invoice editor | No feedback for 2–3 s after Send | Medium |
| Visibility of system status | Bank import | No progress indicator for large CSV files | Low |
| Match with the real world | Credit notes | Finance users call them 'corrections' | Low |
| Match with the real world | Navigation | 'Documents' menu contains only invoices | Low |
| User control and freedom | Invoice list | Deleting a draft has no undo | Medium |
| User control and freedom | Recurring | No way to skip one occurrence | Medium |
| Consistency and standards | Dialogs | Primary button left in dialogs, right on pages | Low |
| Consistency and standards | Dates | DD/MM in lists, MM/DD in the date picker | Medium |
| Error prevention | Payments | Full amount pre-filled for partial payments | High |
| Error prevention | Customers | Archive and delete buttons side by side | Medium |
| Recognition rather than recall | Reports | Filters reset when switching tabs | Low |
| Recognition rather than recall | Invoice editor | Product codes required, names not searchable | Medium |
| Flexibility and efficiency | Invoice list | No keyboard shortcuts for frequent actions | Low |
| Flexibility and efficiency | Bulk actions | Bulk send limited to one page of results | Low |
| Aesthetic and minimalist design | Dashboard | 6 charts, 2 used by participants | Low |
| Help users with errors | Forms | 'Validation failed' without details | High |
| Help users with errors | Login | 'Something went wrong' on expired invite | Medium |
| Help and documentation | Recurring | Help article exists but isn't linked | Low |
| Help and documentation | Roles | No explanation of what each role can do | Medium |
| Accessibility (quick check) | Forms | Errors shown by colour only | High |
| Who | Task | Quote |
|---|---|---|
| P4 | T2 | “I'd expect a ‘repeat monthly’ right next to Send. Is this even possible here?” |
| P2 | T4 | “I'd open the invoice and say ‘correct this line’. Why is it somewhere else?” |
| P7 | T3 | “It says paid. But they only paid half… did I just break the balance?” |
| P5 | T1 | “Validation failed — which field? I'll just try again.” |
| P6 | T5 | “Period of what? Issue date or due date?” |
| P1 | T7 | “I turned reminders on in settings, but this customer still says off.” |
| P8 | T6 | “That was easy. Faster than my bank app.” |
| P3 | T9 | “Accountant or Viewer? I don't know what either can do.” |
| P5 | T10 | “I uploaded the logo… nothing happened?” |
| P2 | T8 | “Found it. The Excel button is a bit hidden but fine.” |
| Priority | Change | Fixes | Impact | Effort |
|---|---|---|---|---|
| 1 | Actions on the invoice page: repeat, credit, record payment with amount | UX-01, 02, 03 | High | M |
| 2 | Field-level error messages with focus management | UX-04 | High | M |
| 3 | Immediate feedback on Send; prevent double submit | UX-09 | Medium | S |
| 4 | Role descriptions in the invite dialog | UX-07 | Medium | S |
| 5 | Clear date filters with presets | UX-05 | Medium | S |
| 6 | One place for reminder settings | UX-06 | Medium | M |
| 7 | Small fixes: SVG logo, export button, wording, button order | UX-08, 10, 12, 14 | Low | S |
After the changes: re-test T2–T4 with 5 new participants; target ≥ 6/8 success and ease ≥ 5. The 10 tasks become UAT cases checked every release.
Sample report. Acme Ledger, its data, people and defects are fictional. Your report shows your product, your tickets and your tools.
More sample reports · Book a 15-minute call