Most failed go-lives in South Australian agencies and councils are not caused by bad code alone. They come from testing that started too late, user acceptance testing (UAT) squeezed into the last week, test data copied from production without anyone asking whether it should be, and no shared view of what “ready” means. This guide explains, in plain words, how SA Government agencies, councils and small businesses can plan software testing and UAT for a new system, a website or a modernisation project: the layers of testing, how to run UAT properly, how to keep personal information out of test environments, and what to write into a tender or contract. It is practical guidance, not legal advice.
Why testing goes wrong on government and council projects
The same patterns come up again and again, whatever the technology:
- Testing is a phase, not a habit. When testing only starts after the build is “finished”, defects are found when they are most expensive to fix and the go-live date is already announced.
- UAT is treated as a demo. A few staff click through the happy path in a meeting, nobody tests the awkward cases, and sign-off is given because the date has arrived.
- Nobody owns the test environment. It is out of date, shares integrations with production, or is unavailable when testers need it.
- Real personal information ends up in test. A copy of production is the fastest way to get realistic data, so it gets used, often in an environment with weaker controls.
- Accessibility, security and performance are left to the end. They are then either rushed or quietly dropped.
- “Done” is never defined. Without entry and exit criteria, every go/no-go meeting becomes an argument.
The layers of testing in plain words
No single kind of testing finds every problem. A sensible plan uses several layers, each owned by the people best placed to do it:
| Layer | What it checks | Usually done by |
|---|---|---|
| Unit and component tests | Small pieces of code behave as intended, including edge cases. | Developers, automated in the build pipeline |
| Integration and API tests | The system talks correctly to finance, CRM, records, identity and payment services, including when they are slow or down. | Developers and testers |
| System and end-to-end tests | Whole journeys work, such as lodging a request, paying a fee or approving a case. | Testers, increasingly automated |
| Regression tests | A change, patch or upgrade has not broken something that used to work. | Automated suite, run on every release |
| Accessibility tests | Pages and forms meet WCAG 2.2 AA with a keyboard, a screen reader, zoom and a small phone. | Testers with accessibility skills, plus automated checks |
| Performance and load tests | The system copes with peaks, such as a rates due date, a grant round opening or a storm event. | Specialist testers or engineers |
| Security tests | Common weaknesses are absent and access controls work; a penetration test for higher-risk, internet-facing systems. | Developers, then an independent tester |
| Data migration tests | Records moved from the old system are complete, accurate and reconcile with the source. | Testers with business data owners |
| User acceptance testing (UAT) | The system supports the real work the business needs to do. | Business staff, supported by testers |
| Operational acceptance | Backups restore, monitoring alerts reach someone, runbooks work and support handover is complete. | The team who will run the system |
The lower layers are mostly the supplier’s job and should be largely automated. UAT and operational acceptance are the buyer’s job: a supplier can help, but cannot accept the system on your behalf.
UAT done properly
UAT is where the business confirms the system is fit for its purpose. It works best when:
- The business owns it. Name a UAT lead from the business area, not from ICT or the supplier, and give testers protected time away from their normal duties.
- Scenarios come from real work. Write them from actual cases: the complicated permit, the customer with two addresses, the refund, the request that is escalated, the end-of-month report. Include what should happen when something is wrong, not only the happy path.
- Testers represent real users. Include front-counter and field staff, people who use keyboard navigation, screen readers or magnification, and anyone who will use the system on a phone or tablet.
- Scripted and exploratory testing are both used. Scripts prove coverage; short exploratory sessions find the problems nobody thought to script.
- Defects are logged consistently. One defect register, with steps to reproduce, severity, screenshots and an owner, shared with the supplier.
- Sign-off is against criteria, not a date. The business owner signs when the agreed exit criteria are met, and records any known issues accepted for go-live.
Test data without privacy risk
Test environments are often less protected than production: more people have access, logging is lighter and data is copied around. The ASD Information Security Manual expects development, testing, staging and production environments to be segregated, and production data not to be used in any non-production environment unless that environment is secured to at least the same level as production.
Privacy obligations still apply in test. State agencies are bound by the Information Privacy Principles Instruction, councils have their own privacy policies and community expectations, and many businesses are covered by the Privacy Act 1988. A personal information leak from a test environment is still a breach (see our guide to the Privacy Act and notifiable data breaches).
Safer options, in order of preference:
- Synthetic data generated to look realistic, including awkward cases such as long names, unusual addresses and accented characters.
- Masked or de-identified copies where synthetic data cannot reproduce a problem, with the masking checked before the data is released to testers.
- Production data only by exception, with a documented approval, a test environment secured to production level, limited access and a deletion date.
Also check that test environments cannot send real emails, text messages, payments or records updates to the public or to production systems. Point integrations at sandboxes, and make every test notification clearly marked as a test.
Accessibility and security testing that stand up to scrutiny
Accessibility
Automated checkers find only part of the WCAG 2.2 AA failures that matter. Combine them with manual checks: keyboard-only navigation, a screen reader on key journeys, 200 percent and 400 percent zoom, and small phone widths. Test forms and error messages most carefully, because that is where residents and customers get stuck. Our WCAG 2.2 guide for SA Government and council websites lists the failures that most often show up in audits.
Security
Build security testing in from the start rather than relying on a single test just before launch:
- Static and dynamic application security testing (SAST and DAST) before the first release and every later release, as the ASD Information Security Manual expects, plus automated dependency scanning in the build pipeline
- Requirements and test cases mapped to a recognised baseline such as the OWASP Application Security Verification Standard (ASVS)
- Access-control tests that prove one user cannot see another user’s records
- An independent penetration test for higher-risk, internet-facing systems, with time set aside to fix findings and retest
A good regression suite also supports the Essential Eight. Patching applications quickly is far easier when you can prove a patch has not broken anything in hours rather than weeks (see our guide to the Essential Eight for SA SMEs doing government work).
Automate what repeats
Manual testing is essential for UAT and exploratory work, but repeating the same checks by hand on every release is slow and error-prone. Automate the checks that run every time:
- Critical journeys end to end with a browser automation tool such as Playwright
- Accessibility rules with an automated engine such as axe-core on every page template
- APIs and integrations with contract and API tests that run against sandboxes
- Performance baselines with a load tool such as k6 or JMeter before major releases
Run them in the delivery pipeline so a failed test stops a release automatically (our CI/CD and infrastructure as code reference architecture shows where these gates sit). Keep the suite small, fast and reliable: a few hundred trusted tests are worth more than thousands that fail at random and get ignored.
Entry and exit criteria, and severity
Agree these before testing starts, so go/no-go is a decision against evidence rather than a debate.
- Entry criteria for UAT: system and integration testing complete, no open critical defects, environment and test data ready, testers trained, scenarios approved.
- Exit criteria for go-live: all planned scenarios run, no open critical or high defects without an approved workaround, accessibility and security findings resolved or formally accepted, data migration reconciled, operational acceptance passed.
| Severity | Typical meaning | Go-live |
|---|---|---|
| Critical | Core function unusable, data loss or corruption, a security or privacy exposure, or a legal obligation cannot be met. | Blocks go-live |
| High | An important function fails and there is no reasonable workaround. | Blocks go-live unless a workaround is approved |
| Medium | A function fails but a reasonable workaround exists. | Can go live with a fix date |
| Low | Cosmetic or minor issue with little effect on users. | Fix in a later release |
Treat accessibility failures that stop someone completing a task as high or critical, not cosmetic.
Writing testing into a tender or contract
If testing is not in the scope, it will not be in the price. In a request for tender, panel RFQ or statement of work, ask for:
- A test strategy covering each layer above, who does what, and the tools and environments used.
- Automated regression tests handed over with the source code, so you can keep using them after the project.
- Accessibility conformance to WCAG 2.2 AA with evidence of manual as well as automated testing.
- Security testing mapped to a named baseline, and the timing of any independent penetration test.
- Test data handling that keeps personal information out of non-production environments by default.
- UAT support: environment availability, defect turnaround times by severity, and help preparing scenarios.
- Defined entry and exit criteria and a warranty period with response times for defects found after go-live.
For modernisation projects, add parallel-run and reconciliation testing for every cutover (see legacy application modernisation for SA agencies and councils). For website moves, add redirect, link and search checks (see the website and CMS migration checklist). And when you are the supplier, a clear testing approach is one of the easiest ways to stand out in an evaluation (see winning SA Government ICT tenders).
The small business version
A small business buying a booking system, an online shop or a customer portal does not need a formal test team, but the same ideas apply at a smaller scale:
- Write down the ten tasks the system must do well and test each one yourself before paying the final invoice.
- Try the awkward cases: a refund, a cancelled booking, a failed payment, a customer on an old phone.
- Ask your supplier what is tested automatically on every change, and who tests updates to plug-ins and integrations.
- Never give a supplier a full copy of your customer list for testing unless there is no alternative and you know how it will be protected and deleted.
A 90-day plan
- Weeks 1 to 2: strategy. Agree the test strategy, environments, test data approach, severity definitions and entry and exit criteria. Name the business UAT lead.
- Weeks 3 to 6: build the safety net. Set up automated unit, API, accessibility and end-to-end tests in the pipeline. Create synthetic test data and connect integrations to sandboxes.
- Weeks 7 to 9: prepare UAT. Write scenarios from real cases, recruit representative testers including assistive technology users, and run a dry run with a small group.
- Weeks 10 to 12: test and decide. Run system, performance and security testing, then UAT and operational acceptance. Hold the go/no-go against the agreed criteria and record accepted known issues.
Questions to ask your supplier or IT team
- Which tests run automatically on every change, and can we see the results?
- Where does our test data come from, and does any of it include real personal information?
- How do you test accessibility beyond an automated scanner?
- What security baseline do you test against, and when is the penetration test?
- What are the entry and exit criteria for UAT and go-live, and who signs them off?
- Will the automated tests be handed over with the code at the end of the project?
Talk to an Adelaide team
If you are planning a new system, website or modernisation and want testing and UAT that give you confidence at go-live, talk to us about software testing and UAT or call 0402 401 584. We can start with a short, fixed-price review of your test strategy, environments and test data approach that produces a prioritised plan, then help you build the automated regression suite and prepare UAT. Learn more about our application development and DevOps and cloud services.