Service assessments

Questions in an alpha assessment

Run through these example questions with your team.

Answer some of these questions as you tell the story of your service.

Designer and content designer

Your designer and content designer be asked questions like:

Have the constraints you’re working within changed? What are the barriers to removing or reducing the impact of those constraints?
How will you continue to ensure the service is scoped in a way that makes sense to users?
What are you doing to make sure users don’t have to provide the same information multiple times?
How do you make sure that the service doesn't exclude any existing or potential users of your service?

Your lead developer will be asked questions like:

What technical choices have you made and why? Including language, framework, deployment, integration and 3rd parties.
What other distinct options have you tried or tested?
How are you managing your open source repository? How is it being shared with other teams who may need to use similar solutions? Including open standards, open data, common platforms and ownership of Intellectual Property.
How do you ensure the service is safe for users, including data privacy, security threats and fraud vectors?
What is the minimum amount of personal information your service needs to collect?
Does your service need an authentication solution?
What plans do you have if your service is unavailable for any length of time?
Have you spoken to your department’s data protection officer about the decisions you’ve made?
Have your technology plans changed?
Is there significant activity planned for modifying or upgrading the technology used?

Your user researcher will be asked questions like:

Who are your users?
What are your users’ lives like?
Why do they need the thing you’re building or buying?
Why has Defence asked you to solve the problem you’re solving?
What do you know about the offline parts of your user’s journey?
How do you know your service joins up with those offline elements?
Who are you inviting to your user research sessions?
What are your continued plans for user research after launch? Is the budget sufficient?

Your service owner and delivery lead will be asked questions like:

What, if anything, is different in your live team compared to the beta team?
How are you engaging people outside of your team? For example service teams across Defence, operations and policy or other government departments.
Has the team disagreed on anything and if so how was that resolved?
How are you using agile methods and how will you continue after launch?
How does governance work? Are there any issues you’ve had to escalate?
What are you doing to make sure other teams know what you’re working on?

Your service owner or performance analyst will be asked questions like:

How are the Key Performance Indicators (KPIs) you identified improving the service?
What existing data sources did you use to decide on your KPIs?
How do you know your service is meeting your users’ needs?
How do you check the service is doing what Defence needs it to do?
What data are you collecting and how are you using it to improve your service?
What management and governance do you have? For example SIRO approval, not collecting personal data or retaining a raw backup data view.
Which KPIs are being published on data.gov.uk and has this changed since launch?
Once live, how will you measure change over time?
How do you know you’ve got appropriate metrics in place to measure the success of the service, based on what you’ve learned during public beta?
How will you continue to use data to improve your service after launch?

Published March 2023