Darllenwch y blog yma yn Gymraeg

In our previous blog we introduced the changes we’re making to GOV.WALES. A really important part of our work is understanding how you, our users, use the site and what you think of the changes we’re making. We do this by running usability testing whenever we can to help us identify any issues early and fix problems before they take root too deeply.

How we conduct usability testing

We’ve adopted Agile principles for the GOV.WALES project and run 2-week sprints. We spend the first week building stuff and then when time allows we run a testing session with 4 or 5 users on the second Wednesday, writing up our findings the following day. We’re then in a good position to inform Friday’s planning session where we decide what to tackle in the next sprint.

Each testing session consists of a number of scenario-based tasks that we ask users to complete. We watch what they do and make a note of things they find challenging – with the site design or the content. Once they’ve completed the task (or had enough of trying), we ask them to give the site a score out of 10 for that task. At the end of the day we write up our observations on sticky notes so we can eliminate duplicates and start to draw out what we’ve learnt.

With the user’s permission, we sometimes record the sessions using Silverback. This tracks the user’s mouse movements and records audio and video. It’s a useful back-up when we think we’ve missed an important nugget of information or just need to resolve some confusion about what a user said or did.

What’s it like to take part?

As users of GOV.WALES will have varying levels of digital experience and capability, we try to involve a wide range of users in our testing. Whether you are proficient or have basic skills, the user testing is nothing to be scared of – as explained by one of our volunteers.

“I was more than happy to take part in user testing – especially as I am not particularly quick with new technology. I’m able to highlight things from my perspective that a ‘Techie’ person would assume everyone knows (some terminology, for example). Experienced users automatically assume certain things about other users which isn’t always the case and, consequently, I felt my input was very useful.

“The whole thing took about 20 minutes. I sat in front of a computer on the .gov.wales website and the point of the exercise is to test how easy some new pages are to use before they went ‘live’.  I was given instructions / tasks to do – e.g. can you go to the consultation on xxxx, which I then had to find.  Then I would be asked to find something else to do with that consultation and maybe asked to go back to the original screen to look for something else.

“It sounds quite straightforward, but when faced with a new website that looks unfamiliar, you really have to look for things and understand the terminology, which not all of us do. At the same time I was being filmed so that my reactions could be monitored and I also had to give a running commentary on what I was doing, how I was doing it and how I felt about each task. At the end of each task I had to score the task out of 10 on how easy I found it.

“It wasn’t especially difficult but once I understood the format, it became easier each time. You don’t have to be technically minded, in fact, it is probably better not to be as the point of the exercise is to see how easy the website is for the public, who can have varying experience of using a website.”

Where do we go from here?

So far, sessions have taken place in meeting rooms in our main office in Cardiff. Usually two of our team attend, one to run the session and one to observe. We asked Santa for a dedicated user lab for Christmas but he must have run out. Or maybe it was too heavy for Rudolph to carry. So instead we’re going to set up another room where we can observe the testing. This will let more of the team watch the session and get a better, direct understanding of what’s working on the site and what’s not. We’re also looking forward to experimenting with some new kit that tracks where the user is looking during the tasks.

We are also trying to run sessions remotely: we’ve used Loop 11 that lets us see where users click when trying to complete a task, and we’ve used Validately that gives us richer information by recording video and audio of users. Tools like these let us reach a whole more users from right across the country but some volunteers aren’t overly keen on using their webcam to record what they do and sharing it with government.

So far, we’ve got about 100 volunteers on our user roster but are always looking for more. If you’d like to help us make GOV.WALES better, please email digital@wales.gsi.gov.uk.

Post by Graham Craig, Corporate Digital Communications Team, Welsh Government