Zak Rowling
Ethical Jobs is a social enterprise to connect job seekers with not-for-profit and community jobs. The company have a strict mission statement that defines an ethical job as one that contributes to a more equitable, just or sustainable world and has a direct, positive impact on the community.
As part of a larger user research and optimisation project, I led desktop and mobile usability testing on the existing website to find opportunities to optimise the website and design high-fidelity solutions. Some of these solutions were designed as 'quick wins' to implement straight away, whereas others were designed for A/B testing activities and further design research.
Despite having a massive volume of daily traffic, the Ethical Jobs product team hadn't conducted usability testing and were unaware of what was working for users and what wasn't. They'd done some user interviews and surveys, but never stress tested the user experience.
This was an opportunity to get an understanding of what users found easy or difficult navigating around job listings, how users interacted with the search tools, how easy or difficult users found applying for a job, amongst other questions.
To ensure the usability tests were an accurate representation of Ethical Jobs' target user base, I delved into existing research as well as Google Analytics demographic data to see how wide or narrow we needed to cast our net. I worked with our recruitment platform to screen users who had searched for jobs, specifically not-for-profit or community jobs in the past two years. Five mobile users and five desktop users were recruited for the test, as is generally recommended.
After finding a pool of test participants, we were ready to see the scene. Test participants were given the following scenario as a lens to complete the test.
You’re a young, newly graduated social worker looking for your first professional counselling job in the Sydney area. You’re hoping to get into the workforce as soon as possible to make a difference.
As this is your very first job out of university, you aren’t sure what to expect from your job search and have a lot of questions about getting into the industry.
The day after you graduate, you turn on your computer, open Google and start searching for jobs. After looking through all the counselling jobs on SEEK and Indeed, you stumble on Ethical Jobs and continue your search.
I designed a set of tasks for users to complete, as well as test questions.
Each of the test participants needed to complete the set task list, while also answering additional questions around what they expected to happen and what they would do next. After each task, users rated how easy or difficult the task was.
After the test was completed, each participant was asked to complete a SUPR-Q survey, based on their experience. This likert scale survey lets researchers know how the experience is rated for usability, trust and credibility, appearance, and loyalty. On rewatch of the sessions, I also counted the number of attempts taken on each task to include in the report.
Insights from each session were recorded in the table below following a 'traffic light' style severity rating, with red meaning major issues, orange meaning minor issues, and green meaning good practices or compliments. Blue insights were used for neutral comments or observations.
The test outcomes were further validated through session replay activities. I watched several hundred recordings of users following a similar experience as the simulated test and recorded insights to support what was found in testing. Session replay sessions were segmented by new and returning users, as well as mobile and desktop traffic.
Here are some of the major insights found from usability testing.
After the test outcomes were reported to the client, I created mobile-first, high-fidelity interface designs in Figma to address the issues. Some of the designs were to be developed without additional testing, while others were decided to be A/B tested to validate them further.
