Top 10 major risks of poor user recruitment: Is your recruitment negatively affecting your research?

Recruiting the right participants for a study is a difficult task and an essential component of the research process. It ensures your user research is valid and the end results (your design changes) are effective.

“Poor user recruitment may have major negative impacts on your research”

It’s well worth the extra time, effort and cost to ensure you recruit representative participants who can provide useful qualitative feedback. Recruiting the right participants is the foundation of effective user research, because your research results are only as good as the participants involved.

When the recruitment of participants for your research is poorly carried out, there is a whole host of negative consequences and potentially a dramatic negative impact on your research and validity of the findings.

Top 10 major risks of poor user recruitment

1. No recruitment at all!

When the agency tells you they can recruit your target users, but it turns out they can’t. This is one we’ve personally experienced. We briefed an agency on what we needed and even gave them the full screener to use and they promised they could deliver. At the last minute, they suddenly pulled out as they realised they were unable to recruit any of our target users.

2. No-shows

The worst thing that can happen on the day of the research and whilst you have your stakeholders and your manager in the observation room is that a user doesn’t turn up. This might happen when people are not carefully selected and their reliability has not been assessed during the recruitment process. However, sometimes things do happen that can’t be avoided – One time a user called us at the last minute to say they wouldn’t be able to make it as they’d just crashed their car on the way to see us! Certain target groups are understandably less reliable (mums often have sick children or last minute childcare issues), in which case you might need to consider recruiting a standby user to stay onsite.

3. Late-shows

Late shows put a lot of pressure on the researcher so need to be avoided as much as possible. There may be bad traffic that day, the bus was late, or the user may simply be poor at time keeping. You should always ask people to arrive earlier to account for these little problems.

4. Uncommunicative participants

Part of good recruitment, is assessing the user’s ability to verbally express themselves. A poor recruitment process can lead to the shortlisting of participants who struggle to express themselves and struggle to communicate their opinions to the researcher.

5. Misinterpretation of your needs

Poor user recruitment is often caused by misinterpretation of your needs due to a lack of expertise in UX research by the recruitment agency. Often agencies don’t clearly understand what is involved in doing user testing / UX research and because of this lack of expertise, they struggle to understand exactly what you need and therefore they fail to recruit the right people.

6. Non-representative sample

If the agency doesn’t understand your needs, they won’t be able to craft an accurate screener. The screener is essential for selecting the right candidates. It may surprise you to hear that many agencies don’t even use a screener, they simply send out a message with your requirements asking for people to let them know if they meet all the criteria. It means it’s a lot cheaper for them to recruit as it takes less time, however, it’s much more likely that users will tell untruths simply to fit the criteria.

7. Brand advocates and bias

Companies who recruit themselves often don’t realise how much they are biasing their own results. A real world example we have just seen, is a retail e-commerce company who are asking for users through their social media channels (along the lines of, ‘love our brand? come in and give your opinion on our website!’). Firstly this type of recruitment attracts people who are already brand advocates and therefore are more likely to give you positive feedback. Yes, that’s nice to hear but not very useful, especially as your aim is to grow your business and attract new customers – what do they think, what will make them switch brands? Also bear in mind that it’s highly likely that users will use your website before they attend the research too as they know they’re going to be using your website. Using a third party keeps this hidden until they attend the session as they won’t know they’ll be using your website until they are at the session and they can’t swot up beforehand!

8. Not enough time to recruit

Working as a UXer means working to tight timescales and an iterative process. It used to drive us crazy that recruitment agencies would need 4 weeks notice to recruit (or they’d turn us away as they were too busy). Fortunately, we have a solution for you, keep reading to find out!

9. Recruiting ‘experts’

Someone slips through who works in web design or who used to work in your sector. They are obsessed with tiny details that ordinary users wouldn’t pick up on and there will be very little of their interview that you can use afterwards.

10. People who just want the money

Their aim is to get through the session as quickly as possible so they can get paid. They don’t interview well as their mind is purely focussed on finishing the tasks quickly as opposed to getting into the mindset. Good recruitment screens out this type of person.

The solution

All of the above can be easily avoided by using good recruitment methods and a thorough user recruitment agency who specialises in UX user recruitment. They are almost impossible to find, and our own bad experiences have led to innovate within the UX industry.

We’re bringing to you our new UX user recruitment agency, I Need Users, founded by UX experts, Keep It Usable. We totally understand your user recruitment needs and your research because we do it ourselves on a daily basis. I Need Users also provides quick, flexible and last minute options to suit your iterative methods.

Need help or advice?

If you’d like to know more about UX participant recruitment and how it can help you, contact our UX experts for free, friendly, no-ties advice.