Outsourced Usability Testing

by Editweapon on September 11, 2008

A few weeks ago I saw a tweet by John Jantsch who had interviewed Bryan Eisenberg of FutureNow .  The post was called Maybe the Best $100 You Ever Spent , and it was about a new service called UserTesting.com.

Here’s how it works:

  • You sign up for user testing, specifying the demographic profile of your target audience and how many user testers you want (one user costs $19, five users cost $95).
  • Users record their screen and voice as they use your website, speaking their thoughts as they browse.
  • You watch and listen to them use your site. Each user’s session – mouse movements, clicks, keystrokes, and spoken comments – is saved as a Flash video for you to watch.
  • You read their review.
  • When I read that post I thought, "Well this will either put me out of business, cause me to cut my rates by 90%, or make my life 900% easier!"

    After running one of my other company’s pages through the UserTesting.com guantlet, I’m happy to report that the latter appears to be the most true.  Why?  Because anyone can watch a user use a website, but *interpreting* usability tests and making recommendations is the secret sauce to being a kick ass information architect / interaction workflow designer, which of course, I am.

    UserTesting.com made the hardest part — finding virgin users to test out UI pages or concepts — about a billion times easier.  I intend to contract UserTesting.com on every project I do from here on out.

    Here’s a video of my reaction to watching a user visit the Jigsaw Magnesium landing page that has had upwards of a 19.8% conversion rate, and is a case study in the Eisenberg’s new book, Always Be Testing oh by the way. ;)


    UserTesting.com Review 1 from editweapon on Vimeo .

    In this video, I (randomly) discuss several aspects of interpreting usability tests.  This would be a subject for a great follow-up post in the future based on the things I’ve learned to look for over the years.

    Watching users is FASCINATING to me, but probably boring as **** to 99% of the population out there.  Plus, it’s 17 minutes long and I cut off the last 12 minutes. So I don’t blame you for falling asleep and drooling on your keyboard, but there’s no F’in way I’m paying for a replacement if the thing breaks!

    Go try out UserTesting.com for yourself.

    UPDATE: In case you’re interested, here’s the tasks I gave my usuability testers…

    Goal: You want to purchase a magnesium supplement.
    – What is your first impression of the landing page?
    – Does the copy make you feel concerned about your own potential for magnesium deficiency? If yes, can you see a way to check your own risk for magnesium deficiency?
    – Does this formula seem effective?
    – What would you say makes it better than other magnesium supplements? – Where else might you look to verify the claims of this product?
    – Go through the process of buying the product (but do NOT click the final "SUBMIT ORDER" button. In the comments field on the last page, please write "This is a test order per Patrick Sullivan. Do not process.")
    – Even though you were asked to walk through the shopping cart, was there enough information for you to make an informed decision on whether or not to purchase it?
    – If this wasn’t a test, would you have made a purchase? Why or why not?

    Demographics :
    Gender: Female
    Age: 56 or older
    Yearly Household Income: Doesn’t Matter
    Expertise with computers and the Web: Beginner
    Other Demographics: Doesn’t Matter

    Previous post:

    Next post: