What is an 'Image Rank' block on OpinionX?
Image Rank is a survey question format on OpinionX which is used to turn a set of images into a ranked list by using a series of head-to-head pair votes. For example:
How are Image Rank scores calculated?
Image Rank on OpinionX is based on the same algorithm as our Pair Rank block, which uses a format called "Win Rate Analysis". For each option, this takes number of wins and divides by its total number of pair comparisons, giving a score between 0-100 (ie. the percentage win rate).
What do the Image Rank exported results look like?
For more information about exports, including examples of exported spreadsheets, please visit the Knowledge Base page on exporting.
Can I change the number of image pairs each participant must vote on?
Yes, you can set your own custom number of pairs for each participant or set your survey to show every pair possible to each participant. Note: only participants who finish your entire survey will be counted on your results page.
How many votes per participant will generate robust results?
There are two formulas you can use to make sure you have a good minimum sample size:
Both are explained below...
Formula 1: Limited pool of participants
This formula tells you how many votes each participant will need to cast so that every possible image pair combination has appeared at least 3 times in total across the whole survey. The formula to calculate this is as follows, where "n" is the number of options you're ranking:
3n(n-1)/2
------------
participants
Here's a breakdown of how this formula works:
For example, if you have 16 options to rank and know that you'll only have 30 people to complete the ranking survey, our formula would be (3(16x15)/2)/30 = 12 votes per person. In this case, when deciding how many votes each participant must cast, I should opt for at least 12 votes or more per person.
Formula 2: Large pool of participants
When you know that you've got a good plenty of participants you can engage, you'll want to get as much data per person without making your survey overwhelming. We'll use a slightly different formula:
3n(n-1)/2
---------------
votes per person
Here's a breakdown of the meaning of this formula:
Whereas before our limiting factor was the number of participants, now our limiting factor is the number of pair votes we will ask each participant to cast.
Let's say we have 40 options we want to rank using the Image Rank format. I usually use 10 votes per participant as my starting point. That formula would then look like: (3(40x39)/2)/10 = 234 participants. If that seems like a lot more than I'll be able to reach, I can try increasing the number of votes to 15, which adds up to 156 participants. As you can see, the number of votes per participant changes your required sample size a lot!
Full Formula Explanation:
Total number of possible pairs is
n(n-1)/2, where n is the number of ranking options.Multiply the total possible pairs by 3 to calculate your target number of total votes.
Divide your target number of total votes by your minimum estimate of participants you expect will complete the survey.
The answer you get is the minimum number of pair votes you should require from each participant.
Some Caveats:
Generally, if the formula results in a number below 10, I would just leave the default 10 votes in place as it is a very reasonable amount to ask from each participant.
If your survey has multiple Image Rank questions, you should consider the total number of votes a participant will have to cast across your entire survey. The higher your total number of pair votes, the lower your completion rate will be. Generally, anything above 40 pair votes within a survey is a lot to expect of any participant (unless you provide a strong participant incentive).
If segmentation is going to be an important part of your analysis, you should substitute the "
participants" variable for your estimate of the total number of people you expect to reach from your smallest key segment. Here's an example:I have 14 ranking options and expect to reach 150 participants, but I plan to segment my results to see the differences between customers on my four different pricing tiers. The segment I'll get the least responses from is Enterprise Customers, but I think I can get at least 16 participants from them. So my formula would be
3(14*13)/2/16= 17 votes per participant.
Can I use multiple Image Rank blocks in my survey?
All OpinionX users can have an unlimited number of Image Rank blocks in their survey, regardless of whether they're a free or paying customer. This apples to all block types on OpinionX.
How many images can I include in an Image Rank question?
There is no limit to the number of images you can include.
Do Image Rank questions work on mobile devices?
Yes. All question types available on OpinionX are designed to be suitable for participants on any web browser -- whether that's a desktop monitor, laptop, tablet or smartphone.
Can I use emojis in my Image Rank captions?
Yes, you can use emojis in the caption sections of Image Ranking options on OpinionX.
I have a question about Image Rank blocks that wasn't answered here!
Send us a message using our live chat or email [email protected] to get a quick answer from the OpinionX team.