Richard Thornton Richard has over 10 years experience in consultancy, operations and commercial strategy positions within the research industry, building up a particular expertise in the online research sector. Read the full biography here.

Will the real Arthur please stand up?

The search for MR's Holy Grail

By Richard Thornton - 6th August, 2010

Like Arthur in the classic 1975 film, The Holy Grail, it seems as though the entire MR industry is searching the lands, high and low, Monty Python style, looking for the research industry’s very own mythical bowl; a universally accepted solution in how to define and measure quality in online sampling and data collection. It is a topic that simply won’t go away, and rightly so.

For some time now, many of the big data buyers, particular in the US, have been pressing hard for more transparency and greater consistency in how sample is sourced and just who on earth these people are taking our surveys, day in day out. At the heart of the issue has been the question “how do we ensure that the people taking surveys are who they say they are?” and hand in hand with this is “how do we guarantee that only unique participants are responding to our surveys on a project by project basis?”. There are also other important requests on the buyer’s wish-list, like quarantining by category and past participation and limiting the number of survey opportunities one can take.

..like Arthur and the Knights of the Round Table....there has been an ultimate failure in seeking out the grail.

In response to the likes of P&G, Coca-Cola, GM, Unilever and the other large buyers holding the purse strings, there have been some commendable efforts in taking up the challenge, although like Arthur and the Knights of the Round Table with the wonderful Trojan Rabbit, there has been a point in each when doubts have started to creep in about whether this is going to work and an ultimate failure in seeking out the grail. It would seem that no one has yet come forth with the right affordable, accessible, all singing and dancing solution. However, is that all about to change with the recent announcement that GfK and Kantar have collaborated in setting-up a not-for-profit online consortium, called Promedius Group. The consortium's offer includes a suite of tools to validate the physical address of online participants as well as ensuring only unique participants can take the same study. Promedius also claims to weed out any duplicates across sample sources and to track category participation using a common standard.

There are a number of reasons as to why this is potentially such a major landmark for the industry. Firstly, for the first time such an initiative is being led by the MR agencies themselves, rather than any single supply vendor. These are the agencies the likes of P&G, Coca-Cola, GM, Unilever, and obviously many others, all buy from so whilst there are already technology products available, all of which do the majority of what Promedius will presumably acheive, there must still be a desire from the end buyers for something over and above these current offerings.

Secondly, the technology behind the service is also a shared effort and brings another MR agency into the frame, with Ipsos working alongside GfK and Kantar to bring the technology to market. This level of knowledge sharing is becoming more commonplace but has never been entered into before on this specific topic.

Thirdly, the technology will be free to implement, so accessible to all, and offered across the MR industry to researchers and agencies alike, with a view to getting adoption from the sample vendor community as well. It remains to be seen what the usage charges are, if any, but if they are nominal then Promedius could be game changing. With the commercial angle taken out of the picture sign-up should be very high leading to possible industry wide acceptance.

Beyond what we know already, it will be intriguing to see what else the Promedius Group accomplishes and whether it will also try to tackle the issue of survey design and best practice. There are still far too many ‘poor quality’ surveys landing on the laps of supply vendors and unfortunately being sent out to participants. The reality in these commercial times that we live in and with increased pressure to deliver profits, ensure healthy margins and survive in what can be a ruthlessly competitive industry, is that the conditions are ripe for suppliers to be scared to say "no".

It should be good news for the industry though as we all strive for more consistent processes, stricter quality guidelines and less black-box mentality in the supply chain. Quality tends to increase in economic downturns as suppliers strive to ‘over-deliver’ to secure new business and retain existing customers whilst buyers have higher expectations in driving more value for the same or smaller budgets. Spending prudently leads to clearer purchasing decisions and the same applies for sample procurement. Yes there is continued discounting and price pressure but everyone should remember the old adage that if you pay peanuts you get monkeys. We all need to make a living and to support each other and the industry more so now than ever before.

If the right quality checks and push-back was applied then I would guess that perhaps as much as a third of all surveys going live into panels would be stopped. It never ceases to amaze me how long people believe they can ask a participant to sit in front of a screen and take a survey for, often a badly designed survey at that. The industry needs to up its game in starting to ‘cap’ the length of surveys, something we have a bee in our bonnet about at my organisation (it’s one of several bees). It can frustrate the hell out of clients but we know it's the right thing to do.

Nothing is too late and there is definitely more good than bad out there. Hopefully the Promedius Group and bodies like ESOMAR, CASRO and others will continue to make a difference. Personally I think the industry has come on leaps and bounds regarding quality and has worked tirelessly to minimize any potential effects to the model that may lead to participant fatigue, attrition and frustration.

There is still work to be done though, especially on the topic of over-solicitation of members on panels. There is a lack of good data to say what the right frequency of invitations should be but it certainly shouldn't be multiple invites per day, even daily invites should be a concern. If profiling is up to date and well populated and the incentive the right balance (affinity vs. value) then 'spamming' can be eradicated. With all the technology and collaborative efforts now pushing us in the right direction, it would be frustrating if greed and commercial pressure stop us reaching our goal. As frustrating as finally getting to the cave with the grail and finding your way barred by the fearsome Rabbit of Caerbannog; or not knowing the average land speed of the swallow. As an industry, sometimes we might need a Holy Hand Grenade and sometimes just the right answer, but we must and shall keep moving towards our goal.

Richard Thornton

Comments on this article

Want to share your thoughts...?

Want to share your thoughts?

NOTE: Please note that this board is moderated, and comments are published at the discretion of the site owner.