 |
Andrew Jeavons
Andrew has worked in the survey software and market research industry for over 20 years in Europe, the USA and the Pacific Rim. He is a frequent writer and presenter on all subjects related to surveys and MR. He is currently Executive VP of SurveyAnalytics LLC USA, a leading provider of DIY MR tools.
Read the full biography here. |
Power to the people
Why market research doesn't need protectionism
7th March, 2011
I enjoyed Jerome Sopocko’s article, he is a very intelligent and interesting person. He makes an interesting case for protecting the MR business. I don’t agree with it though. If any industry has to resort to protectionism then it deserves to die.
There is a lot of noise about ‘DIY research’, most of it seems to be about how it is a ‘threat’ or is somehow wrong. Neither of these is true, unless you want the world to stay the way it is. Progress is always going on, and to decry something for which there is a need is simply not going to work. MR has to embrace change. You can’t stop the future.
I started off as a psychologist, and then I moved into neuropsychology, statistics and eventually software development for the MR industry. I agree that mathematical models are great things. They can have great elegance and power. The fact is that the example that Jerome gives of modeling is, generally speaking, the exception for MR. The vast majority of MR studies contain nothing so elegant or durable. So let’s be clear what we are talking about. While there is some tremendous research and modeling going on in MR, this is not the norm.
I can’t really take the argument that ‘writing surveys’ is a vital justification for ‘professional’ MR. If that is really one of the core unique selling points for ‘professional’ MR, give up now - seriously. Yes, you can bias results via wording. However the assumption than any survey can access some absolute truth is invalid, surveys are very general blunt instruments. The whole idea that we can accurately introspect as to why we do something via a survey is naive psychology of the worst kind. It’s not a question of dumbing down surveys, it is a question of being realistic as to what sort of data you get from a 45 minute survey with huge grids in it. People will always give you answers, but they may not be relevant answers.
The reason why DIY MR is important is that there is a demand for it. MR has adopted a very condescending attitude to DIY research. Just because companies want to do research themselves without the gods of MR blessing their efforts doesn’t mean they are doing the wrong thing. It all comes down to money. There are tens of thousands of companies that don’t have tens of thousands of dollars to give to MR companies for research. Does this mean they can’t research their market place? Just because I can’t afford to buy a BMW doesn’t mean I shouldn’t have a car. It is absurd paternalism to say that companies can’t do their own research. I reject the argument that DIY research is inherently a bad thing. There may be bad DIY research, just as there is bad professional research.
What traditional MR companies have to accept is that making money on the mark up of data collection costs is an old business model. The traditional MR business model generally had a lot of profit and revenue associated with data collection with a relatively small amount of the revenue associated with analytics or modeling. It is clear that MR can’t use this model anymore, or at least it can’t use it all the while. Analytics is the new frontier, whatever we think of the quality of data, we can get the data. What to do with it is the next question. I would suggest that in the long term MR professionals should forget about criticizing survey design and concentrate more on honing their analytical skills.
MR has to adapt, and the DIY movement represents an ideal opportunity for MR companies to take on a consultative role. It isn’t how things used to be, but nothing is the way it used to be. I don’t see a movement by statisticians to disallow the sale of end user statistical software. I don’t hear anything about ‘DIY statistics’ being a bad thing, and that is what most of the people in MR do when it comes to statistics. Professional MR practitioners are often guilty of grievous abuses of statistics because they, either through lack of training or experience in that area, are ‘DIY’ statistics users. How can they therefore complain about ‘DIY’ research?
An example of the DIY nature of the so called MR professionals is the practice performing multiple t-tests on a cross tabulation without adjusting the significance level for each of the t-tests performed. To statisticians this is one of the problems of ‘DIY’ statistics. Yet this is a common practice in MR. MR professionals seldom pay any attention to the assumptions of parametric statistics. I can’t recall when I saw any analytics using non-parametric statistics, yet a great deal of data in MR simply does not conform to the assumptions required by parametric tests. When was the last time you saw someone report a Wilcoxon test result? A Mann Whitney U test anyone?
This is just an example to illustrate that a lot of current MR is DIY. All sorts of disciplines such as statistics, anthropology, psychology and sociology (to name but a few) are plundered at will by so called MR professionals. Various bits of these disciplines are then regurgitated to their clients, often distorted and misunderstood, in the hope of ‘telling a good story’. There is more Scheherazade than science in MR it seems. I therefore find it hard to take the arguments of protectionism from professional ‘MR’ concerning ‘DIY Research’ when many of the ‘professional MR’ skills are obvously DIY.
MR has to embrace reality, and that includes an increase of so called ‘DIY’ research. The paternalism and condescending attitude just looks like a defensive reflex. Maybe all this means is an adjustment to the way MR works with clients, is this such as bad thing? What is certain is that decrying something that is happening already, and for which there is a steadily increasing demand, is a sure way to become irrelevant.
Andrew Jeavons
Comments on this article
Want to share your thoughts...?
It was a pleasure reading such a pragmatic and sober validation of DIY research. The traditional MR companies needs to realize that we are growing a market for them - lowering the bar of entry, so MR becomes relevant for a larger swath of companies. I worked in online advertising back when a "self-service ad platform" was a novel idea, but not relevant for the majority of the market. Fast forward a few years, and it managed to introducing a whole new category of companies to online advertising, and is now a de-facto part of life. MR is next in line, I say! :)
Spot on, Andrew. Companies go DIY not because they don't see the value of market research, but because they don't see what value the MR vendor is contributing to the mix. The simple truth is that data collection has been commoditized and disintermediated from its prior spot in the process flow; any successful modern business now has the ability to collect and manage data.
I work on the client side. I used to work on the vendor side. There are many things that I do myself, because (1) it's cheaper, (2) it's faster, and (3) it's of higher quality. I am happy to work with a partner if you can beat my internal capabilities on two of those three points, or if it's a project that simply exceeds our capacity.
Well said Andrew. I concur with your views that 'protectionism' is the bane for any industry. and DIY is here to stay; the best we can do is to share knowledge so those are proceeding with it, have the basic tools to do it right. Open-source, social networks and wikis are the order of the day. And we need more like you to evangelize :)
I fully agree with your comments Andrew.
However much as we like to berate the ‘Do It Yourself’ trend as something that is going to disappear (or we need to protect ourselves from) this is wishful thinking at best (and misguided). For basic research, DIY is here to stay as tools like Survey Monkey become more powerful and user friendly. Further, we need to embrace the fact that DIY can now even empower us researchers to move more efficiently to those skills that make us indispensable for decision support.
To rise to the challenge, our industry, particularly those of us in the full service area, need to do three things well:
- integrate disparate data sources (integration)
- extract insights by understanding the context (sector expertise, contextualization)
- help deploy the insights within client organisations through workshops, etc (activation)
Market Researchers are increasingly using low-cost DIY mobile survey tools to gather consumer insights using iOS (iPhone, iPod and iPad), smartphones and the new Android tablets. Using products like our own, as well as other mobile survey services enables both market research professionals as well as other people that have no market research expertise to collect information from their guests, clients, meeting attendees, and other consumers.
What is interesting of late is the speed from seeing a need to collecting information. A Fortune 100 company identified a need on a Friday and was live on an iPad at a convention on Monday morning! All for $60 plus the cost of an iPad. They were able to have actionable research with less than an hour spent creating the survey.
Geoff Palmer
Co-Founder
SURVEY ON THE SPOT
www.surveyonthespot.com
Twitter: @surveyonthespot
NOTE: Please note that this board is moderated, and comments are published at the discretion of the site owner.
|
Back to 