Many leading Market Research organizations have built institutions around “Norms” … for Concept Testing, Ad Testing, Communications Tracking, Packaging Research etc.

After a fascinating call today with a client, I was intrigued by their question “Do Norms Matter?”

As we talked about the issue, I was struck by how ingrained the “norms” were as milestones that were either missed, on-norm, or achieved. For this client, norms have become a crutch – where corporate intuition, experience and judgment are secondary to the “normative” threshold.

As an example of their concern, “norms” are rarely challenged, and more likely to be taken as gospel.

Question #1: do the norms always make sense? eg: for a Cheese client, are all the norms from a Category Leader, or is there dispersion amongst the cases – to include other brands in the category.  What if you’re a smaller share marketer, and all/most of the norms are from the category leader (who is more likely to invest in repeat-type methodologies that sustain normative databases than the smaller brands anyways). Are norms as relevant/applicable if you’re being compared to the category leader?

Question #2: how are the norms actioned? Meaning, is the Norm Bar set at 50% (ie an average of the cases), or higher (eg. Top 25% of Cases).  Hopefully, your brand is being compared to some higher standard as a success threshold.

Question #3: have you asked for recent validation? Many of the “norms” quoted today source from the 80′s and 90′s.  In some cases (eg: advertising), norms might not be as relevant today as they were 20 years ago.  It’s a fair question to ask how some norms have changed over time, and to be certain that the standards you are trying to reach, are based on current market conditions/relevancy.

In your experience, when do “norms” matter – and where have they become less relevant?

PS – the guy in the pic is “Norm” (AKA George Wendt) from Cheers.

Last night (with 2 of my 3 kids just about ready for bedtime), I somewhat reluctantly took a telephone survey.  Given that it was a seemingly nice & polite interviewer, on a topic of interest (Travel), by a reputable (Top 10) research firm, and a promised short (5-6 mins), I agreed (Note: I was not asked any disqualification questions).

• After about 8 mins, I asked: “approximately how many questions do you have left?”
• Interviewer: “we’re almost done, and I’ve been trying to get someone to qualify for 3 hours, so this is a really important interview for me to complete” (Red flag alert).
• After about 10 mins, I ask again: “are we almost done, this is longer than you had said it would be”.
• After 12 mins, I politely say “if we are not on our last 2-3 question, I am sorry, but must now end our call”.

I remember being trained as a junior researcher in the late 80′s, and spending 3 nights as a telephone interviewer to round out my training.  It was hard back then, dealing with the challenge of making a 15-20 minute survey about cookies or toilet paper hold interruption value during the golden hours of 5-8pm … and I recall developing a real appreciation for the front line of survey research.

Fast forward to 2010, and I can appreciate that the job of a Telephone interviewer is no less easy, and I’m sure even harder.  Consumers (Note: not “respondents”) are increasingly unwilling to answer the phone, and when they do, are certainly less and less interested to give up their time.

Conclusion #1: All forms of Survey Research are based upon terms & trust.  I was let down, because someone thought that I could be snagged with a promise, and out of guilt would finish the interview.  That was bad for all involved, and made me wonder how many people have had similar experiences, which impact all of us doing survey research.

Conclusion #2: Perhaps nostalgically speaking, I still believe there is a future for telephone surveys, but one where researchers cannot rely on the same tools, approaches and interviewer scripts that existed in the 1970′s and 80′s.  Doing so, will just hasten the demise of that methodological option in the survey toolkit.