Dissecting a survey: Analysis of the Coop Food Stores survey run by Tuck

The Coop Food Stores have partnered with the Tuck School of Business to take a survey of the “primary grocery shoppers” in each household.

This link will likely die in a couple of months, but here’s the current link to the survey.

I picked up a hard copy of the survey in the Lebanon store, and quickly found that I was much more interested in dissecting the survey than actually answering all of the questions. The survey comprised several pages of questions, and some of them required quite a lot of concentration to answer correctly. Some questions seemed repetitious, and other questions seemed to have too much ambiguity for me to feel comfortable answering.

So, on with the show!

The survey started out with a short note to Coop customers. The note described the normal stuff — the parameters of the survey, how data will be “completely confidential” with only “aggregate results” used for Coop improvements, and (of course) the “carrot”: how there will be a drawing for ten $100 gift certificates. I must admit that I was particularly excited that West Lebanon Feed & Supply was on the list of businesses, but confused that the Coop didn’t put themselves on the list as well — I mean, what’s more useful than an extra $100 at the Coop, right?

The first couple of questions were easy, but the third one was a little harder. They asked me to estimate grocery spending at all stores over the last 6 months, including “not only food but also general household supplies,[sic] and health and personal care products.” Then they warned “Note: the total must equal 100%,” before giving a list of 8 stores plus “other”.

First, trying to estimate this breakdown was kind of hard. Just starting with total stores, I estimated I shopped at the Coop, Shaw’s, Price Chopper, BJ’s, Wall-Mart, P&C, as well as two in the “Other” category (Stern’s Produce and Yiping’s Asian Market). I didn’t shop at Hannaford’s or Market Basket, but I now realize that I forgot to list CVS and Dan & Whit’s at all.

With a list of stores set, I still had trouble figuring out how much I’d spent as I don’t keep precise receipts very long, and sometimes I pick up food and supplies for other people (as a favor). In the end I guessed I spend 50% of my money at the Coop, 30% at Price Chopper, and the rest sprinkled around at the rest of the stores.

Question 4 made me think for a while. After all the flexibility of Question 3 in assigning percentages to different chains/stores, #4 made me choose the single coop location at which I shop most frequently. I shop at the Hanover and Lebanon stores about equally, but I’m nearly never out on Lyme Road. Stuck with choosing just one, I went with the (larger) Leb store.

Questions 5 and 6 were loooong. Question #5 comprised a list of statements about the Coop (which should be answered about the store you chose in Q #4), and Question #6 was the same list of statements, but applied to BJ’s Wholesale Club.

I know that surveys have to deal with generalizations, but some of the statements were downright hard to rank.

First, S: “Prices at the Coop are fair.”

Fair to whom? To me? To the producers? To the Coop employees? I wasn’t sure how to answer that one and I started to have flashbacks to when I took the PSAT and SAT. I remember my mother telling me to stop over-analyzing the imprecision in the SAT’s questions, and just fill in the regular, dumb answer they wanted. As for the Coop survey, I gave up and chose “Neither Agree nor Disagree”.

S: “When items are on sale at the Coop, the discounts are deep.”

Sometimes yes, sometimes no. There are currently deep discounts for coop members on Muir Glenn tomato products (still going until May 2nd, I believe), and there are always heavy markdowns on 7th generation products once a year, and on citrus during the winter citrus sale. But at other times the discounts are much smaller.

I started to wonder if the statements were less about facts and more about trying to gauge consumer perception of products at the Coop. I mean, that’s half of marketing, right?

S: “I have a lot in common with others who shop at the Coop.”

Well, sure, I relate to a number of people at the Coop because they’re my friends. But I think that they’re looking for more sociological stuff like income level, politics, crunchiness, dress, etc… With so many possible components to this question it could be tough to answer; of course, lots of people at the Coop drive a Subaru, have 2 kids and a dog, are white, and eat some organic food and some inorganic food while dressing in jeans and polar fleece and bringing their reusable bags to the Coop.

So yeah, I chose “Strongly Agree.”

Some statements were easier than others, such as “I believe the Coop cares about the local community.” and “The selection at the Coop includes some very expensive grocery products.”

Well of course! The Coop has a passel of programs and works with local groups to try to accomplish great stuff in the community. Heck, the top hit on Google for Community Partner + Month is for the Coop Food Stores. As for expensive products, well there’s a no-brainer! There are $100+ bottles of wine, cheeses at over $20/lb, and cuts of meat that are at least in the teens if not over $20/lb. Those items are on the high side for nearly all Coop shoppers.

Moving on to Question #6, dealing with BJ’s, I had a whole passel of questions about the statements:

“Prices are fair”
The same issue as w/the statement when applied to the Coop.

“I can get the same items at lower prices in other stores than BJ’s.”
Well actually yes, I can get items for lower prices, but then again I priced out 50+ ingredients for Milque and Cookies at the Coop, Price Chopper, and BJ’s in a spreadsheet. I believe that most customers don’t shop like a price-matching machine! đŸ™‚

“I believe that BJ’s has environmentally friendly policies.”
Well, BJ’s uses fewer bags than the coop — in fact, they make all people take their stuff in reused cardboard boxes, so they probably use a heck of a lot less plastic than the Coop!

“For what BJ’s offers, the prices they charge are reasonable.”
What about the member fee? For large consumers BJ’s can be more worth it than for small consumers.

“The atmosphere at BJ’s is pleasant.”
I was going to mention that the carts and aisles are larger, which leads to much better cart racing, but I’m not sure how the Tuckies analyzing the survey would feel about that.

“I believe that BJ’s strives to build long-term relationships with its customers.”
Well, yes, in a way — I mean, they do make you buy a year-long membership, which encourages people to take advantage of it as much as possible, and to stick with it for an entire year.

Questions #8 and #9 were two of the most interesting. Question #8 was “Please list the names of up to five products that you think are priced about the same or lower at the Coop than at other stores.”

I haven’t checked these things out, but I went with some veggies (lettuce, often), and bulk foods like spices, brown rice, cornmeal, and (king arthur) flour.

Question #9 was “Please list the names of up to five products that you think are more expensive at the coop than at other stores.”

I went with fish, chicken, ketchup, white bread, pasta, and campbell’s soup.

The thing that people need to remember is that price can seem higher at the Coop because the Coop doesn’t offer the lower grades of certain items, like bleached flour, cheaper chicken parts, lower-grade meats, etc. If you do a straight-across product comparison for things that come in bulk, like King Arthur Flour or brown rice, the bulk bins at the Coop usually win out, but if you’re just looking for the cheapest version of “X,” then the Coop will get undercut.

Question #12 concerned itself with “grocery shopping habits and opinions.”

Part 1 says “A business should give back to the local community.” Now I’m no die-hard libertarian, but I think that businesses shouldn’t have to give back to the community beyond the requirements of their taxes and other legally-required fees. Would I prefer it if they do? Oh, yes. Would their investment in the local community encourage me to patronize them? You betcha!

Perhaps a better statement would have been “I appreciate it when businesses give back…” or “I spend more money at a business if I feel they’re helping the community.”

Part 8 is “I always buy the best.”

Does that mean the best value? The highest-quality product? The highest-priced product?

Part 10 is “Environmental and social responsibility programs increase a company’s costs.”

Well there’s a bag of worms. In some cases some programs can lead toward education and advancement of a community, leading to higher-paying jobs and more income which leads to more money spent at local businesses, including the Coop, and thus in the long term being completely economically beneficial. But many environmental or socially responsible programs take some amount of capital to start and maintain, so they definitely can increase costs.

Question #16 is a small quibble, but I’m surprised that Tuck and the Coop (both rather progressive entities) put down a question of “What is your gender?” followed by only “Male” and “Female” boxes. While discussing this question with a friend, they helpfully pointed out that perhaps the Coop and/or Tuck might be afraid of offending or confusing some group of people if they were to put down an “Other” category.

Question #20 was great: “We’re almost done! Do you have any other feedback for the Co-op?”

I must admit that by this point I was pretty tired with the whole process, so I just scrawled in the box: “Nope. This survey is already long enough!”

Overall, I think that the Coop asked a lot of useful questions of customers about their shopping habits. I’m especially interested in how consumers perceive the Coop as compared to how they perceive BJ’s. I’m also a little curious as to why BJ’s was selected for this comparison instead of a supermarket like Shaw’s or Price Chopper. Do you think it might be because both BJ’s and the Coop have a membership model?

And how about Questions #8 and #9? I’ll have to do some comparisons myself to see if I correctly predicted which products were cheaper/same-price at the Coop and which were more expensive, but I’m going to guess that the average Coop customer is going to have some trouble identifying 5 items in each category off the top of their head.

As always, I wish the Tuck School the best of luck in helping the Coop improve their food stores. As fun as it is to be critical of this work, I reckon that it must be a tricky thing to try to come up with surveys that capture significant amounts of useful information without being so long and dense that no one bothers to complete them.

Update (2010-04-25): I just was poking at the online version of the survey and it used Price Chopper as the 2nd store in the comparison. This seems to indicate to me that they are randomly choosing a 2nd store to compare against the Coop.

— Q

Have you taken the Tuck/Coop survey? Did you author the survey or are working on analyzing the results of the survey? Are you a completely unrelated party, perhaps on an entirely different continent, but feel compelled to throw in your $0.02? Marvy! Leave me a comment below and I’ll try to pay attention to it soon!


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s