Discussion


Note: Images may be inserted into your messages by uploading a file attachment (see "Manage Attachments"). Even though it doesn't appear when previewed, the image will appear at the end of your message once it is posted.
Register Latest Topics
 
 
 


Reply
  Author   Comment  
AndrewM

Registered:
Posts: 2
Reply with quote  #1 
I'm looking for some critiquing on a technique that I've been using to display a particular type of survey question.  I came to it due to a frustration of existing methods used in the market research industry (not least the common technique of calculating means for ordinal data).

A Likert scale is a symmetrical scale to assess a statement.  Usually the scale would be 5 point though 7 would be sensible to use, especially if you're expecting a narrow distribution of results (usually mostly positive or negative).  The scale is usually something like 'strongly disagree, disagree, neutral, agree, strongly agree'.

I've written a long note on my site here.  Summarizing, I took that the viewer wanted to understand 3 elements:

skew - the balance between positive and negative responses
non-neutrality - the proportion of respondents who had either a positive or negative view
Strength of feelings - the split between agree and strongly agree.

Histograms are good at showing the distribution of small numbers of distributions, however I felt their weakness was that you could either arrange so the response-types were aligned or the proportions were aligned, but not both.

Stacked distributions are widely used but these suffer from comparing non-extreme values due to differing bases.  Here is an example:


My 'net stacked distribution' tries to counter this.  Here is the the same data:

As you can see I've rebased the responses to a central, and have excluded the neutral responses.  The neutral can be inferred as inversely proportional to the length of the overall bar.

As noted, I'm looking for a critique.  My nagging worry is that I've missed something & there is a good reason few others are using this approach.
sfew

Moderator
Registered:
Posts: 814
Reply with quote  #2 
AndrewM,

Your "net stacked distribution" certainly makes it easier to see and compare the proportion of combined Strongly Disagree and Disagree responses or the proportion of combined Strongly Agree and Agree responses among the survey questions, assuming that you must keep the questions in their original order. If you want to see which questions received the most favorable responses or which receive the most unfavorable responses, however, sorting them in that manner would make this much easier to see. Some purposes that your "net stacked distribution" doesn't effectively support includes a comparison of "Strongly Disagree" or "Strongly Agree" responses only, or a comparison of "Neither Agree or Disagree" responses only. Other displays would serve these purposes better. This points out the fact that no single display will serve all purposes. To thoroughly explore the results, you need multiple displays.


__________________
Stephen Few
AndrewM

Registered:
Posts: 2
Reply with quote  #3 
Stephen,

Thanks for the feedback.  I agree that sorting would, in many instances, add to the comprehension, either sorting by question or by population group.  One might want to sort by several factors, one of which could include the neutral response.

I had used colour intensity to try to draw attention to the strong agree / disagree responses (the difference between D and E).  Your notes above suggest that this doesn't work for you in the way I intended.  Back to the drawing board on that one I suppose.

There are issues reporting this type of data compared to more objective information, such as revenue or profit. Given the question is to some degree subjective (one person's strongly agree is someone else's agree) we need to show care when making certain comparisons - most notably between geographies as responses can be culturally determined.

Another issue, given that we're reporting a sample of a population, is how to include some form on confidence interval.  Ideas would be welcome.

After posting, I also noticed that it would be better to make the scale symmetrical about the neutral point. Without such the interpretation of the skew becomes harder.

As you note, there is never a single-best way of reporting different data types, and this type of scale is no different.

Again, many thanks
nixnut

Registered:
Posts: 69
Reply with quote  #4 
AndrewM,

Possibly you've already ran across this, but we've discussed this subject earlier in Suggestions for Graphing Survey Data. If you haven't already seen it, maybe some of the comments there are of interest to you.

regards,
nixnut
Previous Topic | Next Topic
Print
Reply

Quick Navigation: