Registered: 1182202224 Posts: 97
Reply with quote #1
In his October/November/December 2011 article, titled "
Benefitting InfoVis with Visual Difficulties? Provocation Without a Cause
," Stephen reviews a research paper from the recent VisWeek 2011 conference which asserts that "visual difficulties" such as chartjunk can actually improve the effectiveness of graphs. However, as Stephen explains, this claim doesn't hold up to scrutiny, as the evidence presented represents leaps of logic based on misappropriated research that was unrelated to data visualization.
What are your thoughts on this research and Stephen's rebuttal? We invite you to post your comments here.
Registered: 1157744699 Posts: 3
Reply with quote #2
Stephen is spot on, in my opinion. I couldn't help but think that the researchers cited perhaps were confusing "visual
abstraction" with "visual distraction/difficulties"? I have seen very complex information presented in an abstract way that both simplified AND amplified the content, while causing one to think more deeply about it. I'm not sure I've EVER appreciated information with distracting elements to the same degree as information simply and elegantly portrayed.
Registered: 1320868097 Posts: 1
Reply with quote #3
I also agree with the points in Stephen's article. The theory that "adding chartjunk will increase the reader's comprehension and recall" is as ill-conceived as the adage that "all publicity is good publicity".
Registered: 1287749814 Posts: 4
Reply with quote #4
Yes, agree with Stephen. What a shame that a paper promoting the use of visual distractions can get such attention. Where are the checks and balances in the review and approval process? I guess if it sounds credible, it must be credible. No need to dive into the details - it's a different point of view - it will attract an audience - we love it. Thankfully, Stephen is keeping them honest.
Registered: 1149528484 Posts: 16
Reply with quote #5
"little research synthesizes the empirical evidence so as to derive the underlying forces that allow some effective graphs to contradict cognitive efficiency assumptions."
Say what? I'm an educated person. I've been reporting for some 15+ years in a variety of tools such as Excel (VBA, too), Cognos, Tableau, Business Objects, and BI Publisher. So that quote, along with some of the other quotes, just sets off my "Full of hot air" meter. It doesn't impress me, it annoys me. It tells me the authors are acting like they know what their talking about when they really, barely, understand what they're talking about. I know that if I produced hard-to-read reports I would be out of a job, no matter how much of their gibberish I quoted as "proof". I've worked with some savvy managers and executive staff and such garbage wouldn't fly. I'm in agreement with Stephen that such nonsense shouldn't be credible and should be used only as an example of what a poor research paper looks like. Additionally, it has me questioning the InfoVis judge's credibility.
Registered: 1320901864 Posts: 3
Reply with quote #6
Thanks Steve for keeping academia honest! Unfortunately the academic community is far from being a good gate keeper of sound scientific research. Dodgy research that has become famous or has adversely affected human lives is not rare nor new. Only a few hit the news after actually getting kicked out (recently:
Duke University cancer researcher who is now sued, Tilburg University psychologist who faked data).
Apropos adding distractions, I am going to smear mud on my windshield to better focus my attention on driving (and I am sure the drive will be memorable).
The viz interface is only a
tool used for the purpose of exploring the data to extract knowledge. Tools should be as invisible and dummy-proof as possible, to avoid distraction from what they are being used for and from misuse. I think that these researchers forgot that they are tool designers -- that also happens to statistics researchers, who get so caught-up with statistical methodology development that they forget that statistics is only a tool and will be used only if it is practical. __________________ Galit Shmueli
SRITNE Prof of Data Analytics and Associate Prof of Statistics & Information Systems
Indian School of Business
Registered: 1320914864 Posts: 2
Reply with quote #7
Is the debate about the Effectiveness (did the viewer get the message), or the Efficiency (can the viewer get the message with minimal effort) of a visualization ?
ChartJunk is detrimental for efficiency, but it is not by definition counterproductive from an effectiveness point of view : if people spend the time to study a graph, they will likely remember more of it than if they quickly glanced over it (same goes for newspaper articles - nobody says you can only spend 3 seconds on a page). Of course, we would like them to remember the substance, not the decoration. In analogy: business people are taught 'effective writing', 'effective presentation', not 'efficient ...' There's always an element of story telling, which is 'a waste of time (or ink, or bandwidth)' for the purists, but helps to get the audience engaged. The same applies to infoviz - but often the creators are carried away by the features of their toolbox and overdo the 'story'. Papers or serious articles have an abstract that helps the reader to assess whether reading the full text is likely to be time well spent. Is there an equivalent for info-viz ? Is a 'loaded' info-viz a (dubious) substitute for an abstract ?
Registered: 1135986598 Posts: 803
Reply with quote #8
Wvdv, In the case of this research paper, the benefits of chartjunk that the authors claim are not clearly articulated in relation to information visualization. It is true that chartjunk is “not by definition counterproductive from an effectiveness point of view” unless you define efficiency as an attribute of effectiveness, which I do. What I agree with, however, is that chartjunk does not necessarily impair comprehension. Whether it does or not depends on the nature of the information and its story. As I wrote in my review of Bateman et al’s study, it is true that comprehension of an extremely simple message that can be stated in a single sentence will not necessarily be harmed by chartjunk if the nature of the embellish clearly supports that message. It is even true that people will often recall the embellished chart better weeks later, but this is rarely useful.
What matters, of course, is that people become engaged with a visualization in a way that leads to the information and understanding that they need. Unless a chart is extremely simple, this requires time, thought, and even commitment on the part of the reader. The person who cares about the information needs a visualization that presents what they need as clearly as possible, without distraction, often accompanied by means of interacting with the information that allows them to ask and answer questions.
When a visualization is created to tell a specific story that someone has already been found in the data, the goal is not to tell it in the fewest possible words or graphical strokes. Whatever is needed to engage the audience’s mind in thinking about the information to the depth that is required is useful—no more, no less. If a photo of the emerging crack in the ice in Antarctica helps to connect with the audience’s values and draw them into a visualization of data about global warming, then it is useful. In this case, as long as it doesn’t get in the way of the data (e.g., as it would if you placed it in the plot background of a graph), it supports meaningful engagement.
I am not a purist in the sense that you’ve described. For me, the bottom line when evaluating an information visualization is its ability to promote effective thinking and/or communication. I am like the writer who looks, not for the fewest words, but the words that express what I’m trying to say effectively. I judge the merits of visualization by the quality of the outcomes, not merely by measures of efficiency. __________________ Stephen Few
Registered: 1325855753 Posts: 7
Reply with quote #9
I recently read a book on design that talked about utility and significance. Utility is (in the broadest sense) how well the thingy works at what it's meant to do (if it's meant to be efficient, is it? if it's meant to be readable, how readable (effective) is it?, etc.). Significance seems much woolier but makes intuitive sense -- it's about how it makes you feel. At risk of sounding like a lunatic, I have a crook knife that I use for carving. It has very high utility (allows me to scoop out a bowl in a way that a straight knife can't), but it also has high signficance -- it's well balanced, it's sharpness makes me grin, I like the wood of the handle.
Such is with information design. This is an area where the graphic designers (who often do not have analytical backgrounds) lack the analytical skills to give a display utility, but they ooze capability at presenting images that resonate. Geeks like me lack their intuition about significance, but are better placed to formulate a display that encourages analytical and critical exploration. More socratic, I guess. I suspect that in many ways this comes down to how people are comfortable thinking. In the past I've used Minard's map of Napoleon's invasion to explain information design -- it's famous and, I think, engrossing (defies the pen of the historian with its brutal eloquence, and all that). Some people look at it and simply can't be arsed; a pictogram with coffins in a pretty field would engage them sufficiently. Temperature, location, attrition, etc., all too much information. Dead people, cold, frost-bitten genitals, etc. much easier to 'feel about' than 'think about'. Unfortunately, the graphic designers are winning... Analytical information design presents too weak a case to those who prefer to feel from the hip. Rant over.
Registered: 1382005469 Posts: 1
Reply with quote #10
I recently reread Stephen's critique of the award given to Bateman et al. for their work on the benefits of information junk, and as a CEO and executive with over 30 years of successful practice at analyzing critical business data, I couldn't agree more with Stephen's views.
In short, chart junk is certainly a powerful propaganda tool. I doubt there was a need for a CHI paper to demonstrate that. But when it comes to helping us understand information, chart junk is just that: junk. The charts used by Bateman et al. were so simplistic that a simple table with 3 to 7 data points would have contained the same information. Of course, experts in propaganda (e.g. Staline or Goebbels) know very well that striking imagery and clichés help people remember such simple data. There's nothing new here. The fact that Bateman et al. chose "memorability" as their efficiency yardstick is symptomatic of an old world culture that favored memory over intelligence. Beautiful minds such as Feynman's have fought such idiocy all of their lives and earned Nobel prizes for their contribution. What makes humankind progress is "understanding", not "memory", especially in our information age where terabytes of data and search engines can compete with living human encyclopedias (a friend of mine diagnosed his cancer himself via google, after 10 consultations with top experts failed; the medical prognosis then confirmed his own findings and he underwent successful surgery…). By Bateman et al.'s standards, Al Gore's "Inconvenient truth" would be considered efficient. However his video was full of misrepresentations (charts had no scale nor legend) and drove an association of British science teachers to prosecute him, resulting in a British court ruling that acknowledged that he had significantly misrepresented scientific evidence (don't get me wrong here, my point has nothing to do with any beliefs about global warming! I'm just saying that lying about data is harmful). That people can remember Gore's movie factoids is no proof that his methods have any legitimacy. __________________ -jr
Registered: 1135986598 Posts: 803
Reply with quote #11
The timing of your response is perfect. I just became aware of a new research paper that was presented at the VisWeek conference this week titled "What Makes a Visualization Memorable?," which is the newest contribution to the chart junk debate. I haven't finished reading it yet, but I've already found several flaws in the first page or so. Once I've finished reading it and thinking about it, I'll write about it in my blog. __________________ Stephen Few