Mjolnir: talks

There are many ways we can get our statistical analyses "wrong". In my talk, I make the case that the most common form of bad stats are miscommunicated stats. I explain why as HCI researchers we have been faring terribly according to this criteria— mostly due to our blind endorsement of the concept of statistical significance. This idea promotes a form of dichotomous thinking that gives a misleading view of the uncertainty in our data and encourages questionable practices such as selective data analysis and various other forms of convolutions to reach the sacred .05 level. While researchers’ reliance on mechanical statistical testing rituals is both deeply entrenched and severely criticized in a range of disciplines—and has been so for more than 50 years—it is particularly striking that it has been so easily endorsed by our community. I explain how we can improve scientific communication by banning p-values, by reporting empirical data using clear figures with effect sizes and confidence intervals, and by learning to provide nuanced interpretations of our results. We can also dramatically raise our scientific standards by pre-specifying our analyses, fully disclosing our results, and sharing extensive replication material online. These are small but important reforms that are more likely to improve science than methodological nitpicking on statistical testing procedures.

Pierre Dragicevic defended his PhD thesis in computer science in Nantes in 2004, after which he worked as a post-doc at the Université Toulouse III, in the IntuiLab company, and at the University of Toronto. In 2007, he joined the Aviz team at Inria in France as a permanent research scientist, where he has been working on HCI and information visualization. He grew unsatisfied with orthodox statistical methods and recently got interested in the literature on statistical reform. In 2013, he organized a local workshop with statistics reformer and statistical cognition researcher Geoff Cumming. In 2014, he co-presented with Fanny Chevalier and Stéphane Huot an alt.chi paper denouncing the current overreliance on p-values. In early 2014, he decided to banish p-values from all his publications and use only estimation—so far with success. The few colleagues he managed to convince so far report an average increase in life satisfaction of 2.1 on a 10-point scale, 95% CI [1.7, 3.4].