Matthew Kay, Steve Haroz, Shion Guha, Pierre Dragicevic, and Chat Wacharamanotham
This workshop produced the intial draft of the Transparent Statistics Guidelines for HCI.
The original CFP for the workshop is below, and position papers from workshop participants are available here.
Call for Participation – Deadline Feb 15
Workshop at CHI 2017, Saturday, May 6, 2017
HCI is large and multidisciplinary, drawing on a variety of statistical practices. However, many of these existing practices have drawn increasing criticism within both HCI and related fields, including but not limited to: over-reliance on particular statistical methods, a lack of transparent reporting, a lack of replication and meta-analysis, few studies published with data or study materials, and inadequate education in statistics. These issues have even reached the popular press in coverage of the replication crisis in social science.
We are running a working workshop to develop concrete guidelines for improving statistical practice in HCI. Participants will work in groups to flesh out guidelines for helping reviewers fairly assess statistical reports in CHI papers, concrete suggestions for changes to review processes, resources for authors, and other relevant guidelines or proposals the group wishes to advance. We will seed writing with draft guidelines developed in the wake of last years’ SIG on Transparent Statistics in HCI. However, we will not be constrained to the outline in these draft documents if discussions take us elsewhere.
These documents will be shared publicly once finalized and we will engage the CHI Executive Committee and/or Program Chairs to discuss the feasibility of suggested changes to the review process, and the possibility of incorporating some of the recommendations into official documents like the SIGCHI reviewer guidelines. We will also continue hosting and drafting future revisions of the documents on a collaborative editing platform.
We are looking for a diverse set of perspectives on quantitative methods in the HCI community to develop these documents (while qualitative methods are important to HCI, our primary focus in this workshop is on improving the use and communication of quantitative work). If you are interested in improving the state of statistical practice in HCI—whether or not you attended last year’s SIG—submit a position statement (at most 2 pages in CHI Extended Abstracts format) containing:
- A short Bio (If there are multiple authors, only include a bio for the one author who wishes to attend).
- A statement of special areas of interest (in HCI, methods, statistics and/or statistical reporting).
- A position statement on improving statistical communication or practice or a comment on our statement on transparent statistics from CHI 2016.
- An indication of which artifact(s) you are interested in contributing to from our draft documents, or a suggestion for another artifact you believe should be included at the workshop.
Submit to chi2017workshop@transparentstatistics.org. The final deadline for submission is Feb 15, 2017.
As much as we would like as many CHI researchers as possible to get involved in this initiative, we may not be able to accept all applicants due to limits in the number of attendees at CHI workshops. If this happens, we will select applications based on expertise, diversity of perspectives in quantitative methods, diversity of HCI domains, and diversity of interest in specific artifacts (to ensure we have people interested in working on each artifact). Our goal is that the participants reflect the diversity in quantitative methods, statistics, and domains across HCI.
The purpose of the position statement is to help us determine specific participants, so we encourage one author per position statement. Position statements with multiple authors should clearly indicate which author will attend. If your position statement is accepted, the attending author must register for both the workshop and at least one day of the conference.