The web site for the book on how to run studies:
Ritter, F. E., Kim, J. W., Morgan, J. H., & Carlson, R. A. (2013). How to run studies: A practical guide to research with human participants. Thousand Oaks, CA: Sage. Released November 2012.
[Sage flyer for it, including link to ask for review copy] [List of libraries with PDF]
[pointer to the free, online version if your library subscribes to Sage Online
(link doesn't always work, but, if your library subscribes to Sage Online, you can find it with the search field)]
[Press release from PSU as a summary]
This web site provides additional resources to help support the book. Even more materials for teachers are available from the authors. The book has been used to teach at Georgia Tech, Florida Institute of Technology, Middlesex (UK), Carlton (Canada), U. of Illinois, CMU, U. of Connecticut, U. of Texas/Houston, and Penn State.
From the preface: There are few practical guides on how to prepare and run experiments with human participants in a laboratory setting. In our experience, we have found that students are taught how to design experiments, and how to analyze data in courses such as Design of Experiments and Statistics. On the other hand, the dearth of materials available to students preparing and running experiments has often led to a gap between theory and practice in this area, which is particularly acute outside of psychology departments. Consequently, labs frequently must impart these practical skills to students informally.
This guide presents advice that can help young experimenters and research assistants to run experiments more effectively and more comfortably with human participants. In this book, our purpose is to provide hands-on knowledge about and actual procedures for experiments. We hope this book will help students of psychology, engineering, and the sciences to run studies with human participants in a laboratory setting. This will particularly help students (or instructors and researchers) who are not in large departments, or are running participants in departments that do not have a large or long history of experimental studies of human behavior. This book is also intended to help people who are starting to run user and usability studies in industry.
Running studies online
We have written a follow up paper about how to run studies online based on running several studies online during the pandemic.
Ritter, F. E., & Ricupero, S. (2023). Running behavioral studies with human participants: Online. Sage Research Methods: Doing Research Online. [case study]
There are also a paper we have since found about how to run studies online.
Grootswagers, T. (2020). A primer on running human behavioural experiments online. Behavior Research Methods, 52, 2283–2286.
Chapter 1, Introduction
Nothing yet
Chapter 2, Preparation
While the book does not cover study design directly, Luke Zhang rightfully recommends this article that notes what's going on with experimental design with respect to running studies.
Joseph E. McGrath, 1981. Dilemmatics: The study of research choices and dilemmas
American Behavioral Scientist. 25: 179 DOI: 10.1177/000276428102500205
The online version of this article can be found at: http://abs.sagepub.com/content/25/2/179
A useful book on study design:
Mitchell, M. L., & Jolley, J. M. (2012). Research design explained (8 edition ed.). Belmont, CA: Wadsworth Publishing. Appendix from Mitchell & Jolley
Home page for book
Student website
Chapter 3, Potential ethical problems
Nothing yet
Chapter 4, Risks to validity to avoid
Example study problems to study. This is a handout for class discussion or homework to note some potential risks to validity for several studies.
Chapter 5, Running a research study
When you run run a study on a computer, our work on RUI and other work on timing does not say as clearly as it should, you should (a) have as few processes as possible running on the the computer that is taking the timing. This computer should also be off the net, so that it does not get timing updates from the net. [Thanks to Ryan Kaulakis to pointing this out.]
The book should note more clearly that it is very useful for the PI to act as a pilot subject, Ray Perez reminds us.
There was an online workshop in May 2020 about running studies online, that may be helpful: Video-chat studies for developmental research: Options and best practices.
This paper notes some implications for running such studies with human-robot interaction.
Lazar, Feng, and Hochheiser (2017), Chapter 15, Working with human subjects, also has some useful informationon on recruiting, running, IRB, and sanitizing data.
Chapter 6, Concluding a session and study
In writing up a study, you will often have to do it with APA style. This is easy enough to learn, but it stlil has to be learned. Here are some links on writing APA style
Psychology.org, based at Jacksonville State
A short course on APA style by the APA
Peter Hancock has recommended, Writing Human Factors Research Papers
A Guidebook, by Don Harris, Ashgate. at $114 to $124 it seems a bit dear. Its ToC is online and looks spot on.
Afterword
Nothing yet
References
Nothing yet
Appendixes
Nothing yet
Other Material
Slides to teach this material previously used at conferences:
Cogsci 2020 pptx and pdf;
Cogsci 2014 ppt and pdf;
Cogsci 2012 ppt and pdf.
Tutorial handout including more material.
Further blurbs and comments about the book
"I need a copy of this for students" Dr. Leslie Blaha, Battlespace Visualization Lab, AFRL.
"I love it." Dr. Jen Bittner, Battlespace Visualization Lab, AFRL.
It is a really nice overview for a quick start and would have helped me, if I had read it before I did my experiments.
I showed it to two of my colleagues, who recently joined our group. They will find it very helpful, I think.
-- Bertram Wortelen, FuE-Bereich Verkehr | R&D Division Transportation
Errata
Jennifer Bittner provided comments on the book and should be included in the acknowledgements.
p. 122, 'back up all your data' should be 'backup all your data'
Last updated, Frank Ritter, 8 Apr 2023.