Checklist of experiment apparatus

  1. Recruit participants
    • Recruit people
    • Make sure they are either paid or are provided with some gift cards (it's important to provide enough motivation to conduct the experiment)
    • Make sure you do not recruit people that have some potential biases such as your close friend ( https://www.nngroup.com/articles/employees-user-test/)
  2. Informed consent
    • Before doing any experiment, it is ethical and it is the first thing in an experiment to let participants know about the experiment (about the possible injury or participant disclosure, etc), and give consent about participating.  Participants should allow knowing their rights, their privacy, and the amount of time they need to spend in the experiment.  If they do not want to participate anymore, they should allow to freely withdraw.   Of course, you should also avoid priming, i.e., asking some questions that may affect their performance or results.  Priming is often a confounding variable for a poorly designed experiment, so please read more about priming.  See example informed consent doc here.
  3. Inform participants before the experiment - for physiological experiments, please inform the participants prior to the experiment, what they should do, e.g., do not drink coffee, do not shower one hour before the experiment, etc.
  4. Demographic sheets
    • In the Participants section, you need to write who do you recruit, their age, their gender, any characteristics that may affect the study results (e.g., experience) and the criteria of inclusion and exclusion.  Usually, you provide this questionnaire to the participant as the second thing, after the informed consent.  See here for example.  I recommend using Google Forms for collecting such data (using iPad or desktop), which is an excellent e-tool to automatically put your data in digital form.  Also, it is environmental-friendly.
  5. Your software or tools - make sure your software logs everything, key point is that you should be able to reconstruct the whole experiment and understand what exactly happens during the experiment.  For example, time log or any measurement tool should be in milliseconds, not seconds.
  6. A checklist of your procedure - make sure you follow the procedure.  This is to avoid human errors which are often the case when there are many tiny steps.  For example, forgetting to instruct your participants not to move their head could completely destroy your whole EEG data.  Or forgetting to measure the baseline physiological data make your data uncomparable.
  7. Webcam, video camera, screen capture.  Very important to capture temporal behaviors of users.
  8. All questionnaires - use only standard questionnaires.  Please do not come up with your own questionnaire.  All standard questionnaires have been scientifically proven while your own is likely to be faulty.
  9. Make sure the setup is clean and quiet, and same across all conditions.  Don't overlook this.