UI之旅－生活总是赏赐不浮躁的人们

These checkpoints are based on the work of Julian Meltzoff (1997).

The checkpoints

• Is the research question clearly stated?
• Does the introduction, statement of the problem, and overview of any literature or previous reports adequately set the background for the reader, and is this material consistent with the research question?
• Is it clear why the study was conducted?
• Given the research question and the background material are the research hypotheses and objectives appropriate and clearly stated?
• Are key terms well defined?
• Is the independent variable appropriate given the question of the study? Are the levels of the independent variable appropriate?
• Is the dependent variable appropriate for the study?
• Are the criterion measures of the dependent variable appropriate, valid, and reliable?
• Are the scoring, rating, and judging procedures valid and reliable?
• Is the measuring apparatus (if any) accurate and reliable?
• Are the controls appropriate? Can the results be affected by variables that have not been controlled? Are the controls or control groups (if used) properly selected?
• Is the research design suitable to meet the objectives of the study? Is the research design appropriate to test the hypotheses and answer the research question?
• Are the methods and procedures clearly described in sufficient detail to be understood and replicated?
• Is the presentation sequence of test stimuli (including any randomisations or counter-balancing) appropriate?
• Are the test participants properly oriented and motivated? What is their understanding of the task? Are the instructions sufficiently clear and precise?
• Are there any signs of experiment bias in the design, data collection, assessment, analysis, or reporting?
• Are the participants properly selected? Is the sample representative and unbiased? Do the procedures adhere to the guidelines for the protection and well being of participants?
• Is the sample size appropriate? Are the appropriate procedures used to assign participants to groups, treatments, or conditions? Are suitable techniques used to establish group equivalence, such as matching, equating, or randomising?
• Does participant attrition occur and if so does it bias the sample?
• Are bad data properly identified and set aside (not included in the final test data set); and are instances of bad data reported and explained as such?
• Have the data been appropriately analysed, sorted, categorised, grouped, prioritised etc.?
• Are descriptive statistics used? Are these accurate?
• Are the inferential statistical tests appropriate? Are the assumptions for their use met? Are there any errors in the calculation or presentation of statistical results?
• Are all graphs correctly labelled (both the X and Y axes)? Are data elements on graphs properly coded, and identified?
• Are tables and figures clearly labelled and accurately presented and referenced in the text? Are results and findings correctly interpreted, properly reported, given meaning and placed in context?
• Are recommendations unambiguous? Do recommendations follow clear usability, human factors, or ergonomics guidelines?
• Are recommendations supported by references to prior literature or to industry standards?
• Is the discussion section of the report reasonable in view of the data?
• Are the conclusions valid and justified by the data?
• Are the generalizations valid?
• Do references (if used) match the citations in the text?
• Have ethical standards been adhered to in all phases of the research?
• What can be done to improve or re-design the study?

Reference

Meltzoff, J. (1997) Critical Thinking About Research: Psychology and Related Fields. American Psychological Association.

Download pdf version

Use this version as a working checklist to evaluate a usability study.

Research checklist (pdf format, 60KB).

via userfocus

 一则读书笔记。
对你的用户做完了测试后，要确认一下自己是否已经弄清楚的这些问题。一个个对照，问一下自己，看看自己的用户测试效果如何。

在uu网看到一篇文章，关于用户满意度调查在一个渐变网站评估中的迭代使用 。

改版的案例是：IBM的易用（EOU）网站（http://www.ibm.com/easy ）。该网站的目的是宣传IBM在产品易用性方面的工作。从1997年由IBM主要为人机交互领域的实践人员提供信息，后来战略转变，被设计成为更广泛的读者群服务，包括界面设计师、信息开发人员、人因从业者、技术记者、程序员、IT经理以及这些专业的教师和学生。这个网站还随着IBM网站设计标准的演变而采用相应的新界面风格。

本文对EOU网站不同设计的三次系列用户满意度调查的方法和用法进行了讨论。挺不错的文章。