Question Checklist

Use the following checklist to assess how well your survey will address your research question.

Checklist:

• Are all of the concepts related to your research question(s) covered with specific items in the survey?
  
• Are the concepts one-dimensional, bi-dimensional, or multi-dimensional?
  
• Ex: one-dimensional – How many hours do you spend studying each week? (“hours” is a one-dimensional variable)
• Ex: two-dimensional - How many hours do you need to spend studying each week to get a good grade? (“hours” and the meaning of a “good” grade are two dimensions)
• Ex: multi-dimensional – How challenging is this course compared to other classes in your major? (“challenging” is multi-dimensional as it can be the amount of work, level of intellectual demand, need to adopt new perspectives, etc.)
  
• Are there items addressing all of the key dimensions of your concepts?
  
• A multi-dimensional concept requires a series of items addressing each dimension that can be combined into a comprehensive measure of the concept. In the example above, just asking if the course is challenging will produce data that can not be interpreted effectively since you will not be able to tell what the respondents are referring to in their answers, and it will mask significant differences between courses.
  
• Are the items organized according to the main concepts?
  
• It is generally a good idea to organize the survey in a way that will seem logical to the respondent. Introducing each section by identifying the concept being measured (e.g., The following items assess how challenging this course is compared to other courses in the major) helps the respondent answer the questions as they were intended. This organization also helps in the data analysis.
• There will be times when concepts overlap, or there are too few items to create a separate section of the survey, but try to arrange the survey so that there is a logical collection of items in each section.
  
• Are there any extra questions that are not required to answer your research question(s)?
  
• This is not the time for a fishing expedition, respect your participants’ time and only ask essential questions.
  

• While questions should be ruthlessly edited to ensure that only those questions directly relevant to the research question are included, there is one exception to this rule. An important way to help determine if your sample is indeed representative of the population is to measure the characteristics that are already know about the population. For example, you would know in advance the gender and class level distributions for Humboldt State students if that is your population of interest. By asking your sample respondents to indicate their gender and class level you can compare your sample with the known characteristics of the population. For instance if half the population is female, and you find that only 10% of the surveys are returned by females, your sample is clearly not representative on that dimension. Demographics can provide valuable insights about how representative the sample you receive is.