Survey Construction Assignment

Overview: For this assignment, groups of students will design a 10-15 item survey measuring a specific psychological construct, administer the survey to 15-20 volunteers, use SPSS to find basic descriptive statistics and conduct reliability and validity analyses for this survey and discuss your results in a brief paper.

Step 1: Understanding your construct

Your group will be assigned one of the following constructs to use as the basis for your survey:

Forgiveness

Your task is to measure forgiveness as a trait. In other words, you want to get a measure of a person’s tendency to forgive across a range of situations and not just for a specific incident. There is some disagreement among researchers about the definition of forgiveness. For our purpose, let’s start with some basic distinctions.

• Forgiveness is generally considered an adaptive trait.
• Forgiveness is not the same as condoning or excusing. Forgiveness is different from reconciliation in that forgiveness is an intrapersonal process, whereas reconciliation is a interpersonal process. 
• Forgiveness researchers generally frame forgiveness as a potential response to transgressions. Transgressions are events that people perceive as violating their expectations and assumptions about how things “ought to be.” 

One group of researchers defined forgiveness as “the framing of a perceived transgression such that one’s responses to the transgressor, transgression, and [precursors] of the transgression are transformed from negative to neutral or positive. The source of the transgression, and therefore the object of forgiveness may be oneself, another person or persons, or a situation that one views as being beyond anyone’s control (e.g., illness, fate, or a natural disaster)”

Fear of Negative Evaluation

Your task is to measure “fear of negative evaluation” as a trait. In other words, you want to get a measure of a person’s tendency to fear negative evaluations across a range of situations and not just in a specific situation. Cognitive-behavioral models of social anxiety (or social phobia) suggest that social anxiety is, in part, a response to perceived negative evaluation by others (not fear of social situations per se as commonly believed).  Research on social anxiety suggests that people with a fear of negative evaluation report:

• feelings of apprehension regarding others’ evaluations
• distress over negative evaluations
• the expectation that others will evaluate them negatively

Research has shown that individuals who are high in fear of negative evaluations tend to avoid evaluative situations, show an attention-bias toward potential evaluative threats, and rate themselves more negatively in evaluative situations than do observers.

Road Rage

Your task is to measure “Road Rage” as a trait. In other words, you want to get a measure of a person’s propensity to get angry while driving not just their anger in a specific situation. There is some disagreement among researchers about the definition of “road rage” or “driving anger”. For our purpose, let’s start with some basic distinctions.

• Driving anger is generally considered a maladaptive trait.
• Anger is not the same as aggression. A driver may get angry but not behave aggressively.
• Driving anger is different from “unsafe or reckless driving”. Although these two constructs may be associated, reckless driving may result from impulsivity or sensation-seeking rather than anger.
• Higher levels of driving anger have been associated with a variety of negative outcomes including: aggressive driving, traffic violations, and impatience with other drivers.

Step 2: Develop a pool of items & Informed Consent statement

After discussing the conceptual definition of your variable and, perhaps, a general measurement strategy, each person in your group should individually brainstorm for items. Then everyone should come together and choose between 10 and 15 items to be used in the final survey. Be sure to consider reverse scored items (if necessary) and the most appropriate response format and anchors (e.g., Likert scale? intensity vs. frequency?).  Using this final set of items, the group should write instructions and construct the formal survey to be used for data collection.  You should include the following consent statement at the top of the page:

Informed Consent

This survey is being conducted for a class project for the Psychology Research Methods course, Spring 2009. Students in the course developed a set of questions to measure the concept of ______ and will be using the data collected to determine the adequacy of the measure. By filling out the survey, you are indicating your knowledge of and consent to the following:

• Your responses are entirely anonymous and your surveys will be destroyed at the conclusion of the class lesson.
• You are free to skip any items you do not wish to answer, and you may stop filling out the survey at anytime.
• Your participation will require less than 5 minutes.

If you agree to these terms, please fill out the survey. Thank you.

Step 3: Collect basic demographic information and create 2 validation questions. Decide on a data collection strategy.

This section should be no more than one-page in length. You should create a brief series of questions to: (a) obtain the basic descriptive information on your participants required by APA (age, race, gender) and any other relevant descriptive information; and (b) get some preliminary evidence for the construct validity of your survey.  You should have at least one question that would provide you with convergent evidence for construct validity and one question that would provide you with discriminant evidence. For instance, if constructing a measure of “leadership”, you might ask if the person currently holds a leadership position in any student organizations (convergent). You would expect your measure to be related to actual leadership positions. On the other hand, a scale measuring the trait of leadership should probably be unrelated to gender (discriminant). If it does relate to gender, then you have possibly created a scale biased by stereotypically masculine (or feminine) leadership styles.

As a group, decide on a strategy for collecting data. Why might it be important for every person in the group to have the same general "script" and "style"? Consider the pragmatic and ethical issues involved with deciding where to get your participants, how to approach them, what to do while they fill out the survey, what to do if they don't want to fill out the survey, and how to collect their completed surveys. Run your plan by me before you leave for the day.

Step 4: Collect & Analyze Pilot Data

Before our next class, each group should administer its survey to at least 15 participants. Bring your raw data (the completed surveys) to class.

• Handwrite a subject number on each survey, and create a subject number variable in SPSS.
• Create all other necessary variables (including appropriate value labels for categorical variables – e.g., gender), and enter your data.
• After entering all the data, run “Frequencies” on every variable.
  • Examine each frequency table to determine if there are any out of range values; if there are, you made a data entry mistake.
    • For example, if your measure has a 1-5 response scale, then any responses other than 1 ,2, 3, 4, or 5 indicate a data entry mistake.
    • For example, if you coded gender as 1=man, 2=woman, and 3=transgendered, then any value other than a 1, 2, or 3 in the frequency table indicates a data entry error.
    • If you have an error, find the subject number and refer back to the original survey to correct the mistake.
• Next, determine if any of your items need to be reverse scored and, if so, do the appropriate recoding.
• What is the internal consistency of your scale?
  • Are there any problematic items?
    • Remember that your decisions about the measure need to based on both statistics and critical thinking.
  • Come get me when you reach this point in the assignment and let’s chat together about your data and your conclusions.
• Create your scale.
• Determine the mean and standard deviation for your scale.
• Use the appropriate statistical analyses to determine the relationship of scores on your measure to each of your validation questions. Try to figure this out on your own, but if you have problems or want to double-check your choices, I'm happy to help.
• Check to see if your measure is related to age and/or gender.  If you have a sufficiently racially diverse sample, you should also check to see if there are any race differences.
• Be sure that every member of the group has a copy of the data file and knows how to do the above analyses.

Step 5:  Write Up Your Results (INDIVIDUAL)

This portion of the assignment is to be completed individually. You should not collaborate with other group members at all on this portion.

Based on the information gathered in steps 1 – 4, your final task is to write 3 or 4 informal paragraphs about your work. Include the following 3 sections (modeled after APA style).

• Method
  • Describe who participated in your study (n, age, race, gender). Describe the three measures (yours and the validation questions) participants completed. (Be sure to report on the original version of your measure. If you made changes based on the reliability statistics, you can discuss that below.)  Describe the questionnaire you created by providing the following information: number of items, response format, an example item. Also provide a copy of the entire measure (either cut and pasted into this document, or uploaded as a separate document at the same time you upload this paper to BB (load/save both documents before submitting the assignment).
• Results
  • Use the following sub-headings:
    • Reliability: report the internal consistency of your scale. If you decided to drop an item from the scale because of poor reliability, first report the alpha for all items, then discuss which item and why you removed it, then report the alpha for the final scale.
    • Validity:  Report the findings for the other 2 variables and comment briefly on whether the results do or do not support the construct validity of your measure.
    • Other Analyses : basic descriptive for your measure (e.g., the overall mean and SD). Report whether there were or were not any significant differences as a function of age, race or gender and provide details (with appropriate statistics) for significant results. If you did not have sufficient variability in  race or gender to make a comparison, report that.
• Future Directions
  • Taking all of your results into consideration, briefly discuss the extent to which your measure has adequate reliability and construct validity. Give some suggestions for what you would do next if you continued to develop this measure. Be specific. Also, briefly discuss the extent to which the representativeness of your sample is or is not a problem. Again, be specific (e.g., what group of people did your sample miss, and why and how might that group of people respond differently to your measure?) Remember, representativeness might not be an issue for some constructs or for some groups. If you can't think of a reason an under-represented group might score differently, then it might not be an issue.

The final paper should be submitted to Blackboard.