Construct Reliability & Validity Assignment

Part I.

Establishing the construct validity of a scale/test is a process of collecting evidence about what the scale measures. Specifically, construct validity is typically evaluated by looking at the patterns of correlations of the scale in question with a variety of other measures. Correlations with some measures provide convergent evidence (i.e., what it is measuring) while others provide discriminant evidence (i.e., what it is not measuring). The goal of this assignment is to evaluate the construct validity of a self-report survey of “Hypercompetitive Attitudes”

Target Construct

Hypercompetitiveness –  The need to compete and win at any cost as a way of maintaining self-worth.  The construct includes manipulation, aggressiveness, exploitation, and denigration of others across a wide range of situations (Horney, 1937 as cited in Ryckman et al., 1990).

Measure of Target Construct

HAS scale (Hypercompetitive Attitudes Scale).  Ryckman, R. M., Hammer, M., Kaczor, L. M., & Gold, J. A. (1990). Construction of a Hypercompetitive Attitude Scale. Journal of Personality Assessment, 55(3), 630-639.

Convergent Evidence?

If the HAS really measures hypercompetetiveness as defined above, then we would expect it to be correlated with similar or theoretically related constructs.

Discriminant Evidence?

If the HAS really measures hypercompetetiveness as defined above, then we would NOT expect it to be as strongly correlated (if at all) with less similar or theoretically unrelated constructs.

Your Task

Form groups of three and look through these web sites. Find one measure that you believe would provide convergent evidence for the construct validity of the HAS. Find one measure that you believe would provide discriminant evidence for the construct validity of the HAS. Each group will report back to the class. As a class, we will decide which to use based on everyone’s suggestions. Hint – pick fairly short tests because you have to take both of the online tests. Also, there must be scoring instructions for the test.

After we've chosen which measures to use, take each of the two online test. Provide your test scores (anonymously) using the scoring sheets I will give you.

Part II.   SPSS Transformations and Reliability (Internal Consistency)

You each completed a brief survey pertaining to hypercompetitiveness. I have posted the complete data in an SPSS file on Blackboard (HASdata.sav). For this task, you will need to download and then open this data file and complete the tasks and questions below.  Please write down the answers to the numbered questions and hand them in during class.

Remember, the point of running internal consistency analyses is to see if the scales we are interested in are reliable – do the individual items on a given scale appear to be measuring the same thing for our sample? We already know that, conceptually, the items sound like they belong together. This internal consistency statistical analysis is going to provide a numerical answer to the question that we can combine with our conceptual answer.

Since gender is a nominal (a.k.a. categorical) variable, an internal consistency analysis isn't applicable (think about why). Also, an internal consistency analysis isn't possible because gender is a single-item variable.

A.  Reverse-Coding

Consider the items to be included in a single scale and decide if you need to “recode” any variables such that higher numbers indicate the same attitude for every item.

• With a pen and paper, label all the items you want to recode. 
  • It is standard for higher scores on an item to indicate “more” or “better” of the name you are intending to eventually give the scale.
  • So, reverse score items so that higher scores mean more competitive attitudes.
• In SPSS, go to “Transform” and “Recode.”  Choose to recode into “different variables.”
  • Move each variable you want to recode into the main box. 
  • Click on the first item in the recoding list and type in a new name in the box on the right.  I’d recommend that the new name be the same as the old, appended with an “r”
    • (e.g., H1 = H1r). 
  • Click “change.”  Repeat for each item that needs to be recoded.
  • Next, go to “old and new values.”  Type in the lowest possible response in the left box (in this case, “1”) for the old value, and type in the highest possible response (in this case, “5”) in the right box for new value.
  • Repeat so that you recode the entire scale.
  • When you are finished entering old and new values your box should look like this:
    • 1 --> 5
    • 2 --> 4
    • 3 --> 3  (this is necessary because you are creating a new variable. Without this, a 3 would be missing in the new variable)
    • 4 --> 2  (this is necessary because the arrow only runs one way. So, 2 --> 4 above does not recode a 4 into a 2)
    • 5 --> 1
• Click continue and okay to complete the recoding procedure.
• There is no output for this procedure. Rather, look at the end of the data file to see the new variables. If you have made a mistake, you can simply delete the new variable(s) and repeat the recoding procedure.

1.  Write down the item numbers you chose to recode.

B.  Computing internal consistency

To compute internal consistency, look under “Analyze-Scale-Reliability analysis”

• Move all of the variables that you think are part of one scale into the Items box (e.g., all of the hypercompetitiveness items).
  • Be sure to move the recoded, but not the original items, over where relevant.
• Click on the “Statistics” box and select the following: (Descriptives for)  “Scale If Item Deleted”
• Click continue and okay to run the reliability analysis.
• Look first at the bottom of the output for Cronbach’s coefficient alpha (a) to see the overall reliability. Then look at item-total correlations of each item  and alpha if item deleted. Is there a problematic item?
  • When you get to this point, let me know -- I'd like to check your work and understanding as well as address any problems with the data set.

2.  Report the internal consistency for the entire scale (format:  alpha = .xx) and provide a comment about the reliability (adequate, high, low).

C. Scale Creation

Assuming the internal consistency was sufficient, you now need to create the hypercompetitiveness scale (i.e., create a single variable that summarizes all 26 scale items).

• Go to “Transform” and “Compute” to create a new variable from existing data.
  • Enter the new variable name in the box on the left (e.g., HSCALE). Variable names may not contain spaces or characters.
  • Notice there is an equal sign between the variable name box on the left and the equation box on the right.
  • Click on the equation box to the right of the equal sign, and then click on “All” in the “Functions” box. 
    • Find “MEAN” and click on the small up arrow ^ box to move that function into the equation box.
  • The equation box should now say “MEAN(?,?)". You will be replacing the question marks and comma with a list of the hypercompetitiveness items.
    • By double-clicking on the variable name (e.g, H1) in the variable list at left, enter each of the hypercompetitiveness scale items into the function for MEAN. Separate the items from one another with commas. Delete the question marks and extra comma.
    • Be sure to use the recoded items where appropriate.
• When finished, you should have created the following equation:

HSCALE = MEAN(H1,H2,H3r,H4,H5r,H6r,H7,H8,H9,H10r,H11,H12,H13r,H14,H15r,H16r,H17,H18r,H19r,H20r,H21,H22,H23,H24r,H25r,H26r)

• Click okay to create the variable. If you get an error message check to see that you have enclosed all the items within the parentheses, all the item names are accurate, and the items are separated from one another with a comma.
• There is no output for this analysis. Check the end of the data file to see your newly created variable. Note that, because the original response scale could range from1 to 5, the new HSCALE variable should also have scores falling between 1 and 5 (no values higher or lower).

3.  Using your knowledge of SPSS from Psych Stats, figure out how to get SPSS to provide the mean and standard deviation for your new HSCALE variable.  Report those statistics.

Compute bivariate correlations between HSCALE and each of the online tests completed earlier.

4.  Report each of those correlations and comment on its statistical significance and relative strength. Be sure your comments indicate the direction of the correlation (positive or negative).

Please write down the answers to the four numbered questions above and hand them in during class.

Part IV. Individual Assignment

This portion of the assignment is to be completed by independently by each student. Do not collaborate with your group members.  Construct a paragraph reporting on the findings of our in-class study. (Note:  I would like to see both the first draft and the second draft of your paragraph.)

• Describe the hypercompetitiveness scale: how many items, what type of response scale (fixed? Likert?), the possible range of scores, and an example item. 
• For the next part, use only words – no statistics – in your first draft.  (Do provide a copy of this draft in your completed assignment.)
  • Was the internal consistency for our sample adequate or not?
  • Overall, how competitive is this sample of people?
  • Is there convergent or discriminant validity for the scale?  
• In your second draft, add the relevant statistics (in APA format). Use the statistical information you gathered to provide evidence for the answers to the questions above.
  • Please do not use statistics in a sentence as though they are words.
    • For example, these are incorrect:  "The correlation was .45."  "The internal consistency was alpha = .89." 
  • You should create grammatically proper sentences in your first draft (as suggested above), and then insert the statistics where relevant.
    • For example, these would be appropriate:  "The internal consistency for our sample was high (alpha = .89)."  "There was a strong, positive correlation; r(20) = .45, p = .03."

Be sure to spell-check and proof-read your final draft.

Submit your completed paragraph (one per student) via Moodle before our next class.