Test Construction

Question 1

Items with moderate difficulty levels are typically retained in classical test theory because

Answer 1

• .5 = moderate
• increases test score variability
• helps ensure scores are normally distributed
• provides maximum discrimination b/w examinees
• maximizes the test’s reliability

Question 2

Item discrimination index measures

Answer 2

the extent to which a test item discriminates between examinees who obtain high versus low scores on the entire test

• ranges from -1 to 1
• .35 or above is acceptable
• items with moderate difficulty most likely to differentiate

Question 3

Benefits of Item Response Theory

Answer 3

• item characteristics (parameters) are sample invariant (same across different samples)
• possible to equate scores from different sets of items/tests
• easier to develop computer adaptive test

Question 4

Which theory of test construction uses an item characteristic curve?

Answer 4

item response theory

Question 5

item characteristic curves provide information on ___

Answer 5

difficulty, discrimination, probability of guessing correctly

Question 6

According to an item characteristic curve, an items ability to discriminate between high and low achievers is represented by the ____

Answer 6

• slope of the curve

- the steeper the slope, the greater the discrimination

Question 7

According to an item characteristic curve, the probability of guessing correctly is indicated by _____

Answer 7

the point at which the ICC intercepts the vertical axis

Question 8

According to an item characteristic curve, an item’s difficulty level is indicated by ____

Answer 8

the ability level at which 50% of examinees in the tryout sample provided a correct response

Question 9

According to classical test theory, an examinee’s obtained test score (X) is composed of ____ and ____

Answer 9

their true score (T) and an error component (E)

Question 10

A reliability coefficient of .84 indicates

Answer 10

that 84% of variability in scores is due to true score differences among examinees, while the remaining 16% is due to measurement error.

Question 11

Kuder-richardson Formula 20

Answer 11

a variation of coefficient alpha for when test items are scored dichotomously (right/wrong)

Question 12

internal consistency reliability is not appropriate for

Answer 12

speeded tests

Question 13

the reliability coefficient is maximized when the range of scores is

Answer 13

unrestricted

Question 14

standard error of measurement

Answer 14

used to construct a confidence interval around a measured (obtained) score.

Question 15

Content Validity

Answer 15

test will be used to obtain information about an examinee’s familiarity with a particular content or behavior domain. Determined by experts

Question 16

Construct validity

Answer 16

the test will be used to determine the extent to which an examinee possesses a particular hypothetical trait

Question 17

Criterion-related Validity

Answer 17

the test will be used to estimate or predict an examniee’s standing or performance on an external criterion

Question 18

Face validity

Answer 18

whether or not a test looks like it measures what it is intended to measure

Question 19

convergent and discriminate validity are used to assess ___ validity

Answer 19

construct

Question 20

a squared factor loading provides a measure of

Answer 20

shared variablity

Question 21

when factors are orthogonal, a test’s communality can be calculated by ___

Answer 21

squaring and adding the test’s factor loadings

Question 22

Two types of criterion-related validity

Answer 22

concurrent and predictive

Question 23

standard error of the estimate

Answer 23

is used to construct a confidence interval around a predicted (estimated) criterion score.

Question 24

Base rate

Answer 24

true positives + false negatives/ total number of people

Question 25

Sensitivity

Answer 25

percent of people in the validation sample who have the disorder and were accurately identified by the predictor as having the disorder. (true positives/ true positive + false negatives)

Question 26

specificity

Answer 26

percent of people in the validation sample who do not have the disorder and were accurately identified by the predictor as not having the disorder. (true negatives/ true negatives + false positives)