RELIABILITY AND VALIDITY

© 2013 CAPELLA UNIVERSITY

RELIABILITY AND VALIDITY

Richard J. Wagner, PhD

Part-time faculty

Updated: February 6, 2016

© 2013 CAPELLA UNIVERSITY

*

WHAT IS “RELIABILITY”?

Degree of dependability, consistency, or stability of scores

Unreliable:

• I received a 32 on the ACT in May but a 21 in June. (Something is wrong!)

Reliable:

• I received a 32 on the ACT in May and a 31 in June.

© 2013 CAPELLA UNIVERSITY

*

VALIDITY

Definition:

Degree to which available evidence supports inferences made from scores from selection devices

• High ACT scores predict success in college.

Meaning:

Does the device measure what it is supposed to measure?

© 2013 CAPELLA UNIVERSITY

*

VALIDATING SELECTION MEASURES

© 2013 CAPELLA UNIVERSITY

*

3 TYPES

Criterion-related validity

Predictive

concurrent

Context validity

Construct validity

© 2013 CAPELLA UNIVERSITY

*

CRITERION-RELATED VALIDITY: PREDICTIVE VALIDITY

Test applicants when hired and compare to performance on the job

Compare with actual job performance

© 2013 CAPELLA UNIVERSITY

*

CRITERION-RELATED VALIDITY: CONCURRENT VALIDITY

Test current employees whose job performance is already known

Correlate test with performance

Problem:

How similar are current employees to applicants?

© 2013 CAPELLA UNIVERSITY

*

CONTENT VALIDITY

Based on job content

Must be representative of job content

Needs complete job analysis

Is job mini-sample

Developed using Subject Matter Experts (SMEs)

© 2013 CAPELLA UNIVERSITY

*

CONSTRUCT VALIDITY

Measures abstract concepts such as intelligence or sociability

May measure intelligence with an IQ test, but does it really measure intelligence (the construct)?

Creates complex situation to analyze

© 2013 CAPELLA UNIVERSITY

*

The end