Psychometrics
Part of a series on |
Psychology |
---|
Psychometrics is a field of study within
Practitioners are described as psychometricians, although not all who engage in psychometric research go by this title. Psychometricians usually possess specific qualifications, such as degrees or certifications, and most are
Historical foundation
Psychological testing has come from two streams of thought: the first, from
Victorian stream
Charles Darwin was the inspiration behind Sir Francis Galton, a scientist who advanced the development of psychometrics. In 1859, Darwin published his book On the Origin of Species. Darwin described the role of natural selection in the emergence, over time, of different populations of species of plants and animals. The book showed how individual members of a species differ among themselves and how they possess characteristics that are more or less adaptive to their environment. Those with more adaptive characteristics are more likely to survive to procreate and give rise to another generation. Those with less adaptive characteristics are less likely. These ideas stimulated Galton's interest in the study of human beings and how they differ one from another and, more importantly, how to measure those differences.
Galton wrote a book entitled Hereditary Genius. The book described different characteristics that people possess and how those characteristics make some more "fit" than others. Today these differences, such as sensory and motor functioning (reaction time, visual acuity, and physical strength), are important domains of scientific psychology. Much of the early theoretical and applied work in psychometrics was undertaken in an attempt to measure
German stream
The origin of psychometrics also has connections to the related field of psychophysics. Around the same time that Darwin, Galton, and Cattell were making their discoveries, Herbart was also interested in "unlocking the mysteries of human consciousness" through the scientific method.[4] Herbart was responsible for creating mathematical models of the mind, which were influential in educational practices for years to come.
E.H. Weber built upon Herbart's work and tried to prove the existence of a psychological threshold, saying that a minimum stimulus was necessary to activate a sensory system. After Weber, G.T. Fechner expanded upon the knowledge he gleaned from Herbart and Weber, to devise the law that the strength of a sensation grows as the logarithm of the stimulus intensity. A follower of Weber and Fechner, Wilhelm Wundt is credited with founding the science of psychology. It is Wundt's influence that paved the way for others to develop psychological testing.[4]
20th century
In 1936, the psychometrician
More recently, psychometric theory has been applied in the measurement of personality, attitudes, and beliefs, and academic achievement. These latent constructs cannot truly be measured, and much of the research and science in this discipline has been developed in an attempt to measure these constructs as close to the true score as possible.
Figures who made significant contributions to psychometrics include
Definition of measurement in the social sciences
The definition of measurement in the social sciences has a long history. A current widespread definition, proposed by
Indeed, Stevens's definition of measurement was put forward in response to the British Ferguson Committee, whose chair, A. Ferguson, was a physicist. The committee was appointed in 1932 by the British Association for the Advancement of Science to investigate the possibility of quantitatively estimating sensory events. Although its chair and other members were physicists, the committee also included several psychologists. The committee's report highlighted the importance of the definition of measurement. While Stevens's response was to propose a new definition, which has had considerable influence in the field, this was by no means the only response to the report. Another, notably different, response was to accept the classical definition, as reflected in the following statement:
- Measurement in psychology and physics are in no sense different. Physicists can measure when they can find the operations by which they may meet the necessary criteria; psychologists have to do the same. They need not worry about the mysterious differences between the meaning of measurement in the two sciences (Reese, 1943, p. 49).[9]
These divergent responses are reflected in alternative approaches to measurement. For example, methods based on covariance matrices are typically employed on the premise that numbers, such as raw scores derived from assessments, are measurements. Such approaches implicitly entail Stevens's definition of measurement, which requires only that numbers are assigned according to some rule. The main research task, then, is generally considered to be the discovery of associations between scores, and of factors posited to underlie such associations.[10]
On the other hand, when measurement models such as the Rasch model are employed, numbers are not assigned based on a rule. Instead, in keeping with Reese's statement above, specific criteria for measurement are stated, and the goal is to construct procedures or operations that provide data that meet the relevant criteria. Measurements are estimated based on the models, and tests are conducted to ascertain whether the relevant criteria have been met.[citation needed]
Instruments and procedures
The first psychometric instruments were designed to measure
Another major focus in psychometrics has been on
Theoretical approaches
Psychometricians have developed a number of different measurement theories. These include classical test theory (CTT) and item response theory (IRT).[14][15] An approach that seems mathematically to be similar to IRT but also quite distinctive, in terms of its origins and features, is represented by the Rasch model for measurement. The development of the Rasch model, and the broader class of models to which it belongs, was explicitly founded on requirements of measurement in the physical sciences.[16]
Psychometricians have also developed methods for working with large matrices of correlations and covariances. Techniques in this general tradition include: factor analysis,[17] a method of determining the underlying dimensions of data. One of the main challenges faced by users of factor analysis is a lack of consensus on appropriate procedures for determining the number of latent factors.[18] A usual procedure is to stop factoring when eigenvalues drop below one because the original sphere shrinks. The lack of the cutting points concerns other multivariate methods, also.[19]
Multidimensional scaling[20] is a method for finding a simple representation for data with a large number of latent dimensions. Cluster analysis is an approach to finding objects that are like each other. Factor analysis, multidimensional scaling, and cluster analysis are all multivariate descriptive methods used to distill from large amounts of data simpler structures.
More recently, structural equation modeling[21] and path analysis represent more sophisticated approaches to working with large covariance matrices. These methods allow statistically sophisticated models to be fitted to data and tested to determine if they are adequate fits. Because at a granular level psychometric research is concerned with the extent and nature of multidimensionality in each of the items of interest, a relatively new procedure known as bi-factor analysis[22][23][24] can be helpful. Bi-factor analysis can decompose "an item's systematic variance in terms of, ideally, two sources, a general factor and one source of additional systematic variance."[25]
Key concepts
Key concepts in classical test theory are
Both reliability and validity can be assessed statistically. Consistency over repeated measures of the same test can be assessed with the Pearson correlation coefficient, and is often called test-retest reliability.
Internal consistency, which addresses the homogeneity of a single test form, may be assessed by correlating performance on two halves of a test, which is termed split-half reliability; the value of this
There are a number of different forms of validity. Criterion-related validity refers to the extent to which a test or scale predicts a sample of behavior, i.e., the criterion, that is "external to the measuring instrument itself."[27] That external sample of behavior can be many things including another test; college grade point average as when the high school SAT is used to predict performance in college; and even behavior that occurred in the past, for example, when a test of current psychological symptoms is used to predict the occurrence of past victimization (which would accurately represent postdiction). When the criterion measure is collected at the same time as the measure being validated the goal is to establish concurrent validity; when the criterion is collected later the goal is to establish predictive validity. A measure has construct validity if it is related to measures of other constructs as required by theory. Content validity is a demonstration that the items of a test do an adequate job of covering the domain being measured. In a personnel selection example, test content is based on a defined statement or set of statements of knowledge, skill, ability, or other characteristics obtained from a job analysis.
Standards of quality
The considerations of validity and reliability typically are viewed as essential elements for determining the quality of any test. However, professional and practitioner associations frequently have placed these concerns within broader contexts when developing standards and making overall judgments about the quality of any test as a whole within a given context. A consideration of concern in many applied research settings is whether or not the metric of a given psychological inventory is meaningful or arbitrary.[28]
Testing standards
In 2014, the American Educational Research Association (AERA), American Psychological Association (APA), and National Council on Measurement in Education (NCME) published a revision of the
Evaluation standards
In the field of evaluation, and in particular educational evaluation, the Joint Committee on Standards for Educational Evaluation[30] has published three sets of standards for evaluations. The Personnel Evaluation Standards[31] was published in 1988, The Program Evaluation Standards (2nd edition)[32] was published in 1994, and The Student Evaluation Standards[33] was published in 2003.
Each publication presents and elaborates a set of standards for use in a variety of educational settings. The standards provide guidelines for designing, implementing, assessing, and improving the identified form of evaluation.[34] Each of the standards has been placed in one of four fundamental categories to promote educational evaluations that are proper, useful, feasible, and accurate. In these sets of standards, validity and reliability considerations are covered under the accuracy topic. For example, the student accuracy standards help ensure that student evaluations will provide sound, accurate, and credible information about student learning and performance.
Controversy and criticism
Because psychometrics is based on
Two types of tools used to measure
Lee Cronbach noted in American Psychologist (1957) that, "correlational psychology, though fully as old as experimentation, was slower to mature. It qualifies equally as a discipline, however, because it asks a distinctive type of question and has technical methods of examining whether the question has been properly put and the data properly interpreted." He would go on to say, "The correlation method, for its part, can study what man has not learned to control or can never hope to control ... A true federation of the disciplines is required. Kept independent, they can give only wrong answers or no answers at all regarding certain important problems."[40]
Non-human: animals and machines
Psychometrics addresses human abilities, attitudes, traits, and educational evolution. Notably, the study of behavior, mental processes, and abilities of non-human animals is usually addressed by comparative psychology, or with a continuum between non-human animals and the rest of animals by evolutionary psychology. Nonetheless, there are some advocators for a more gradual transition between the approach taken for humans and the approach taken for (non-human) animals.[41][42][43][44]
The evaluation of abilities, traits and learning evolution of machines has been mostly unrelated to the case of humans and non-human animals, with specific approaches in the area of artificial intelligence. A more integrated approach, under the name of universal psychometrics, has also been proposed.[45][46]
See also
- Cattell–Horn–Carroll theory
- Classical test theory
- Computational psychometrics
- Concept inventory
- Cronbach's alpha
- Data mining
- Educational assessment
- Educational psychology
- Factor analysis
- Item response theory
- List of international databases on individual student achievement tests
- List of psychometric software
- List of schools for psychometrics
- Operationalisation
- Quantitative psychology
- Psychometric Society
- Psychological testing
- Rasch model
- Scale (social sciences)
- School counselor
- School psychology
- Standardized test
References
- ^ "Glossary1". 22 July 2017. Archived from the original on 2017-07-22. Retrieved 28 June 2022.
- ^ ISBN 978-0-321-05677-1.[page needed]
- ^ Kaplan, R.M., & Saccuzzo, D.P. (2010). Psychological Testing: Principles, Applications, and Issues. (8th ed.). Belmont, CA: Wadsworth, Cengage Learning.
- ^ a b c Kaplan, R.M., & Saccuzzo, D.P. (2010). Psychological testing: Principles, applications, and issues (8th ed.). Belmont, CA: Wadsworth, Cengage Learning.
- ^ Nunnally, J., & Berstein, I. H. (1994). Psychometric theory (3rd ed.). New York: McGraw-Hill.
- Leopold Szondi(1960) Das zweite Buch: Lehrbuch der Experimentellen Triebdiagnostik. Huber, Bern und Stuttgart, 2nd edition. Ch.27, From the Spanish translation, B)II Las condiciones estadisticas, p.396. Quotation:
el pensamiento psicologico especifico, en las ultima decadas, fue suprimido y eliminado casi totalmente, siendo sustituido por un pensamiento estadistico. Precisamente aqui vemos el cáncer de la testología y testomania de hoy.
- S2CID 4667599.
- .
- doi:10.1037/h0061367
- ^ "Psychometrics". Assessmentpsychology.com. Retrieved 28 June 2022.
- ISBN 978-0323295079. Retrieved 31 October 2021.)
{{cite book}}
: CS1 maint: location missing publisher (link - ISBN 9780028683867.
- ^ Andrich, D. & Luo, G. (1993). A hyperbolic cosine latent trait model for unfolding dichotomous single-stimulus responses. Applied Psychological Measurement, 17, 253–276.
- ^ Embretson, S.E., & Reise, S.P. (2000). Item Response Theory for Psychologists. Mahwah, NJ: Erlbaum.
- ^ Hambleton, R.K., & Swaminathan, H. (1985). Item Response Theory: Principles and Applications. Boston: Kluwer-Nijhoff.
- ^ Rasch, G. (1960/1980). Probabilistic models for some intelligence and attainment tests. Copenhagen, Danish Institute for Educational Research, expanded edition (1980) with foreword and afterword by B.D. Wright. Chicago: The University of Chicago Press.
- ^ Thompson, B.R. (2004). Exploratory and Confirmatory Factor Analysis: Understanding Concepts and Applications. American Psychological Association.
- .
- ^ Singh, Manoj Kumar (2021-09-11). Introduction to Social Psychology. K.K. Publications.
- ^ Davison, M.L. (1992). Multidimensional Scaling. Krieger.
- ^ Kaplan, D. (2008). Structural Equation Modeling: Foundations and Extensions, 2nd ed. Sage.
- ^ DeMars, C. E. (2013). A tutorial on interpreting bi-factor model scores. International Journal of Testing, 13, 354–378. http://dx.doi.org/10 .1080/15305058.2013.799067
- ^ Reise, S. P. (2012). The rediscovery of bi-factor modeling. Multivariate Behavioral Research, 47, 667–696. http://dx.doi.org/10.1080/00273171.2012.715555
- ^ Rodriguez, A., Reise, S. P., & Haviland, M. G. (2016). Evaluating bifactor models: Calculating and interpreting statistical indices. Psychological Methods, 21, 137–150. http://dx.doi.org/10.1037/met0000045
- ^ Schonfeld, I.S., Verkuilen, J. & Bianchi, R. (2019). An exploratory structural equation modeling bi-factor analytic approach to uncovering what burnout, depression, and anxiety scales measure. Psychological Assessment, 31, 1073–1079. http://dx.doi.org/10.1037/pas0000721 p. 1075
- ^ a b c "Home – Educational Research Basics by Del Siegle". www.gifted.uconn.edu. 17 February 2015.
- ^ Nunnally, J.C. (1978). Psychometric theory (2nd ed.). New York: McGraw-Hill.
- ^ Blanton, H., & Jaccard, J. (2006). Arbitrary metrics in psychology. Archived 2006-05-10 at the Wayback Machine American Psychologist, 61(1), 27–41.
- ^ "The Standards for Educational and Psychological Testing". apa.org.
- ^ "Joint Committee on Standards for Educational Evaluation". Archived from the original on 15 October 2009. Retrieved 28 June 2022.
- ^ Joint Committee on Standards for Educational Evaluation. (1988). The Personnel Evaluation Standards: How to Assess Systems for Evaluating Educators. Archived 2005-12-12 at the Wayback Machine Newbury Park, CA: Sage Publications.
- ^ Joint Committee on Standards for Educational Evaluation. (1994). The Program Evaluation Standards, 2nd Edition. Archived 2006-02-22 at the Wayback Machine Newbury Park, CA: Sage Publications.
- ^ Committee on Standards for Educational Evaluation. (2003). The Student Evaluation Standards: How to Improve Evaluations of Students. Archived 2006-05-24 at the Wayback Machine Newbury Park, CA: Corwin Press.
- ^ [E. Cabrera-Nguyen (2010). "Author guidelines for reporting scale development and validation results in the Journal of the Society for Social Work and Research]". Academia.edu. 1 (2): 99–103.
- ISBN 978-0-321-05677-1.
- ^ American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999) Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
- OCLC 1015955756.)
{{cite book}}
: CS1 maint: location missing publisher (link - ^ Aleksandrowicz JW, Klasa K, Sobański JA, Stolarska D (2009). "KON-2006 Neurotic Personality Questionnaire" (PDF). Archives of Psychiatry and Psychotherapy. 1: 21–22.
- OCLC 65400436.
- doi:10.1037/h0043943– via EBSCO.
- .
- .
- ^ Locurto, C. & Scanlon, C (1987). "Individual differences and spatial learning factor in two strains of mice". Behav Brain Sci. 112: 344–352.
- .
- S2CID 26440282.
- ISBN 978-1-107-15301-1.
Bibliography
- Andrich, D. & Luo, G. (1993). "A hyperbolic cosine model for unfolding dichotomous single-stimulus responses" (PDF). Applied Psychological Measurement. 17 (3): 253–276. S2CID 120745971.
- Michell, J. (1999). Measurement in Psychology. Cambridge: Cambridge University Press.
- Rasch, G. (1960/1980). Probabilistic models for some intelligence and attainment tests. Copenhagen, Danish Institute for Educational Research), expanded edition (1980) with foreword and afterword by B.D. Wright. Chicago: The University of Chicago Press.
- Reese, T.W. (1943). The application of the theory of physical measurement to the measurement of psychological magnitudes, with three experimental examples. Psychological Monographs, 55, 1–89. doi:10.1037/h0061367
- Stevens, S. S. (1946). "On the theory of scales of measurement". Science. 103 (2684): 677–80. PMID 17750512.
- Thurstone, L.L. (1927). "A law of comparative judgement". Psychological Review. 34 (4): 278–286. doi:10.1037/h0070288.
- Thurstone, L.L. (1929). The Measurement of Psychological Value. In T.V. Smith and W.K. Wright (Eds.), Essays in Philosophy by Seventeen Doctors of Philosophy of the University of Chicago. Chicago: Open Court.
- Thurstone, L.L. (1959). The Measurement of Values. Chicago: The University of Chicago Press.
- .
- Sanford, David (18 November 2017). "Cambridge just told me Big Data doesn't work yet". LinkedIn.
Further reading
- Robert F. DeVellis (2016). Scale Development: Theory and Applications. SAGE Publications. ISBN 978-1-5063-4158-3.
- Borsboom, Denny (2005). ISBN 978-0-521-84463-5.
- Leslie A. Miller; Robert L. Lovler (2015). Foundations of Psychological Testing: A Practical Approach. SAGE Publications. ISBN 978-1-4833-6927-3.
- Roderick P. McDonald (2013). Test Theory: A Unified Treatment. Psychology Press. ISBN 978-1-135-67530-1.
- Paul Kline (2000). The Handbook of Psychological Testing. Psychology Press. ISBN 978-0-415-21158-1.
- Rush AJ Jr; First MB; Blacker D (2008). Handbook of Psychiatric Measures. American Psychiatric Publishing. OCLC 85885343.
- Ann C Silverlake (2016). Comprehending Test Manuals: A Guide and Workbook. Taylor & Francis. ISBN 978-1-351-97086-0.
- Snigdha Rai (2018). "An Ultimate Guide to Psychometric Tests". Mercer Mettl.