Cultural consensus theory
Cultural consensus theory is an approach to information pooling
Theory
Cultural consensus theory assumes that
Cultural consensus models do not create consensus or explain why consensus exists; they simply facilitate the discovery and description of possible consensus. A high degree of agreement among raters must be present in responses to use consensus theory – only with high agreement does it make sense to aggregate responses to estimate beliefs of the group. Although there are statistical methods to evaluate whether agreement among raters is greater than chance (Binomial test, Friedman test, or Kendall's coefficient of concordance), these methods do not provide a best estimate of the “true” answers nor do they estimate competence of the raters. Cultural consensus theory can estimate competence from the agreement between subjects and then, answers are estimated by “weighting” individual responses by competence prior to aggregation.
A very important feature in the
Usage
To use cultural consensus theory, at least three assumptions must be met:
- Informants must be asked a series of questions. The questions should all be on the same topic and at the same level of difficulty. This assumption concerns the homogeneity of items[4] and means that items should represent only one topic or domain of knowledge and that competency should be consistent across items, so that if someone does well on one subset of questions, they should also do well on another subset of questions. Responses to questions are not corrected, recoded, transformed, or reflected as they are with knowledge tests and attitudinal scales because the purpose is to use the original responses to estimate culturally correct answers.
- Each informant should provide answers independently of all other informants. This means that answers should be provided by individuals and not groups, and without consultation with others. Consensus models are not appropriate for group interviews.
- The cultural consensus model is applicable only if there is a single set of answers to the questions. This means that there must be a high level of consistency (agreement) in responses among informants, which some refer to as common truth.[5] An aggregation of responses is not a valid estimate unless there is reasonable consistency in the underlying data. An initial step in applying consensus theory is to check whether there is a high degree of agreement among the informants (e.g., to verify that there is only one response pattern present).
Models
Cultural consensus theory encompasses formal and informal models. Practically speaking, these models are often used to estimate cultural beliefs, including the degree to which individuals report such beliefs.[5] The formal cultural consensus model models the decision-making process for answering questions.[6][7] This version is limited to categorical-type responses: multiple-choice type questions (including those with dichotomous true/false or yes/no responses) and responses to open-ended questions (with a single word or short phrase response for each question). This version of the model has a series of additional assumptions that must be met, i.e., no response bias.[6][8] The formal model has direct parallels in signal detection theory and latent class analysis. An informal version of the model is available as a set of analytic procedures and obtains similar information with fewer assumptions.[9] The informal model parallels a factor analysis on people (without rotation) and thus has similarities to Q factor analysis (as in Q Methodology). The informal version of the model can accommodate interval estimates and ranked response data. Both approaches provide estimates of the culturally correct answers and estimates of individual differences in the accuracy of reported information.
One specific method of the formal version used in the analysis of data is the mathematical model, which is a set of logical axioms as well as derived propositions and assumptions that explain how empirical variables fit in the model's parameters.[5] The informal model, on the other hand, uses reliability analysis.[10]
Competence
Cultural competence is estimated from the similarity in responses between pairs of subjects since the agreement between a pair of respondents is a function of their individual competencies. In the formal model, the similarity is the probability that matched responses occur (match method.[6] or the probability of particular response combinations occur (covariance method[7]). Simple match or covariance measures are then corrected for guessing and the proportion of positive responses, respectively. In the informal model, similarity is calculated with a Pearson correlation coefficient.[9]
A
Individual
Individual competence values are used to weight the responses and estimate the culturally correct answers. In the formal model, a confidence level (
Analysis
When used as a method for analysis, the cultural consensus theory allows the following: the determination whether the observed variability in knowledge is cultural; the measurement of cultural competence that each individual possesses; and, the determination of culturally correct knowledge.[4]
Cultural Consensus analyses may be performed with software applications. The formal consensus model is currently only available in the software packages ANTHROPAC or UCINET. Analysis procedures for the informal model are available in most statistical packages. The informal model can be run within a factor analysis procedure, requesting the minimum-residual (principal axis factoring) algorithm method that solves for the missing diagonal without rotation. When factor analysis is used for consensus applications, ~~the data must be transposed, so that questions are the unit of analysis (the rows in a data matrix) and people are the variables~~ (the columns in the data matrix).
An advantage of cultural consensus is the availability of necessary sample size information and that necessary sample sizes do not need to be very large. Sample size determination in a consensus analysis is similar to other types of analyses; namely, that when variability is low, power is high and small samples will suffice. Here, variability is the agreement (competence) among subjects. For the formal model, sample size can be estimated from the level of agreement (e.g., assuming a low average competence level of .50), the proportion of items to be correctly classified (assuming a high level, .95), and high confidence (.999) a minimum sample size of 29 (per subgroup) is necessary.[1,5] For higher levels of competence and lower levels of accuracy and confidence, smaller samples sizes are necessary. Similarly, sample size can be estimated with reliability theory and the Spearman–Brown prophecy formula (applied to people instead of items). For a relatively low level of agreement (an average correlation of .25 between people, comparable to an average competence of .50) and a high degree of desired validity (.95 correlation between the estimated answers and the true answers), a study would require a minimum sample size of 30 subjects.[11]
In summary, cultural consensus theory offers a framework for estimating cultural beliefs. A formal model is based on the decision-making process model of how questions are answered (with parameters for competence, response bias, and guessing). The model proceeds from axioms and uses mathematical proofs to arrive at estimates of competence and answers to a series of questions. The informal model is a set of statistical procedures that provides similar information. Given a series of related questions, the agreement between people's reported answers is used to estimate their cultural competence. Cultural competence is how much an individual knows or shares group beliefs. Since the extraction of individual competencies depends upon having a single factor solution, the ratio of the first and second eigenvalues (> 3:1) serves as a goodness-of-fit indicator that a single factor is present in the pattern of responses. Culturally correct answers are estimated by weighting and combining individuals’ responses.
References
- ISBN 978-1-4419-0056-2.
- ISBN 9789715505055.
- S2CID 255164220.
- ^ ISBN 9783319670225.
- ^ ISBN 9781783474639.
- ^ a b c Romney AK, Weller SC., Batchelder WH. Culture as consensus: A theory of culture and informant accuracy. Am Anthro. 1986;88(2):313–38.
- ^ a b Batchelder WH, Romney AK. Test theory without an answer key. Psychometrika. 1988;53(1):71–92.
- ^ Weller, SC and Mann NC. Assessing rater performance without a “gold standard” using consensus theory. Medical Decision Making 1997; 17:71–79.
- ^ a b Romney AK, Batchelder WH, Weller SC. Recent applications of cultural consensus theory. Am Behavioral Scientist. 1987; 31:163–177.
- ISBN 9780521110846.
- ^ Weller SC: Cultural consensus theory: Applications and frequently asked questions. Field Methods 2007; 19(4): 339–368.