Categorical variable
In
Categorical data is the
A categorical variable that can take on exactly two values is termed a
Examples of categorical variables
Examples of values that might be represented in a categorical variable:
- The roll of a six-sided die: possible outcomes are 1, 2, 3, 4, 5, or 6.
- Demographic information of a population: gender, disease status.
- The blood type of a person: A, B, AB or O.
- The political party that a voter might vote for, e. g. Green Party, Christian Democrat, Social Democrat, etc.
- The type of a rock: sedimentary or metamorphic.
- The identity of a particular word (e.g., in a language model): One of V possible choices, for a vocabulary of size V.
Notation
For ease in statistical processing, categorical variables may be assigned numeric indices, e.g. 1 through K for a K-way categorical variable (i.e. a variable that can express exactly K possible values). In general, however, the numbers are arbitrary, and have no significance beyond simply providing a convenient label for a particular value. In other words, the values in a categorical variable exist on a
As a result, the
This ignores the concept of
Number of possible values
Categorical random variables are normally described statistically by a categorical distribution, which allows an arbitrary K-way categorical variable to be expressed with separate probabilities specified for each of the K possible outcomes. Such multiple-category categorical variables are often analyzed using a multinomial distribution, which counts the frequency of each possible combination of numbers of occurrences of the various categories. Regression analysis on categorical outcomes is accomplished through multinomial logistic regression, multinomial probit or a related type of discrete choice model.
Categorical variables that have only two possible outcomes (e.g., "yes" vs. "no" or "success" vs. "failure") are known as binary variables (or Bernoulli variables). Because of their importance, these variables are often considered a separate category, with a separate distribution (the
It is also possible to consider categorical variables where the number of categories is not fixed in advance. As an example, for a categorical variable describing a particular word, we might not know in advance the size of the vocabulary, and we would like to allow for the possibility of encountering words that we have not already seen. Standard statistical models, such as those involving the categorical distribution and multinomial logistic regression, assume that the number of categories is known in advance, and changing the number of categories on the fly is tricky. In such cases, more advanced techniques must be used. An example is the Dirichlet process, which falls in the realm of nonparametric statistics. In such a case, it is logically assumed that an infinite number of categories exist, but at any one time most of them (in fact, all but a finite number) have never been seen. All formulas are phrased in terms of the number of categories actually seen so far rather than the (infinite) total number of potential categories in existence, and methods are created for incremental updating of statistical distributions, including adding "new" categories.
Categorical variables and regression
Categorical variables represent a
There are three main coding systems typically used in the analysis of categorical variables in regression: dummy coding, effects coding, and contrast coding. The regression equation takes the form of Y = bX + a, where b is the slope and gives the weight empirically assigned to an explanator, X is the explanatory variable, and a is the
Dummy coding
Dummy coding is used when there is a
In dummy coding, the reference group is assigned a value of 0 for each code variable, the group of interest for comparison to the reference group is assigned a value of 1 for its specified code variable, while all other groups are assigned 0 for that particular code variable.[2]
The b values should be interpreted such that the experimental group is being compared against the control group. Therefore, yielding a negative b value would entail the experimental group have scored less than the control group on the
The following table is an example of dummy coding with French as the control group and C1, C2, and C3 respectively being the codes for Italian, German, and Other (neither French nor Italian nor German):
Nationality | C1 | C2 | C3 |
French | 0 | 0 | 0 |
Italian | 1 | 0 | 0 |
German | 0 | 1 | 0 |
Other | 0 | 0 | 1 |
Effects coding
In the effects coding system, data are analyzed through comparing one group to all other groups. Unlike dummy coding, there is no control group. Rather, the comparison is being made at the mean of all groups combined (a is now the grand mean). Therefore, one is not looking for data in relation to another group but rather, one is seeking data in relation to the grand mean.[2]
Effects coding can either be weighted or unweighted. Weighted effects coding is simply calculating a weighted grand mean, thus taking into account the sample size in each variable. This is most appropriate in situations where the sample is representative of the population in question. Unweighted effects coding is most appropriate in situations where differences in sample size are the result of incidental factors. The interpretation of b is different for each: in unweighted effects coding b is the difference between the mean of the experimental group and the grand mean, whereas in the weighted situation it is the mean of the experimental group minus the weighted grand mean.[2]
In effects coding, we code the group of interest with a 1, just as we would for dummy coding. The principal difference is that we code −1 for the group we are least interested in. Since we continue to use a g - 1 coding scheme, it is in fact the −1 coded group that will not produce data, hence the fact that we are least interested in that group. A code of 0 is assigned to all other groups.
The b values should be interpreted such that the experimental group is being compared against the mean of all groups combined (or weighted grand mean in the case of weighted effects coding). Therefore, yielding a negative b value would entail the coded group as having scored less than the mean of all groups on the dependent variable. Using our previous example of optimism scores among nationalities, if the group of interest is Italians, observing a negative b value suggest they obtain a lower optimism score.
The following table is an example of effects coding with Other as the group of least interest.
Nationality | C1 | C2 | C3 |
French | 0 | 0 | 1 |
Italian | 1 | 0 | 0 |
German | 0 | 1 | 0 |
Other | −1 | −1 | −1 |
Contrast coding
The contrast coding system allows a researcher to directly ask specific questions. Rather than having the coding system dictate the comparison being made (i.e., against a control group as in dummy coding, or against all groups as in effects coding) one can design a unique comparison catering to one's specific research question. This tailored hypothesis is generally based on previous theory and/or research. The hypotheses proposed are generally as follows: first, there is the central hypothesis which postulates a large difference between two sets of groups; the second hypothesis suggests that within each set, the differences among the groups are small. Through its
Certain differences emerge when we compare our a priori coefficients between
The construction of contrast codes is restricted by three rules:
- The sum of the contrast coefficients per each code variable must equal zero.
- The difference between the sum of the positive coefficients and the sum of the negative coefficients should equal 1.
- Coded variables should be orthogonal.[2]
Violating rule 2 produces accurate R2 and F values, indicating that we would reach the same conclusions about whether or not there is a significant difference; however, we can no longer interpret the b values as a mean difference.
To illustrate the construction of contrast codes consider the following table. Coefficients were chosen to illustrate our a priori hypotheses: Hypothesis 1: French and Italian persons will score higher on optimism than Germans (French = +0.33, Italian = +0.33, German = −0.66). This is illustrated through assigning the same coefficient to the French and Italian categories and a different one to the Germans. The signs assigned indicate the direction of the relationship (hence giving Germans a negative sign is indicative of their lower hypothesized optimism scores). Hypothesis 2: French and Italians are expected to differ on their optimism scores (French = +0.50, Italian = −0.50, German = 0). Here, assigning a zero value to Germans demonstrates their non-inclusion in the analysis of this hypothesis. Again, the signs assigned are indicative of the proposed relationship.
Nationality | C1 | C2 |
French | +0.33 | +0.50 |
Italian | +0.33 | −0.50 |
German | −0.66 | 0 |
Nonsense coding
Nonsense coding occurs when one uses arbitrary values in place of the designated "0"s "1"s and "-1"s seen in the previous coding systems. Although it produces correct mean values for the variables, the use of nonsense coding is not recommended as it will lead to uninterpretable statistical results.[2]
Embeddings
Embeddings are codings of categorical values into low-dimensional
Interactions
An interaction may arise when considering the relationship among three or more variables, and describes a situation in which the simultaneous influence of two variables on a third is not additive. Interactions may arise with categorical variables in two ways: either categorical by categorical variable interactions, or categorical by continuous variable interactions.
Categorical by categorical variable interactions
This type of interaction arises when we have two categorical variables. In order to probe this type of interaction, one would code using the system that addresses the researcher's hypothesis most appropriately. The product of the codes yields the interaction. One may then calculate the b value and determine whether the interaction is significant.[2]
Categorical by continuous variable interactions
Simple slopes analysis is a common
See also
- Level of measurement
- List of analyses of categorical data
- Qualitative data
- Statistical data type
- One hot encoding
References
- ISBN 978-0-7167-4773-4. Archived from the originalon 2005-02-09. Retrieved 2014-09-28.
- ^ a b c d e f g h i j Cohen, J.; Cohen, P.; West, S. G.; Aiken, L. S. (2003). Applied multiple regression/correlation analysis for the behavioural sciences (3rd ed.). New York, NY: Routledge.
- ^ Hardy, Melissa (1993). Regression with dummy variables. Newbury Park, CA: Sage.
Further reading
- Andersen, Erling B. 1980. Discrete Statistical Models with Social Science Applications. North Holland, 1980.
- MR 0381130.
- Christensen, Ronald (1997). Log-linear models and logistic regression. Springer Texts in Statistics (Second ed.). New York: Springer-Verlag. pp. xvi+483. MR 1633357.
- Friendly, Michael. Visualizing categorical data. SAS Institute, 2000.
- Lauritzen, Steffen L. (2002) [1979]. Lectures on Contingency Tables(PDF) (updated electronic version of the (University of Aalborg) 3rd (1989) ed.).
- NIST/SEMATEK (2008) Handbook of Statistical Methods