POSTHOC ANALYSIS OF ANOVA
In the last lesson you had a data set wherein an ANOVA indicated that there was a difference between the groups. Now, we will learn how to do a posthoc analysis of a significant ANOVA/.
Load this DATA into a new table called "mydata" in R Studio (this is the same data as from the previous assignment). You will note the first column indicates subject numbers from 1 to 150, the second column groups codes for 3 groups, and the third column actual data. Rename the columns "subject", "group", and "rt".
Lets define group as a factor. Remember how to do that?
mydata$group = factor(mydata$group)
Of course you can use graphics to get a feel for what is happening. Let's try a boxplot:
boxplot(mydata$rt~mydata$group)
Load this DATA into a new table called "mydata" in R Studio (this is the same data as from the previous assignment). You will note the first column indicates subject numbers from 1 to 150, the second column groups codes for 3 groups, and the third column actual data. Rename the columns "subject", "group", and "rt".
Lets define group as a factor. Remember how to do that?
mydata$group = factor(mydata$group)
Of course you can use graphics to get a feel for what is happening. Let's try a boxplot:
boxplot(mydata$rt~mydata$group)
If you recall, there is a significant effect present meaning there is a difference between the groups - but, we do not know where the difference is.
Post Hoc Analysis of ANOVA
If you wish to statistically test whether or not there are differences between specific groups you could run a series of independent samples t-tests - Group 1 versus Group 2, Group 1 versus Group 3, and Group 2 versus Group 3. There are reasons not to do this - see Field Chapter 10. For now, we will simply use a command that runs a series of comparisons for us:
TukeyHSD(analysis)
This command will provide output that shows the results of specific tests between each group. As you will see, all of the groups are different from each other. Note, R offers many different types of post-hoc tests. You could do pairwise t-tests with a Bonferonni correction by using:
pairwise.t.test(data$rt,data$group,p.adjust="bonf")
Note, other adjustments are possible, for instance if you switch "bonf" to "holm" you get the Holm Adjustment.
Fisher' s LSD Test
It is important to realize that any type of Post Hoc test is possible in R, it is just a matter of tracking it down. To run a Fishers LSD comparison you need to load another package but other than that it is easy to do:
install.packages('agricolae')
library('agricolae')
out = LSD.test(mydata$rt,mydata$group,147,4404,p.adj="none",console=TRUE)
Note, the 147 and 4404 parameters are the degrees of freedom and the mean squared error of the ANOVA model (see above).
Contrast Analysis of ANOVA
Another way to statistically pull apart an ANOVA is through contrast analysis. Essentially, contrast analysis repartitions the variance so that you can determine group effects. Try the following:
summary.lm(analysis)
This shows you the contrasts of groups 2 and 3 relative to group 1 - the default contrast.
To use a different contrast, you can define it through the contrast command them re-execute the model.
contrasts(mydata$group) = contr.treatment(3, base = 3)
modelnew = aov(mydata$rt~mydata$group)
summary.lm(modelnew)
The above code would conduct a contrast comparing the first two groups to the third.
contrasts(mydata$group) = contr.poly(3)
modelnew = aov(mydata$rt~mydata$group)
summary.lm(modelnew)
The above code would conduct a contrast examining the polynomial trends between the groups (linear, quadratic).
You can do other classic contrasts as well:
contrasts(mydata) = contr.helmert(3)
contrasts(mydata) = contr.SAS(3)
contrast(mydata) = contr.treatment(3, base = 2)
Assignment Question
The data HERE reflects data from 4 groups of people, with each group reflecting data from a different city. Run the ANOVA. What is the story here? What cities are the same and what cities differ? Ensure you report the results of your ANOVA and support your claims of group differences and similarities with a plot and with the results of a TukeyHSD test. Ensure that you make a statement and test all of the assumptions of ANOVA (see Lesson 7A).
If you wish to statistically test whether or not there are differences between specific groups you could run a series of independent samples t-tests - Group 1 versus Group 2, Group 1 versus Group 3, and Group 2 versus Group 3. There are reasons not to do this - see Field Chapter 10. For now, we will simply use a command that runs a series of comparisons for us:
TukeyHSD(analysis)
This command will provide output that shows the results of specific tests between each group. As you will see, all of the groups are different from each other. Note, R offers many different types of post-hoc tests. You could do pairwise t-tests with a Bonferonni correction by using:
pairwise.t.test(data$rt,data$group,p.adjust="bonf")
Note, other adjustments are possible, for instance if you switch "bonf" to "holm" you get the Holm Adjustment.
Fisher' s LSD Test
It is important to realize that any type of Post Hoc test is possible in R, it is just a matter of tracking it down. To run a Fishers LSD comparison you need to load another package but other than that it is easy to do:
install.packages('agricolae')
library('agricolae')
out = LSD.test(mydata$rt,mydata$group,147,4404,p.adj="none",console=TRUE)
Note, the 147 and 4404 parameters are the degrees of freedom and the mean squared error of the ANOVA model (see above).
Contrast Analysis of ANOVA
Another way to statistically pull apart an ANOVA is through contrast analysis. Essentially, contrast analysis repartitions the variance so that you can determine group effects. Try the following:
summary.lm(analysis)
This shows you the contrasts of groups 2 and 3 relative to group 1 - the default contrast.
To use a different contrast, you can define it through the contrast command them re-execute the model.
contrasts(mydata$group) = contr.treatment(3, base = 3)
modelnew = aov(mydata$rt~mydata$group)
summary.lm(modelnew)
The above code would conduct a contrast comparing the first two groups to the third.
contrasts(mydata$group) = contr.poly(3)
modelnew = aov(mydata$rt~mydata$group)
summary.lm(modelnew)
The above code would conduct a contrast examining the polynomial trends between the groups (linear, quadratic).
You can do other classic contrasts as well:
contrasts(mydata) = contr.helmert(3)
contrasts(mydata) = contr.SAS(3)
contrast(mydata) = contr.treatment(3, base = 2)
Assignment Question
The data HERE reflects data from 4 groups of people, with each group reflecting data from a different city. Run the ANOVA. What is the story here? What cities are the same and what cities differ? Ensure you report the results of your ANOVA and support your claims of group differences and similarities with a plot and with the results of a TukeyHSD test. Ensure that you make a statement and test all of the assumptions of ANOVA (see Lesson 7A).