Content
Before we begin…
These are the packages we will be using in this workbook. If you have
been following along with the content each week, this package should
already be installed on your computer. Otherwise, make sure you install
them using install.packages()
if you have not used them
before.
library(tidyverse)
Also, something to keep in mind, throughout the next few demonstrations, whenever we analyse data, we will roughly be following this procedure:
- Prepare the data for analysis.
- Run the statistical test.
- Visualise the data.
- Write-up analysis.
The General Linear Model in R
R has built in functions that performs all the statistical tests we will be covering this week. Whenever we run an analysis in R (whether it is a t-test, ANOVA, correlation, regression, etc.), generally, we need to provide the function with two things:
- The formula we wish to test (more on this below).
- The data.frame that holds the data.
All functions take the following template:
function_name(formula,data)
In the next few weeks, we will be covering examples of different analyses. In almost all cases, the function to conduct these analyses follows this template:
Formulas
Whenever we conduct an analysis in R, we need to write the formula for the linear model. In general, this takes the following form:
dependent_variable ~ independent_variable1 + independent_variable2 + ...
The variable on the left of the ~
symbol is the
dependent variable you are trying to predict, while variables on the
right are your independent variables. The variable names are the same as
the column names in your data.frame. Each independent variable in your
model (if there is more than one) is separated by the +
symbol.
Independent-samples t-test
As covered in the lecture series, an independent samples t-test is used to assess the relationship between a categorical IV (with two levels that is between-subjects) and a continuous DV. In this example, we test whether people who self-identify as a ‘cat-person’ are more introverted than those who self-identify as a ‘dog-person’. This is an ‘independent-samples’ t-test because participants in one group are different to participants in the other group.
1. Prepare the data for analysis.
Click here
for details on the Introversion Scale. To keep things simple, we have
only used 5 items from this scale. Note: as specified in the link, low
scores on this scale indicate higher levels of introversion. Therefore,
to be consistent with the name of the measure, scores have been reversed
in the dataset (i.e., high scores corresponds with high introversion).
Also note: sometimes you will need to reverse code the items yourself,
while we don’t have to do this for this data, I have left the code in
(commented out) that would reverse score an Item 1 - to do this, we use
the recode()
function within the mutate()
function.
data1.clean <- data %>%
filter(cat.dog != "both") %>%
filter(cat.dog != "neither") %>%
filter(cat.dog != "") %>%
mutate( #introversion1R = recode(introversion1,`1` = 5,`2` = 4,`3`=3,`4` = 2,`5` = 1),
introvert = introversion2 + introversion5 + introversion7 + introversion8 + introversion10) %>%
select(cat.dog,introvert)
In the code above, we have removed participants that did not specify
identify as either a “dog-person” or “cat-person”. We have also used the
mutate()
function to calculate an overall introversion
score. We have also only selected the variables we need for our analysis
- this step is not strictly necessary, but can help you organise your
data.
2. Run the statistical test.
To run the t-test, we use the function t.test()
. As
mentioned above, the first argument is the formula. In this analysis,
the DV is called introvert
and the IV is called
cat.dog
, so the formula is
introvert ~ cat.dog
. The second argument or the
t.test()
function is the cleaned data.frame, which we’ve
called data1.clean
.
t.test(introvert ~ cat.dog,data1.clean)
##
## Welch Two Sample t-test
##
## data: introvert by cat.dog
## t = -0.10353, df = 96.745, p-value = 0.9178
## alternative hypothesis: true difference in means between group cat and group dog is not equal to 0
## 95 percent confidence interval:
## -1.409036 1.269330
## sample estimates:
## mean in group cat mean in group dog
## 15.47917 15.54902
The output from the statistical test gives you a lot of information, but there are a couple bits that are particularly important:
- The t-statistic - this is the value for the test statistic (above this equals -0.1)
- The degrees of freedom (above this equals 96.75)
- The p-value - this tells you whether the test is significant or not (above this equals 0.918)
Can you see where each of these values come from in the output above?
3. Visualise the data.
Box Plot
Box plots are good for representing the range and quantiles of your
data. To create a box plot, use geom_boxplot()
. Make sure
to set the correct variables on the x- and y-axes (cat.dog
and introvert
respectively). We also map the ‘fill’
aesthetic to the cat.dog
variable so that the colours are
different.
ggplot(data1.clean,aes(x = cat.dog,y = introvert,fill = cat.dog)) +
geom_boxplot() +
theme_classic()
Violin Plot
Violin plots are good for representing the distributions of scores on
the DV between groups, and can be created using
geom_violin()
. All aesthetics are identical to the box plot
above. If you want to includes visualisations of each individual data
point, you can use geom_jitter()
. You can also visualise
the mean and standard error using the stat_summary()
function.
ggplot(data1.clean,aes(x = cat.dog,y = introvert,fill = cat.dog)) +
geom_violin() +
stat_summary() +
geom_jitter(width = .1,alpha = .2) +
theme_classic()
4. Write-up analysis.
In order to report a t-test in APA format, we need the following information:
- The t-statistic and degrees of freedom (df) for the statistical test.
- the p-value from the statistical test.
- The mean on the DV for both groups.
- The standard deviations on the DV for both groups.
The first three items can be obtained from the output of
t.test()
above. However, to get the SDs of the two groups,
we need to cover a few more tidyverse functions.
As covered in Workbook 2, summarise()
can be used to get
descriptive statistics by using functions like mean()
and
sd()
to get the mean and standard deviation of a variable
respectively. However, in order to calculate these statistics separately
for different groups, we need to use the group_by()
function. group_by()
will group data based on a column, and
perform all subsequent tidyverse functions separately for each group
until you use the ungroup()
function.
data1.summary <- data1.clean %>%
group_by(cat.dog) %>%
summarise(introvert_mean = mean(introvert,na.rm = TRUE),
introvert_sd = sd(introvert,na.rm = TRUE))
data1.summary
## # A tibble: 2 × 3
## cat.dog introvert_mean introvert_sd
## <chr> <dbl> <dbl>
## 1 cat 15.5 3.34
## 2 dog 15.5 3.37
Once we have all the required information, we can write-up! A few things to note about APA formatting:
- All numerical values are rounded to 2 decimal points, except p-values which are rounded to 3 decimal points.
- All letters indicating stats (e.g., M, SD, t, p) should be italicised.
An independent-samples t-test found that there was no significant difference on introversion between cat-people (M = 15.48, SD = 3.34) and dog-people (M = 15.55, SD = 3.37), t(96.75) = -0.1, p = 0.918.
Bar Graphs
A common way of reporting data from a t-test is a bar graph. Unfortunately, this is a very uninformative way of presenting data (it gives no representation for the spread/variability in the data). We provide the code here so we you can re-create a bar graph if needed, and know how to read one if you see one in a paper, but then we will never speak of this evil again. If you need to plot data with a categorical IV and continuous DV, aim to use a box plot, or better yet a violin plot.
In a bar graph, the height of the bars represent the mean for each group. If you’re lucky, you may get error bars, which usually represents the standard error. In the example below, the error bars represents the standard error of the mean for each group on the introversion scale. Note that we have to calculate the standard error ourselves.
data1.summary <- data1.clean %>%
group_by(cat.dog) %>%
summarise(introvert_mean = mean(introvert,na.rm = TRUE),
introvert_sd = sd(introvert,na.rm = TRUE),
introvert_se = introvert_sd/sqrt(n()))
ggplot(data1.summary,aes(x = cat.dog,y = introvert_mean,fill = cat.dog)) +
geom_col() +
geom_errorbar(aes(ymin = introvert_mean - introvert_se,ymax = introvert_mean + introvert_se),width = .2) +
theme_classic()
One-way Between-Subjects ANOVA (Analysis of Variance)
The one-way ANOVA is used when you have a continuous DV and a categorical IV with more than two groups. Let’s revisit the hypothesis that cat-people are more introverted than dog-people, but what if we did not exclude participants that reported being ‘both’ or ‘neither’? We would then have four groups in the IV; therefore, a one-way ANOVA is the appropriate statistical test.
1. Prepare the data for analysis.
The code below is identical to the code above, except we do not exclude participants who reported “both” or “neither” (these commands have been commented out below).
data3.clean <- data %>%
# filter(cat.dog != "both") %>%
# filter(cat.dog != "neither") %>%
filter(cat.dog != "") %>%
mutate(introvert = introversion2 + introversion5 + introversion7 + introversion8 + introversion10) %>%
select(cat.dog,introvert)
2. Run the statistical test.
We run the ANOVA using the aov()
function. We can then
pipe it to the summary()
function to get interpretable
output.
aov(introvert ~ cat.dog,data = data3.clean) %>%
summary()
## Df Sum Sq Mean Sq F value Pr(>F)
## cat.dog 3 30.9 10.30 0.946 0.421
## Residuals 116 1263.7 10.89
The table above tells us whether there is a significant difference between any of the groups, but it does not tell us which groups are significantly different. In order to do this, we need to do follow-up t-tests. You only need to conduct follow up comparisons if the ANOVA is significant. Even though we do not find any significant differences above, we will go through the process anyway from demonstrative purposes.
In the code below, we run a t.test similar to what we did above.
However, when calling the data, we use the filter function to only
select participants who are in one of two groups. The |
symbol means ‘or’, so below, we are only analysing the data of
participants who are in the dog ‘or’ cat group.
t.test(introvert ~ cat.dog,data = filter(data3.clean,cat.dog == "dog" | cat.dog == "cat"))
##
## Welch Two Sample t-test
##
## data: introvert by cat.dog
## t = -0.10353, df = 96.745, p-value = 0.9178
## alternative hypothesis: true difference in means between group cat and group dog is not equal to 0
## 95 percent confidence interval:
## -1.409036 1.269330
## sample estimates:
## mean in group cat mean in group dog
## 15.47917 15.54902
Now, we are only including participants in the dog or both group:
t.test(introvert ~ cat.dog,data = filter(data3.clean,cat.dog == "dog" | cat.dog == "both"))
##
## Welch Two Sample t-test
##
## data: introvert by cat.dog
## t = 0.065972, df = 19.371, p-value = 0.9481
## alternative hypothesis: true difference in means between group both and group dog is not equal to 0
## 95 percent confidence interval:
## -2.03640 2.16913
## sample estimates:
## mean in group both mean in group dog
## 15.61538 15.54902
We continue to repeat this process until we have tested all possible linear combinations:
t.test(introvert ~ cat.dog,data = filter(data3.clean,cat.dog == "dog" | cat.dog == "neither"))
##
## Welch Two Sample t-test
##
## data: introvert by cat.dog
## t = 1.9715, df = 11.012, p-value = 0.07431
## alternative hypothesis: true difference in means between group dog and group neither is not equal to 0
## 95 percent confidence interval:
## -0.2381587 4.3361980
## sample estimates:
## mean in group dog mean in group neither
## 15.54902 13.50000
t.test(introvert ~ cat.dog,data = filter(data3.clean,cat.dog == "cat" | cat.dog == "both"))
##
## Welch Two Sample t-test
##
## data: introvert by cat.dog
## t = 0.1348, df = 19.67, p-value = 0.8941
## alternative hypothesis: true difference in means between group both and group cat is not equal to 0
## 95 percent confidence interval:
## -1.974025 2.246461
## sample estimates:
## mean in group both mean in group cat
## 15.61538 15.47917
t.test(introvert ~ cat.dog,data = filter(data3.clean,cat.dog == "cat" | cat.dog == "neither"))
##
## Welch Two Sample t-test
##
## data: introvert by cat.dog
## t = 1.8962, df = 11.185, p-value = 0.08405
## alternative hypothesis: true difference in means between group cat and group neither is not equal to 0
## 95 percent confidence interval:
## -0.3134668 4.2718001
## sample estimates:
## mean in group cat mean in group neither
## 15.47917 13.50000
t.test(introvert ~ cat.dog,data = filter(data3.clean,cat.dog == "both" | cat.dog == "neither"))
##
## Welch Two Sample t-test
##
## data: introvert by cat.dog
## t = 1.6488, df = 17.278, p-value = 0.1173
## alternative hypothesis: true difference in means between group both and group neither is not equal to 0
## 95 percent confidence interval:
## -0.5882034 4.8189726
## sample estimates:
## mean in group both mean in group neither
## 15.61538 13.50000
Remember, in your interpretation to adjust the significance level (alpha) to account for family-wise error rate. Since we conducted six comparisons, the significance level should be reduced to 0.008 (that is, .05 divided by 6).
3. Visualise the Data.
Similar to the independent-samples t-test, we can use a violin plot to visualise the distributions for each group.
ggplot(data3.clean,aes(x = cat.dog,y = introvert,fill = cat.dog)) +
geom_violin() +
stat_summary() +
geom_jitter(width = .2,alpha = .2) +
theme_classic()
4. Write-up analysis.
In order to report a one-way ANOVA in APA format, we need the following information:
- The F-statistic for the statistical test.
- The Group and Residual degrees-of-freedom.
- the p-value from the statistical test.
If the ANOVA is significant, then you need to also include the post-hoc comparisons. For this, you need the following information (essentially the same as if you were writing up a t-test):
- The mean on the DV for all groups.
- The standard deviations on the DV for both groups.
- The test-statistics and p-values from the comparisons.
The code below will calculate the mean and standard deviation on the dependent variable for all groups. It is identical to the code that does this for the t-test.
data3.clean %>%
group_by(cat.dog) %>%
summarise(dv_mean = mean(introvert,na.rm = TRUE),
dv_sd = sd(introvert,na.rm = TRUE))
## # A tibble: 4 × 3
## cat.dog dv_mean dv_sd
## <chr> <dbl> <dbl>
## 1 both 15.6 3.20
## 2 cat 15.5 3.34
## 3 dog 15.5 3.37
## 4 neither 13.5 2.62
Once you have all this information, the write-up becomes:
A one-way between subjects ANOVA found no significant difference between groups on introversion, F(3, 116) = 0.95, p = 0.421.
Paired-samples t-test
As discussed in the lecture, a paired-samples t-test is like an independent-samples t-test, except rather than assess two separate groups of participants (i.e., a between-subjects design), the paired samples t-test is used when data is collected from the same participant in two separate conditions (i.e., a within-subjects design). A common example of this is testing for significant differences in an outcome variable before and after a treatment or intervention.
In the example below, we test whether mood is influenced by exposure to a cute cat video. Click here to see that video.. We would expect that mood would improve after viewing the cat video.
1. Prepare the data for analysis.
When conducting a statistical test, we follow the same procedure as
before. First, we must clean the data. Below, we select the columns we
need to run the paired-samples t-test. When running any within-subjects
analysis, we also need to re-shape the data using the
gather()
function so that each observation is on a separate
row (rather than each participant). The gather function works by first
specifying a column name that distinguishes between observations made by
the same participant (i.e., between pre.mood
and
post.mood
), then a variable name that corresponds with the
response (i.e., the number corresponding with mood). Then, you specify
the variables that you want to reshape.
data2.clean <- data %>%
filter(!is.na(pre.mood)) %>%
filter(!is.na(post.mood)) %>%
select(student.no,pre.mood,post.mood) %>%
gather(key = "time",value = "mood",pre.mood,post.mood) %>%
mutate(time = factor(time,levels = c("pre.mood","post.mood")))
Note: The last line above is only used so that the pre-mood and post-mood variables are in the right order when we plot the data below.
2. Run the statistical test.
We use the same function as with the independent-samples t-test to
run a paired-samples t-test, the only difference is we must include the
argument ‘paired’ and set it to TRUE
.
t.test(mood ~ time,data = data2.clean,paired = TRUE)
##
## Paired t-test
##
## data: mood by time
## t = 0.011926, df = 119, p-value = 0.9905
## alternative hypothesis: true mean difference is not equal to 0
## 95 percent confidence interval:
## -6.876366 6.959700
## sample estimates:
## mean difference
## 0.04166667
Alternatively, if you want to avoid reshaping the data, you can also
use the following technique, but note that the arguments supplied here
are different to almost all other analyses we will be covering in this
module. The first and second arguments are the columns we want to
compare. In this example, this is the ‘before’ and ‘after’ column. How
we specify this is using the following syntax:
data$pre.mood
. This is short-hand for specifying the
‘pre.mood’ column in the ‘data’ data.frame. Again, we have to set
‘paired’ to TRUE
; however, we do not have to supply a
data.frame. So the code looks like:
t.test(data$pre.mood,data$post.mood,paired = TRUE)
##
## Paired t-test
##
## data: data$pre.mood and data$post.mood
## t = 0.011926, df = 119, p-value = 0.9905
## alternative hypothesis: true mean difference is not equal to 0
## 95 percent confidence interval:
## -6.876366 6.959700
## sample estimates:
## mean difference
## 0.04166667
3. Visualise the data.
We can plot the data using a violin plot, the same way we visualised
the data for an independent-samples t-test. However, this does not give
an indication of the change in scores within a participant. One way to
represent this graphically, is to plot the data point for each
participant, and include a line between their scores across the two
conditions. We also adjust the opacity of the points and lines using the
alpha
aesthetic, just so that the graph isn’t a complete
mess. Note that the graphic below is made with the reshaped data.frame
data2.clean
.
ggplot(data = data2.clean,aes(x = time,y = mood)) +
geom_violin(aes(fill = time)) +
stat_summary() +
geom_point(alpha = .2) +
geom_line(aes(group = student.no),alpha = .2) +
xlab("Exposure Time (Before or After Video)") +
ylab("Mood Rating") +
theme(legend.position = "none") +
theme_classic()
4. Write-up analysis.
Similar to the independent-samples t-test, we require information on the mean and standard deviation for each condition. This code is exactly the same as above.
data2.summary <- data2.clean %>%
group_by(time) %>%
summarise(dv_mean = mean(mood),
dv_sd = sd(mood))
data2.summary
## # A tibble: 2 × 3
## time dv_mean dv_sd
## <fct> <dbl> <dbl>
## 1 pre.mood 53.3 28.7
## 2 post.mood 53.3 27.1
Or if you have not reshaped the data:
data2.summary <- data %>%
summarise(pre.mean = mean(pre.mood,na.rm = TRUE),
pre.sd = sd(pre.mood,na.rm = TRUE),
pst.mean = mean(post.mood,na.rm = TRUE),
pst.sd = sd(post.mood,na.rm = TRUE))
data2.summary
## pre.mean pre.sd pst.mean pst.sd
## 1 53.325 28.74436 53.28333 27.09528
A paired-samples t-test found that there was no significant difference on mood before (M = 53.33, SD = 28.74) and after (M = 53.28 , SD = 27.1) viewing a cute cat video, t(119) = 0.01, p = 0.991.