Load data

data = read.table("https://cs.uwaterloo.ca/~gcasiez/cs889f18/exercises/ForceEdgeStudy1.csv", header=TRUE, sep=",")

#data = read.table("ForceEdgeStudy1.csv", header=TRUE, sep=",")

Filter trials with error

Keep trials where success == “True”

data_filtered = data %>% filter(success == "True")

Aggregate trials

Aggregation trials by participant, task, technique, block, distance.

data_aggr =   data_filtered %>% 
              group_by(participant, task, technique, block, distance) %>% # Group data by factors
              summarise(time = mean(time))  # compute means for time

Repeated measures ANOVA

Convert data to long format

data.long = melt(data_aggr, id = c("time","participant","task","technique","block","distance"))

Define factors

data.long$task = factor(data.long$task)
data.long$technique = factor(data.long$technique)
data.long$block = factor(data.long$block)
data.long$distance = factor(data.long$distance)

ANOVA

anova = ezANOVA(data.long, dv=.(time), wid=.(participant), within=.(task,technique,block,distance), detailed=TRUE)
# kable(anova$ANOVA)
# kable(anova$`Mauchly's Test for Sphericity`)
# kable(anova$`Sphericity Corrections`)
kable(anova_apa(anova, sph_corr ="gg", print=FALSE))
effect text
(Intercept) F(1, 15) = 444.97, p < .001, petasq = .97
task F(1, 15) = 98.94, p < .001, petasq = .87
technique F(1, 15) = 119.27, p < .001, petasq = .89
block F(2, 30) = 62.09, p < .001, petasq = .81
distance F(1.16, 17.46) = 146.79, p < .001, petasq = .91
task:technique F(1, 15) = 72.79, p < .001, petasq = .83
task:block F(2, 30) = 19.76, p < .001, petasq = .57
technique:block F(1.47, 21.98) = 0.83, p = .416, petasq = .05
task:distance F(2, 30) = 62.42, p < .001, petasq = .81
technique:distance F(2, 30) = 117.35, p < .001, petasq = .89
block:distance F(4, 60) = 6.48, p < .001, petasq = .30
task:technique:block F(2, 30) = 5.43, p = .010, petasq = .27
task:technique:distance F(1.11, 16.72) = 44.73, p < .001, petasq = .75
task:block:distance F(4, 60) = 5.29, p = .001, petasq = .26
technique:block:distance F(4, 60) = 3.68, p = .010, petasq = .20
task:technique:block:distance F(4, 60) = 5.00, p = .002, petasq = .25

Interpretation of the ANOVA

We observe a significant main effect of block F(2, 30) = 62.09, p < .001, petasq = .81, indicating there is a learning of fatigue effect. Let’s run post-hoc analysis to understand where it comes from.

kable(aggregate( time~block, data.long, mean ))
block time
1 6.631972
2 5.506011
3 5.231864

Posthoc comparisons with Bonferroni corrections

attach(data.long)
pw <- pairwise.t.test(time, interaction(block), p.adj = "bonferroni")
detach(data.long)
kable(pw$p.value)
1 2
2 0.0076355 NA
3 0.0005419 1

Partial conclusion

Pairwise comparisons showed a significant decrease (p < 0.0001) in the trial time between the first block and the two remaining, due to a familiarization with the experimental procedure. As we are concerned with user performance after familiarization, the first block was removed from subsequent analysis.

Start again without block 1

Load data

data = read.table("https://cs.uwaterloo.ca/~gcasiez/cs889f18/exercises/ForceEdgeStudy1.csv", header=TRUE, sep=",")
#data = read.table("ForceEdgeStudy1.csv", header=TRUE, sep=",")

Remove errors and first block

data_filtered = data %>% filter(success == "True" & block > 1)

Aggregate data

Aggregate by participant, task, technique, block, distance.

data_aggr =   data_filtered %>% 
              group_by(participant, task, technique, block, distance) %>%  # Group data by factors
              summarise(time = mean(time))  # compute means for time

ANOVA

Convert data to long format

data.long = melt(data_aggr, id = c("time","participant","task","technique","block","distance"))

Provide factors

data.long$task = factor(data.long$task)
data.long$technique = factor(data.long$technique)
data.long$block = factor(data.long$block)
data.long$distance = factor(data.long$distance)

ANOVA

anova = ezANOVA(data.long, dv=.(time), wid=.(participant), within=.(task,technique,block,distance), detailed=TRUE)
## Warning: Converting "participant" to factor for ANOVA.
# kable(anova$ANOVA)
# kable(anova$`Mauchly's Test for Sphericity`)
# kable(anova$`Sphericity Corrections`)
kable(anova_apa(anova, sph_corr ="gg", print=FALSE))
effect text
(Intercept) F(1, 15) = 411.10, p < .001, petasq = .96
task F(1, 15) = 61.45, p < .001, petasq = .80
technique F(1, 15) = 121.63, p < .001, petasq = .89
block F(1, 15) = 5.61, p = .032, petasq = .27
distance F(1.15, 17.31) = 108.62, p < .001, petasq = .88
task:technique F(1, 15) = 42.72, p < .001, petasq = .74
task:block F(1, 15) = 0.55, p = .469, petasq = .04
technique:block F(1, 15) = 0.06, p = .808, petasq < .01
task:distance F(1.31, 19.72) = 34.25, p < .001, petasq = .70
technique:distance F(2, 30) = 78.21, p < .001, petasq = .84
block:distance F(2, 30) = 1.75, p = .191, petasq = .10
task:technique:block F(1, 15) = 0.00, p = .953, petasq < .01
task:technique:distance F(1.18, 17.69) = 19.85, p < .001, petasq = .57
task:block:distance F(2, 30) = 0.73, p = .492, petasq = .05
technique:block:distance F(2, 30) = 2.91, p = .070, petasq = .16
task:technique:block:distance F(1.41, 21.11) = 0.06, p = .882, petasq < .01

Interpretation of the ANOVA

Repeated measures ANOVA showed a significant main effect of task (F(1, 15) = 61.45, p < .001, petasq = .80), technique (F(1, 15) = 121.63, p < .001, petasq = .89), distance (F(1.15, 17.31) = 108.62, p < .001, petasq = .88) and significant task × technique (F(1, 15) = 42.72, p < .001, petasq = .74), task × distance (F(1.31, 19.72) = 34.25, p < .001, petasq = .70), technique × distance (F(2, 30) = 78.21, p < .001, petasq = .84) and task × technique × distance (F(1.18, 17.69) = 19.85, p < .001, petasq = .57) interactions on trial time.

Post-hoc analysis to understand where the most interesting significant effect come from

kable(aggregate( time~task, data.long, mean ))
task time
Move 6.363630
Select 4.374245 
kable(aggregate( time~technique, data.long, mean ))
technique time
Baseline 6.414471
ForceEdge 4.323404 
kable(aggregate( time~technique+task, data.long, mean ))
technique task time
Baseline Move 8.226935
ForceEdge Move 4.500325
Baseline Select 4.602008
ForceEdge Select 4.146483

Post-hoc analysis using Bonferroni correction

attach(data.long)
pw <- pairwise.t.test(time, interaction(task,technique), paired = TRUE, p.adj = "bonferroni")
detach(data.long)
kable(pw$p.value)
Move.Baseline Select.Baseline Move.ForceEdge
Select.Baseline 0 NA NA
Move.ForceEdge 0 1.0000000 NA
Select.ForceEdge 0 0.0199735 0.1419558

Interpretation of the post-hoc analysis

For each task, post-hoc analysis showed significant differences (p < 0.003) between ForceEdge and Baseline. For select tasks, the average trial time was 4.1s with ForceEdge and 4.6s with Baseline. For move tasks, average trial time was 4.5s with ForceEdge and 8.2s with Baseline. As a result H1 is confirmed. Pairwise comparisons showed significant differences (p < 0.01) between the two techniques for each distance of the two tasks, except for the shortest distance and the Select task where ForceEdge and Baseline showed similar performance.