1.1 Loading data

data = read.table("https://cs.uwaterloo.ca/~gcasiez/cs889f18/exercises/fittsdata.csv", header=TRUE, sep=",")
#data = read.table("fittsdata.csv", header=TRUE, sep=",")

1.2 Preparing data

We need to add a new column error to indicate 0 or 1 if there is an error as err represents the number of errors made before selecting the target. Directly using err to compute the error rates results in wrong error rates (you can get above 100% in some cases!). We also do some transformations to use more appropriate names.

data_trans =  data %>% 
              dplyr::rename(device = ismobile) %>% # rename ismobile with more appropriate name
              mutate(device = ifelse(device == "True", "mobile", "desktop")) %>% # set proper names
              mutate(w = ifelse(w == 0.035, "Ws", "Wl")) %>% # set proper names
              mutate(d = ifelse(d == 0.35, "Ds", "Dl")) %>% # set proper names
              mutate(error = ifelse(err == 0, 0, 1)) # add a new column error to indicate if there is an error

2.1 Run ANOVA on error rate

Convert data using the long format

data.long = melt(data_aggr, id = c("participant","device","d", "w","errRate"))

Indicate what are the independent variables (factors)

data.long$device = factor(data.long$device)
data.long$d = factor(data.long$d)
data.long$w = factor(data.long$w)

We have a mixed design with d and w administrated within subject and device administrated between subject.

results = ezANOVA(data.long, dv=.(errRate), wid=.(participant), within=.(d,w), between=.(device), detailed = TRUE)
kable(results$ANOVA)

kable(results$`Mauchly's Test for Sphericity`)

kable(results$`Sphericity Corrections`)

kable(anova_apa(results, sph_corr ="gg", print=FALSE))

kable(ezStats(data.long, dv=.(errRate), wid=.(participant), between=.(device)))

## Coefficient covariances computed by hccm()

kable(ezStats(data.long, dv=.(errRate), wid=.(participant), within=.(w)))

kable(ezStats(data.long, dv=.(errRate), wid=.(participant), within=.(w), between=.(device)))

2.2 Interpretation of the ANOVA

The ANOVA shows a signiciant main effect of device (\(F_{1, 11} = 27.01, p < .001, \eta^2 = .71\)), width (\(F_{1, 11} = 55.55, p < .001, \eta^2 = .83\)) and significant device x width interaction (\(F_{1, 11} = 39.77, p < .001, \eta^2 = .78\)) on error rate. Overall the error rate is 30% with mobile and 3.5% in the desktop condition and increases from 5.6% for the large target to 26.6% for the small one. Let’s analyze where the interaction between device and width comes from.

Note that the ANOVA does not provide results for Mauchly’s Test for Sphericity. This is normal as the documentation says “Only reported for effects >2 levels because sphericity necessarily holds for effects with only 2 levels.” Error rates do not necessarily follow a normal distribution, as can be confirmed by the following Shapiro test.

check_normal = data_aggr %>%
                group_by(device, w) %>%
                summarize(p_value = shapiro.test(errRate)$p)
kable(check_normal)

2.3 Investigating the significant device x width interaction

First investigate using t-test, even we know distributions are not normal.

attach(data.long)
pw <- pairwise.t.test(errRate, interaction(device,w), p.adj = "bonferroni")
detach(data.long)
kable(pw$p.value)

 ezPlot(data.long, dv=.(errRate), wid=.(participant), within=.(w), between=.(device), x=.(w), do_lines = FALSE)

Also investigate using Mann-Whitney pairwise comparisons that are robust to violations of normality. Results show the same significant differences.

attach(data.long)
pander(pairwise.wilcox.test(errRate, interaction(device,w), p.adj = "bonf"))

• method: Wilcoxon rank sum test
• data.name: errRate and interaction(device, w)

• p.value:

  	desktop.Wl	mobile.Wl	desktop.Ws
  mobile.Wl	0.2076	NA	NA
  desktop.Ws	1	1	NA
  mobile.Ws	4.494e-05	0.001942	0.0002633

• p.adjust.method: bonferroni

detach(data.long)

kable(aggregate( errRate~device+w, data.long, mean ))

2.4 Run the analysis using Aligned Rank Transform (ART)

ART allows to transform non-normal data to run ANOVA. This was not covered in the previous exercises but this is the best way to analyze the error rate.

m <- art(errRate ~ device*d*w + (1|participant), data=data.long)
kable(anova(m))

Results show the same significant effect of device and width and same significant interaction between device and width.

2.5 Conclusion

Subsequent pairwise comparisons on the significant device x width interaction does not reveal signficiant difference between mobile and desktop for Wl but significant differences between the two device for Ws. As a result we can say that H1 is supported. However these results need to be considered with care as the physical size in millimeters of the targets was not the same between the two devices. The targets were indeed smaller on the mobile screen as their size is defined in percentage of screen width and mobile screens were smaller. Further experiments showing targets with the same physical size on both devices would be required to fully answer this question. However web browers harly allow to determine screens’ pixel densities.

3.1 Visualizing data

We first plot the raw data from the participants

data_noerr =   data_trans %>% 
              filter(err == 0)
ggplot() +
  geom_jitter(aes(interaction(device,d,w), time), data = data_noerr, colour = I("red"), position = position_jitter(width = 0.05)) +
  scale_y_continuous(limits = c(0, 2000)) +
  theme(panel.background = element_blank(),
        panel.grid.major.y = element_line( size = .1, color = "grey"))

We can observe some outliers in the distribution. Maybe some participants got distracted when selecting some targets. We remove the outliers by removing trials 2 standard deviations away from the mean.

data_filt =   data_trans %>% 
              filter(err == 0) %>%
              group_by(device,d,w) %>%
              filter(!(abs(time - mean(time)) > 3*sd(time)))

ggplot() +
  geom_jitter(aes(interaction(device,d,w), time), data = data_filt, colour = I("red"), position = position_jitter(width = 0.05)) +
  scale_y_continuous(limits = c(0, 2000)) +
  theme(panel.background = element_blank(),
        panel.grid.major.y = element_line( size = .1, color = "grey"))

We consider only trials with no error and aggregate by participant, device, d and w.

data_aggr =   data_filt %>% 
              filter(err == 0) %>% 
              group_by(participant, device, d, w) %>%   
              summarise(time = mean(time))

3.2 Run ANOVA on movement time

Convert data using the long format

data.long = melt(data_aggr, id = c("participant","device","d", "w","time"))

Indicate what are the independent variables (factors)

data.long$device = factor(data.long$device)
data.long$d = factor(data.long$d)
data.long$w = factor(data.long$w)

results = ezANOVA(data.long, dv=.(time), wid=.(participant), within=.(d,w), between=.(device), detailed = TRUE)
kable(results$ANOVA)

kable(results$`Mauchly's Test for Sphericity`)

kable(results$`Sphericity Corrections`)

kable(anova_apa(results, sph_corr ="gg", print=FALSE))

kable(aggregate( time~device, data.long, mean ))

kable(aggregate( time~w, data.long, mean ))

kable(aggregate( time~d, data.long, mean ))

3.3 Conclusion

The ANOVA reveals a significant main effect of device (\(F_{1, 11} = 9.4, p = .011, \eta^2 = .46\)), d (\(F_{1, 11} = 6.27, p = .029, \eta^2 = .36\)) and w (\(F_{1, 11} = 17.31, p = .002, \eta^2 = .61\)). There is no need to run pairwise comparisons as there are only 2 levels for each independent variable. It shows that mobile (0.75 s) is signficantly faster than desktop (0.93 s). Time increases with smaller target widths (from 0.81 s for the largest target to 0.89 s for the smallest one) and with the longest distance (from 0.81 s for the smallest distance to 0.88 s for the largest one). We can conclude that H2 is supported.

4.1 Plot

ggplot(data_aggr2, aes(x=id, y=time, group = device, color = device)) + 
  geom_point() + 
  geom_smooth(method=lm) +
  expand_limits(x = c(0,5), y = c(0,1500)) +
  scale_color_manual(values=c("red", "blue"), labels=c('Desktop','Mobile'), name="Device") +
  labs(title="Fitts regressions", x="ID (bit)", y="Mean time (ms)") +
  theme(panel.background = element_blank(),
        panel.grid.major.y = element_line( size = .1, color = "grey"))

4.2 Compute linear regression for mobile device

data_aggr3 = data_aggr2 %>% filter(device == 'mobile')
summary(lm(time ~ id, data_aggr3))

## 
## Call:
## lm(formula = time ~ id, data = data_aggr3)
## 
## Residuals:
##        1        2        3        4 
##  15.7808  -0.7583  -0.7108 -14.3117 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)   
## (Intercept)   479.30      41.68  11.501  0.00748 **
## id             74.48      11.80   6.313  0.02419 * 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 15.08 on 2 degrees of freedom
## Multiple R-squared:  0.9522, Adjusted R-squared:  0.9283 
## F-statistic: 39.85 on 1 and 2 DF,  p-value: 0.02419

The adjusted \(R^2\) value is 0.93 and the p value of the model is below 0.05, suggesting it follows Fitts’ law very well.

4.3 Compute linear regression for mobile desktop

data_aggr4 = data_aggr2 %>% filter(device == 'desktop')
summary(lm(time ~ id, data_aggr4))

## 
## Call:
## lm(formula = time ~ id, data = data_aggr4)
## 
## Residuals:
##      1      2      3      4 
## -14.50  29.34  27.50 -42.34 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)  
## (Intercept)   680.50     117.56   5.788   0.0286 *
## id             72.00      33.28   2.163   0.1630  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 42.55 on 2 degrees of freedom
## Multiple R-squared:  0.7006, Adjusted R-squared:  0.5509 
## F-statistic:  4.68 on 1 and 2 DF,  p-value: 0.163

The adjusted \(R^2\) value is 0.55 and the p value of the model is above 0.05, suggesting it does not follow Fitts’ law very well. The visual inspecting of the graph suggest participants took longer than expected to select the target with the lowest index of difficulty. Further investigations would be required to understand why.

4.4 Conclusion

We can only partially validate H3 from the results of the experiment.