CS 854: Advanced Topics in Computer Systems

Experimental Performance Evaluation

Fall 2022

 

 

 

 

 

Home Video Logistics Schedule
Projects Resources Integrity

 

 

 

Statistics and Hypothesis Testing

If you already have a good understanding of all of the concepts outlined below you can skip doing this. Where you start and how much time you spend will depend on how much background you have in statistics. E.g., I would guess that all of you can skip the parts about mean, median and mode.

I'm making use of quit a bit of material that is available on Khan Academy taken from their treatment of
AP (Advanced Placement) College Statistics

Part I: Statistics

For each of the topics below the high level topic (numbered bullet point) contains a bunch of subtopics. The high level topic and link provided so you can get access to more information and/or background you might need to cover the subtopic I've listed under each topic. The subtopics are denoted with letters of the alphabet.

You should be sure that you already know or that you cover at least all of the subtopics listed.

1. Summarizing quantatitive data
   1. Interquartile range (IQR) [Duration 6:11]
2. Introduction to confidence intervals
   1. Confidence intervals and margin of error [Duration 11:44]
   2. Conditions for valid confidence intervals [Duration 6:43]
   3. Introduction to t statistics [Duration 4:25]
3. Modeling data distributions
   1. Calculating percentile [Duration 3:37]
   2. Analyzing a cumulative relative frequency graph [Duration 5:54]
4. Bootstrapping
   1. Wikipedia - Please read Sections 1, 2 and 3.
   2. Bootstrap confidence intervals (by Jeremy Orloff and Jonathan Bloom)

If you would prefer to read some notes (rather than watching videos) here is a link to some MIT Open Courseware which seems quite good.
It's an inverted class which means that the course notes are quite good and useful.
Introduction to Probability and Statistics
You can find more readings by clicking on the "Readings" link on the left side of the home page.

Part II: Hypothesis Testing

1. Significance tests (hypothesis testing)
   1. Idea behind hypothesis testing [Duration 7:48]
   2. Examples of null and alternative hypotheses [Duration 5:41]
   3. Comparing P-values to different significance levels [Duration: 4:40]
   4. Estimating a P-value from a simulation [Duration: 4:44]
   5. Using P-values to make conclusions [Exercise]
2. Another source for significance tests
   1. Null Hypothesis Significance Testing I
   2. Null Hypothesis Significance Testing II
   3. Null Hypothesis Significance Testing III

 

 

 

Home Video Logistics Schedule
Projects Resources Integrity

 

 

 

Last modified: Fri Oct 21 15:47:23 EDT 2022