Empirical Validation of RE and SE Methodss

e.g., Hypothesis: using method M allows the production of better SW than
using method N

first must decide what is "better SW"


extremes

Controlled Experiment				Case Study
in lab						in industry
e.g.						e.g.
20 groups apply M				get an industrial group to
20 groups apply N				apply M to their next
to problem p to produce SW s			problem P to produce SW S
						as group in order
control						compare what happens now
make groups as indistinguishable as possible	to what happened in the past
so that ONLY possible cause of difference in	possibly using N
SW produced by group is the difference in
M and N.

internal validity				external validity

+s & -s						+s & -s

+	statistical validity			+	realism
-	toyness					-	no statistical strength

in the middle
Mining an organization's project database for data about lots of projects
to answer questions
-	no control

+	in lieu of control, might be able to find enough data to isolate an
	issue
+	realism
+	might get statistical validity and strength if there are enough
	data points for any issue

Be careful of drawing invalid conclusions from experiment and data
Look at Sobel & Clarkson paper.