Ideally, we would like to know whether the differences in these measures between any of the runs are statistically significant. Such differences would indicate that evolution might genuinely be treading a different path, for no other reason than the different seed used for the random number generator when the runs commenced. The choice of a statistical test for this task was not immediately obvious. We wished to avoid parametric tests, as we did not want to make assumptions about the population parameters (for example, there is no reason to suspect that any of the measures we are looking at are normally distributed across all possible evolutionary runs).
We therefore chose a non-parametric method--a randomisation version of the paired sample t test (described in [Cohen 95]). For each measure of interest, this test will tell us, for each run, which other runs produced significantly different results. The test can indicate whether two samples are related without any reference to population parameters. The procedure used was as follows: