Michael Crawley - Introduction to Statistics using R

 

p.1

The truth is that there is no substitute for experience: the way to know what to do is to have done it properly lots of times before. 

 

p.1~2

It is essential, therefore, tha tyou know:

 

- which of your variables is the response variable?

- which are the explanatory variables?

- are the explanatory variables continuous or categorical, or a mixture of both?

- what kind of response variable have you got - its it a continuous measurement, a count, a proportion, a time-at-death, or a category?

 

These simple keys will then lead you to appropriate statistical method:

 

1. The explanatory variables (pick one of the rows):

    (a) All explanatory variables continuous : Regression

    (b) All explanatory variables categorical : Analysis of Variance (ANOVA)

    (c) Some explanatory variables continous some categorical : Analysis of Covariance (ANCOVA)

 

2. The response variable (pick one of the rows):

    (a) Continuous : Regression, ANOVA or ANCOVA

    (b) Proportion : Logistic Regression

    (c) Count : Log linear models

    (d) Binary : Binary logistic analysis

    (e) Time at death : Survival Analysis

 

p.2

The key concept is the amount of variation that we would expect to occur by chance alone, when nothing scientifically interesting was going on. If we measure bigger differences than we would expect by  chance, we say that the result is statistically significant. If we measure no more variation than we might reasonably expect to occur by chance alone, then we say that our result is not statistically significant. It is important to understand that this is not to say that the result is not important. Non-significant differences in human life span between two drug treatments may be massively important (especially if you are the patient involved). Non-significant is not the same as 'not different'. The lack of significance may be due simply to the fact our replication is too low.

 

On the other hand, when nothing really is going on, then we want to know this. It makes life much simpler if we can be reasonably sure that there is no relationship between y and x. Some students think that 'the only good result is a significant result'. They feel that their study has somehow failed if it shows that 'A has no significant effect on B'. This is an understandable failing of human nature, but it is not good science. The point is that we want to know the truth, one way or the other. We should try not to care too much about the way things out. 

 

p.3

Karl Popper was the first to point out that a good hypothesis was one that was capable of rejection. He argued that a good hypothesis is a falsifiable hypothesis. Consider the following two assertions:

 

A: there are vultures in the local park.

B: there are no vultures in the local park.

 

Both involve the same essential idea, but one is refutable and the other is not. Ask yourself how you would refute option A. You go out into the park and you look for vultures. But you do not see any. Of course, this does not mean that there are none. They could have seen you coming, and hidden behind you. No matter how long or how hard you look, you cannot refute the hypothesis. All you can say is 'I went out and I didn't see any vultures'. One of the most important scientific notions is that absence of evidence is not evidence of absence.

 

Option B is fundamentally different. You reject hypothesis B the first time you see a vulture in the park. Until the time that you do see your first vulture in the park, you work on the assumption that the hypothesis is true. But if you see a vulture, the hypothesis is clearly false, so you reject it.

 

p.3~4

The p-value is not the probability that the null hypothesis is true, although you will often hear people saying this. In fact, p-values are calculated on the assumption that the null hypothesis is true. It is correct to say that p-values have to do with the plausibility of the null hypothesis., but in a rather subtle way.

 

As you will see later, we typically base our hypothesis testing on what are known as test statistics: you may have heard of some of these already (Student's t, Fisher's F and Pearson's chi-squared, for instance): p-values are about the size of the test statistic, or a value more extreme than this, could have occurred by chance when the null hypothesis is true. Big values of the test statistic indicate that the null hypothesis is unlikely to be true. For sufficiently large values of the test statistic, we reject the null hypothesis and accept the alternative hypothesis.

 

Note also that saying 'we do not reject the null hypothesis' and 'the null hypothesis is true' are two quite different things. For instance, we may have failed to reject a false null hypothesis because our sample size was too low, or because our measurement error was too large. Thus, p-values are interesting, but they do not tell the whole stofy: effect sizes and sample sizes are equally important in drawing conclusions. The modern practice is to state the p value rather than just to say 'we reject the null hypothesis'. That way, the reader can form their own judgement about the effect size and its associated uncertainty.

 

p.5

The object is to determine the values of the parameters in a specific model that lead to the best fit of the model to the data. The data are sacrosanct, and they tell us what actually happened under a given set of circumstances. It is a common mistake to say 'the data were fitted to the model' as if the data were something flexible, and we had a clear picture of the structure of the model. On the contrary, what we are looking for is the minimal adequate model to describe the data. The model is fitted to data, not the other way around. The best model is the model that produces the least unexplained variation (the minimal residual deviance), subject to the constraint that the parameters in the model should all be statistically significant.

 

p.7

It does not matter very much if you cannot do your own advanced statistical analysis. If your experiment is properly designed, you will often be able to find somebody to help you with the stats. But if your experiment is not properly designed, or not thoroughly randomized, or lacking adequate controls, then no matter how good you are at stats, some (or possibly even all) of your experimental effort will have been wasted. No amount of high-powered statistical analysis can turn a bad experiment into a good one. 

 

p.9~10

Most statisticians work with alpha = 0.05 and beta = 0.2. Now the power of a test is defined 1 - beta = 0. under the standard assumptions. This is used to calculate the sample sizes necessary to detect a specified difference when the error variance is known (or can be guessed at).