Significance testing recap

From the lectures you may recall the concept of a hypothesis. Often in quantitative social science research, you will deal with a null hypothesis (H0) and an alternative hypothesis (H1). The way we do hypothesis testing in most of social science research is that we assume that the null hypothesis is true and that we look at the probability of our data under the null hypothesis. If this probability is very small, we reject the null hypothesis and at least temporarily accept the alternative hypothesis. Usually we take a significance level, denoted by the \(\alpha\) parameter, of 0.05 or 0.01. We then compare the p value that we find to the significance level against which we testing.

Given that we have tested against a significance level of 0.05 and found a p value of 0.03, do we accept the null hypothesis or the alternative hypothesis? What does a p value of 0.03 mean?