Difference Between Statistical significance and Practical significance

Introduction

Statistical significance refers to lesser chance of sampling error affecting the mean-differences. Statistical significance comes from within the data used and confidence of the analyst in the result. In other words, statistical significance reflects the low probability that an observed data has been arrived at by chance.

For determining statistical significance, significance level is used. P-value is the probability that the test statistic being computed would acquire a value equal to or less than the fixed value or significant level called ‘α’. If the P-value is equal to or less than α, then the data is said to be statistically significant at level α. Thus if α=.05 then the result is significant at P<.05.

Differences

i. Statistical significance hints that a probability of relationship between two variables exists, where s practical significance implies existence of relationship between variables and real world scenario.

ii. Statistical significance is mathematical and sample-size centric. Practical significance arises out of applicability of the result in decision making. Practical significance is more subjective and depends upon external factors like cost, time, objectives, etc. apart from statistical significance.

The above differences may be understood in the light of an example. In a survey arranged by school-authority of a district on participation in sports by school-going boys and girls, it is found that 60% of boys and 57% of girls participate in outdoor sports. Thus the survey shows a 3% difference between school-going boy-participants and girl-participants in outdoor sports. Now the point is how much significance this 3% difference has statistically as well as practically. Statistical significance of this 3% depends upon the size of data used in determining the percentage of boys and girls participate in sports. If a sufficiently big sample size is used then the difference is statistically significant, and if a very small sample size is used then the difference is statistically insignificant. Thus bigger the sample size more is the statistical significance of a computed figure.

On the other hand practical significance of this 3% difference arises if decision is made or action is taken or needs to be taken on the basis of this 3% difference. If cost permits, the authority may consider promoting girl students participation in sports in order to bring about more gender parity in outdoor sports. In this case the 3% difference though small, may be practically significant.

We can think of another scenario, where the difference is 40%. If the sample size is big enough then this 40% difference is significant both statistically as well as practically, as 40% is too big a difference to warrant for an immediate action on the part of the authority to fix the huge imbalance. However if the sample size is small enough, then the 40% difference is neither statistically nor practically significant though the figure 40% is big enough.

Summary:

i. Statistical significance refers to the unlikelihood that the result is obtained by chance, i.e., probability of relationship between two variables exists. Practical significance refers to the relationship between the variables and the real world situation.
ii. Statistical significance depends upon the sample size, practical significance depends upon external factors like cost, time, objective, etc.
iii. Statistical significance does not guarantee practical significance, but to be practically significant, a data must be statistically significant.

References:

1. Practical Significance vs Statistical Significance: available at http://www.moresteam.com

2. Practical Significance versus Statistical Significance: available at http://atrium.lib.uogelph

Sharing is caring!