SciPy - Statistical Significance Tests

Statistical significance tests helps find out whether results we see in data are real or just happened by chance. They are used to check if differences or patterns in data are meaningful. SciPy’s scipy.stats module helps to perform these tests using simple Python functions. These tests are useful in research, experiments and data analysis.

Before running any test, it's essential to understand following key terms:

• Null Hypothesis (H₀): There is no effect or no difference.
• Alternate Hypothesis (H₁): There is an effect or a difference.
• p-value: probability that observed result is due to chance.
• alpha (α): threshold (usually 0.05), maximum acceptable risk of rejecting H₀.

If p-value ≤ α -> reject H₀ (there is a significant effect)
If p-value > α -> fail to reject H₀ (no significant effect found)

Commonly Used Statistical Significance Tests

Some of widely used statistical significance tests are listed below. Each test serves a specific purpose in analyzing data and checking for meaningful patterns or differences.

1. Hypothesis Test

Hypothesis testing is a way to make decisions or predictions using data. It helps check if a certain assumption (called a hypothesis) about a population is likely to be true based on sample data.

Example: This example tests whether average weight of a product is significantly different from 50 grams using sample data.

Output

T-statistic: -0.0882
P-value: 0.9339
The null hypothesis is not rejected. No significant difference from 50.

Explanation:

• A list of sample weights is tested against value 50 using a one-sample t-test with stats.ttest_1samp() which calculates how different sample mean is from 50.
• Then it checks p value against alpha = 0.05 and prints whether difference is statistically significant or not based on that.

2. T-Test

T-test is a type of hypothesis test used to compare averages (means) of two groups. It helps find out if difference between two group means is real or just happened by chance. The t-test works best when data is small and follows a shape similar to a normal distribution (called t-distribution).

Example: A restaurant wants to test whether adding chipotle sauce to a dish has affected its average sales. Two sets of sales data are collected: one with chipotle and one without.

Output

T-statistic: 0.1846
P-value: 0.8552
The null hypothesis is not rejected. Chipotle sauce does not have a significant effect on sales.

Explanation:

• Code uses a two-sample t-test (ttest_ind) to compare sales with and without chipotle sauce to see if their averages are significantly different.
• It checks p-value against alpha = 0.05 and prints whether difference in sales is statistically significant or not.

3. Kolmogorov Smirnoff Test

Kolmogorov–Smirnov test is a statistical test used to check whether a dataset follows a specific distribution. It is often used to test if data is normally distributed or uniformly distributed or to compare two distributions.

Note: This test is valid only for continuous distributions.

Example: In this example, a dataset of 1000 values is randomly generated from a uniform distribution. The one sample KS test is used to check whether data really follows a uniform distribution.

Output

K–S Test Statistic: 0.0277
P-value: 0.4194
The null hypothesis is not rejected.
Conclusion: The data likely follows a uniform distribution.

Explanation:

• The code generates 1000 random values from a uniform distribution, then uses kstest() to check if the data actually follows a uniform distribution.
• It compares p value with alpha = 0.05 to decide whether to reject or accept null hypothesis and prints result.

4. Normality Tests (Skewness and Kurtosis)

Normality tests help check whether data follows a normal (bell-shaped) distribution. Two common indicators are:

Skewness: Measures symmetry of the distribution.

• 0 means perfectly symmetrical
• Negative = left-skewed, Positive = right-skewed

Kurtosis: Measures how heavy or light the tails are compared to normal.

• Normal distribution has kurtosis ≈ 3
• Higher = heavy tails, Lower = light tails

Example: This example checks if the given data is normally distributed by calculating skewness and kurtosis using scipy.stats.describe().

Output

Skewness: -0.2234
Kurtosis: 3.0771

Explanation:

• describe() returns summary statistics including skewness and (excess) kurtosis.
• We add 3 to kurtosis to get standard kurtosis which is compared against 3.
• If skewness ≈ 0 and kurtosis ≈ 3, data likely follows a normal distribution.

Related Article

• Linear Algebra - SciPy linalg
• Differention Equation with ODEINT Function
• SciPy - Integration
• SciPy Interpolation
• Spatial Distance Matrix