Hypothesis Testing

What is hypothesis-testing?

Hypothesis testing is when an assumption is made and evidence is collected in favor of the assumption. In Statistics, hypothesis-testing is performed using sample data to draw inferences regarding a given population parameter or population probability distribution. An analyst decides which hypothesis-testing methodology to be catered to on the basis of the nature of data used and the analysis reason. 

The sample data used while testing a hypothesis might come from a large population or a process of data generation. The term ‘population’ is used to refer to this sample data irrespective of its source.  

The stats library of Python allows you to perform hypothesis-testing. There is a Python package, named ht, exclusively available for carrying out hypothesis tests in Machine Learning. 

Hypothesis-Testing Terminology

There are some commonly used terms in hypothesis testing that one must know before conducting a hypothesis test. Those commonly used terms are:

Null Hypothesis: An assertion concerned with a population that you want to test. It is called the ‘null hypothesis in the sense that it assumes the contrast of our belief.  For instance, if we are testing the presence of a specific ingredient in hair oil, our null hypothesis would state the absence of that ingredient in the oil. It is denoted by H0.

Alternative Hypothesis: The opposite of a null hypothesis is the alternative hypothesis in a test. Evidence for approval of this hypothesis is to be found through a hypothesis test. For the example mentioned above of hair oil, the alternative hypothesis would state the ingredient’s presence in the oil. It is denoted by HA.

Null and alternative hypotheses are always mutually exclusive of each other. 

Test Statistic: A random variable computed from a random population sample is known as a test statistic. It is used in a hypothesis test to decide the rejection of the null hypothesis. For example, the test statistic for a t-test is t-statistic. 

P-value: The probability of getting a value of the test static at least as extreme as the computed value in the null hypothesis of the hypothesis test is known as p-value. The smaller the p-value, the stronger is the evidence in support of the alternative hypothesis. 

Significance Level: The significance level represents the probability of rejecting the null hypothesis when it is true. It is denoted by alpha (α). For example, a significance level of 0.01 marks a 1% risk of drawing an inference even though there is no actual difference.

Steps of Hypothesis-Testing

The entire process of testing a hypothesis is carried out in the following order of events:

1. State the Null and Alternative Hypotheses: The very first step that an analyst takes is to specify the null and alternative hypothesis for the corresponding hypothesis test.

2. Formulate an Analysis Plan: The second step is to map out an analysis plan depending on the nature of the data and the reason behind the analysis.

3. Analyze the Sample Data: In the next step, the analyst carries out the formulated plan to analyze the sample data based on the determined hypothesis.

4. Determine plausibility of Null Hypothesis: The last step that comes is to decide whether to reject the null hypothesis or state the plausibility of the null hypothesis for the given sample data. The decision made is determined by the p-value and significance value of the hypothesis test. 

Need of Hypothesis-Testing in Machine Learning

All the discoveries made based on hypotheses need to pass the hypothesis test. The result of a hypothesis test is used to assert the statistical significance of a finding. As mentioned in the hypothesis-testing introduction, the result of a hypothesis test tells us whether the statement of the null hypothesis or the statement of the alternative hypothesis best reflects the sample data of the population parameter. 

In Machine Learning, when we train an ML model, we must be confident about the population itself first. Thus, to analyze the nature and trend of a population, we need to conduct a hypothesis test as it tells whether a speculated hypothesis about the population is true. 

Basis of Hypothesis

The entire hypothesis revolves around the basics of normalization and standard normalization. A variable is normally distributed when its curve is a normal bell-shaped curve representing the equal mean, median, and mode. On the other hand, we say that a variable has standard normal distribution when its mean 0 and the standard deviation is 1. 

Types of Hypothesis Test

Four types of hypothesis tests can be performed. These types are:

• Z-test: This test concludes whether two population means are different for known variances and a large volume of sample data. The test static, in this case, is z-static and the formula used is:

Z= X- µₒs

Here, Z = Z-test

X = sample average

µₒ = mean

s = standard deviation

• T-test (Student t-test): Another hypothesis-testing tool differentiates between the means of two groups when their variances are not given. The test static, in this case, is t-static. It is further categorized into three kinds which are as follows:

• One sample t-test

• Independent two-sample t-test,

• Paired sample t-test

t-tests have common applications in the field of Data Science, CS Research, and ML.

• ANOVA test: ANOVA stands for Analysis of Variance. This test reports a statistically significant result when one of the groups differs significantly from the overall mean of the groups. It can be one-way, two-way, or n-way. The test static here is F-static.

• Chi-square test: This hypothesis test is used to test the independence of two variables. It is sensitive to the size of the test sample. A chi-square test holds a comparison between expectations and model results. The test static in this type of test is chi-squared static. 

Alpha Risk & Beta Risk

In statistical hypothesis-testing, there might be a risk of rejecting the null hypothesis when it is true. This risk is referred to as alpha risk. The other term for alpha risks is Type-I error and false positive. The amount of alpha risk is mainly governed by the size of the sample used. A large sample size corresponds to low alpha risk. 

In contrast to alpha risk, there is a beta risk, commonly known as Type-II error. It is the likelihood of approving the null hypothesis when it is actually false, i.e., it is a false negative. 

Real-World Examples of Hypothesis Tests

The real-world examples in various fields that involve hypothesis-testing are given below:

Example-1: Hypothesis tests are done in different clinical trials to check the effectiveness of a new drug for curing a certain symptom. 

Example-2: To test the presence of a specific ingredient in any cosmetic item, food item, etc. 

Example-3: A hypothesis test can be used to determine whether a new type of manure will increase soil fertility. 

Example-4: Hypothesis tests are useful to find evidence in favor of a certain marketing strategy by analyzing the business growth.

Example-5: A hypothesis test can be run to determine whether an ML model fulfills the requirement in concern.

Likewise, endless real-world examples can be cited to show the implementation of hypothesis tests.

About the Course – Hypothesis Testing

The planned curriculum of this course will highly benefit the enthusiastic learners seeking opportunities in the field of Machine Learning. The hypothesis-testing course of Great Learning will help the learners to build a strong base in hypothesis-testing. Taking up this course enables you to get an insight into of Machine Learning Analytics job. Another advantage of this online course is that it is going to cost you nothing except for your time and dedication. 

The content of this online course is well structured to give a boost to beginners. It starts with a descriptive hypothesis-testing introduction and ends with a quiz to test your skills. On qualifying for the quiz, you can claim your free course completion certificate. Hence, along with learning new skills, you get a certificate to upgrade your CV and LinkedIn profile. This course contains two hours of video content presenting lectures on the basics of hypothesis-testing, including a description of student t-test and paired tests, for example.