Prior Probability

Understanding Prior probability and its study is important as it helps us to combine the new information with the past data to make better decisions and improve accuracy. Prior probability forms as a foundation of Bayesian Theorem which allows us to integrate the new data with the old data to improve the estimation accuracy.

In this article, we will take a deeper look into prior probability, its applications in various fields and some examples for understanding it better.

What is Prior Probability?

Prior probability is defined as the initial assessment or the likelihood of the event or an outcome before any new data is considered. In simple words, it tells us about what we know based on previous knowledge or experience.

For example, let's take a situation in which we have data which describes that in a month 30% of the days are rainy, then the prior probability of rain on any random day of that month is 30%. Prior probability plays a significant role in Bayesian Statistics, which allows the prior probability to combine with the new data to produce new understandings that would eventually help in improving the accuracy.

Prior probability is used in various fields like machine learning, and medical diagnosis where the decisions can be taken from the data available. Also, prior probability allows us to change or update our beliefs as and when the new data is made available.

Significance of Prior Probability in Bayesian Statistics

In Bayesian statistics, prior probability plays an important role because it represents the initial beliefs based on the available or past data before any future data is considered this makes the foundation of the Bayesian inference. Bayesian methods combine the prior probability with the likelihood of the observed data, that in result produces the posterior probability that reflects the updated knowledge.

👁 Prior-Probability

This approach of the Bayesian methods helps in improving the overall estimation accuracy with the limited data and prevents overfitting. In this way prior probability enables adaptive learning as it updates continuously as the new data arrives.

The expression for the Bayesian Theorem is defined as follows:

P(A|B) = P(A)P(B|A) / ∑ P(A_i)P(B|A_i)

where,

• P(A|B) is posterior probability which is defined as the probability of event A given that event B has occurred.
• P(B|A) is likelihood which is defined as the probability of event B given that event A has occurred.
• P(A) is prior probability which defines the initial probability of event A before any event B is considered.
• P(B) is marginal likelihood which is defined as the total probability of event B under all circumstances.

It can be calculated as , if there exists multiple mutually exclusive events .

Classification of Priors

In Bayesian statistics, priors are classified based on the amount of information content in it. Below discussed are some of the types of priors:

• Informative Priors
• Weakly Informative Priors
• Non-informative Priors
• Improper Priors

Informative Priors

These kind of priors have detailed knowledge or are decided from the expert opinions. These priors are chosen based on the past or historical data, or under expert guidance. These priors have a significant impact on the posterior distribution. These kind of priors are useful only when we have strong information that can drive the analysis.

Weakly Informative Priors

These kind of priors are in-between informative and non-informative priors. They have some prior knowledge but cannot eventually influence the posterior distribution. These priors provide some regularization and also prevents from noise fitting but it still allows the data to influence the posterior distribution. Normal prior with the high variance can be considered as weakly informative priors.

Non-informative Priors

Non informative priors are also known as uninformative priors. These kind of priors have very little or practically no prior knowledge about the parameter. They have a minimum influence to posterior distribution, allowing the data to primarily drive the inference. Uniform prior is an example of non-informative prior that assigns equal probabilities to all the possible outcomes, reflecting the lack of prior knowledge.

Improper Priors

These are non-informative priors but it does not integrate over one parameter space, i.e. they do not have a valid probability distribution. These kind of priors are still used in Bayesian statistics as long as the resulting posterior distribution remains proper which integrates over one parameter space. These priors are generally used for parameters with unbounded ranges such as it uses 1/θ over a parameter θ.

Applications of Prior Probability

Various application of Prior Probability includes:

• Medical Diagnosis: Prior probabilities are used in medical diagnosis in order to determine the likelihood of the disease before testing.
• Spam Filtering: Prior probabilities are used in email filtering to classify an email as a spam or not spam based on the pervious historical data.
• Financial Forecasting: Investors consider the prior probability in order to assess the risk of the investments before considering market trends and data.
• Machine Learning: In the field of Machine Learning, prior probabilities are integrated with various algorithms in order to improve model performance and its accuracy.

Solved Problems on Prior Probability

Problem 1: Given that 2% of the emails are spam. What is the prior probability of an email being spam?

Solution:

As it is given that 2% of the emails are spam, so the prior probability of an email being spam is given below:

∴ P(Spam) = 2% = 0.02

Problem 2: Consider a medical scenario where the historical data shows that 1% of the total population has the certain disease, then find the prior probability of the patient having certain disease.

Solution:

As it is given that 1% of the total population has the certain disease, so the prior probability of the patient having certain disease is expressed below:

∴ P(Disease) = 1% = 0.01

Problem 3: Imagine the situation where the bank wants to assess the risk of a borrower defaulting on a loan. Historically, 5% of borrowers default on their loans. The bank uses a credit scoring system that correctly predicts defaults with 85% sensitivity (true positive rate) and correctly predicts non-defaults with 90% specificity (true negative rate). If a borrower receives a positive result on this credit scoring system indicating potential default then find what is the probability that the borrower will actually default on the loan?

Solution:

Given,

• Prior probability that a borrower defaults is 5%
• Sensitivity of the credit scoring system is 85% which means that it correctly identifies 85% of those who will default
• Specificity is 90% which means that it correctly identifies 90% of those who will not default

Using these values, we calculate the overall probability of positive result from the credit scoring system:

∴ P(Positive) = P(Positive | Default) . P(Default) + P(Positive | No Default) . P(No\ Default)

Now, substituting the values given, we get

∴ P(Positive) = (0.85 x 0.05) + ((1-0.9) x (1-0.05))

∴ P(Positive) = (0.85 x 0.05) + (0.1 x 0.95)

∴ P(Positive) = 0.0425 + 0.095

∴ P(Positive) = 0.1375

Now, using Bayes Theorem we find the posterior probability

Substituting the values in the above expression

∴P(Default∣Positive) = (0.85×0.05)/0.1375

∴P(Default∣Positive) = 0.0425/0.1375

∴ P(Default | Positive) = 0.3091

Therefore, after a positive result from the credit scoring system, the probability that the borrower will actually default on the loan is approximately 30.91%.