Philosophy and Data Science – Thinking Deeply About Data
Part 2: Epistemology
After reading this article, I hope that you will have a practical understanding of how thousands of years of deep thinking about knowledge applies to your daily work as a data scientist.
This is the second installment of a series on how concepts of philosophy have helped me in my work as a data scientist. The first article is on determinism (link); a specific metaphysical theory. This article will cover multiple schools of thought that fall under the philosophical field called epistemology.
Epistemology is the study of what we can know and how we can know it. It is the study of knowledge itself. This ties very nicely with data science, since we are trying to gain knowledge from data!
Epistemology is the study of what we can know and how we can know it. It is the study of knowledge itself.
Here is what we will cover:
- Inductive vs. deductive reasoning
- Skepticism
- Pragmatism
Inductive vs. deductive reasoning
Reasoning is how we rationalize and defend knowledge. It is the reason we know something. There are multiple types of reasoning, but I think the most common (and the most applicable) are deductive and inductive reasoning.
Deductive reasoning
Deductive reasoning creates a conclusion that is the logical result of the premises. In deductive reasoning, the premises and the arguments create a closed system. If the premises are correct and the logic is free of fallacies, we have an airtight reason to believe the knowledge we propose ( the ‘therefore’ or ‘then’ portion of the argument).
In other words, if we assume the premises to be true, the conclusion of a deductive argument is knowledge! Of course, the battle ground for deductive arguments (assuming no logical errors) is whether or not the premises are actually true.
Below is a very simple example of deductive reasoning:
Notice that if we accept that all ducks have wings and that Huey is a duck, we must accept that Huey has wings. To do otherwise would be logically inconsistent.
As data scientists, we use deductive reasoning when discussing the assumptions necessary for our model to be valid. After making the deductive argument, we typically go on to show evidence that supports the premises we propose. The example below may feel familiar to you:
We propose the assumptions that are necessary for our predictive models or analysis to be valid and claim that if they are all met, our results are valid as well. We then work to demonstrate that we have reason to believe that the assumptions are comprehensive and correct.
Inductive reasoning
Deductive reasoning happens in a sterile space where we create our premises. But inductive reasoning happens in the messy world!
Inductive reasoning also involves premises and conclusions, but the premises are often evidence instead of logical propositions. Inductive reasoning is when we make conclusions based on our observations of the world.
An example of inductive reasoning is: for all observed history, the sun has come up in the morning, therefore it will come up tomorrow. A defining difference between deductive and inductive reasoning is that with inductive reasoning, we can be wrong without creating a logical contradiction.
A defining difference between deductive and inductive reasoning is that with inductive reasoning, we can be wrong without creating a logical contradiction.
The difference between induction and deduction is subtle here. The inductive premise is ‘the sun has come up every morning’ a similar, but deductive premise would be ‘the sun comes up every morning.’
Just because the sun has risen for as long as we have observed, doesn’t mean that the sun has to rise tomorrow.
Hopefully, you are able to see that sure knowledge becomes more challenging under deductive reasoning; since our conclusion does not have to follow our premises. This problem is called ‘the problem of induction,’ which we will discuss further in the skepticism section.
Machine learning is inductive. Meaning that from available information, we induce relationships/knowledge of the world around us. We have observational reasons to make specific predictions, but our predictions being wrong does not create logical contradictions.
The fact that machine learning models use inductive reasoning leaves them susceptible ‘the problem of induction.’ The root of this problem is the fact that inductive premises do not invariably lead to the argument’s conclusion. This can cause legitimate concern regarding the validity of the knowledge we gain. Just because we’ve seen something happen many times before doesn’t mean we will necessarily see it again.
A big proponent to the philosophical school of skepticism is the problem of induction. Skepticism would state that all ML models are useless because they could all be wrong. However, despite the challenges of inductive reasoning, we know that our models are often very helpful and practical — this view point lines up with the philosophy of pragmatism. We will first discuss skepticism and then pragmatism!
Skepticism
Skepticism is a philosophical stance characterized by doubt or suspension regarding knowledge. When it comes to epistemology, one of the more compelling arguments for skepticism comes from the previously mentioned problem of induction. I’ll talk a little bit more about the problem of induction, then I will move on to discuss how a skeptic perspective can be useful in the field of data science.
The problem of induction
Nissam Taleb’s books ‘Fooled by Randomness’ and ‘The Black Swan’ both discuss the problem of induction at length. The later book’s name is an homage to the problem (if this problem seems interesting, I recommend both books to you!).
A common illustration of the problem of induction is the ‘black swan’ example. The example goes as follows: if you see one thousand swans and all of them are white, you might conclude that all swans are white. Your world would be flipped upside down when you see your first black swan!
Taleb discusses examples of investment traders who make trading decisions based on what has happened rather than what could happen. The result is that they have success for some time, until they encounter the metaphorical ‘black swan,’ an unprecedented or extremely rare event that erases all of their gains and causes huge losses. His main point is to avoid the problem of induction by taking into account all things the could happen rather than just what has happened.
The problem of induction can logically lead to skepticism:
How can skepticism make you a better data scientist?
While skepticism can be a crippling philosophy that leads to inaction because of overwhelming doubt, skepticism in small doses can help you be smarter in your data science work. We will talk about pragmatism in the next section – which is an answer to the paralyzing side effect of skepticism.
But first, here are the ways I think a skeptic perspective can help improve the quality of your data science work.
Skepticism:
- Prevents you from believing everything you see – Because of skepticism, we rigorously search for evidence of our models’ validity. We look at out-of-sample and out-of-time performance instead of training performance. If our error is suspiciously low, we look for leakage or other ‘unfair’ advantages that our model is getting in the training and validation process that it would not have in production. Because of this perspective, we make more robust and defensible models and we have a better idea of how our models will perform when in use.
- Helps ensure you communicate model limits to users – When you have the problem of induction in mind, you are a lot more likely to clearly communicate a model’s limits to its users. You are more likely to use phrases like ‘our model is valid if the future is similar to the past’ or ‘our model will likely make good predictions on similar input ranges that we have seen in the past.’ This is helpful because it communicates to the users when they should use the model and when they should be wary of a model’s outputs. It also helps preserve your reputation by carefully setting expectations. If something crazy happens and your model’s predictions become useless, the model users won’t be completely thrown off and your credibility will be preserved!
- Helps manage the risks associated to your model having misses – In other words, some reasonable skepticism can prevent you or your business partners from going ‘all in’ on your model’s predictions. You understand that there are possible future circumstances that would cause your model to make bad predictions, and you take that into account when considering how to use your model in the overall strategy. When the model suggests big changes, you may start with smaller changes in the direction of the model’s recommendations, understanding that the model may not fully capture the knowledge you want it to.
- Keeps you from taking everything your model says as true – Sometimes models give counter-intuitive results. A skeptic perspective will cause you to doubt this ‘knowledge’ and investigate further. Without a skeptic perspective, you might say ‘that is what the data say, it must be right!’ I’ve been in this situation multiple times in my career. A couple of years ago, I was working on a project where my company discounted the price of a product at a time when the demand for the product dropped unrelated to the discount. The model I built on the data suggested that the lower the product price, the less we would sell! The model was picking up on the unrelated trend. I shutter to think how a meeting with my boss would’ve gone if I didn’t have a skeptical point of view that caused me to dig further into the illogical result and discover the true cause!
Pragmatism
Pragmatism is the epistemological school of thought that counters skepticism by suggesting we have the ‘right to believe’ things that we have evidence for. The justification for this ‘right’ comes from the perspective that facts only become useful (knowledge) if we use them. So, if we are paralyzed by skepticism, we have no knowledge because we are making no use of the facts that we have. Pragmatism emphasizes the practical consequences of our beliefs or knowledge.
Pragmatism emphasizes the practical consequences of our beliefs or knowledge.
William James (a leader in pragmatic thinking) famously asserted that facts are not truth, facts just are. When we find a useful way to use facts, we have truth and knowledge.
James uses a person lost in the woods to illustrate the pragmatic perspective. There is a person lost in the woods that finds a trail. They can be filled with skepticism because they don’t know that the trail leads to civilization, so they sit on the ground and starve. Or they can be pragmatic and believe that the trail leads somewhere, and follow it to civilization.
It is important to note here, that pragmatism does not propose that we use random or unfounded beliefs to make decisions. Pragmatism requires that beliefs are backed by sufficient evidence.
Spectrum of confidence in knowledge
I see our confidence in knowledge as being on a spectrum. From total skepticism – no confidence in any knowledge, to total dogmatism – complete confidence in knowledge, even in the absence of evidence.
In the data science application, under total skepticism we believe that our machine learning models or data analysis tell us nothing useful about the world, because there is always a chance that we could be wrong. Under total dogmatism – we are completely confident that the our model is correct, and the predictions from the model will be accurate even if we don’t have compelling evidence.
On the extreme left (total skepticism), we are disabled with doubt that prevents us from making any decisions. On the extreme right (total dogmatism), we are supremely over confident which leads us to over bet on our predictions. We should be somewhere in the middle on the ‘confidence in knowledge spectrum.’ Meaning we should believe things that can be useful only when the evidence suggests it. We shouldn’t be more pessimistic or optimistic than the evidence permits. I believe that this balance on the spectrum is pragmatism.
How can pragmatism make you a better data scientist?
Pragmatism:
- Helps you focus on results — Pragmatism is all about the impact of our knowledge or beliefs. If a model gives ‘interesting’ results that are not useful in solving a problem the model is not useful. Pragmatism can help us stay aligned with our business partners by keeping us focused on results-driven data science.
- Keeps you asking ‘so what?’ – Has someone ever told you an interesting fact and you thought or said ‘so what?’ I’ve definitely been on both sides of that question! If we approach data science with a pragmatic perspective, we will always have an answer to that question, because we will believe that not having an answer to it means that we don’t have anything valuable to share.
- Helps avoid ‘analysis paralysis’ – I’m pretty sure every data professional has been here before. ‘Analysis paralysis’ is the state where you continue to gather information that doesn’t help you come to a decision. The end result is that you have a mountain of facts, but no action. This is the bane of a pragmatist! A healthy pragmatic perspective will help you only gather information that contributes to making a decision – thus avoiding ‘analysis paralysis’.
- Gives a basis for understanding a model’s value – Under pragmatism, a model is only as good as the decision it helps us make. If we have a model that has very low error, but doesn’t have a big impact to the organization, under pragmatism, it doesn’t have much value. Pragmatism also suggests that we should try to understand the impact of the model. E.g., the model helped save $2.3 million dollars in waste last year or the model correctly diagnosed 500 patients earlier allowing for better treatment outcomes. Having a focus on a model’s value can help us optimize our efforts to maximize added value!
- Gives a practical basis for comparing models – Often, we choose some error metric when comparing models. Whichever model has the lower validation error, we say is better. Pragmatism suggests that we should consider a model’s impact on decision making to compare the models rather than the error. Perhaps we have two models, one has lower error than the other, but both of the predictions result in the same decisions. Under pragmatism, these two models are equivalent!
- Allows us to ‘believe’ our model is useful and therefore use it – With pragmatism, we avoid crippling skepticism in our model. We don’t have to spend the rest of our lives validating and perfecting a model before putting it to use. We have the right to believe it is useful (and therefore, use it for decisions) when we follow data science best practices to validate the model.
Conclusion
Various philosophies of epistemology can be helpful to adopt in your work as a data scientist. Understanding the difference between deductive and inductive reasoning can help us understand what assumptions we are making when developing models and analysis. Skepticism can help us search for rigorous evidence that our models are useful. And pragmatism can help us accept the evidence we have found to take beneficial action. I hope this article has expanded your thoughts about how deep epistemological ideas can help you become a better data scientist!
Share This Article
Towards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program.
Write for TDS
