AI and explainability, a non-problem?
The use of AI/machine learning in decision-making is often considered problematic due to its opacity. This consideration is not unreasonable. When making a decision, it is generally assumed that the decision should be explainable to the subject of the decision. If the decision cannot be explained, we have no meaningful way of holding the decision-maker accountable.
Because of the opacity that frequently occurs with AI decision-makers, many, like the EU commission on ethical AI, argue that a ‘principle of explainability’ should apply to all AI. In short, developers need to make it possible to understand how an AI makes its decisions in order to meet the requirement of ethical AI.
But is it? In a recent article, Scott Robbins, a former computer engineer and current PhD-candidate focusing on the ethics of AI, argues that ‘the principle of explainability’ (he calls it ‘explicability’) is “misguided”. He is concerned that this principle might prevent us from reaping the benefits of AI, and give rise to redundancies in its development. I find his argument quite compelling, and in this post I will summarize his argument in an easy to digest-way.
Why the principle of explainability is misguided
Robbins argument follows from the observation that explainability is a function of the decision to be made, not the agent/entity making the decision. “We do not require everyone capable of making a decision to be able to explain every decision they make. Rather, we require them to provide explanations when the decisions they have made require explanations”. The result, he argues, is that many AI decision-makers are redundant.
How so? The simple reason is that, in most cases, applying the principle of explicability “assumes that we have a list of considerations that are acceptable for a given decision”, but if we already know the acceptable considerations “then we could just hard code them into traditional automation algorithms rather than let the ML algorithm take the role of decision-maker”. In other words, there is no need to use an AI to learn rules by looking at historical data when we already know the rules beforehand. If we do, we can make a traditional algorithm, like a decision-tree.
For example, consider an AI that decides whether to give someone a loan. If it rejects a loan application, it is reasonable to ask why it decided to do so. Was it because the applicant had a high debt-to-income ratio, or was it because they were black? If we investigate and find that it was because of high debts, we say that it made the right decision. If we find that it was because the applicant was black, or a proxy thereof, we say that it is wrong. However, in doing so we explicate the acceptable considerations for accepting or rejecting loan applications, and having done so, the AI is redundant. We could instead make an algorithm with the rule “if debt-to-income ratio exceeds 2, reject loan, otherwise accept”.
Another consequence of the principle of explainability is that in low-stakes decisions, those where we do not require explanation, the development of AI is hampered by having to make it explainable. For example, in the promising field of NLP (natural language processing), it is not clear that an AI must be explainable. If we want to translate a web page from one language to another, it is sufficient that the translation is good (whatever that means), not that the choice of specific translations can be accounted for.
Similarly, requiring explainability means that AI decision-makers could be developed for tasks that might as easily (and more cheaply) have been done by decision trees, i.e. by focusing on the considerations that must hold for the decision to be ‘right’.
In short, the principle of explainability is misguided, and we might add, misleading, because it confuses the decision-maker with the decision. Focusing on the latter will be more fruitful and can also provide a path for non-explainable AI.
In conclusion, I find Robbins paper a highly valuable contribution to the literature on explainability and AI. I particularly like the simplicity of his argument and the way it defuses a lot of the mystique concerning opaqueness in AI, which is sorely needed in the literature. This is not to say that there are no unresolved problems. First, there is the problem of unintended consequences. Even if we let AI make decisions in low-stake situations, there might be consequences down the line where accountability is needed. Second, there might be opportunity costs by not letting AI make decisions. For example, if someone develops an AI that makes policy proposals to mitigate global warming, can we afford to say no? Lastly, there might be gray areas where the property of requiring explanation is itself disputed. In some cases, there might be people who say that a decision requires an explanation and others say it don’t. Neither of these are problems concerning Robbins paper, but they show that shifting our perspective from AI to the properties of decisions leaves ample room for further theorizing.
Robbins paper can be read here. Here is the link to his website, where his other papers are published. I first heard about Robbins from his appearance on the Philosophical Disquisitions podcast hosted by John Danaher which you can find here or other places where podcasts are found.