“I don’t know,” says the trustworthy algorithm

"I don't know," says the trustworthy algorithm

Inaugural lecture professor Pınar Yolum

AI systems must be accountable, transparent, fair, explainable, and ethical, it is often demanded. But according to professor Pınar Yolum, that is not realistic. Instead, she advocates for trustworthy AI: systems that demonstrate certain properties that allow people to trust their results. For example, by sometimes answering, "I don't know." Yolum delivered her inaugural lecture on July 2.

AI has become an integral part of our daily lives. This brings about wonderful things, such as improving healthcare, facilitating complex calculation or planning tasks, or reducing energy consumption and food waste. But there is also a downside: AI can exhibit biases against certain groups and be used for wrong purposes, such as warfare and the spread of misinformation.

Where does it go wrong? According to Pınar Yolum, the newly appointed professor of Trustworthy AI at the department of Computing & Information Sciences, it mainly lies in the AI techniques we develop. In her inaugural lecture on Tuesday, July 2, she argued that the systems we build today may resemble how humans reason and argue, but they do not replicate how humans attempt to do these tasks. Yolum: "AI systems can perform complex calculations much faster than humans. But at the same time it is tricky, because they can't apply all the checks and balances we do as humans. This means that AI can make mistakes that humans, with their broader understanding of situations, would likely avoid."

Golden properties

In response, many people say that we need to develop AI that is accountable, transparent, fair, explainable, and ethical. According to Yolum, this is not realistic: "For some reason, we accept institutions that are not transparent, people who do not act ethically, and a world that is not fair. But when it comes to AI, the systems must possess all these golden properties. That is an unattainable standard to live up to."

Instead, Yolum advocates for trustworthy AI. "Just as people show they are trustworthy by exhibiting certain behaviors, doing certain things, or refraining from others, an AI system should act in the same way." According to Yolum, this can be achieved by giving AI more response options, such as "I don't know". "A phrase I wish men would say more often," jokes Yolum. "Current AI systems are designed to provide an answer. If there is no clear answer, the system chooses quite arbitrarily, increasing the likelihood of providing an incorrect answer and thus becoming less reliable. If AI systems better understand their expertise or limitations, they can hand certain tasks back to the user."

We accept institutions that are not transparent and a world that is not fair. But AI systems must possess all golden properties?

For instance, Yolum's group developed an AI app that acts as a personal privacy assistant on your phone or computer. "This system learns the user's privacy preferences and can then make recommendations, such as not sharing certain photos or locations. The algorithm can also explain, both in text and visually, why the user might want to keep a particular photo private." The researchers found that when the system could say "I don't know" for certain photos and leave the choice to the user, the other predictions became more accurate and thus more reliable.

Yolum envisions future AI as a trustworthy employee to whom you delegate tasks because they have demonstrated reliability, integrity, and helpfulness. "We are currently in a situation where we have AI perform tasks that we then have humans check because mistakes are made. I believe we should move towards AI that we can confidently assign tasks to, knowing that the system will say 'I don't know' if there are uncertainties. Then you truly collaborate with AI as a team, with everyone doing what they do best."

According to Yolum, it is a political choice to develop AI in this way. "Computer science is a field where most funding comes from industry. And at the end of the day, the industry says, 'It's nice that you're helping people better protect their privacy, but what's in it for me? I want to collect and process data.' I therefore hope that in the future, more public funding will be available for this type of research."

Five response options for trustworthy AI

According to Yolum, there are five response options that trustworthy AI must be able to provide:

• I don't know
  "AI is currently unable to reflect on its own knowledge. It cannot assess whether it might be making a mistake. However, reflecting accurately on how much we know of something essentially reflects competence and improves trustworthiness."
  
• Here's why
  "There is already AI that can explain how it arrived at a particular decision, but not the background of that decision. AI also cannot judge whether an explanation is useful for the user. New AI should be able to do this more intuitively. This reflects willingness to collaborate and improves trustworthiness."
  
• I got this
  "The AI system not only has the expertise to answer a question but also the autonomy to decide which factors are important and which are not. The controversial aspect of this is that the AI can perform an action that was not explicitly requested. Using this autonomy responsibly improves trustworthiness."
  
• Fair enough
  "Rather than always pushing for its own utility, own benefit, an AI system should be able to understand and respect others' choices and say 'Fair enough'. Having and exhibiting such empathy improves trustworthiness."
  
• With all due respect
  "The AI can assess whether the task it is given might be harmful to others or inappropriate in a certain situation and then decide not to perform the task. Knowing that the AI system has this integrity increases trustworthiness."