Dealing with bias with the help of standards

Image by Gerd Altmann from Pixabay

In TV shows, it is always the middle of the night when the crime scene investigators arrive, wearing gloves and protective suits to avoid contaminating the evidence. Usually, they start off by assessing the scene, noting the location and condition of the body, as well as noticing obvious evidence, like blood spatter, weapons, or signs of a struggle. What they don't tell you in the show is that at this point the outcome may already be compromised as the CSIs are introducing cognitive bias into the investigation. <_o3a_p>

That is why researchers at University College London (UCL) are currently looking into ways of harnessing artificial intelligence (AI) to improve forensic methodology. They are using eye-trackers to study the gaze patterns of forensic scientists to compare against verbal accounts of what the investigators said they looked at and in what order they collected evidence. The UCL researchers are feeding the data they collect into a machine-learning system to evaluate current investigations and possibly develop new and improved methodologies. <_o3a_p>

Cognitive bias happens when we create our own "subjective reality" from our perception of a specific situation or piece of information. The different kinds of cognitive bias can significantly impact crime scene investigations in several ways. For instance, investigators might focus on evidence that supports their initial theory while overlooking or dismissing evidence that contradicts it. If they believe that particular suspect is guilty, they might give more weight to evidence that confirms this belief and ignore evidence pointing to another suspect. This kind of cognitive bias is called confirmation bias.<_o3a_p>

The presence of certain contextual information, such as knowing a suspect has a criminal record, can influence how forensic examiners interpret ambiguous evidence. This is known as contextual bias. Closely related to this is stereotype bias, such as racism, misogyny and ageism, which reflects overgeneralized beliefs or assumptions about a group of people based on shared characteristics. <_o3a_p>

The anchoring effect occurs when investigators rely too heavily on the first piece of information they receive (the "anchor") and fail to adjust their thinking in light of new evidence. For instance, if the first witness they interview suggests a specific suspect, they might be less open to considering other suspects. <_o3a_p>

An example of availability bias, also known as availability heuristic, is investigators giving undue importance to evidence or scenarios that are more readily available in their memory, often because they are more recent or emotionally charged. <_o3a_p>

Sampling bias could occur in witness selection, evidence collection - for example, focusing on fingerprints and ignoring DNA evidence) - or suspect identification, if the pool of suspects is not representative of the population, among others. <_o3a_p>

Cognitive bias has played a significant role in some of the worst cases of miscarriages of justice. In 1989, for instance, five teenagers were wrongfully convicted of assaulting a jogger in Central Park, New York. Confirmation bias and contextual bias played significant roles in their wrongful convictions. The real perpetrator confessed in 2002. In the UK, hundreds of sub-postmasters were wrongfully accused of theft, fraud and false accounting due to faults in the Post Office's Horizon IT system. Cognitive biases, such as authority bias and confirmation bias, led to these wrongful convictions.<_o3a_p>

Of course, bias is not confined to the justice system. We are all conditioned by our environments and experiences and carry with us different kinds of social, political or values-based baggage. Sometimes our horizons are not as broad as we would like to think and as a result, the vast volumes of data used to train algorithms are not always sufficiently variegated or diverse. More often than not, there is actual human bias in data or algorithms, which simply look for patterns in the data we feed it: garbage in, garbage out.<_o3a_p>

The good news is that bias can be detected and mitigated quite easily. The bad news is that it can be difficult to get to the bottom of how algorithms are making decisions in order to solve the problems, as more often than not they operate within a "black box". It is one of the most important challenges we face, as algorithms are increasingly at the centre of our daily lives, from search engines and online shopping to job applications and bank loans. <_o3a_p>

Unintended biases in AI systems can have significant consequences. Unlike traditional IT systems, AI systems have the capability to learn, which offers great potential but also significant risks. The IEC and ISO committee for AI, SC 42, has developed a portfolio of international standards that cover a range of these challenges, from trustworthiness to addressing ethical considerations in applications, such as eavesdropping and of course, bias.<_o3a_p>

SC 42 approaches the problem from multiple perspectives, considering the application, developer, and deployer viewpoints. For instance, alongside SC 42 work on trustworthiness, they offer comprehensive standards for the data quality used in AI systems. Their standards also address computational aspects and AI governance. These efforts are unified through a management system standard (MSS) that organizations can use for auditing and conformity assessment of their systems.<_o3a_p>

Addressing issues such as bias from the outset is crucial. The entire portfolio of ISO/IEC AI standards integrates ethical considerations. Additionally, SC 42 provides standards that work across different vertical domains. This means collaborating with individual IEC and ISO technical committees to develop AI solutions for specific applications. SC 42 also supports IEC and ISO committees by providing foundational standards to ensure their applications are responsible, trustworthy, ethical and deliver the intended benefits.<_o3a_p>