Governments can learn from Silicon Valley as they tackle AI and data science

Artificial IntelligencePolicyResearchSocial Sciences

Researchers have examined how governments around the world have modernised their digital infrastructure in recent years by learning lessons from Silicon Valley technology companies, yielding tips for adapting to future advances in AI and data science.

A new study from the Oxford Internet Institute (part of the University of Oxford) and the London School of Economics examines how governments have gradually adopted Silicon Valley's way of working.

This shift offers valuable lessons as the new UK Labour government looks to embrace AI and data science in public services. However, the study also highlights ongoing challenges, such as outdated IT systems, reliance on conservative tech providers, and the dominance of big tech companies.

The study, titled 'The Political Economy of Digital Government: How Silicon Valley Firms Drove the Adoption of Data-Science and Artificial Intelligence in Public Management,' was published in the journal Public Money and Management. It shows how governments in the US, UK, and Australia were slow to modernise public services due to their dependence on traditional large-scale computer firms, known as "systems integrators." Public officials also hesitated to adopt Silicon Valley methods, which held back progress for decades. The study suggests that to better harness AI, the public sector must avoid repeating these past mistakes.

Professor Helen Margetts from the Oxford Internet Institute explains: 'Our research looks at how government contracts for information systems have evolved over time. For years, public officials, working with highly traditional global tech firms, mistakenly believed that Silicon Valley's methods didn't apply to the public sector. But this mindset has now changed, driven by the growth of massive platform companies, the adoption of Silicon Valley technologies like cloud computing and AI, and the realisation that these companies are often better at cybersecurity and managing sensitive information.'

The rise of Silicon Valley technologies and cloud services has reduced the influence of traditional systems integrators, embedding these new tools across government departments.

Patrick Dunleavy, Emeritus Professor of Political Science and Public Policy at the London School of Economics, adds: 'Over the next decade, Silicon Valley and platform companies' technologies will shape how governments handle data-intensive tasks. Digital managers within the government will play a crucial role in either speeding up or slowing down cultural change and in developing the expertise needed to manage diverse technology contracts.'

While these technologies offer significant benefits for improving public services and the online experience of citizens, the researchers urge government officials to proceed carefully and ensure that these firms are held accountable.

Professor Margetts concludes: 'A major challenge for the new government will be building the expertise and capacity needed to fully leverage AI and address the challenges it brings. They'll also need to effectively manage their relationships with both traditional tech providers and Silicon Valley giants while regulating the technology market. Our research offers valuable insights to help them learn from past mistakes.'

Read the full paper, 'The political economy of digital government: How Silicon Valley firms drove conversion to data-science and artificial intelligence to public management', published by the journal Public Money and Management.