AI in the Public Sector: Stumbling forward and learning from Brilliant Failures
Artificial Intelligence (AI) offers many opportunities for optimising public services. Taking advantage of these opportunities often involves trial and error. When innovating in the complex field of AI in government, it is often (near) failures that drive progress. A Brilliant Failure is a well-prepared attempt to achieve something with a different outcome than planned. Failures are brilliant when lessons are learned and experiences are shared with others. We find opportunities to learn at moments when things go wrong or (in the case of success) could have gone wrong but did not. This may be because you were lucky, but also because you consciously made the right decision based on reflection, knowledge, collaboration, etc.
Since 2022, the Institute of Brilliant Failures , in collaboration with the Netherlands AI Coalition , presented the award for the most brilliant failures in the field of AI/Data Science in the public sector. The Dutch AI Coalition and the Institute of Brilliant Failures want to continue to emphasise the importance of experimenting with AI and promote the idea that (brilliant) failure is inevitable in the responsible development and application of AI.
The jury selects winners based on the VIRAL formula, developed by the Institute of Brilliant Failures, taking the following factors into account: Vision: The extent to which the failed project is based on a detailed vision of AI implementation. Commitment: The extent to which people have committed themselves to making the project a success and to helping others. Risk management: The extent to which people have succeeded in finding the right balance between avoiding unacceptable risks and daring to take acceptable risks. Approach: The extent to which people prepared, collaborated and utilised available knowledge. Learning: The extent to which people learned from this project and shared or can share their knowledge with others.
The 2024 prize for most brilliant failure was won by the Province of South Holland in collaboration with Geronimo.AI, a young company that combines geodata with AI. The project aimed to investigate whether it is possible to use existing road-related data to predict the optimal maintenance moment more accurately than the current “expert judgement” method, i.e. based on asphalt expertise. This question was also exploratory in nature: what is already possible in the field of data-driven working and predictive models, and is this useful for road maintenance? The desired results were not achieved, mainly due to (poor) data quality. The dataset was too small and inconsistent to make a reliable prediction within a timeframe that is “actionable” for the province. Unfortunately, nothing else could be done with it except to take the lessons learned to heart and share them within the network, which has been done. For example, the algorithm developed has been shared with the Algorithm Register for the government. An important lesson is to accept that a dataset is important and that AI does not yet possess all human domain knowledge.
The ceremony for the 2024 prize winning brilliant failure
The Institute of Brilliant Failures has developed a methodology for learning from (possible) brilliant failures, based on recognition of recurrent universal patterns (archetypes) of failure. These are being used to identify, record and share lessons learned. The nominated projects for the AI Brilliant Failure Award have been analysed and the description of the projects, including lessons learned, coupled to these failure patterns can be found in the public version of the ‘early learning’ system BriMis (www.brimis.nl).
This approach can be useful for anyone who is working with AI, including those working in or for the public sector.
For more information, contact Paul Iske, Chief Failure Officer at the Institute of Brilliant Failures and Chief Dialogues Officer at YB Inspire
