“AI is perhaps the most talked about technology of our time, promising to transform fundamental aspects of how we live, work and play. The capability of AI to perform increasingly more tasks that were previously the sole preserve of humans will free people up to focus on higher value, more fulfilling activities. However, this also creates a sense of unease about our very usefulness in the future. Either way, intelligent systems will play an ever increasing role in determining New Zealand’s future prosperity, security and social cohesion.”  AI Forum (May 2018), p.14

New and emerging technologies such as artificial intelligence (AI) will disrupt people’s lives in many ways. That’s why all New Zealanders have a stake in how these technologies might enhance our future well-being and how the complex societal challenges they pose will be addressed.

Commenting on the AI Forum’s recent report, Rod Oram, RNZ’s Business Commentator on Nine-to-Noon, was clear that AI fundamentally challenges what it means to be human. It is also likely to change how we understand and relate to nature, how we organise the economy, etc. And Oram reckons that the report underplays both the possible benefits and challenges of AI!

When a technology challenges such fundamental categories, there’s plenty of scope for the sort of public outcry we had in the 1990s over genetic modification (GM). This is why the NZ government needs to be thinking carefully about its approach to the governance of new and emerging technologies.

Governance lessons from the genetic modification

In the mid-1990s, public opposition to GM crops led to a Royal Commission. The Royal Commission considered around 10,000 submissions and identified a major gap in New Zealand’s governance system: there was no body responsible for focusing on the big questions raised by disruptive technologies. These questions – e.g. about the meaning on being ‘human’ or of ‘natural’ – go well beyond what government departments, expert working groups, industry bodies or research institutes can address. One of the Royal Commission’s key recommendations was that the government strengthen the science governance system by establishing Toi te Taiao: the Bioethics Council to proactively engage with the public on such questions.

Engaging on the big questions: The Bioethics Council model

The government of the day accepted the need for a body to make ‘upstream’ public engagement [1] more central to decision-making and governance for potentially disruptive technologies and established the Bioethics Council in 2002.

Its role was to promote and participate in forms of deep public engagement on the cultural, ethical and spiritual questions raised by emerging biotechnology [2] before the technology was widely adopted. This would provide a mechanism for public values to shape these technologies, associated legal and regulatory systems, and enable anticipatory governance that would:

• make decisions more legitimate
• enable large numbers of citizens to engage with the big questions in dialogue with experts and with each other
• improve the quality of decisions by making sure that the problems, issues and solutions that citizens see, but experts often miss, were considered.

The Bioethics Council developed state-of-the-art approaches to online and face-to-face engagement which were very well received by the thousands of New Zealanders who took part, won international prizes and were recommended by the OECD as best practice governance. Despite its successes, the Bioethics Council was disestablished by the National-led government in 2009 against the wishes of many organisations [3] and department staff. This reopened the governance gap identified by the GM Royal Commission, a gap that still exists.

‘Responsible innovation’ for science and technology

While the National-led government dismantled key parts of NZ’s science, technology and innovation system between 2009 and 2011, European policy makers have continued to grapple with how to deal with new and emerging, potentially disruptive technologies.

Conventional governance approaches based, for example, on evidence-based regulation and codes of responsible conduct in science can’t do the job because such novel technologies have wide-ranging and uncertain present and future impacts. And it’s not sufficient to focus on the end-products of innovation (e.g. a new GM food crop) as public dialogues about new technologies consistently raise questions about the purposes and motivations for the innovation itself. Why do it? Who might benefit? Will the benefits and risks be shared? Do we as a society want it?

In summary, a framework for responsible innovation [4] must not just include a consideration of products, but also of purposes; not just consider what we don’t want from science and innovation but what we want them to do. Responsible innovation means we have to think and reflect on what sort of future(s) we want science and technology to bring forth, what challenges we want these to meet, and what values these futures will be anchored in.

The European experience can give us pointers about what features a responsible governance system for new and emerging science and technology should have. It should be:

• Anticipatory – describing and analysing intended and potentially unintended impacts that might arise e.g. economic, environmental, social etc
• Reflective – reflecting on underlying purposes, motivations, and potential impacts
• Deliberative – opening up questions to a broad collective deliberation through processes of dialogue, engagement and debate, inviting and listening to perspectives from publics and diverse stakeholders
• Responsive – using reflective processes to both set the direction and influence the direction and pace of innovation through participatory and anticipatory governance.

The responsible governance of science and innovation must be a holistic approach across the innovation ecosystem, with important roles for established players – universities, institutes, research funders and policy-makers – and for a new governance body.

A new public engagement body for today’s disruptive technologies

Our experience with disruptive technologies – John Pennington was the Manager of the Bioethics Council secretariat from its inception until it was disestablished and Simon Wright was its Senior Advisor – has taught us that the public is extremely interested in engaging with the big questions and that good engagement processes can deliver well considered and useful insights for anticipatory governance and decision making.

We also believe that a full-time permanent organisation is needed for engaging the public on today’s disruptive technologies. It’s staff would need a deep knowledge of public engagement approaches, especially dialogue and deliberation, and the rationales behind them. It would also need the motivation and support to continue exploring and experimenting with the new and expanding range of online and face-to face engagement processes. Such an organisation would need to have many of the institutional features of the Bioethics Council [5]. It would:

• Be independent of Ministers and their agencies and operate under a Terms of Reference.
• Have no role or stake in promoting the technology being discussed [6]. In this sense, it would different to the AI Forum and the Data Futures Partnership.
• Have members with no interest in the technology being discussed and with no subject-specific expertise. In this sense, members would be expected to learn about and grapple with the issues just like members of the public. It would also have significant Maori membership. On all these counts, it would be different to the Law Commission’s Centre for Law and Policy in Emerging Technologies.
• Report to relevant Ministers depending on technology and issues being considered
• Have a permanent secretariat with significant expertise in all aspects of public engagement
• Have sufficient budget (the Bioethics Council received $1.5M per year in the period 2002-9).

If a body were to be established to promote, encourage and undertake public engagement on new and emerging technologies, such an organisation could also undertake public engagement on a wide variety of societal concerns, e.g. climate change. Its mandate would need to include research into public engagement that is happening beyond officially sanctioned exercises. As a part of its work, it would adopt a whole systems approach to public engagement; this would include ongoing systematic mapping of NZ public engagement with e.g. new and emerging technologies so as to capture emerging citizen-led and grassroots forms of engagement. This broader approach would provide additional social intelligence about public values and concerns that may present barriers, and reveal untapped forms of social innovation that could be harnessed to improve well-being and ensure that new technologies serve human needs and purposes.

Endnotes

[1] The idea that the government needs a specialist public engagement agency with no stake in issues under consideration has been proposed by Associate Professor Jennifer Lees-Marshment, a political marketing expert at the University of Auckland, based on interviews with over 50 political leaders. https://researchspace.auckland.ac.nz/handle/2292/23242.

[2] Biotechnology was seen as the disruptive technology of the 2000s.

[3] The Catholic, Anglican, Methodist and Presbyterian Churches valued the Bioethics Council’s public engagement processes, which, unlike standard policy processes, allowed citizens to respectfully explore spiritual perspectives, and campaigned hard to have the Bioethics Council reinstated.

[4] The Engineering and Physical Sciences Research Council, the UK’s largest research council, and the EU Horizon 2020 funding framework require researchers when applying for funding to use their respective ‘Responsible Innovation’ approaches.

[5] Further information is available from the Bioethics Council’s archived website – http://ndhadeliver.natlib.govt.nz/delivery/DeliveryManagerServlet?dps_pid=IE1074184.

[6] The Science, Society & Sustainability (3S) Research Group, University of East Anglia report to the UK Energy Research Centre (UKERC) 2017.

---

Also published on Medium.