Scottish Government Seeks National Data Sharing Platform

As they define in the CivTech Challenge briefing the Scottish Government are seeking a solution for a national Data Exchange Platform.

CivTech is a Scottish Government programme that brings the public, private and third sectors together to build innovative solutions to shared challenges.

The goal of this particular challenge is to address:

Overview

The Scottish Government Digital Directorate have made a commitment to Scottish Ministers about exploring solutions to allow a strategic approach to data sharing across the public sector.

• DG Communities have convened stakeholders from across Scottish Government to establish a programme of work leading to a business case delivering a step change in public sector data sharing.
• The Scottish Local Government Digital Office have been developing a Local Government Data Platform to allow more systematic sharing of data between Local Authorities and Scottish Government.

Stakeholders

The Challenge Sponsor team will be made up of staff from the following Scottish Government Divisions: Chief Data Office, Data Architecture Team and The Information Governance team, Digital Citizen Division, with key stakeholders being:

• End users across Scottish Government, Local Authorities and NHS Scotland, and the wider public sector.
• Information Asset Owners, Data protection officers, Analysts supporting policy and operational decisions.
• Product Owners and Software developers who use data programmatically.

Solution Design

Depending on the specific service areas chosen, the solution may have to integrate into

• The Data Architecture team’s chosen metadata cataloguing solution.
• The Local Government Data Platform.
• The National Services Scotland (NSS) Near-Time Data Service.
• Microsoft in the Cloud (M365) platform, aligning with the ongoing National Information Governance Plan.

Use Case Scenarios

The specific service areas involved in the Challenge will likely include one or more of:

• A justice use case liaising with Police Scotland, Scottish Courts & Tribunal Service, the Scottish Prisons Service, the Justice Division of Scottish Government, and the Judiciary.
• A Health & Care use case liaising with the relevant Scottish Government, Local Authorities and NHS Personnel.
• A National Statistics use case liaising with the relevant Scottish Government and Local Authorities.

Technology

While not limited to the following technologies, the Challenge Sponsor is particularly interested in proposals that explore the following:

• AI – including Large Language Models, chatbots, and more conventional Natural Language Processing.
• Distributed Ledger Technology – peer-to-peer, cryptography, Blockchain.
• Multi-Party Computation and other privacy preserving techs.
• Cloud-agnostic and hybrid solutions.

Requirements

There are thousands of requests for data to be shared across multiple organisations in the Scottish Public Sector, feeding a wide range of different statistics and analysis which in turn provide the evidence to inform policy and operational decisions. In order to make sure regular decisions are informed by the latest information the process might be more automated for gaining approval for reusing public data and demonstrating this approval for each subsequent update.

• Organisations at different stages of the data value chain have separate legal obligations for its protection, so that they are obliged to maintain their own processes and infrastructure for managing data access, rather than inheriting from a parent organisation. This means there is a lack of a defined, shared process or standards for approving data sharing requests and a way to document this. Because specific legal obligations and rights may vary, the lawful basis for sharing might ideally be tracked across multiple organisations and transformations of the data.
• The incentives for data owners to approve sharing do not align with the public benefit, causing widespread risk aversion. Meanwhile, often the skills and culture of organisations focus on the risks of data sharing rather than the public benefits and an informed weighing of these. This might be addressed by making best practice, precedents, and the latest tools from the Informational Commissioner’s Office, more readily available and accessible to everyone involved in the approvals process.
• Data owners and data protection officers are more comfortable sharing data between organisations, regardless of how technical the use for public benefit. The Challenge Sponsor outcome here will be existing information governance approvers agreeing implementation of the solution.
• Once data sharing has been approved, every transfer of data will be handled programmatically, without duplicative manual handling or exchanging messages – enabling near-time sharing via Application programming interface (APIs) for seamless public services. The Challenge Sponsor Outcome here will be automation of an inter-organisational data pipeline – which may already have approvals in place before the Challenge.
• Organisations made up of siloed teams are still able to share data in pursuit of tackling big Challenges, like child poverty.
• Privacy preserving protocols can be approved, in place of simple data transfers. The Challenge Sponsor Outcome here will be that the prototype solution is able to describe privacy preserving protocols in a rigorous way that can be automatically checked.
• Newer tools from the Information Commissioner’s Office are widely understood and used across the public sector, like delegation of approvals or organisation certification.

Commercialization Opportunity

This problem exists anywhere where separate legal entities have responsibilities for managing agreements for data sharing, and have no trusted third party willing to administer these approvals.

The UK and global public sector will share the same issues and represent a route to scaling out a Scottish solution. Even if the same infrastructure were shared at these different levels, a business model could still be made from providing services for the users of this infrastructure – with a competitive advantage from intimate knowledge of its development.