Sharing data

Datasets

We identified different systems that provide, create, or read planning data, and systems that use planning data.

Source data

These systems provide source data:

  • Local Authority Gazetteer
  • Digital Land Planning Data Platform
  • Local Authority GIS

Create data

These systems create planning application data:

  • PlanX
  • Planning Portal

Process data

These process planning application data:

  • BoPS
  • Other back office system
  • Consultee System
  • Consultee Portal
  • ODP Register

Generated from data

  • Central government reports
  • Other data users

Data strategies

We examined potential data strategies for systems to share data. We concluded on a hybrid stategy, detailed below.

Centralised data storage

A centralised data store would give the benefit of one point of truth. Data would flow from external systems to a central database. Centralised data store

Individual system data storage

Each system stores data in a format unique to its system, and data transfer would be agreed between systems and vendors. Individual data store

Proposal: hybrid strategy

An open API standard for planning application information data exchange. Suppliers would send data and pull data to and from endpoints. Hybrid data store

Data volume and replication latency

It is a challenge in distributed systems where multiple databases are used to store and access data, especially in real-time applications where data consistency and availability are critical.

Replication latency is the amount of time after an update is made for it to be replicated across all systems.

Most councils use one system to record planning applications, and as such any data strategy would have to examine data transfer issues around this. A conventional nightly batch update would likely meet this issue, with systems having a known 24 hour period of latency. As an API develops, this could be lowered using push techniques.

An API would therefore have to send and recieve ‘diffs’, or just send changes to data, between start and end times, in order to not overload any nightly tasks.

The largest amount of data transferred would be documents attached to planning data. These would likely be transferred between systems using signed URLs.

Service level agreements

In adopting a common API, service level agreements would likely be entered into between vendors. This would detail rate limits, latency agreements, and response times.

Adoption

By proposing an API based approach, we hope to help vendors adopt such a scheme ahead of legislation.