Print This Post

Universal Service Bus – Beyond SDI

From informal mashups on the web to the Internet of Things, the widespread interconnection of services, databases, and applications is rapidly becoming a dominant aspect of our society. While some have expressed concern about the implications of such “hyper-connectivity”, there is no question that it will continue. The only real question is how to best achieve and manage it.

This article argues that the technical side of Spatial Data Infrastructures must be seen as a kind of Universal Service Bus, with the objective of enabling rapid interconnection of data and services to create new multi-party applications spanning multiple jurisdictions and large geographic areas. Any other interpretation of Spatial Data Infrastructure technology can be forgotten.

A Universal Registry Service

The objective of any service bus is to provide a universal mechanism whereby providers of data and services can register their offerings in such a manner that data and service consumers can easily find and integrate with or consume them. Because it applies to any service and any data type, we will call this a “Universal Registry Service”. We will also assume that it is universal in the sense of supporting open standards for both the service interfaces and for the underlying data model.

To be effective, such a Universal Registry Service must be highly extensible, and yet it must still provide direct support for key data types such as the geometry (e.g. location, shape) and interconnectedness (topology) of information concerning the real world. Users will want to find services based on various criteria, including geospatial, classification, associations, and other properties; for example, a typical query might be “find me a service for showing the distribution of crime rate and educational facilities, and which can assist us with computations on this information for the State of California”. Moreover users may be interested in data processing services, or data access services, or both – a user may go looking for data to find services, or go looking for services to find data. The extensibility of the Universal Registry Service cannot be over emphasized. Registries developed only as the basis of data and/or service catalogues such as CSW-ISO or CSW-INSPIRE are far too limiting for this purpose. What is required is a registry service with an extensible data model such as CSW-ebRIM.

CSW-ebRIM provides a complete, non-relational data model that makes it easy to capture information about web services, data stores, documents, and sensor data streams, including all of the associated supporting artifacts (e.g. schemas, CRS definitions, units of measure). CSW-ebRIM is able to express relationships between various objects such as data, services, and other artifacts, and it provides powerful classification mechanisms that make it easy to create and exploit all manner of taxonomies and hierarchies. CSW-ebRIM is also a completely spatial data model, enabling any data object to have one or more geospatial properties; in addition, it makes schema definition simpler and schema migration much easier than conventional relational or geo-relational models.

A Universal Service Bus

The key infrastructure for the sharing of data and for the wide area integration of data and services is the Enterprise Service Bus (ESB). Once a service is discovered (e.g. through the Universal Registry Service – as in the diagram below), the consumer accesses the service by sending messages across the ESB. In many cases, of course, one would like this process of discovery and integration to be both automatic and re-usable. In many cases, the objective of discovery and integration is to build a multi-agency application. The Universal Service Bus (USB) — which we define as a component that provides interactive access to and orchestration of services, together with automated data publication/subscription — should make that easy. Even in cases where the objective is simply data access, consumers should be provided with the ability to request the data once and have the data stay automatically in synch with the source, at least until the consumer decides that different data is required. The USB should provide a maximum of utility and services to the community.

A Universal Service Bus and Registry Service

A Universal Service Bus and Registry Service

The Universal Service Bus must enable providers of data and services to easily “advertise” service endpoints and determine if they can subscribe to that service, make interactive requests, or both. Ideally, consumers of data and services can create subscriptions by selecting from the advertised offers, possibly adding additional constraints such as constraining the data to a particular region, or to a particular time windows, etc. Subscriptions can then be either push (e.g. automatically detect changes at the source) or pull (e.g. based on time markers) the desired data to the consumer. Whenever the conditions of a subscription are met, data changes should be automatically transferred from publisher to consumer. No action should be required by either party. Data should be deliverable from database to database, or GIS to GIS, and may involve dynamic data transformations applied en route, such as change of coordinates or change of schema. The process should be largely transparent to both parties. There should be no need for the publisher to maintain lists of subscribers, and no need for custom security handling. All data transfers should be reliable, secure, fine-grained and automatic.

We should be going beyond conventional notions of SDI and geo-database replication to think in terms of Universal Service Buses and Universal Registry Services. This is the means to faster and more cost-effective deployment of multi-agency applications.