Much of the management of the physical environment has to do with various forms of permits or interests. Think about building or excavation permits, mineral claims, oil or gas exploration leases, or construction permits. There are permits or licenses to draw or consume water, or to discharge substances into the air or into the water. There are interests in cutting trees or planting them, and rights to plant crops or to harvest fish or operate a fish farm. There are insurance claims for weather damage. The ownership of land involves interests in a land parcel or a building unit.

These and many other permits or interests have several things in common. First, they clearly have a component which is related to location. Land parcels, water parcels, fish farm locations, etc., all require geographic information about position, and usually extent as well. Second, this spatial component is just one of many information elements, and typically not the most important one. Third, the balance of the information is determined by a well-defined information model. Forms for permits, licenses, and interests usually have unambiguous fields, many of which are often mandatory. Form fields for information about the lessee, owner, or permit holder would, of course, be required, but there are typically some numeric fields for information such as area of land, volume of water, concentration of pollutants, and so forth. Records without these fields would be of little use either to the permit holder, the issuer, or to the government or private entity that must track the collective impact of the permits, licenses, etc., that have been issued. A formal information model is essential for standardizing and describing such forms and, yes, at least one or two of the fields are spatial.

A fourth item of commonality is the importance of documents associated to each permit, license, lease, etc. including, of course, the “permit” document itself. A building permit will reference a set of drawings, and applicable zoning regulations. A solar cadastre will reference a solar energy survey that validates the value of the cadastre. A harvest permit may reference a forest value assessment.

Documents such as permits, licenses, leases, etc., are very often legal, or at least regulatory, instruments – making maintenance of an audit trail for any changes to their information content mandatory. In addition to any associated documents, the “permit” holder and the issuer need to track who made what changes and when. Furthermore, there is usually a governance component associated with any of these changes. Permit are typically requested, reviewed, and either authorized or denied. Exploration leases expire, or their terms are violated causing them to be voided by the issuer. Management of the life cycle state of these documents, in the context of an auditable trail of changes, is a fifth common element between them.

So… how do registries meet these requirements? And how is a registry different from a database?

A registry, of course, is fundamentally a database, in the sense that it persists information (the permits, associated documents, etc.); however, it is much more than that. Out of the box, a registry is a web service – meaning that, when deployed, a registry can easily be connected to by a variety of clients over the Internet, both publishers and consumers. It also fits quite perfectly into cloud deployments, and into situations where there are a variety of issuer authorities ranging from large corporations and governments, to the smallest municipal office without an IT department of any kind.

A registry provides an information meta-model for the data it contains. In the case of the Galdos INdicio™ Registry, the meta-model is based on the OGC CSW-ebRIM specification; this specification provides natural constructs such as registry objects (with properties), taxonomies (classifications), collections (packages), and object relationships (associations). Users can quickly create their own custom information models, which can include one or more spatial properties such as location or extent, and immediately deploy them using web-based clients. Spatial properties do not need to be special; they are just part of the information model. With a registry, you can locate and re-use the information models of others, not by reading UML diagrams or text documents, but simply by downloading and using them – immediately.

A registry provides life cycle status for every registry object built in; no matter whether that object represents a permit, a license, or an organization. Furthermore, the set of life cycle states, and the allowed state transitions between them, are completely under the control of the agency that deployed the registry; for example, the agency may allow a permit to be transitioned from Submitted to either Denied or Withdrawn, but may not allow it to be transitioned directly from Submitted to Deprecated. Registries can support almost any governance model, keeping track of all registry changes through the built-in audit trail, and also tracking who made what changes and when.

One of the coolest things about a registry is that, unlike a relational database, you get what you build. A relational database is built from an information model, translated using well established UML or ER tools and techniques into the database table structure; however, when you come to query it – you need to re-translate back, from the tables that the tools created, to the model itself. Sometimes this is easy enough – but sometimes it is quite complex. With a registry, there is no such difficulty – what you created in your meta-model is exactly what you use to make a query request. What you build is what you get!

Of course, you could build a database application in place of a registry, but it will take longer and cost more. You could buy a Geographic Information System, and integrate it with a Document Management System, and layer a Permit System on top of that, using all the “best practices” in your approach, but you will most assuredly end up with a system which is vastly more expensive to build, harder to maintain, and more complicated to operate. So… when you are thinking about what you need for your next information system – keep it simple. Buy a Registry!

Having got this far, this seems like a good point to try and guess how things might evolve from here.

Will GIS now go from strength to strength?  Will we see engineering and construction companies shifting from engineering design software to GIS?  Or will we see a continuation of these awkward architectures where things are designed in one environment and then transformed for no functional reason and stored in another environment, mostly for the purpose of visualization?  Could a lighter weight, lower cost (i.e. free) presentation environment eliminate this second storage model and thus simplify the enterprise system architecture?  Could there be a fundamental change brewing in the industry?

I have long harangued on the fact that most data integration projects or SDI’s have focused on the wrong target (national data), when they should be focused on cities (http://www.galdosinc.com/archives/what-is-an-sdi, http://www.galdosinc.com/archives/the-new-sdi-–-spatial-data-infrastructure).  Urban data is enormously more valuable, is constantly changing, and the simple act of sharing can save hundreds of millions (even billions) of dollars each year in every country on the planet.  Cities are the land of designed infrastructure (whether located inside or outside the city proper), and the critical target of SDI’s should be the integration of design data across projects, across the city, and over the life cycle of the city and its component infrastructures.

Does this require GIS as we have traditionally understood it?

Most such infrastructure is best understood in three dimensions (augmented by 2D views of course), and 2D symbolization, map drawing, and analysis (all cornerstone technologies of GIS), while not irrelevant, are clearly NOT the key technologies.  The world of (3D) infrastructure is about engineering design and engineering analysis.  Why translate into another storage model if you don’t need to, and if it does not provide much benefit?  Can’t your BIM oriented design software provide you the 2D views when you need them?  Would an engineering company seriously start designing, or making design modifications to, their infrastructure in a GIS?  I don’t think so.

It is not Enterprise GIS that we should be thinking about in our cities, but Enterprise Infrastructure Model(s).

I believe that this viewpoint is underscored by the need and the increasing interest in what I would call wide area engineering.  This is not engineering as understood in the discipline siloed past, but rather a form of systems engineering applied to a large area such as a major airport, a large city, or a sea port.  Such engineering must not only deal with a range of conventional engineering disciplines, but must also take into account many other sciences, including ecology, biology, and aerodynamics.  While there are clearly limits to modeling and simulation (the heart of all engineering) for such complex systems, it would be foolish to continue development without taking advantage of the tools and techniques that are available to us.  In wide area engineering, the integration of data – design data – and simulation and modeling data from a variety of different domains and organizations is required, and on an ongoing basis.  This does not really sound like the world of GIS.

Perhaps a sea-change in the industry is close at hand?

This post was also published as a column in V1 Magazine on September 11, 2011.

Just so that there is no confusion, let me start with an obvious definition. A feature (or feature type) is a model of some entity in the real world. It can be quite abstract or quite concrete. It usually has an associated list of properties, which are defined by the feature type or feature schema. A feature type implies a controlled vocabulary, meaning an agreed to list of terms, with specific meanings, and usually with a specific list of properties. Often, features have a geometric location or extent, in which case we would refer to them as geospatial features.

A KML Placemark, regardless of whether it has Point, LineString, or Polygon geometry, is in general NOT a feature. It is a piece of geometry – a polygon, for example – to which may be attached a label, and possibly other attributes. It has no defined type, as such, and asking “find me all of the roads within 50 km of a selected road” is not a well-defined question.

A ShapeFile can imply a feature through the .dbf Entity Type, although this is much abused, and cannot generally be relied upon.

DGN files, as we have already noted, are without features, and simply provide geometric elements with annotation (labels) and possibly additional attributes. It is important to understand that the presence of annotation does not a feature type make, since it simply attaches text to the geometry instance.

So… if so many popular encodings don’t use features, why do we need them?

If you are only doing map or drawing visualization, you very often do NOT need them, or you only need a much weaker version, what one might call a map, drawing, or image feature, meaning “something I can see on this map”. Placemarks are a perfect example of this notion of a “map feature”. This is not, however, what is meant by “feature” in this article. Placemarks are location centric, not feature or object centric.

Without features, almost any kind of computation on geometric objects (whether designed or natural) is somewhere between extremely difficult and impossible. Think about how you would compute (without features) such things as:

1. The distance that something is from a road.
2. The distance from something to the nearest runway.
3. The area of a user-specified polygon that is within an airport zoning surface.

The computational software simply cannot “locate” the object in question if it does not know what it is. Computations that depend on the properties of the object (feature) are, of course, equally impossible.

Without features, lots of “vanilla” queries are completely impossible. “Find all houses within 10 meters of a given road” as just one trivial example.

Without features, any kind of geo-demographic analysis is also completely impossible. Imagine trying to compute the best location for a store or a bank branch (or your next house) without being able to refer to kinds of things, and kinds of value distributions?  All of this requires features.

Without features, how could one perform any type of simulation?  How would you simulate things like traffic flows? building heat loss? urban heat islands?

Not only are features (feature types) essential, but it does not take too much introspection to see that standard feature types are also required. Since none of us live “on an island” – well, actually, many of us do! – we need to share information with one another. The rapid transit line goes to an airport. Power for the transit line comes from a hydroelectric company. Water for the airport comes from the local community water supply. Everything is connected. To analyse and engineer any of these systems, we need to, indeed we must, share information – and that is impossible without features and, increasingly, without standard feature models.

The route to standard feature models (e.g. agreement on a classification scheme for feature types, and possibly attributes associated with each type) often starts with a standard means of encoding feature types and feature instances. One key standard in this respect is GML (Geography Markup Language), which uses XML Schema to define application-specific vocabularies, vocabularies that can then be used to test instances of these vocabularies for conformance (“Is this really a road?”).

GML has given rise to a number of languages which define standard feature types for particular domains, and we can anticipate more of the same in the future. These languages include things like CityGML, IndoorML, GeoSciML, and AIXM, to name only a few.

I believe we have just scratched the surface in the area of standard features and feature sharing.

Don makes a number of good points in his blog.  It is the data that matters, and the apparent complexity of GML (I will use GML since I am thinking about geospatial data) stems from the complexity of data models, and not from GML itself.  If you have a simple shape file (meaning a .shp file, a .prj file, and a .dbf file) and translate it to GML, you will have a much simpler, single, GML file that you can read like a comic book, even if you know little or nothing about XML or GML.  The complexity is in your data, and in the data models that you build.  Keep them simple!

If we compare aging standards such as Shape, or VMap, or S57, GML looks sparkling in its simplicity.  It is just that you don’t normally look at a Shape file or an S57 file; you look at a rendering of it through a tool interface.  As the tools by Safe and others improve, you will come to think in the same way about GML as you do about a shape file.  We will also see, and are starting to see already, content level models (e.g. GML Application Schemas like CityGML, DIGGS, and AIXM) that further simplify your universe, and to use these you will never need to look at angle brackets again.

The mainstreaming of XML makes me think back to an adage I promoted in the early days of GML – namely “if you have a GML (XML) face, should you also have an XML heart?”  What does that mean?

If GML (XML) is the lingua franca of data exchange, perhaps we should start managing, storing, and operating on it that way.  This does not mean that we need to store everything as text (although that is great for archival purposes), but it does mean that more GML-aware storage, processing, and management (“the GML heart”) would make a great deal of sense.  Why keep translating back and forth to a relational data store that is NOT GML (XML) friendly?  This is painful, even for objects, in spite of all of the work that has gone into J2EE.  Going from a GML (XML Schema) -aware data store to in-memory objects is much, much easier, since XML Schema is much closer to an object model than a relational model.

At Galdos, we have developed a Web Feature Service, INscape (http://www.galdosinc.com/technologies/products/inscape) that runs on the very powerful EMC Documentum XDB, which is a native XML database.  With this approach, we have none of the headaches of schema mapping that we encounter with relational stores (we also provide INscape on Oracle and ArcGIS Server), and we can be immediately up and running with new and complex GML schemas.  Need to deploy large geospatial datasets at home or in the clouds?  Think about getting a GML heart.  It could be the easiest thing you have ever done – and nary an angle bracket in site!

In the RESTful architectural style, interfaces are not the objective. Rather, the idea is to focus on the identification of resources and the provision of resource representations or encodings. GML began life, somewhat in contradistinction to the interface view, enshrined in CORBA (Common Object Request Broker Architecture). In a talk given in 1999 on xGML (my pre-cursor to GML), I felt compelled to defend a data encoding (read “resource representation”) point of view, with a slide entitled “Why encoding” and another entitled “Interfaces vs Encoding”.

GML was conceived as a means of representing resources that model objects in the physical world – what, in GML, are called Features. Every GML feature must have an Identifier. The original notion of identifier was much like that of RDF, from which may ideas in GML are borrowed, namely that the id was a local identifier of the object or feature that became global by virtue of its location path (thus composing an HTTP URI for the feature resource).

GML thought a lot about linking geographic entities to one another. Originally, the idea was that any property in GML (remember that GML properties conflate attributes and relationships) could be supplied with an xlink:href, the value of which “identified” the resource which was the value of the property. Early GML talks (circa 2001) talked about linking resource based on geographic scale (refinement), generalization, and the sharing of geometry. Almost every talk had a slide like the following:

The introduction of the WFS (and other OGC services) began to steer the ship in the direction of a greater focus on interfaces, and on the management of GML in relational databases (which did not support the xlink:href construct). This got many people thinking that somehow there was a conflict between GML and REST, but I think we can see that this could not be further from the truth.

Using the http implementation of the RESTful architectural style, we can see that GML fits rather nicely into a world of GET, PUT, and POST. If I create a geographic object and wish to assign it a particular resource identifier on the web (URI), I can easily do this with an http PUT method something like:

       PUT http://www.myworld.org/smallLakes.xml#pl02
       <abc:Lake gml:id = “pl02”>
              <abc:surfaceArea>100.2</abc:surfaceArea>
              <abc:extent>
                     <gml:Polygon srsName = http://www.epsg-registry.org/export.htm?gml=urn:ogc:def:crs:EPSG::4326>
                           <gml:exterior>
                                  <gml:LinearRing> … </gml:LinearRing>
                           </gml:exterior>
                     </gml:Polygon>
              </abc:extent>
              <abc:locatedIn xlink:href = http://www.parksOfAmerica.org/directory.xml#p45
       </abc:Lake>

Someone else can request this feature using a simple GET operation, as in:

       GET http://www.myworld.org/smallLakes.xml#pl02

Since GML provides a formal description of all such geograohic resources through an application schema, which is also a resource, we can get the structure for the GML representation of an abc:Lake (or other feature types), just by requesting the schema with another GET operation, such as:

       GET http://www.myworld.org/canadianGeographicfeatures.xsd

How to process the GML returned in these GET operations is quite straightforward. If a schema is returned, parse it for use in later query construction or use it to validate returned GML content. If XML (GML) is returned, parse it and possibly style it for presentation (e.g. to SVG or KML) or inspect the URI’s it contains (e.g. locatedIn property in the example above) and decide if these need to be requested, and so on. This process can be repeated many times, depending on the application processing semantics.

So GML fits just fine into a RESTful world!

These are clearly key concepts of our evolving understanding of the GeoWeb, of the importance of integrating systems that support the functions of sensing, analysis, visualization, and action (control, design) in order to deal with some aspect of the world around us, such as the management of air traffic, or the development of urban infrastructure.  The need for systems that span organizational boundaries, and which function in a collaborative fashion without a single locus of control, seems central to our approach to many of the world’s problems, especially those that are the most vexing and complex.

This concept of the Geoweb as a “directed” web of systems or as “the internet with a purpose” is clearly more than just a technical challenge.  It will require, in many domains, a change in how we think about the process of design and “acting in the world”, and a change in the kinds of software and systems required to support that acting.  We might, for example, require collaboration frameworks operated by cities or private entities that facilitate the interaction of all of the stakeholders in the development of a city, from developers to architects to urban planners, and which integrate functionality for permitting, building, and structure design, as well as the design of whole city blocks or even of entire cities.  Such collaboration envisions a level of integration of organizations and their information systems which does not exist today.

I don’t believe that this is about making the design process faster, or about minimizing the labour effort.  Rather, I believe it is about more effective design, where we use information technology including, of course, geographic information in its widest sense, to develop urban infrastructure that is more energy efficient and more planet friendly, while providing us with a more human and liveable surrounding.  Clearly this is about collaboration as much as it is about geography, and one might just as easily speak of Collaborative Design as Geo-Design or, as in the GeoWeb, to see the word Geo to mean the world (global, globality), hence implying collaboration as its essential element.

GeoWeb 2011 – Smart World Applications addresses these issues of collaboration and the integration of systems head-on.

It should be noted as well that this first convergence is about so-called unstructured data, meaning that while the data does have internal structure, this structure has little to do with the information content, and is all about the mechanisms for data transmission, presentation, and distribution.  We might call this “Convergence Part I – Unstructured Data”, as I believe we are entering into a new era of convergence, one that I will label “Convergence Part II – Structured Data”.  While, one might have thought that these would proceed in the opposite order (structured data was already data), the issues with structured data are actually more complex, as they center on meaning.

Where might such structured data convergence be taking place?  Can we learn anything from the history of unstructured data convergence?

It might be helpful to look at what the convergent technologies in Part I (prior to convergence) allowed us to do.  Television enabled the wide area transmission of moving pictures.  Radio enabled the transmission of audible sound and speech without wires and over large areas.  Telephone enabled the transmission of speech (via wires) from one person to another, while photography enabled the capture and recording of visual scenes.  We can see the Part I convergence in terms of enabling the acquisition and transmission of information so that it could be reconstituted for remote presentation.  Convergence meant that the individual networks established for these purposes for each of the separate technologies (e.g. telephone, radio, etc.) could be used to carry other kinds of information (e.g. pictures), eventually resulting in a more or less single network able to carry all of the different kinds of media.

What might we anticipate for Part II?  What sort of convergence are we talking about here?

My claim is that this second convergence is all about the management of information concerning events or projects (a project is simply an event of long duration), whether that is an aircraft landing, the segmentation of airspace to enable a public airshow, urban renewal in the inner city, the construction of a new highway, a G20 summit or Olympic Games event, a terrorist explosion, or an environmental catastrophe caused by an oil spill.  For all of these events, we can exploit common technologies and standards that enable the structured description of the event or project, controlled communication of the information to other participants, integration of that information into the participant data stores and applications, and presentation of the information in a manner that is useful within the context of the event or project.

Note that a structured data convergence implies that we are exploiting common tools and technologies that are aware of the “meaning” of data and not merely its structure.  This is the essence of structured data.  For many, this will immediately bring forth visions of ontologies and a reasoning infrastructure.  I, too, believe we will get there, but that this is another generation away.  For now, we should be working with ontology “lite”, meaning schemas, classification hierarchies, and associations.  More will come later.  Even with these few items in place, an enormous amount of convergence is possible.  We can have common notions of a data advertisement, subscription, and publication.  We can define standard means of encoding geometric and topological information, observations, and authorized features.  We can define generic interfaces for insert/update/delete that work across wide area networks and against a spectrum of datastore technologies and vendors.  We could do all of these things using different approaches in each area – one approach for air traffic management, another for the management of electrical generation and distribution, another for urban planning and transportation, and yet another for public safety and security … OR we could move along the path of convergence.

As an alternative to the “core plus extensions” or “complete with profiles”, I would like to propose the dictionary or library model for GML.

In this model, we think in terms of a natural language like English or French, which has a grammar and a dictionary of words in the language. The grammar says something about how these words go together. The dictionary is just a big list of words, and their definitions, the definitions being written using sentences from the language that comply with the grammar. We don’t get too excited about the size of the dictionary as we go about our daily business using an appropriate subset of the dictionary.

If we apply this to GML (or other extensible encoding standards), we see that the grammar corresponds to the basic GML model (e.g. how we serialize an object as XML, how we serialize a class as XML Schema), and includes things like the “object-property-value” pattern, remote reference pattern, etc. The rest of GML is a then nothing more than a dictionary of words in the GML language. If we look in the dictionary we see lots of words, since GML covers lots of ground – but we can expect things like Point, LineString, TimeInstant, and so forth. Just as in English, we can think of there being many such dictionaries, some being very complete and covering “most of the language” like the Oxford Dictionary of English, or being very specialized and covering only a narrow domain like Aircraft Mechanics. These dictionaries might be constructed from one another in various ways. For example, I might make the “Pocket Dictionary for Travellers” from the Oxford Dictionary of English, but just extracting the words I think travelers would like to know about and use.

This Dictionary Model for GML is similar in many ways to the use of object libraries in programming languages. If I write a program, it will frequently use other programs and components, written by others, that I access through a program library (e.g. API). When my program integrates these other programs and components, it does not “import” the entire program library or libraries, but only those programs or components that are actually used by my program. In using a Dictionary Model for GML, I would only use the schema components in the dictionary that my “program” (my application schema) actually uses. Of course, the naïve use of XML schema parsers does not support this model, but that is something that can be fixed by other means, and should not impact the GML standard.

How do we decide what words get into the dictionary? What is the governance process? Remember that we are talking about only the core GML elements here, and not those in derived dictionaries for particular application domains. Clearly we do need a process which ensures that any word (i.e. schema component) that we add to the dictionary is compliant with the GML model, that it correctly uses any other “words” from GML, and that it is semantically correct. This seems like a task for the OGC’s GML SWG; however, other groups within the OGC and elsewhere could submit word definitions for inclusion in the dictionary.

Should we couple governance with the use of XML namespaces? I would argue no. Namespaces in XML exist to disambiguate terms (elements, attributes) which are spelt the same, but which have different meanings. If the definition of Point in GML is not changed from one version to the next, then there is no reason for the namespace to change. Since many schema components in GML use elements, types, attributes, etc., from other schema components, there is no logical way to decompose GML into different namespaces, except using purely arbitrary means, like splitting coverages from geometry, or topology from geometry. Note that if one realized coverages alone, one would have had to introduce points, lines, polygons, etc., in coverages. The interdependency between these categories surely means that creating namespace boundaries, as a means of governance, does not make sense. I see no problem in recognizing governance boundaries, based on domains of expertise, or other more or less arbitrary criteria. I just do not see enshrining them using namespaces.

This is not to say that all of GML must forever be in a single namespace. For example, if in the future we change the meaning of an element (e.g. Point), but we wish to keep the name, then the new Point, must clearly live in a new namespace.

Using this dictionary model (please don’t confuse this discussion with gml:Dictionary elements) for GML enables a much more fine grained modularity than will be possible with any interpretation of “core plus extensions”. Let’s move in this direction.