A platform that aggregates a region’s tourism data has to make sources talk that were never meant to talk to one another: cultural-heritage registries, public-transport timetables, environmental sensors, municipal cartography, editorial content. Exposing an API is the easy part; the hard part is deciding which protocols should carry flows that differ in cadence, volume and consistency model. I am gathering material for an analysis of this kind inside the SMARTOUR project (a PON “Smart Cities and Communities” project on the cultural-heritage strand, with field trials in preparation around Salerno), and what follows is a survey of the protocols available as of January 2017.

Context

The first constraint of a regional system is that the sources are given. A heritage authority publishes its datasets in one format; a transport operator exposes another; environmental sensors already speak their own protocol before the project even starts. Integration adapts the sources instead of rewriting them. The problem is no longer to design one monolithic API, but to map a set of protocols, each with its own semantics.

It helps to keep two axes apart. The first is the interaction pattern: synchronous request/response against asynchronous publish/subscribe. The second is the nature of the data: registry-like and slow to change (a monument, a timetable) against telemetric and high-frequency (an environmental reading, a visitor count). The two axes do not coincide, and forcing them onto a single protocol costs dearly on one or the other.

The synchronous protocols

For registry and lookup data, REST over HTTP remains the option with the lowest integration cost: cacheable, inspectable, spoken by any client. The known limit is request fan-out when a view composes many resources — the familiar point-of-interest card that pulls down registry data, opening hours, events and photographs.

There were two alternatives available and mature enough to deserve evaluation. gRPC reached 1.0 on 23 August 2016, with stable bindings for C++, Java, Go, Python, Ruby, Node and C#; it runs on HTTP/2 and Protocol Buffers, and makes sense on internal server-to-server links, where the contract is fixed and latency weighs. Its drawback, in January 2017, is browser support: it needs a proxy, because the framed HTTP/2 stream is not reachable directly from client code. GraphQL takes the fan-out head-on: it lets the client declare the fields it needs. The specification has been in public development since 2015, and the reference edition is the one from October 2016. The server pays the price: HTTP caching no longer works per URL, and a single query can turn into a backend load that is hard to bound.

None of the three replaces the others. REST for public, cacheable exposure; gRPC for internal paths; GraphQL where client-side composition is the dominant constraint.

The asynchronous protocols and telemetry

Sensor data inverts the pattern. The source produces when it has a value, not when a client asks, and consumers can be many and intermittent. Here the publish/subscribe model comes naturally.

MQTT is the reference candidate for bandwidth-constrained telemetry. Version 3.1.1 has been an OASIS standard since October 2014, and in July 2016 ISO/IEC JTC1 approved it as ISO/IEC 20922:2016 — not a formal detail, because in a consortium with public partners an ISO standard shortens the argument over the choice. MQTT has a minimal frame overhead and three quality-of-service levels; the downside is that it leaves most of the semantics to the application layer (payload schema, topic-based routing), which therefore has to be disciplined upstream.

Where the requirement is system-to-system messaging rather than embedded telemetry, AMQP 1.0 offers a richer model of routing and reliability; it has been an OASIS standard since 2012 and ISO/IEC 19464 since 2014. The tightest requirement decides: small frames and constrained devices push towards MQTT, delivery guarantees and elaborate exchange topologies towards AMQP.

The critical point: cartography is not data like the rest

The geospatial component is where naive integration costs the most. A region already has its mapping services, and the Open Geospatial Consortium (OGC) standards are the common ground: WMS for maps rasterised on the fly, WMTS for pre-generated, cacheable tiles, WFS for queryable vector geometries. Treating these flows as generic REST endpoints and reinventing their conventions means giving up interoperability with the GIS tools that already speak them.

On the geolocated-sensor side, the OGC SensorThings API Part 1 (Sensing) was approved as an OGC standard in February 2016. It is relevant here because it holds the two axes above together: it exposes observation data over REST with JSON encoding and OData query conventions, and provides for MQTT to subscribe to updates. It is the one place where synchronous lookup and asynchronous telemetry coexist inside a single standardised data model, which is why it should be evaluated explicitly before setting out to build a bespoke abstraction.

Deployment and orchestration

How the service is delivered also bears on the choice of protocols. As of January 2017 the settled direction is containerisation: Docker brought swarm mode into the core with 1.12 (announced in June 2016, GA on 28 July), and Kubernetes reached 1.5 in December 2016. For a consortium whose partners run different infrastructure — some on-premise, some on public cloud — a container as the unit of deployment reduces portability friction more than any agreement on formats.

The open-source orientation here follows a practical criterion. Every component that enters a public platform with a multi-year life has to be weighed for maturity, licence, community health and maintenance sustainability beyond the funding period. A proprietary protocol or a broker without an active community becomes a deferred cost that lands on the body inheriting the system.

Limits

This is a map of options, not an architectural decision. The real selection depends on measurements I do not yet have in January 2017: real flow volumes, update frequencies of the public sources, field bandwidth constraints, the skills of the partners who will maintain the system. The rule I apply while gathering the material is not to introduce a protocol until a flow requires it for a property the others do not provide — a service level, a delivery guarantee, a geospatial convention. Each additional protocol is one more surface to document, monitor and keep alive.

The technological-scouting work this survey comes out of feeds into deliverable D1.3 of SMARTOUR’s OR 1.3 objective, coordinated by noze with 01 Sistemi: the operational start is described in the noze insight https://www.noze.it/en/insights/smartour-start/.

https://grpc.io/blog/ga-announcement/ https://spec.graphql.org/October2016/ https://www.oasis-open.org/2016/07/19/oasis-mqtt-internet-of-things-standard-now-approved-by-iso-iec-jtc1/ https://www.iso.org/standard/69466.html https://www.oasis-open.org/standard/amqp/ https://www.ogc.org/standards/wms/ https://docs.ogc.org/is/15-078r6/15-078r6.html

Cover image: The Greek Doric temples of Paestum seen across the archaeological site: two large ochre limestone temples with intact colonnades… — photo by Miguel Hermoso Cuesta, CC BY-SA 3.0 — https://commons.wikimedia.org/wiki/File:Templos_Paestum._01.JPG