Marketing Middleware and Data Layer

Term “marketing middleware” was recently pioneered by marketing technologist and blogger Scott Brinker (at least in context of marketing technology). Search for term “marketing middleware” on web leads to Scott Brinker’s posts on Marketing Technology Landscape Supergraphic and Tag management software as marketing middleware. Scott’s marketing technology landscape post is one of the most talked and referenced article this year in marketing technology space. I wrote my view on this subject with stress on the API Platforms as enabler for marketing and digital transformation.

What is Marketing Middleware

Marketing Middleware acts as glue and integrates various marketing products (here on we will call them components). Concept is quite similar to traditional middleware. The key here is to avoid exponential number of point-to-point integration between n number of components.

Lets take following illustration as an example of point-to-point integration approach, we have 4 components. Hence we need 6 point-to-point connections and 12 actual point-to-point integration. Extending this analogy, with 5 components you will need 10 connections and 20 integration. It is not uncommon these days to have upto 10 components which means you are looking at 45 connections and 90 integration. In nutshell, a point-to-point connection approach requires a lot of effort in integration with drawback of “tightly coupled” connections between components.

Unlike the point-to-point integration, a middleware solution requires less effort in integration and offers “decoupled” or “loosely coupled” connections between components. Think point-to-point solution like everyone driving their own car to go different places on a route. A middleware is more like a bus people can get in and get down anywhere on the route. In layman terms, a middleware acts as middle-man and hides the integration complexity. Each component only talks to middleware and middleware communicates with other components. Following is highly simplified version of middleware.

Actual mechanism how a middleware works depend on type of middleware. Due to loose coupling, middleware enables to add new components or remove/version existing components without disrupting any existing integration. There are currently 5 classes of marketing technology products which can be considered as middleware,

1. Data management platforms (or customer data platforms)
2. Tag management systems (or data layer based middleware)
3. User management systems (SSO)
4. Cloud connectors and ESB
5. API management systems

Data Layer as Middleware

Traditional Tag management solutions (TMS) acted as glue between page and marketing tags. Depending on url, TMS load rules will inject the appropriate tags into the page. With invent of data layer the role Tag management systems is changed a lot. I have written extensively about data layer and emerging W3C’s Customer Experience Digital Data Layer (CEDDL). In nutshell,

A data layer is a standard way to format data within a web page. It is collection of one or more JavaScript Objects (JSO) holding information or important signals about page and user.

and most interesting bit,

It also makes data reusable and act as single source of truth for all analytics and marketing tags. Using data layer tags can communicate and share data on the web page.

I think real deal here is the data layer and not Tag management system itself. In absence of a standard data layer, Tag management systems are playing a middle-man role. Most Tag management systems have developed their own data layer, corresponding data layer enrichment, sharing and activation protocols.

Data Enrichment

Data enrichment process takes both online and offline data from various sources and places them into data layer. Data can be attributes about visitor, audience segment, product or campaign. Data layer can also be enriched with derived attributes. A good example of derived attributes will be lifetime value of visitor based on first interaction timestamp. Enriched data can be stored in browser’s local storage or cookies for future use and can be refreshed every time user re-visits website. Currently both Tealium and Ensighten support data layer enrichment using first party data. In addition, tags can also enrich data layer (add, update or append new attributes to data layer). Often page data and attributes are already populated in data layer.

Data Sharing

Data layer is accessible to all tags. As data layer is specific to Tag management solutions and tags have no idea about data layer convention, a data sharing protocol will be explicitly implemented in Tag management system. This includes data mapping, data translation and data transformation utilities provides by Tag management systems. For instance, Tealium provides data mapping toolbox, extensions for transforming data layer object like data flatten, data conversion, etc.

Data Activation

Once data is enriched and shared this data can be used for activation or action across all digital touch-points. Using enriched data layer one can deliver relevant messaging and content. For instance once visitor attributes are populated in data layer we can use these visitor attributes to define tag load rules. This means you can conditionally load display ad tag if visitor attribute (previous display ad interactions) match your criteria.

Tag management systems as evolved Middleware

However, current data sharing based integration between marketing tags is quite lightweight, and you can do very limited things (enrich, share and action). In addition, with emergence of a standard data layer (W3C’s CEDDL), this lightweight integration can be easily achieved without a Tag management system given that all tag providers start supporting CEDDL.

So what future holds for Tag management systems?

To answer this question we have to look how most of non-marketing Middleware solutions evolved into service-oriented Enterprise Service Bus (ESB). Messaging was one of they key aspect of the Middleware solutions, but ESB went beyond messaging and introduced orchestration, routing, transformation, mapping, adaptation, event handling, versioning, mediation, etc.

This is how I believe Tag management systems will evolve. Data layer (“messaging” equivalent or data sharing) will become standard commodity. Tag management systems will be more involved in the data enrichment, data transformation, data security, data privacy and data activation (the differentiator factors). Luckily the W3C’s CEDDL specification already describes data access control mechanism to support both data privacy and data security. It is more likely that Tag management systems will marshal these security and privacy rules. In addition Tag management systems will offer orchestration, loading, event handling and versioning (the commodity stuff). In this avatar, Tag management systems will be more architecturally and functionally evolved like ESB.