Dawn of a new era: Why systems of engagement require systems of intelligence

As the world shifts from physical to virtual assets and methods of engagement, there is an increasing need for systems of intelligence – and the data platforms that enable them – to deliver contextually relevant systems of engagement, alongside more traditional systems of record.

The 451 Take

Businesses need a digital platform that can leverage existing investments in transactional systems while providing additional capabilities for engagement and insight. Figure 1, below, shows the future of digital transformation to be the fusion of systems of record coupled with data and analytics for actionable insight, combined with engagement-driven applications to improve process efficiency. Engagement-driven applications paired with data will provide a more dynamic experience. However, the newest investments in systems of intelligence provide the power. The augmentation of systems of record with the combination of systems of engagement and systems of intelligence will ultimately drive and enable digital transformation.

Figure 1

451 Research has previously described how intelligence, agility and customer-centricity are the three imperatives of digital transformation being adopted by organizations as they seek to transform how they serve customers, employees, and partners; support continuous improvement in business operations; disrupt existing businesses and markets; and invent new businesses and business models.

As we have also previously described, in order to drive digital transformation, new enterprise applications need to be intelligent, agile and customer-centric. Furthermore, they must be designed specifically to serve and address engagements that occur in the virtual world with virtual assets, rather than the physical world with physical assets. These new applications (or systems of engagement) involve the three imperatives of digital transformation – intelligence, agility and customer-centricity – and are being developed and adopted to replace or augment traditional transactional applications (or systems of record) that have been used to record the results of interactions that occur in the physical world.

Machine-learning-powered applications play an important role in the evolution of these emerging applications. It is these systems of intelligence that support new systems of engagement, enabling enterprises to communicate and transact intelligently with the customers through operational applications and other engagement channels.

Figure 2, below, describes the relationship between systems of engagement and systems of intelligence, as well as the difference between the latter and more traditional business intelligence tools and reports (for completeness, let's call these 'systems of analysis') that drive human decision-making and help power the systems of intelligence.

Figure 2 – Systems of record, engagement, analysis and intelligence

Perhaps the best way to describe how the systems of record, engagement, analysis and intelligence interact is via example. Consider a traditional retail store – the engagement with the customer has, to date, been performed by the employee, who may make suggestions and recommendations and answer questions.

Assuming a purchase is made with cash, the transaction also occurs in the physical world, before details of both the transaction and the engagement are recorded in the retailer's systems of record – the ERP or financial and CRM applications. Systems of intelligence improve individual experiences with data that is updated constantly (e.g., transactions, events, contexts, interactions and behaviors) and tied to a unique identity for each customer, in order to build a complete customer profile. Then that information and identity must be turned into prescriptive insight using machine-learning-based algorithms to identify customer opportunities and determine how to best engage with customers across multiple channels and devices.

Software is also being used to replace the engagement role played by the employee in a brick-and-mortar environment. Chatbots and digital assistants may answer questions and make suggestions, while potential purchases may be recommended and personalized offers made based on previous transactions.

Smart bots and robotic process automation (RPA) are also potential elements of the digital platform. Because the new model for interaction is conversation, smart bots can be embedded from open protocols like SMS or email for better customer engagement processes. Not all bots use machine learning, but it is a critical element for improving the accuracy of the conversation. Smart bots will also further enhance the capabilities in digital transformation initiatives, but today they are most popular in customer-engagement use cases that enable personalized, structured responses, and in automating services and business workflows.

These interfaces between the enterprise and the customer – the application, digital assistants, smart bots and chatbots – are the new systems of engagement, but they need to be driven by intelligence. Specifically, new systems of engagement are enabled by rules engines, decisioning systems, recommendation engines, natural language processing, image recognition and other forms of artificial intelligence, including machine learning and deep learning algorithms.

Figure 3, below, illustrates how deterministic, rules-based approaches still add value by targeting segments such as high-value customers and anonymous visitors. However, they don't provide 1:1 algorithm-driven recommendations, which require a more cognitive approach.

Determinism works well when tracking and using data such as clicks, time spent, mouse movement, hovers, scroll and inactivity. It can incorporate each visitor's referring site, geolocation, industry, and online ad or email campaign source.

Figure 3 – Data-driven experiences must take a hybrid approach

However, the cognitive approach applies algorithmic, predictive machine learning for an even stronger engagement strategy. The deeper data and improved algorithms provide the ability to factor in individual affinity, segment and survey-response data, and overall intent, resulting in greater relevancy and effectiveness.

Machine-learning algorithms can self-learn to adjust or adapt based on any factor or combination of factors in each visitor's personal interests/preferences. While an algorithm can be based on a visitor's behavior or geolocation, it can also be based on key company variables, such as inventory levels and manufacturer incentives.

These systems of intelligence enable the enterprise to engage intelligently with the customer through operational applications and other systems of engagement. In order to do so, these systems of intelligence must operate in real time. It is not enough to generate intelligence via traditional business intelligence tools and reports (systems of analysis) used by humans to make data-driven business decisions. Instead, the emerging systems of intelligence provide the automation of data-driven decisions, delivered via operational applications.

Data is the fuel of intelligence and engagement

As noted above, in an online retail environment, every aspect of that engagement needs to be performed by the application itself. That puts huge demand on the application to be responsive in real time, and generates enormous volumes of data that potentially need to be stored, processed and analyzed. What we've seen in recent years is that the cost of storing and processing data has decreased enormously, thanks in part to open source software like Hadoop and NoSQL databases, but driven primarily by advances in the cost of storing and processing data in commodity hardware.

High-performance processors and memory advances have driven down the cost of utilizing scale-out architecture to store and process huge volumes of data, particularly unstructured data that would previously have been ignored because the cost of storing and processing it in traditional scale-up data-warehouse environments was so high. As such, data sources like server log and web log data are now providing greater insight in terms of how customers are engaging with online applications.

This reduction in the cost of processor performance has opened up opportunities for enterprises to change their approaches to analyzing data. Traditionally, enterprises had transactional systems that acted as the systems of record, and then that data was extracted and loaded into a separate system of analysis (typically a data warehouse) to generate business intelligence.

The reason for that separation was not process elegance, but because analyzing data in the transactional environment would slow that environment down to the extent that it wasn't actually usable as a transactional environment.

Greater processor performance is removing that limitation, to the extent that it is now possible to perform analytics on transactional data. However, the demands of new interactive applications mean that it is now actually essential to incorporate intelligence – the result of analytic processing – into the operational application.

Platforms for intelligent engagement

So what are the underlying platforms that support the new systems of intelligence and deliver new systems of engagement? There are multiple ways of delivering that combination of interactivity, scalability, performance and availability, and there is no one right way of doing it.

Certainly, we see enterprises investing in Hadoop (and its ecosystem) as a platform for low-cost data storage and processing, especially to serve as a data science workbench, enabling analysts and data scientists to explore new business models and ways to engage with customers – with the associated machine-learning platforms and advanced analytics tools and projects, of course.

We also see enterprises investing in NoSQL and NewSQL databases to deliver the high levels of performance and availability required for interactive applications, through which to operationalize the results of the data-science work in Hadoop. It is this combination of processing large volumes of historical data and rapid processing of real-time data that is essential to delivering results. For example, a company's data science team will process and analyze large volumes of historical data in order to identify the best algorithms for making online offers in a retail application. This is done via existing systems of analysis. However, this needs to be combined with real-time information about a customer and their recent browsing and search history in order for the recommendation to use the algorithm to create a targeted offer that could encourage that customer to make a purchase, or prevent them from churning.

This is automated through systems of intelligence and then delivered to the user via systems of engagement. In addition to building their own platforms based on Hadoop and NoSQL/NewSQL (among other things) to deliver insight-driven experiences, enterprises are also turning to prepackaged customer data platforms, illustrated in Figure 4, that harness the power of data and add a layer of predictive machine-learning intelligence to achieve real-time 1:1 capability in less than 20 milliseconds. The deeper data and improved algorithms now enable individual affinity, segment and survey-response data to be factored in, along with overall intent, resulting in greater relevancy and effectiveness.

Figure 4 – Information, identity and insight feed contextual experiences

By and large, most enterprise applications are built on a more traditional relational software stack. Delivering systems of engagement and intelligence in existing databases is by no means impossible, and due to sunk costs and investments in skills, we continue to see many enterprises looking to extend the life of their existing data processing platforms. Additionally, there will continue to be a role for the relational database to support the transactional systems of record. However, we do see an increasing number of enterprises looking at emerging data processing and analytics technologies, as well as customer data platform providers, as part of overall digital transformation efforts.

New Alert Set

"My Alert"

Failed to Set Alert

"My Alert"