The Knowledge Perspective
The Open Group recently published its Open Agile Architecture™ (O-AA) standard. It is the response of the enterprise architecture community to the increasing traction of the agile software movement, not only with software developers, but also with business executives. It is an important contribution to agile methods.
Meanwhile, the problem space is changing. Increasingly, enterprise systems are dealing, not just with information, but with knowledge.
This article looks at why knowledge processing is important, how traditional and agile methods of enterprise architecture have evolved in The Open Group, and how they can evolve further to architect knowledge-based systems.
More and more enterprises are using techniques such as data science and artificial intelligence. Their computer systems are processing, not just information, but knowledge.
What is knowledge, and how is it different from information? There is no clear distinction between them. Both can be represented by computer data. Knowledge data is richer, with more inter-relationships, so that each individual piece of information has a context. This makes it a better basis for estimates and projections.
The fact that I am browsing a garden product supplier's website is a piece of information. Add to that the information that I recently searched the web for "best flavor tomato," and some knowledge of tomato varieties, and the supplier might project that I would be interested in Gardeners Delight tomato seeds, and display a special offer pop-up. This kind of knowledge processing is commonplace in Internet shopping and increasingly in other applications such as (some would say unfortunately) politics. Most enterprises could take advantage of it to improve their marketing capabilities, and its use is likely to spread to other areas.
It has taken a lifetime for this to become possible. As a child in the 1950s, I remember being taken to see a computer that, it was claimed, could translate from French to English. What this meant in practice was that it had a look-up table giving the English equivalents of a few French words. Today there is a freely-available web service that will provide a fairly good translation of pretty much any French sentence. That is a measure of how far we have come.
Our challenge now is to make good use of knowledge processing in enterprise systems.
Back in the 1950s, J Lyons & Co was a conglomerate whose businesses included restaurants. Its Lyons Electronic Office (LEO) was a digital computer that ran accounting, inventory, payroll, and other applications. It was probably not much more powerful than the computer I saw, but it was the first to support business operations, the fore-runner of today's powerful and complex enterprise systems.
The word architecture was used from about 1960 to describe the problems of designing complex information processing hardware and software. Information Technology (IT) architecture gradually evolved as a computing discipline that addressed the organisation of digital computing systems supporting enterprise business operations.
Buildings architects communicate between clients who want and are paying for buildings, the contractors who are constructing them, and the suppliers of materials. They draw up plans that show the clients how the buildings will meet the requirements, show the contractors what needs to be done, and list the materials to be used. Then they organize and oversee the work. In the same way, IT architects communicate between stakeholders in the client enterprises and their IT suppliers and developers.
It was in the 1990s that the X/Open company, which later became part of The Open Group, reviewed the existing architecture frameworks and chose TAFIM from the the US DoD to be the basis of its TOGAF standard. With more than 87,000 certified enterprise architects and trainers worlwide, TOGAF is now the most widely adopted open enterprise architecture framework.
TOGAF defines a vocabulary containing terms such as "application" and "platform" that IT architects can use to communicate, and describes an Architecture Development Method (ADM) that they can use to draw up plans for systems, and to organize and oversee their implementation.
It is well suited to business IT systems such as those pioneered by J Lyons and Co. At its heart is the idea that you look at how the business works before deciding what applications and data stores are needed to support its operations, and then specifying the computing platform on which they will run. This is done in its Business Architecture, Information Systems Architecture, and Technology Architecture phases. The figure shows how this might have applied to Lyons' restaurant business.
As the interdependence of enterprise IT and business operations became more apparent, TOGAF became an enterprise architecture framework with a wider scope than just IT. Design of information systems and supporting technology, though important, was only one aspect of this. A large corporation might have many divisions and departments, each a small enterprise in its own right. Maintaining efficient technology support for business in such a corporation required a number of collaborating architecture teams. TOGAF gave them a common language and methodology, so that they could collaborate effectively.
Agile Software Development
Although its scope was wide, TOGAF did not apply to all IT work. It did not address the design and delivery of digital products, though aspects of it may have been applicable to that. Representatives of product vendors participated in Open Group discussions, but their aims were primarily to define and maintain standards that would enable markets for their products, rather than to agree common approaches to product architecture. The scope of TOGAF was the design of the enterprise itself. As more enterprises had products that were digital, and used digital technology to deliver and support all their products, the limitations of this became increasingly apparent.
At first, the development and delivery of digital products used similar methods to those used for manufacturing physical products, optimised for large-scale production of identical units. The digital market demanded a different kind of scaling, the ability to produce large numbers of different versions of a product to deliver new features and appeal to a range of consumer tastes. Eventually, new "agile" methods emerged. Arguably, they are comparable in importance to assembly-line methods in manufacturing, delivering scalability (of a different kind) with faster production at lower cost. They combine product development, delivery and support as devops, and enable "continuous delivery" of new product versions.
Agile software practices evolved in the 2000's, following the principles of the Manifesto for Agile Software Development that was published in February 2001. Ten years later, the Scaled Agile Framework (SAFe)® appeared, seeking to codify the principles, practices, and competencies for achieving business agility using lean business methods and agile software development. There is a large body of instruction and guidance on agile methods, from a number of sources. There is also a wide range of supporting products and services.
The Open Agile Architecture Standard
Now, The Open Group has added O-AA. This has a different scope to TOGAF, including digital product architecture, where agility is particularly needed. It is positioned as a complement to TOGAF, not as a replacement. But, like TOGAF, it defines a vocabulary and describes methods for architects to use.
Its vocabulary is radically different from that of TOGAF. There are some common core terms, such as architecture and capability, but O-AA discards many TOGAF terms and instead has new ones, such as customer experience, lead time, and outcome that are related to digital enterprise agility. Some of the agility terms, such as emergent and intentional (as applied to design and architecture), are also found in the SAFe vocabulary but, although there is some overlap, O-AA and SAFe are significantly different.
The methods and techniques described in the O-AA standard are also radically different to those of the TOGAF ADM. The TOGAF ADM has nine phases and a continuous requirements management process. The O-AA identifies three perspectives, defines a set of building blocks, sets out axioms for the practice of agile architecture, and describes a number of agile work practices, such as guardrail and continuous architecture refactoring. Again, there is some overlap with the agile practices of SAFe, but significant difference.
The O-AA Perspectives
The perspectives provide an interesting basis for comparison. Architects look at their systems from many different perspectives; this is one of the reasons why a well-architected system is superior to one that "just grew". Business, information systems, and technology are three perspectives that the TOGAF standard identifies as being of primary importance, and an architect following its ADM looks at the enterprise from each of them in turn.
O-AA also identifies three perspectives as being of primary importance, but they are subtly different from those of TOGAF. They are: experience, which defines value from a client perspective, work system, which defines the ability of the enterprise to deliver client benefits efficiently, and technical, which covers the software and hardware parts of the enterprise. The architecture team views the system from these perspectives simultaneously. The idea of using perspectives to define sequential work packages has gone, but the essential concept of perspective remains.
From each perspective, the O-AA recommends a specific approach to the definition of building blocks. From the experience perspective, it recommends pursuing product discovery in parallel with a combination of traditional market research and customer/user experience research. From the work system perspective, it recommends shaping the enterprise’s organization in a way that mirrors its intentional product and software architectures. From the technical perspective, it recommends domain-driven design. The figure shows the three perspectives and the views they give of the building blocks, as they might relate to the use of computers by a modern restaurant.
The Knowledge Perspective
This certainly helps architects be more agile, but the O-AA does not help them to create the systems based on knowledge processing that today's enterprises need.
Architects should view knowledge-based systems from a knowledge perspective. Seen from this perspective, the primary building blocks are the sources of data, metadata describing the data, and reasoning modules that create useful business knowledge.
These can use deductive reasoning to reach conclusions based on arithmetic and logic. For example, in a restaurant system, they could determine the most popular dishes by calculating how frequently they appear in orders. They can also use inductive reasoning, based on techniques such as deep learning and knowledge graphs, to generalize or extrapolate from specific cases to assert new propositions. For example, by looking at a customer's past orders, and noting that they include no dishes containing meat, they might assert that the customer is probably vegetarian. Knowing what dishes are popular, and how many customers are vegetarian, helps managers plan future menus.
Because knowledge is represented by data, the knowledge perspective is inherently data centric.
Architecting Knowledge-Based Systems
For a knowledge-based system, knowledge should be a top-level perspective in the architecture development.
It will not be the only top-level perspective. The Experience perspective is crucial for any enterprise that delivers digital products or services, or that uses digital technology to interact with its customers - in other words, for nearly every enterprise. The work system and technical perspectives should also be at the top level. Much of traditional business architecture could now be viewed from the work system perspective, but some artifacts, such as a high-level capabilities map, should still be developed from a separate business perspective. The information systems perspective will remain important for enterprises that use off-the-shelf applications and enterprise databases.
Definition of building blocks from a knowledge perspective should not be a phase in a "big, up-front design". New sources of data can be added as business operations change. Reasoning modules can be added, changed and removed as different kinds of knowledge become more or less useful, and as technical advances enable knowledge to be deduced, or induced, in new ways. The knowledge perspective fits well with agile methods, and is a natural approach to agile architecture.
Computer systems have grown in power and fallen in cost. Information processing has become essential to support business operations. Business IT systems have grown in scale and complexity. Enterprise architecture evolved as a discipline to manage this complexity.
Digital technology became an increasingly important component of enterprise products and services. The market demanded faster delivery of new functionality. Agile software development methods were adopted to achieve this. Enterprise architecture has adapted to this increased agility.
Systems are now powerful enough to deliver knowledge processing. Digital products and services are becoming knowledge-based. This can improve enterprise effectiveness, but there are ethical and legal issues. Enabling knowledge-based enterprise operation, and the development and delivery of knowledge-based products and services, while maintaining the legal and ethical character of the enterprise, is enterprise architecture's next challenge.
Meeting the challenge does not mean abandoning traditional well-tried methods, or losing the more recently acquired agility. But it does mean embracing new knowledge-processing technology. Most importantly, it means treating knowledge as a primary perspective from which to view enterprise systems.