Keywords

1 Introduction

An anticipatory System acts not only on the basis of past and present states but also on the anticipated future states of itself and the environment [1]. Anticipatory systems are complex and intelligent. As a result they present a great deal of challenge in understanding and even greater in attempting to construct them. So far, anticipatory systems, intelligent systems (AI) and a variety of natural systems such as biological, social systems, etc., have been studied separately and have resulted in separate treatments and analyses. These studies, although very rich and extensive (for an extensive list of related work see references in the compendium by Vladimir Arshinov and Christian Fuchs [2]), still need to help fill knowledge gaps in handling present day systems that integrate diverse elements. For example they involve cooperative working of several intelligent agents, or joint working of biological and mechanistic systems. It is generally known that the primary task in integrating such systems is to address the optimization of communication, computation, and information utilization; however, there is a need for models and methodologies that deal with all these aspects in a way that is transparent across different system types (e.g., machines, biological, social, etc.). Even within communication between similar entities, the merging of communication and computation has enabled distributed intelligent systems where communicating entities act on the basis of the “meaning” of information transacted thus dealing with the intent as opposed to the content. In general the important part played by communication among the various entities needs to be addressed from the perspective of meaning and intent rather than merely transmission and reception. In the work presented here we have drawn upon our past work [3–5] and attempted to utilize some of those concepts to unify the understanding and visualization of systems with diversified intelligence. The central model that facilitates this is the Information Engine concept. Our presentation begins with an overview of the information engine model and then we show how it applies in a simple fashion to various systems such as an anticipatory system, multi-agent system, hybrid brain-machine composite, etc. An important concept derived from the information engine mechanization is that of Logical Work that is used to define an intelligence metric that is meaningful across diverse systems.

2 Information Engine

An Information Engine is set up as consisting of an information source, a processing agent, and a sink. Detailed discussions of the Information Engine including optimality are available in our earlier work [3]. Here is a brief overview. This engine is modeled in a fashion similar to a heat engine where the engine takes heat from the source and converts some of it into mechanical energy and discards the rest of the heat to the sink, the mechanical energy is then used to drive external applications. The performance of the heat engine is analyzed in terms of physical parameters such as temperature, pressure, volume, entropy, etc. Modeling of the information engine is motivated by Carnot’s engine of the second law of thermodynamics, which is known to be an optimal heat engine. Carnot’s engine is characterized in terms of temperature and entropy. In order to formulate the information engine, we define a parameter Information-Need as analogous to temperature and assign the physical entropy (the sum of Shannon, i.e. the statistical entropy and the Kolmogorov/Algorithmic entropy) to information fragments. Although metrization of information-need continues to be a research topic, some more details are discussed in our prior work [3]. This engine is seen to work between the information source and sink to transform the random ensemble entropy (Shannon entropy) of the source information to algorithmic entropy (Kolmogorov entropy), which gives the information needed to directly effect the action to drive the desired application in a way analogous to the conversion of heat to mechanical work by a heat engine working between a heat source and a heat sink. Considering Semiosis as the action of extracting meaning from the information ensemble, such an information engine is representative of a semiotic loop.

Consider two systems S1 and S2 that can communicate and work cooperatively. Further, consider that the system S2 is assigned an application task for which it has insufficient information that can be obtained from a source S1. The corresponding information engine cycle is shown in Fig. 1. The engine starts at the point d with S1 being a source and S2 being a processor of information. During the leg d–a S2 recognizes that it needs a certain fragment of information from S1 to proceed with the assigned task i.e., the target application. Data containing this information is obtained by S2 from S1 during a–b. This is followed by the leg b–c during which the acquired data is processed by S2 to generate the necessary algorithm or compacted information necessary for actuation. During b–c the processing converts some of the Shannon entropy that comes from the uncertainty in the acquired data to Algorithmic (or Kolmogorov) entropy that may be useful to the target application. The next leg c–d denotes the reduction of entropy due to selection of the algorithms generated in the processing during b–c and utilized by the system for actuation. The remaining Shannon entropy is discarded and the cycle restarts by evaluating further information need through the leg d–a. Thus the useful information made available by the system to the application is equal to the algorithmic entropy extracted by the system for use by the application as represented by the leg c–d.

Fig. 1
figure 1

Information cycle

Full size image

3 Intelligent Systems

There is a large number of Intelligent Systems in existence today, their numbers as well as variety is increasing. A single system may consist of several diverse subsystems with various levels of capability and autonomy collaborating with each other. Architectural diversity is a fact of life. In general an intelligent system strives to extract useful information from the given raw information ensemble. The “intelligence” of system is characterized by the effectiveness with which it does the above extraction that in turn is concerned with the efficacy of information transactions and processing to enable taking an action.

An intelligent system can thus be modeled as an information engine which would transform the raw information to a form that can be directly utilized by the target application. As described in the previous section such an information engine would consist of four fundamental processes, recognition of information need, acquisition of data, processing of the data for information generation, and the separation of the algorithmic entropy and its use by the application for actualization of its need. These four processes when represented in the space of Entropy/Information Need, form a cycle analogous to a thermodynamic cycle (engine) represented in the Entropy/Temperature space. Since this engine is modeled in terms of fundamental notions such as Shannon and Kolmogorov entropy, using it to construct measures of intelligent processing of information in systems is very attractive. Analogy of the Information Engine with thermodynamic cycles prompts us to consider the area of the cycle that consists of the product of the Information Need and the Entropy to be the logical work performed by the information engine in driving the target application. Logical work becomes a representation of the value produced by the information engine and when normalized with respect to the information input into the system, it leads to a measure of intelligence.

4 Semiosis in Intelligent Systems

In intelligent systems actions are dependent upon meanings arrived at by interpretations rather than merely the raw data that is a priori available. Thus, semiosis as the process of meaning extraction plays a vital role in intelligent systems. Although there are some excellent expositions on semiotics in the context of complex systems that offer detailed discussions on semiosis, the information engine representation ties the concepts together in the form of a concise fundamental architecture. In discussions of Complex Semiotic Systems for example Cliff Joslyn [6] states that, “semiotically closed systems maintain cyclic relations of perception, interpretation, decision, and action with their environments”. It is easily seen that these relationships are exactly those described by the four legs of our information engine. Howard Pattee [7] in his discussion of Semiotic Controls, has discussed the relationship and distinction between dynamics and semiotics as being similar to the formal and the functional and goes on to discuss the epistemic cut between them. Pattee says, “we must ask not what we mean by information but what the information itself means in the physical world”. This coincides with our approach that meaning is equivalent to action. Considering the end action to be the proof of the meaning, it is representative of the meaning itself. The information content of the meaning is such that it fulfils the need of the action. Meaning is purposive. The information engine combines the formal or dynamic effort of information collection and processing with the functional or purposive aspect of the algorithmic entropy and information need. Intelligent systems expend their dynamic effort towards purposive goals. This is accomplished through processing of information and selection of the results that are capable of neutralizing the need. All four legs of the information engine are essential for an intelligent system to function and therefore provide a good basis for modeling an intelligent system.

5 Logical Work Representation of Intelligence

Having said that an intelligent system is purposive, we can use this property to measure intelligence. Modeling an intelligent system by an information engine allows us to compare intelligence with the working efficiency of this engine. Although evaluating the efficiency of a real system is expected to be fairly complicated, studying an ideal cycle such as described above adds useful insights to the construction of measures of intelligence. In Carnot’s cycle of thermodynamics, which motivated our Information Engine model, the area of the rectangle describing the process in temperature-entropy space yields the mechanical work output by the engine [8]. In a similar fashion, the area of the rectangle representing the information engine process in the information need—entropy coordinates can be considered to be the Logical Work output of the information engine. This is justified by the following reasoning. Work performed by an information engine must identify with the functional value of the evolved algorithm to the specific target application and the intrinsic complexity of the algorithm itself. An information processing system that generates a piece of information with high complexity which is capable of satisfying a large portion of the information need is associated with a large measure of logical work. Thus the product of the neutralized information need and the Kolmogorov complexity of the algorithm generated is a suitable measure of the logical work output of the information engine.

There are three concepts of efficiency relating to intelligence that emerge from the above model. Two of these relate to information generation, and information utilization. The two together determine the third one that represents a measure of the system intelligence.

5.1 Information Generation Efficiency

The efficiency of information generation is the ratio of the algorithmic information generated to the total information input to the system. In terms of the corresponding entropies this may be written as Information generation efficiency:

$${\text{L}} = \left( {{\text{H2}} - {\text{H1}}} \right)/{\text{H2}} = 1- {\text{H1}}/{\text{H2}}$$
(1)

5.2 Information Utilization Efficiency

The information utilized depends upon the utility of the generated information. Thus this is the product of the algorithmic entropy and the need that it satisfies.

$${\text{U}} = \left( {{\text{H2}} - {\text{H1}}} \right) \cdot \left( {{\text{N1}} - {\text{N2}}} \right)$$
(2)

This is to be compared with the possible utilization had the total need of N1 been fulfilled. This would be:

$${\text{V}} = \left( {{\text{H2}} - {\text{H1}}} \right) \cdot {\text{N1}}$$
(3)

Thus, Information utilization efficiency:

$${\text{M}} = \left( {{\text{N1}} - {\text{N2}}} \right)/{\text{N1}} = 1- {\text{N2}}/{\text{N1}}$$
(4)

It is observed that the information utilization efficiency is similar to the efficiency of heat to mechanical work conversion in the Carnot’s cycle. This points to the value of a piece of information for an application to be analogous to the mechanical work performed by a heat engine. For actuation an application always seeks meaning in the information.

5.3 Information System Efficiency

This is an indicator of the value of the algorithmic information generated to the application in relation to the information value that was provided in the raw data to the system. Information system efficiency:

$${\text{J}} = \left( {{\text{H2}} - {\text{H1}}} \right) \cdot \left( {{\text{N1}} - {\text{N2}}} \right)/{\text{H2N1}}$$
(5)
$$= {\text{Info}}.{\text{Generation}}\,{\text{eff}}.{\text{X}}\,{\text{Info}}.{\text{Utilization}}\,{\text{eff}}.$$
(6)

We consider Information System Efficiency to be an indicator of the measure of intelligence in a system. As seen above this can be factored into the capabilities of the system to generate algorithmic information and that to utilize from it what is needed.

As an example of such factorization consider a search engine that processes the enormous data available to narrow down to a set of information fragments that contain the supplied keywords. However, within the members of the generated set, one that is the best match to the context is figured out as part of the utilization.

As an observation of the information cycle diagram it is noticed that cycles described by tall rectangles would be associated with small generated entropy, characteristic of simple algorithms, however satisfying large needs. On the other hand, cycles described by broad rectangles are associated with handling large statistical entropies and that come up with relatively large algorithmic entropy and correspondingly complex action but satisfy modest needs. One must take cognizance of this composition while constructing intelligent systems.

6 Anticipatory Systems

Robert Rosen defines [1] an anticipatory system as “a system containing a predictive model of itself and of its environment, which allows it to change state in accord with the model’s predictions pertaining to a future instant”. Based on the information engine model we examine an anticipatory system to understand the relationship between its mechanics and the intelligence in its functioning. Rosen’s definition, as seen by his modeling relation, may be broken down into two major structural characteristics of an anticipatory system S2 working in cooperation with another natural system S1. First, S2 possesses a predictive model of S1, and second, at any instant, change of state occurs in S2 as a function of its predictions about S1. Since updates of the model of S1 in S2 necessarily involve communication with S1, it is seen that the cooperative working of S1 and S2 at once fits the information engine model. Various aspects of this model are discussed in detail in our previous work [4]. Here we would like to first highlight the way in which the anticipatory nature of the system is reflected in the information engine model. After this we discuss a high level architecture applicable to the construction of an anticipatory system.

6.1 Anticipation as the Final Cause

Referring to the diagram of information engine described in the section above, the system S2 obtains information from S1, which is the environment here, during the leg a–b. This is a data gathering process. However, it needs to be in consonance with the need to be fulfilled. Thus it depends upon goal related causality (Aristotle’s “final cause”) that is part of the leg d–a that projects the need. Once S2 has the raw data, it processes it in conformity with its goals and in association with the information that it may already possess about S1 in the form of a model of S1; this happens in the leg b–c by way of conversion of the Shannon entropy to Kolmogorov entropy. Then during the leg c–d, the separated algorithm is made available for anticipatory action; this is a part of the intelligence but one that does not involve anticipation. Anticipation depends upon the recognition and estimation of need, and this happens during the leg d–a. Optimality of the system is also closely tied with this function since for minimization of the data acquisition and processing effort, the data to be collected must be maximally useful for the final task.

6.2 Architectural View of Anticipatory Systems

Anticipatory Systems involve prediction, communication, processing and interpretation in a knowledge environment. In the past since all these individual areas progressed independently architecting the combined systems escaped serious attention. Here we touch upon a few aspects that are useful in engineering such systems.

As seen from the information engine representation, architecting an anticipatory system calls for engineering four broad categories of functions. It calls for estimating information needs, acquisition of data from all relevant sources, intelligent processing of information, and planning and commitment to appropriate actions. Estimation of information needs as well as intelligent processing of data require knowledge resources. Correspondingly, the total infrastructure may be factored into the following three categories:

  1. 1.

    Service plane: Sets up communication with all entities, acquires data as needed, registers service requests, etc.

  2. 2.

    Knowledge plane: Provides knowledge assets, carries out all cognitive processing, maintaining knowledge bases, and updates models of self and other entities.

  3. 3.

    Action plane: Does planning, decides on commitments, and effects actions.

This concept of the planes enables the infrastructure to be implemented using any combination of hardware, software, firmware, or network assets. This is similar to the control and user plane concepts used in present day telecommunication networks. The action plane can be thought of as part of control plane, however, we prefer to separate it because it has more of an AI flavor while a control plane is related more to data transport. Although the knowledge plane has been talked about [9], its use does not seem to be quite so prevalent yet. For an implementation example, the interested reader is referred to the analytics of the author’s patent on an adaptive phone for users with dementia or other mental impairments that illustrates the use of knowledge plane involving cognitive processing [10].

Further to the resource categorization in the three planes, following aspects merit special attention while architecting for performance.

  • Modeling of various system entities including external world representation necessary for estimation of information need, acquisition, and processing

  • Efficient maintenance of knowledge databases and models in the system

  • Use of domain ontologies consistent across the entire system

  • Efficient information acquisition and interpretation using robust communication protocols

  • Cognitive processing consistent with the task, data, and the knowledge bases.

  • Efficient planning to commit from the outcome of cognitive processing

  • Monitoring intelligence measures of the system

Discussions of the above considerations in relation to multi-agent systems are available in [5]. Further expansion in terms of actual design would depend upon the specific task situation.

7 Hybrid Cooperative Systems

Since machines are generally not considered as effective as natural systems in anticipatory capabilities, a composite working of machine and natural system can have significant merits over the two individually. In “The Limits of Intelligence” Douglas Fox [11] has projected an analysis to show that it is unlikely that human brain by itself may evolve considerably further and suggests that a good way to enhance intelligence would be through augmentation with internet. Undeniably, although presently difficult to implement, in the future it is quite conceivable that human brain could be closely coupled with a computing system or other machines to increase the total intelligence. Will it increase intelligence? And how to optimize the enhancement? To explore answers to this we must examine and analyze the information transactions and processing that are expected to occur in the composite system and then optimize the cooperative working. We cast this problem as a cooperative multi-agent system and apply the information engine model. Formal treatment of communication and cooperation among intelligent agents has been addressed by many authors, for example see Haddadi [12]. Reliability aspects of such systems are considered in [5]. Here we would like to consider the intelligence aspect of the cooperative working.

We consider the machine and the brain as two intelligent agents and set up an information engine between them to examine their cooperative working. To show that intelligence is increased, we need to observe that now logical work can be performed in both agents provided they both have the capability to a) identify their information need from the other and b) they can process the information received from the other to perform the global service. The contribution of each to the total will be equal to the logical work performed by each which in turn is determined by the product of the amount of raw information processed and the need fulfilled by each. The enhanced computational capabilities of present day and future machines with large processing capability suggest that they can make a large contribution to the logical work. However, according to the information engine model we see that this will happen only if the machine knows what its information need is. This can come either from the biological brain in the hybrid system or in a purely mechanistic AI system it would come from a proxy module. Thus in the hybrid system, the biological part could be assigned the executive function of factoring the information need of which it will process some and others would be passed to the machine. Notice that if the biological part were to pass all the information need for processing to the machine the two together will form only one information engine that is distributed over multiple locations. Although in either case the total logical work and correspondingly the intelligence may be increased, the multiple engine structure may be preferred from robustness and reliability perspective. Evidently much further work is needed in this open area of research.

8 Conclusion

In this paper we have examined the working of an anticipatory system as a semiotic loop involving information transaction as well as processing for meaning extraction in the context of an application task. For this analysis we used the information engine concept. Based on this we defined Logical Work, a notion of work in logical spaces that is analogous to the notion of mechanical work in physical systems. We proposed using logical work as the basis of a unified measure of intelligence that can be applied to diverse systems ranging from natural to mechanistic and hybrid systems. The system intelligence was shown to consist of two factors called the information generation efficiency and the information utilization efficiency. Each of these can be expressed in terms of the statistical entropy of the input data, the algorithmic entropy of the output information, and the information need that relates to the goal of satisfying the application. This kind of modeling demonstrates the epistemic cut between the dynamic and semiotic parts of the intelligent system; in fact the logical work output and therefore the intelligent system functioning itself requires both parts. The application of these principles to anticipatory systems and hybrid bio-mechanistic systems is illustrated.