Electronics and Information Engineering: A New Approach to Modelling 1880–1950

Abstract

Most of the literature on mathematical modelling has been devoted to the natural sciences and economics; comparatively little has been written on the specific characteristics of modelling for engineering and technology. This chapter will briefly examine some such specifics in the context of information engineering, by which is meant here the engineering disciplines of electronics, telecommunications, signal processing and control. It is claimed that there are some very significant differences between modelling for engineering—at least for information engineering—and modelling in the natural sciences.

1 Introduction

A number of the major scientific figures of the nineteenth century considered deeply the question of modelling: what, precisely, a model is, and how it relates to reality. These figures (Maxwell, Boltzman, Hertz, Lord Kelvin, for example) have been discussed in depth elsewhere, and nothing more will be said on this. (For an interesting overview, however, see Monk 2012). Towards the end of the nineteenth century, electrical and telegraph engineers turned their attention both to modelling the phenomena they observed in the new technologies and to designing systems that would behave in the desired way. Although the mathematical techniques they used were mostly well established, such engineers developed novel ways of developing and employing them. Oliver Heaviside , for example, as well as simplifying and re-casting Maxwell’s equations in the vector form that subsequently became universal, developed his operational calculus (effectively equivalent to Laplace transforms) in order to model the behaviour of electrical circuits. By the early decades of the twentieth century the whole array of Fourier techniques was being developed, including convolution in the time domain as an equivalent to multiplication in the frequency domain.

Feedback circuits presented a particular challenge, as they were known to become easily unstable. Researchers at Bell Labs studied this problem in detail, resulting in the Nyquist stability criterion of 1932 and Bode’s monumental work on circuit design some years later. Karl Küpfmüller in Germany carried out similar, but rather less well-known work.

Some of the most impressive, and still under-estimated, techniques involved graphical tools for design. The Nichols Chart removed the need for difficult computation of closed-loop behaviour based on open-loop modelling or experimental recording, while the Smith Chart, although it will not be discussed here, provided a similar resource for engineers concerned with transmission lines (Bissell 2012). Such charts now form an integral part of computer tools—not, now, to replace calculation, but because they are still unsurpassed as ways of presenting information to the skilled engineer.

2 Oliver Heaviside: Changing the Paradigm

Until comparatively recently, Heaviside was largely neglected in the history of technology, but in the last 25 years or so several significant books and a number of articles have appeared (Nahin 1988, 2002; Mahon 2009) . Heaviside was a strange character, largely self-taught, and he engaged in sometimes vituperative exchanges with those with whom he disagreed—particularly with Sir William Preece, chief engineer of the British Post Office.

Born in 1850, Heaviside’s major contribution to the development of modelling in information engineering was his operational calculus , mostly published in the 1880s and essentially an application of the Laplace transform . Heaviside used the operator p ⁿ to represent the nth derivative in a differential equation, thus transforming an nth order differential equation into an nth order algebraic one—exactly as the D-operator is sometimes taught today. Using this technique, as well as some other quite advanced mathematical methods, he revisited a result derived by William Thomson (Lord Kelvin) on modelling the transmission of telegraph signals. Kelvin had neglected the self-inductance of the transmission line (which was valid, as signalling speeds at that time were sufficiently low for inductance to be negligible), but as speeds increased, the inductance of the cable played a major role, significantly distorting the signals, much to the puzzlement of practising engineers. Exploiting his mathematical expertise, which most electrical engineers at that time were unable to follow, Heaviside came to the counter-intuitive conclusion that loading the line periodically with additional inductance would greatly reduce the problem. Devices to do this were subsequently introduced in practice by M I Pupin and G A Campbell, and became vital to long-distance telecommunication cables .

Heaviside’s operational methods received a lot of criticism at the time, and were slow to be accepted. This is hardly surprising, since although the methods were based implicitly on Fourier and Laplace transforms—and some even earlier results—Heaviside was quite happy, for example, to expand his p-operator as an infinite series, multiply several infinite series together and even—in a famous paper correcting Kelvin’s analysis of the cooling of the earth—use the square root of p! Most mathematicians at the time were baffled that apparently correct results could be obtained in this way and Heaviside himself remarked that the notion of the square root of p is “unintelligible by ordinary notions of differentiation” (although in fact it can be justified with recourse to the gamma function). Among Heaviside’s other achievements were to popularize vector calculus and hence to cast Maxwell’s equations in their now familiar form. But because of his often cavalier approach to mathematical rigour, he ultimately became alienated from the scientific establishment; and even though he had been elected a Fellow of the Royal Society, one of his papers was famously rejected by them in 1894 as being insufficiently rigorous. He died in poverty and obscurity in 1925.

The claim that Heaviside ‘changed the paradigm’ is based on two partially conflicting observations. First, although having left school at 16 with only an elementary knowledge of mathematics, he was happy to develop advanced techniques for solving practical problems, many of which were quite beyond the competences of most electrical or telegraph engineers . Note, however, that although he did not pursue formal education beyond the age of 16, he was in the top 1 % of the candidates for the College of Preceptors school-leaving examinations (Mahon 2009). So, in a sense he was crucial to the mathematicisation of communications engineering . Second, he believed that rigorous proofs could be left to others: if his techniques worked, then engineers could use them. This approach to mathematics has coloured information engineering ever since, and the tension between the two observations is still to be found in the teaching of engineering mathematics today, when students and professional engineers often query the usefulness of the formal mathematics taught at university. Finally, although Heaviside did use his operational methods for certain analytical problems (for example, the cooling of the earth problem mentioned above), his methods were also clearly oriented towards synthesis and design, as in the inductive loading of cables. This latter trend marked much subsequent development of modelling for information engineering, and will now be explored in more detail in the following sections .

3 The Development of Linear Systems Theory

Heaviside’s operational calculus was given a rigorous foundation by 1920, particularly through the work of T J I’A Bromwich and J R Carson (Bennett 1979) . Bromwich related Heaviside’s work explicitly to Fourier analysis and contour integration, thus justifying his techniques to the satisfaction of mathematicians. Carson also linked the frequency domain and time domain approaches. Figure 1 illustrates this in a modern form.

Fig. 1

Linear system input-output relations in a modern form

A time-invariant linear system (that is, one that obeys the principle of superposition) can be modelled in the frequency domain by its frequency response or transfer function. Multiplying the input spectrum by the system frequency response gives the output spectrum. Alternatively, as Carson showed, the system can be modelled by the convolution integral. Convolving the input signal as a function of time with the impulse response (the ideal response of the system to a delta function) gives the output signal. In fact, Carson wrote the convolution integral in terms of the step response of the system, which he called the indicial admittance, but this is essentially identical .

By the 1920s these approaches were being used for various electrical and telecommunications problems, and electrical and telecommunications engineers were becoming highly adept in moving between the time- and frequency domains as necessary both to understand system behaviour and to design devices such as filters. One important advance was made by the German Karl Küpfmüller , although again it met with some resistance at the time (Bissell 1986, 2006). Küpfmüller realised that important conclusions could be drawn about system behaviour without any knowledge about its component parts. In particular, a model of a perfect so-called ‘brick wall’ filter (a perfectly rectangular frequency response, with constant gain or attenuation in the pass band, and complete rejection of all other, out-of-band, frequencies) allowed important general conclusions about the limiting behaviour of filters in general, whatever their implementation. Although his name is little known outside Germany, in his native country he is considered to be one of the great founding fathers of information engineering .

4 Filters

By the 1920s there was a need to be able to synthesise filters to a particular specification in order to separate out the various channels in frequency division multiplexing systems. Some of the most important advances were made by Campbell, Foster and Zobel (USA) and Cauer and Wagner (Germany). One of the most accessible introductions to their ideas is still that of Guillemin (1935), where full bibliographic references can be found. Guillemin had studied with Arnold Sommerfeld in Germany, and was a major conduit of German work on filter design to the English-speaking world.

The emphasis of modelling by now was increasingly on design, rather than analysis. Such wave filters were modelled, like transmission lines, as sequences of lumped passive elements (particularly capacitors and inductors, but sometimes resistors and transformers), but the key was to develop various so-called canonical forms, which could be used to synthesise a particular required filter characteristic. The first significant attempt to do this was by the American R M Foster , who used partial fraction expansions as the basis of the mathematical model, but far more wide-reaching was the work a few years later by the German Wilhelm Cauer , who used continued fractions, and was the first to put circuit synthesis on a sound mathematical basis (Cauer et al. 2000). Figure 2 shows some basic topologies of filter sections which could be linked directly to the corresponding mathematical model. A series of transformations enabled lowpass topologies to be converted to bandpass or highpass topologies simply by manipulating the diagrammatic structure, which was directly isomorphic with the mathematical model. For further information on the history of circuit design, and the mathematical techniques involved, see Belevitch (1962) and Darlington (1999).

Fig. 2

Some capacitor/inductor topologies for filter design. (Source: Wikipedia, Electronic Filter Topology. Available under the Creative Commons Attribution-ShareAlike License. Accessed 10 May 2013)

A few words should be said here about realisability . The ideal ‘brick wall’ filter is non-realizable: it is simply impossible in the real world to have an infinitely steep cut-off after a perfectly flat passband. By the 1930s various realisable approximations had been discovered, again exploiting complex mathematical ideas for synthesis and design, and then simplifying the design approach so as to avoid the need for the electronics engineers to carry out—or even fully understand—the underlying mathematics .

Figure 3 shows some normalised frequency response plots for various filter synthesis techniques. (The Butterworth filter is named after its inventor in 1930, while the other two are named after particular mathematical functions exploited in the design.) Note the various trade-offs: it is possible to have a very flat passband (Butterworth) if you can accept a more gentle cut-off; or a sharper cut-off at the expense of ripple either within the passband or outside it (Chebyshev); or an extremely sharp cut-off (elliptic) but there will be ripple over the whole frequency range.

Fig. 3

Some realisable filter approximations. (Source: Wikipedia, Butterworth Filter. Available under the Creative Commons Attribution-ShareAlike License. Accessed 11 May 2013)

These standard designs can easily be transformed into practical circuits using widely available diagrams, tables, and other accessible practical aids. At the risk of labouring the point, all this is very different from modelling in the natural sciences or economics. All the modelling effort here is directed towards design, but also in order to result in a set of relatively easy and barely ‘mathematical’ techniques (compared with the mathematical effort in deriving and proving results such as those in Fig. 3) that can be applied in practice by the circuit designer.

5 Electronic Feedback Circuits

Feedback circuits had been used in electronics since the early part of the twentieth century, and it was well known that they could become unstable. Sometimes this was a desirable quality—for the design of an oscillator circuit for radio transmission, for example, but in other cases it was very troublesome. In 1927 Harold Black (1898–1983) realised that by means of negative feedback, an amplifier with very low distortion could be realised, something required for transcontinental telephony. He famously sketched his idea on that day’s New York Times (Fig. 4).

Fig. 4

Black’s feedback amplifier. Schematic diagram (a) and Black’s original sketch (b) (Reprinted with permission of Alcatel-Lucent USA Inc)

Black discovered that although such a design could become unstable, its behaviour was not in accordance with the naïve model of the time, a version of the Barkhausen criterion developed for modelling oscillators. The assumption was that instability would occur when the overall loop gain was  > 1 and the input and feedback signals were in-phase, thus reinforcing the signal indefinitely each time around the loop. The problem was resolved in 1932 by Black’s colleague at Bell Labs, Harry Nyquist (1889–1976). Nyquist realised that you had to take account of the frequency response of the system. If you plotted this on polar coordinates of amplitude and phase, stability was determined when the frequency response curve encircled a certain critical point—in Nyquist’s original model (1, 0) but later revised to (− 1, 0) because of a slight generalisation of the model, so that the curves were plotted in a different way. This is illustrated in Fig. 5.

Fig. 5

Unstable (a) and stable (b) Nyquist plots. (Source: Bissell and Dillon (2012), p 56)

The distance of the curve from the critical point is a semi-quantitative measure of the closeness to (in)stability, as will be discussed further in the following section: note, for example, that if it is possible to decrease the gain in (a) sufficiently, the curve will shrink, ultimately into the stable region.

6 Feedback Control Systems

At the same time as electronics and communications engineers were developing an understanding of linear systems, feedback loops and stability, engineers dealing with control systems were running up against similar problems. Although invented in the latter half of the nineteenth century, servomechanisms underwent a huge development during the early part of the twentieth century for such widely differing applications as ship steering and differential analysers, and the same issues of stability arose. Towards the end of the 1930s, and even more during WW2, there was a coming together of telecommunications and control engineers, as well as mathematicians, particularly in such US wartime centres of R&D such as Bell Labs and MIT. Other important work was carried out in the UK, Germany and the USSR, but it was the American results which determined to a large extent the later development of what became known as classical control theory, so this brief account will be restricted to the main players in the USA.

The history of automatic control has been well documented: see Bissell (2009) for a short account and references to other sources. Basically, a number of researchers realised that the feedback model of Fig. 6, essentially that used by Nyquist in his analysis, could be applied to any other linear feedback system, even to control systems in which the variables might be flow rate, temperature, position, velocity and so on, rather than electrical waveforms.

Fig. 6

A feedback system showing the closed-loop transfer function

The problem was, given a knowledge of the open loop transfer function H, how could one determine the closed loop transfer function in order to apply the Nyquist criterion and, if necessary, make changes to the system to ensure desired performance. Yet again, an ingenious mathematical analysis was turned into a straightforward design tool. Nathaniel Nichols (1914–1997) realised during his WW2 work on gun control servos that it was possible to derive closed-loop amplitude and phase loci for any open-loop function. Plotting these loci in amplitude (decibels) and phase (degrees) form resulted in the famous chart shown in Fig. 7. For each open loop point defined on the rectangular grid, a corresponding close-loop frequency response point could be read from the curved lines as amplitude and phase. And that was not all. The closeness of approach of the open-loop locus to the critical point (halfway up the vertical axis in the figure) gave a direct measure of the transient behaviour of the system and the degree of (in)stability. A compensating network or controller could then be designed to be inserted into the feedback loop and shift the locus to a region with a desirable closed-loop dynamic response. The uncompensated closed loop frequency response, very close to the critical point, and the compensated one much further to the right of the critical point, are included in Fig. 7.

Fig. 7

Redrawing of the original published form (1947) of the Nichols Chart. (Source: Bissell (2009). A history of automatic control. In: Nof, Shimon Y. ed. Springer handbook of automation. Springer handbook series (LXXVI). Heidelberg, Germany: Springer Verlag, pp. 53–69)

The approach to control system design just outlined was developed just before and during WW2, mainly in the US, but also in the UK and to a lesser extent elsewhere. It reached the public domain immediately after the war, but many control engineers from mechanical or process engineering backgrounds found it difficult to understand or accept. The idea that you could talk about the frequency response of mechanical systems or even chemical process plant—where the frequencies involved might be fractions of a hertz—took considerable time to assimilate. The immediate post-war control engineering literature is littered with reports of meeting discussions where participants were still bewildered, or with arguments about the precise best way to plot graphs or specify gain and frequency.

7 System Identification

Our final example of the special nature of modelling in information engineering is what is known as system identification . Assuming, as throughout this chapter, that a system can be modelled sufficiently closely as a linear system, how might we obtain a suitable model in order to design a controller? The obvious approach, as is common in the natural sciences, is to make models of each element—such as an electric motor, a valve, a pump, a hydraulic cylinder, and so on—and then combine them into an overall model. While this is sometimes done, more often the system to be controlled may not consist of easily modelled components. Furthermore, strict analysis using Newton’s or Kirchhoff’s laws can result in an overly complicated or high-order model. When this is the case, direct input-output testing can often identify an appropriate model.

For example, it may be possible to subject the system to an input step change in variable, and from the response directly deduce a model of an appropriate order. A second possibility is to subject the system to a frequency response test—that is, apply an input sinusoid, wait until any transient has died away, and record the output sinusoid. Repeating this over the appropriate range of frequencies gives a direct measure of the frequency response. A third possibility is to apply white noise to the input and then correlate the output with the input: this also results in a knowledge of the system transfer function. All these techniques are valid because the input contains, in principle, all frequencies, and so can identify the complete system frequency response. In the early days of system identification such testing required significant manual input, but it is now highly automated, with much more complicated techniques using computer algorithms to match a model of desired order to the test results. Discussion here has been limited to simple linear cases, but ultimately techniques for non-linear systems were also developed.

Note that all of these methods use a ‘black box’ approach: no knowledge of the internal constituent parts of the system is necessary to obtain a model. And very often it is possible to obtain an adequate lower-order model of a system which, if analysed in terms of the physical behaviour of its individual components, would result in an unwieldy higher-order model. Furthermore, neither the Nyquist nor the Nichols plots discussed above require an analytical model of the overall system; a model derived by system identification suffices for the ultimate design.

8 The Mathematical Education of Information Engineers

Over the period under discussion the mathematical training of information engineers changed radically, reflecting the increasing use of the approaches outlined in the previous sections. Clearly, any detailed historical account of the development of the teaching of engineering mathematics would be impossible here. However, it is worth briefly mentioning one of the pioneers of the new curricula from the 1930s onwards: Ernst Guillemin (1898–1970) at MIT, who published six seminal works in three decades (Bissell 2008). His first, two-volume work, Communication Networks was considered by many at the time to be too ‘advanced’ for a supposedly introductory text, introducing transient and steady-state response; network theory; the Heaviside approach; and Fourier analysis—in other words the very material needed to understand the developments of the previous few decades outlined in this chapter. In his preface to the first volume he is unapologetic, and comments: “Methods are frequently designated as advanced merely because they are not in current use. To the student the entire field is new; the advanced methods are no exception. If they afford better understanding of the situation involved, then it is good pedagogy to introduce them into an elementary discussion. It is well for the teacher to bear in mind that the methods which are very familiar to him are not necessarily the easiest for the student to grasp.” In Volume 2 (1935) he was also one of the first to stress synthesis, “which has been an important motivating influence in the enlargement of our view and process with regard to network theory, not only had its inception in the field of communications but owes its development almost wholly to workers in that field. Nevertheless, these ideas and principles are too general in nature to remain confined to one field of application […] ” (Fig. 8).

Fig. 8

The most important volumes of Guillemin’s pedagogical output

Over a period of several decades Guillemin’s pedagogical approach was highly influential in the USA and elsewhere, breaking much new ground on what to teach and how to teach it, although not uncontroversial. His teaching philosophy is clearly presented in Guillemin (1962). While not shirking from presenting students with advanced mathematics, he was also a great proponent of heuristic arguments in order to firmly ground the theory in engineering practice. But, as noted in Sect. 2 of this chapter, tensions of this nature still remain in the teaching of mathematics to information engineers.

9 Conclusion

This short chapter has aimed to present some of the major special characteristics of the way models are used in information engineering , in contrast to much of the literature on modelling in the natural sciences or economics. These characteristics include:

• 1. The primary aim of the modelling is for system synthesis or design, rather than analysis or explanation.

• 2. Many of the models are based on quite complicated mathematics, such as complex analysis and Fourier and Laplace transforms , and thus were not immediately accepted by practising engineers when they were introduced.

• 3. Practising engineers had to cope with considerable changes over the period outlined in this chapter, accepting increasingly more sophisticated models of electrical, electronic or control systems than they had been used to, and learning new languages with which to discuss their design processes.

• 4. The new models were converted into much simpler form for the use of engineers, particularly graphs and charts which, often isomorphic with the mathematical foundations of the techniques, were able to hide the complexities of the underlying models from practitioners.

The history of modelling in information engineering is thus a complicated story of both mathematicisation and demathematicisation. With the advent of the digital computer, it became possible to carry out very complex engineering calculations automatically. Yet the graphical techniques presented above—as well as many others not mentioned here—remain an essential part of the user interface owing to the succinct and accessible way in which they present ideas.