Keywords

1 Introduction

Recently, multilayer neural network models, in particular, a deep learning model have been researched very sprightly. The performance has been extremely improved in the specialty of image/sound recognition. The internal mechanism of recognition system is revealed more clearly; self-learning integrated circuit (IC) chips have also been realized. However, these models are operating on a general-purpose Von Neumann model’s computer with the application system. There are minority studies that construct an analog parallel hardware structure using a biological information processing mechanism. We proposed the neural network machine learning model with a pure analog network electronic circuit. Such a model will develop original recognition or prediction system using the analog neural electronic circuit. In the field of multilayer learning models, many useful models such as image recognition or phenomenon prediction have been proposed. And there are many hardware realization models such as an image/sound sensor or computer by parallel circuit have been enlargement.

1.1 Analog Hardware Neural Network

The main strong point of an analog machine learning network, its operation is by the real-time linear system, not due to the clock frequency behavior. On another side, a digital system behavior is due to the clock operation by central processing unit (CPU) based on a Neumann computer. As another researchers’ study, innovative analog neural models were proposed [1, 2]. In the complete analog circuit, one element is the implementation of analog data saving unit, keeping the analog numerical value for a while time using analog memory [3]. The dynamic random access memory (DRAM) can be memorized in the condenser memory in short period of time, because it was performed in the generic complementary metal oxide semiconductor (CMOS) [4]. However, when the charge is maintained for a long time in the capacitor, it needs the processing system to keep the numerical value data in the memory. It is required the refresh process. Capacitors reduce the electric charge over time. It is not difficult to get back the electric charge of a condenser by refresh process utilization in the general digital binary memory. Nevertheless, in the situation using analog memory, the refresh process is not easy because memorizing data is linear analog data, it needs the time analysis system of electronic charge reducing curve. Other researchers proposed the memorize methods of connecting weights, the floatage gate type device [5] and magnetic substance memories [6].

1.2 Pulsed Neural Network

Pulsed neural networks receive great number pulses as learning data and change the connecting weights due to the number of pulses. Such networks can maintain their connecting weights after learning by the number of pulses and outputs the signal depending on the input value [7]. Nevertheless, it is necessary the long time for learning because great number pulses are required before complete learning. For example, the time interval average of the pulse is 10 uS and 100 pulses are needed before finished the learning procedure, and it needs 1 mS for finished learning approximately.

1.3 How to Realize the Variable Connecting Weights?

In our former study, proposed model was a movement detection biomedical vision model using analog general electronic circuits. The suggestion model is composed of four layers. There are differentiation circuit, difference circuit and multiple circuits in each layer for detecting pure motion output. From a technical standpoint, the proposed model makes possible to elucidation of the artificial vision system mechanism, which can detect the target object, motion and velocity by the design and simulation using an analog network electric circuit [8, 9].

On the other hand, attempted to be realize was a multi-layered hardware neural network using an analog electronic circuit. In the machine learning and neural network field, many practical use models have been proposed. But, these models are based on a digital general Neumann computer. There are few studies in the practical study about analog learning neural networks. Early, analog circuit network models were configured of the difference circuits, multiple circuits and solid resistance. It is not easy to change the value of solid resistance for the learning process.

The first constructed model was a hardware neural network using variable resistance elements as the connecting weights. In the learning operation process, each resistance value needs to be adjusted by hand. Next, multiple circuits were used as the connecting weights. Multiple circuits can calculate the products of a two input signals. One is an input signal value. Another input as the connecting value is considered.

In the former study, three-layered neural network analog electronic circuits were designed. The model used multiple circuits by opamp and metal-oxide-semiconductor field-effect transistor (MOSFET) as the connecting weights. The connecting weights vary easily by controlling the input signal. The model has two input units and one-output unit with three layers. After the learning process, the model worked Exclusive OR (EX-OR) logic as the simulation program with integrated circuit emphasis (SPICE) simulation, this is a linear inseparable problem [10, 11].

2 Neural Network by Analog Electronic Circuit

2.1 Neural Network Using Solid or Variable Resistance Elements

Early analog neural circuits’ models were configured of the difference circuits, multiple circuits and solid resistance. It is not easy to change the value of solid resistance for the learning process. In past research, a hardware neural network was constructed using variable resistance. This variable resistance means the connecting weights of network. The network has nine units in the input layer, three units in the middle layer and three units in the output layer. The system was able to recognized simple patterns by pure analog circuits. However, in the learning process, each resistance value needs to be adjusted by hand.

2.2 Neural Network Using Multiple Circuits

In a former study, multiple circuits were utilized as the connecting weights. The connecting weights could be easily changed by controlling the input signal. Figure 23.1 shows the 2-input and 1-output neural circuit. It means the structure of one neuron. There are three input units, two input signals and one threshold value. The input unit calculates the product of two voltages, input signal value and connecting weights. The connecting weights can be easily changed by operating the voltage of the MOSFET gate signal in Fig. 23.1.

Fig. 23.1
figure 1

Neuron model by multiple circuits

Full size image

3 Perceptron Network by Analog Circuits

Next, a learning neural network was constructed. It is a two-input unit and one-output unit basic perceptron model with a feedback circuit. The diagram of this model is indicated in the upper half of Fig. 23.2. In the diagram, multiple circuits are identified as “Mul”. Additional circuits are indicated as “Add” and subtraction circuits as “Sub”. Figure 23.2 also shows the perceptron network of analog electronic circuits. The SPICE simulation result is shown in Fig. 23.3 [12]. It is shown that the convergence time is about 900 μS, when the learning process is finished.

Fig. 23.2
figure 2

Block diagram and learning circuit of perceptron

Full size image
Fig. 23.3
figure 3

Convergence output of perceptron by SPICE simulation

Full size image

4 AC Operation Neural Circuit

In the previous chapter, the hardware learning network is explained. Although, in the situation of setting up a network, there is one problem. The operating range of input and output-voltage level is limited. Moreover, the circuit behavior is sometimes instability because of the multiple circuit features using a semiconductor. It is said “circuit limitations.” One reason is a semiconductor specificity. Not all semiconductors are manufacturing equally. Another reason is the output-voltage limitation of semiconductor element.

The alternative current (AC), not direct current (DC) was used as a transmission signal. Shown is the AC operation of one neuron unit in Fig. 23.4. There are two components in the alternative current. On the alternative currents, current flows with two elements, voltage and frequency. The input signal of the neural circuit is the input voltage of the alternative current. The connecting weights of the neural network are the frequency of the alternative current. The parameter of the circuit is decided as the capacitance and resistance (CR) coefficient. CR circuit has good stability compared to a semiconductor analog circuit using the frequency characteristics of capacitor and resistance. As the result, this circuits outputs the approximately products of voltage and frequency.

Fig. 23.4
figure 4

AC operating neural model

Full size image

Figure 23.5 is the graph of input frequency and output voltage by AC neural circuits. Output voltage is the root mean square (RMS) value. In the network, it is added the two input alternative currents by an additional circuit. Additional circuit outputs a wave modulation. This additional circuits works that the outputs increase satisfactory in the general-purpose frequency range, 3–30 kHz. Figure 23.6 is the output value of the same neural circuit by a two-dimensional graph. The frequency range is from 3 to 30 kHz and the voltage range is 5–10 V. It was confirmed that the output RMS voltage value is also monotonically increasing.

Fig. 23.5
figure 5

Output AC operation neural circuits

Full size image
Fig. 23.6
figure 6

Output behavior of AC operation neural circuit

Full size image

4.1 V-F Converter Circuits

A learning AC operation neural network with a feedback circuit was designed. The error value feedback signal needs to be converted into a connecting weight. Because, these circuits use the frequency data as the connecting weights. The initial input signal is DC current. A voltage-frequency converter circuits was used when generating the connecting weight. V-F converter circuits were used to calculate the frequency data from the error value. The unit generates alternative current as the connecting weight. It is used the backpropagation (BP) learning process using AC feedback circuit. It has to convert from DC voltage to AC current with frequency, after receive the DC current by a rectifier circuit. Figure 23.7 shows V-F converter circuit characteristics, NJM4151, New Japan Radio Co., Ltd. (JRC). The input voltage and output frequency are monotonically increasing in the logarithmic scale.

Fig. 23.7
figure 7

V-F converter circuit characteristics NJM4151, New JRC

Full size image

4.2 The Behavior of the Neural Circuit by AC Transmission

In the AC operation neural network, first amplifier circuit generates the AC current. Input signal is AC current and the gain of the amplifier is input DC current. Two AC currents after each amplifier circuits are added by an additional circuit. The modulated wave is generated in the output of the additional circuit. The first layer’s output of this neural network is modulated wave. In the learning process of the AC neural circuit, the feedback error signal needs to be converted to a connecting weight by AC current. The products difference error signal and input signal means the correction error signal. The difference between the output signal and the teaching signal means the difference error signal [13, 14].

Figure 23.8 shows the diagram of AC operation learning neural network model unit. These circuits are composed of an amplifier, additional circuit, inverter, subtract circuit, rectifier circuit and V-F converter. The initial input signal and connecting weights are both DC current. The V-F converter units generate the frequency from the input DC current. The output of the amplifier circuit is the AC current. Input signal is AC current and the gain of the amplifier is input DC current. It is shown the AC operation learning neural circuits except the V-F converter in Fig. 23.9.

Fig. 23.8.
figure 8

Diagram of AC operation learning neural network model unit

Full size image
Fig. 23.9
figure 9

Whole circuit of AC operation learning neural network

Full size image

Two AC currents after each amplifier circuits calculate the sum by an additional circuit shown in Fig. 23.10. Each signal is different in voltage and frequency. The modulated wave is generated in the output of the additional circuit. The output of inverse circuit is the phase inversion modulated wave. The input of the second amplifier circuits is phase inversion modulated wave. The teaching signal means the gain of second amplifier circuit. The second adder circuit calculates the sum of the second amplifier circuits output and the first adder circuits output. The input of second amplifier circuits is the output of phase inversion circuit. It means the phase inverted teaching signal. The output of the second adder circuit is the difference of the first adder circuits’ output and teaching signal, it is the error value. Thus, the subtract circuit does not need to be used to calculate the error value. The output of rectifier circuit is the DC current, converted from the modulated wave of the second adder circuits’ output. Figure 23.11 shows the simulation results after rectifier circuits. There is some distortion in the rectifier signal, but it is not a big problem.

Fig. 23.10
figure 10

Two input signals of AC feedback neural model

Full size image
Fig. 23.11
figure 11

Simulation results after rectifier circuits of AC feedback neural model

Full size image

The correction quantity of the connection weights is the DC current, output of the rectifier circuits. The subtract circuits calculate the difference of initial connecting weights and correction quantity. The output of subtract circuits is new connecting weights. Figure 23.12 shows the new connecting weights by an AC feedback neural model. It is shown that the new connecting weight’s value is almost constant.

Fig. 23.12
figure 12

Simulation results of new connecting weights by AC feedback neural model

Full size image

The input of V-F converter circuits is the subtract circuits’ output and the output of V-F converter is the AC current by a frequency signal. The input of first amplifier circuits is AC current by a frequency signal and input DC current. At last, first amplifier circuits generated the AC current. The input value means the RMS voltage of AC current. The connecting weights of the neural network are the frequency of the AC current. The series of behaviors means BP learning operation, feedforward and feedback process.

The graph showing the relationship between learning count time and the frequency of output is shown in Fig. 23.13. Frequency f1 and f2 means the each connecting weights wiring from each input unit. It shows each frequency convergences to constant value approximately. The learning count is only 5 or 6 times; it is quite small compared to other learning model. Another merit is fast learning speed; however, the performance of network is inferior compared to another digital learning network model. But this model refers to biomedical neuron model. Therefore, this proposed method outputs soft and smooth nonlinear signal.

Fig. 23.13
figure 13

Learning count time and the frequency of output

Full size image

5 Conclusion and Future Work

5.1 Conclusion

An analog neuron model by multiple circuits was designed using operational amplifier and MOSFET instead of using variable resistance. The operation of the network was confirmed by electronic circuit simulation. Next, one unit of analog neural network was constructed by an AC current operation circuit. The input of first amplifier circuits is AC current by a frequency signal and input DC current. There are two components in the alternative current. The input value means the RMS voltage of AC current. The connecting weights of the neural network are the frequency of the AC current. Two AC currents after each amplifier circuits calculate the sum by an additional circuit.

In the final learning process of the network model, the input of V-F converter circuits is the subtract circuits’ DC output and the output of V-F converter is the AC current by a frequency signal. In the SPICE simulation results, the frequency converges to a constant frequency and finishes only several learning processes. In the experimental result, frequency converges to 4 kHz and 1 kHz, respectively. The learning count is only 5 or 6 times; it is quite small compared to other learning model. Another merit is fast learning speed; this model refers to biomedical neuron model. Especially, a deep learning model has been developed very rapidly in the research area of image/sound recognition. Recently, multilayer neural network models, in particular, a deep learning model have been researched very sprightly. The performance has been extremely improved in the specialty of image/sound recognition. It is awaited high performance artificial intelligence model using the learning system using analog circuit in the near future.

5.2 Future Scope Using Deep Learning Model

Deep learning model is one kind of machine learning model. The performance of the recognition is getting better more and more. This model is used for practical purposes in the field of image detection, video/sound recognition. This model will be developed in the field of advanced technology, self-driving, robotics and artificial intelligence. In the original BP, learning neural network is three-layer structure. However, the structure of in the general deep learning model, there are nine layers. And there are also three layered sub-networks, like a convolution network [15]. Furthermore, learning algorithm in the field of deep learning model uses a stacked auto-encoder. This algorithm can detect the feature data and abstract expression data from the input image using large quantity learning data. The proposed AC operation circuits prompt the possibility for flexible structure neural network like a deep learning model [16].

If AC operation learning network system developed in the direction of multi-layered network model like a deep learning, many applications will be expected. It is one of the kinds of soft computing. The output data is soft and smooth, difficult to realize in the digital processing. In the field of image recognition and speech recognition, the proposed AC operation model suggests the chance of making the electrical circuit model of deep learning [17]. The model will develop the artificial intelligence unit under the environment of automated operation with tough and beneficial for fault tolerance network. Increase the number of units, large-scale system development and creating integrated circuit are future problems.