Keywords

1 Introduction

The traditional approach of artificial intelligence algorithm usage forces centralised schema. This approach is simpler, but in some cases, it makes the high load on a central server. One solution is to transfer tasks from the central server to smaller processors or microcontrollers on edge, and this is called edge computing. This approach reduces network traffic since edge nodes upload reduced data instead of complete input data [1].

In the last few years, many microcontroller producers have worked on artificial intelligence implementation on microcontrollers. Some of them have developed special libraries with AI functions, but the others have implemented special hardware with enhanced AI capabilities.

In this paper, a low-cost Chinese SoC ESP32 is tested with feed-forward neural networks on two different development environment, Arduino IDE and MycroPython. Espressif System, a company from China introduced ESP32 before three years. It does not have machine learning capabilities. It is manufactured using a 40 nm process by TSMC (Taiwan Semiconductor Manufacturing Company) [2]. Many projects have been developed with ESP32 SoC like water pumping solar system [3], or monitors for Liquefied petroleum gas [4]. Some authors suggested that ESP32 will play one of the major role in future Internet of Things devices development [5].

In the second section, some papers that deal with machine learning usage and ESP32 are presented, but in the third section, some development environments for ESP32 are presented. In the fourth section, neural networks forward propagation times with a different number of neurons in the hidden layer are measured. Finally, in section five, discussion and conclusion can be found.

2 ESP32 and Machine Learning

ESP32 is used for various machine learning tasks, but usually only for measuring and collecting data for dislocated machine learning servers. On the other hand, some papers deal with ESP32 usage in projects that have machine learning algorithms implemented. In the next two paragraphs, papers that deal with these two different approaches are analysed.

2.1 Machine Learning on the Cloud

Zidek et al. used ESP32 SoC with different sensors for wireless devices input data optimization. Data was accumulated into the packet and then the hole packet was sent to the cloud service [6].

Rosato and Masciadri have chosen Logistic Regression to detect sitting person with the device based on ESP32 SoC. They made monitoring system and the key component with ESP32 was installed under the person chair. Data has been analysed on the server, and ESP32 has been wirelessly connected to it [7].

Islam et al. designed small device that can record electrical activity of the human heart using electrodes placed on the skin. This device has been based on the Analog Devices AD8232. They used ESP32 SoC to send data to the server. Linux server has been used to analyse data with machine learning algorithms [8].

Fernoaga et al. used ESP32 to take pictures of gas, electricity or water counters. These devices have had implemented cameras and they have sent pictures to the cloud. Numbers from the counters are recognised by neural networks from supplied pictures [9].

Komarek et al. presented a project that provide a layer that uses MQTT (MQ Telemetry Transport) for interfacing sensors and nodes. They used ESP32 SoC to send data to other nodes and the cloud for the ambient intelligence system [10].

Chand et al. used ESP32 SoC for a person behavior and status prediction in one room. They used ultrasonic sensors and data from sensors are sent to a server. The main goal of the server was to decide whether it is normal or abnormal situation. This decision system is based on machine learning algorithms [11].

2.2 Machine Learning on an ESP32

Espressif Systems company developed ESP-WHO framework for face detection and recognition. This framework is based on Multi-task Cascaded Convolutional Networks model and new mobile architecture - MobileNetV2 and it is available on the GitHub. The main difference between this framework and previously mentioned projects is in the fact that ESP-WHO has face detection and recognition algorithms on the ESP32 SoC [12, 13].

Kokoulin et al. used ESP32 SoC for the system that process video stream and detect presence of faces or silhouettes. Only images with faces or silhouette are sent to the central face recognition server. The main goal of that system were network traffic reduction as well as central face recognition server computing load reduction. Estimated network traffic decrease up to 80–90% [14].

3 ESP32 Development Environments

ESP-IDF (Espressif IoT Development Framework) is the official development framework for the ESP32 SoC. ESP-IDF is the most powerful framework, and it can be used with Linux and Windows OS. Because of ESP32 popularity some other tools are adapted to ESP32. Most popular are Arduino IDE and MicroPython, and they will be analysed.

On the other hand, there are lots of development environments that can be used for ESP32 projects development. KB-IDE is open IDE available on GitHub for ESP32 SoC boards. It supports visual programming similar to Scratch, Arduino style programming, and the official ESP-IDF framework for more experienced users (https://kbide.org/). Lua programming language can be used with ESP32, too. With ChiliPeppr ESP32 Web IDE, a browser can be used for editing/uploading Lua code to ESP32 device. The ChiliPeppr IDE has a serial port JSON (JavaScript Object Notation) Server that can be used locally or remotely to let browser communicate directly to serial port and ESP32 (http://chilipeppr.com/esp32). ESP32 is supported by PlatformIO. It is a cross-platform code builder and library manager. It supports more than 200 development boards, 15 development platforms and ten frameworks.

3.1 Arduino IDE

Arduino IDE is java application used to write programs for Arduino compatible boards. It can be used with ESP32, but first some additional software components have to be downloaded before use. The Arduino IDE supports simplified versions of C and C++ languages. Arduino is well known Italian company with lots of microcontroller boards that are licensed under the LGPL (Lesser General Public License) or the GPL (General Public License). In Fig. 1, the Arduino IDE can be seen.

Fig. 1.
figure 1

Arduino IDE

Full size image

3.2 MicroPython

MicroPython is an implementation of a Python programming language optimised to run on some microcontrollers. Some of the core Python libraries are included, but it includes modules that allow low-level hardware access to the programmer. MicroPython was created by Australian programmer Damien George. It does not have rich IDE, but on their web page spycraft IDE is suggested. In Fig. 2, uPyCraft IDE can be seen.

Fig. 2.
figure 2

uPyCraft IDE

Full size image

4 Neural Networks Forward Propagation Speed

Teerapittayanon suggests that their Distributed Deep Neural Networks model can reduce the communication cost by a factor of over 20x but to achieve this, so-called “edge” or “end” devices have to use simple Machine Learning models and process data from sensors [15]. Zhang describes keyword spotting device based on Cortex-M7 based STM32F746G-DISCO development board. Keyword spotting devices are typical examples of edge devices that are based on tiny microcontrollers with limited memory and compute capability. They require real-time response and high accuracy for good user experience [16]. Lai quotes that “Deep Neural Networks are becoming increasingly popular in always-on IoT edge devices performing data analytics right at the source, reducing latency as well as energy consumption for data communication”. Lai used Arm Cortex-M processor board and achieved 4.6X improvement in runtime/throughput and 4.9X improvement in energy efficiency with CMSIS-NN kernels [17].

There are lots of examples where a neural network is trained on a powerful computer and then deployed to tiny microcontroller to analyse some input data. In this paper ESP32 is tested for that task. The simple neural network is programmed with random weights and different number of nodes and data propagation time is measured.

4.1 Data and Methods

Neural networks with 50 input nodes and ten output nodes are programmed with Arduino IDE and MicroPython. The number of hidden nodes in one hidden layer is between 50 and 200. A logistic sigmoid activation function is used (Fig. 3).

Fig. 3.
figure 3

Neural network with one hidden layer

Full size image

Setup for testing is in Table 1.

Table 1. Testing setup
Full size table

4.2 Microcontrollers

Two ESP32 based microcontrollers are used for testing purposes. ESP32 WROOM is in Fig. 4, and PYCOM WIPY 3.0 is in Fig. 5. The main difference between them is that PYCOM WIPY 3.0 has MicroPython installed and ready to be used. ESP32 WROOM can be used with Arduino IDE without any setup.

Fig. 4.
figure 4

ESP32 WROOM (Arduino)

Full size image
Fig. 5.
figure 5

PYCOM WIPY 3.0 (MicroPython)

Full size image

Both boards have Xtensa dual-core 32-bit LX6 microprocessor operating on 240 MHz. ESP32 WROOM has only 520 KB RAM, but PYCOM WIPY 3.0 has 520 KB and 4 MB additional memory. WiFi and Bluetooth modules characteristics are same, but ESP32 WROOM has serial to USB controller implemented on board.

4.3 Testing

There are different ways to test the speed of neural network data propagation, but in this research digital oscilloscope is used. Some authors used implemented functions milis() and micros() on both platforms but measuring with oscilloscope is better solution because some processes in microcontroller cannot affect measuring.

To measure the time, GPIO (general-purpose input/output) output of the microcontroller is connected to an oscilloscope. Simple code in Arduino C and MicroPython with tasks listed in Table 2 is prepared. This code has been used to measure frequency of pin output rise and fall but these periods are too small to have any effect on measuring accuracy. Periods can be seen in Table 3. All code listings can be found on GitHub.

Table 2. Code in Arduino C and MicroPython for pin output rise and fall
Full size table

The results are in Table 3.

Table 3. Rise/fall period (Arduino C and MicroPython)
Full size table

Oscilloscope screenshot can be seen in Fig. 6.

Fig. 6.
figure 6

Oscilloscope screenshot

Full size image

In the next step, neural network code has been implemented two times after GPIO ON and GPIO OFF code on both platforms. The flowchart can be seen in Fig. 7.

Fig. 7.
figure 7

Flowchart

Full size image

Results are in Table 4. Arduino C can use two floating-point number precisions, and both of them are used in measuring. New MicroPython firmware support double-precision floating-point numbers but it was unavailable to us, and in Table 4 there are only four rows concerning MicroPython.

Table 4. Final results
Full size table

Results can also be seen in Fig. 8.

Fig. 8.
figure 8

Periods for different platforms and hidden nodes

Full size image

5 Discussion and Conclusion

It can be seen that Arduino C based neural network has significant lower data propagation latency than MicroPython based neural network. Some authors indicated this, but they have tested propagation speed with integers. On the GitHub platform number of counts running for 10 s for three different setups can be found. MicroPython on Teensy 3.1 (96 MHz ARM) counts to 1,098,681 for 10 s, but Arduino C on Teensy 3.1: (96 MHz ARM) counts to 95,835,923 for the same period. Author used milis() function for time measuring [18].

Arduino IDE is faster platform than MicroPython for neural networks development on edge devices. This research can be expanded with other development environments, especially with ESP-IDF. Some authors suggested that floating-point neural networks can be replaced with integer neural networks when cost is primary design concern, so it would be interesting to test performance and speed with integer neural networks [19, 20].