Finding order using chaos: Synchronization of spiking oscillators helps build physical reservoirs

Engineers at the Tokyo Institute of Technology (Tokyo Tech) have demonstrated a computational approach using a ring network of coupled spiking oscillators with chaotic dynamics implemented on analog hardware. This new approach is based on the emergence and pattern formation phenomena that occur under “incomplete synchronization” within chaotic dynamics. In the future, it could significantly impact conventional methods for pattern detection commonly used in artificial neural networks on digital hardware counterparts.

In recent times, algorithms based on artificial intelligence (AI) are finding various societal applications such as personalized healthcare, autonomous driving, smart cities and precision farming. The amount of computing capability necessary to deploy such algorithms is increasing. Therefore, research initiatives are looking at alternative AI approaches taking inspiration from existing natural systems.

One approach is physical reservoir computing, where an ensemble of dynamical elements exploiting physical phenomena is used to map input data onto a high-dimensional space. The merit of this method is the reduced need for training algorithms that require a large amount of processing power. These reservoirs can often be implemented by very simple physical systems and do not require complex architectures as is the case with neural networks. The circuit that was used in this study, named the Minati-Frasca circuit and initially discovered and developed by researchers at the Universities of Trento and Catania in Italy, is highly elementary, involving only five passive and two active components, while showing rich spiking behavior [1]. “These circuits are truly remarkable and are a natural candidate for physical reservoir computing,” says Dr. Hiroyuki Ito, head of the Nano Sensing Unit where the study was conducted.

The experiments performed by the researchers at Tokyo Tech included tuning the chaoticity and coupling strength within a ring network of Minati-Frasca circuits (Fig. 1a). Initially, at low values of a suitable control parameter, the network showed periodic spiking, followed by highly irregular behavior when this parameter was increased (Fig. 1b, left). In combination with sweeping the coupling strength, this operation revealed a rich variety of ways in which the network synchronizes, meaning that nodes within it show similar behavior as can be observed in their waveforms (Fig. 1b, right). Considering the network as a whole, the emergence of synchronization patterns with a preferential synchronization of some node pairs over others, a situation known as incomplete synchronization, can be observed within chaos (Fig. 1c). Furthermore, in the case of this particular network, this region reaches maximal width near the edge of chaos, which is the boundary between periodic and chaotic regions of operation.

The researchers at Tokyo Tech then introduced two additional factors to influence the “route-to-synchronization,” namely, the injection of coherent noise into each node within the network (Fig. 2a, left) and splitting of the network into two different populations (Fig. 2b). The former showed that the extra noise significantly reduces the synchronization of the network in the periodic region, while, in the chaotic region, the area of incomplete synchronization (Fig. 2a, right) shifts and the synchronization of nodes that are not structurally adjacent is enhanced. This indicates that the network can respond to external stimuli in a complex manner. The latter experiment split the network into two halves, one operating within chaos (A) and the other within periodicity (B). The route-to-synchronization under this condition was examined with a coupling strength sweep, producing a striking diversification of the synchronization behaviors between the two populations. While the synchronization strength steadily increased within the chaotic half, the periodic half showed non-monotonic effects, i.e., multiple minima appeared when sweeping the coupling strength (Fig. 2b right). In addition, upon investigating the synchronization patterns in detail, an adversarial behavior was revealed, showing an initial synchronization of the periodic half which was then overtaken by the chaotic half, followed by a final overall synchronization between both halves. This effect further underlines the generative potential of this network. In essence, a binary split of two populations shows a highly simplified scenario of the input perturbations that this network could be exposed to when used for physical reservoir computing.

As such, the researchers considered the network in their study and proposed to use it for implementing reservoir computing in the future by exploiting the various phenomena that were described above (Fig. 2c). “Coming from a background of machine learning, the couplings within the network reminded me of working with neural networks. However, at first I was not able to understand the implications of changing dynamics and chaos, since conventional AI algorithms tend not to have their innate dynamical activity,” says Jim Bartels, one of the lead authors of this study. “I realized that exploiting these dynamics for computation could fit well into the field of reservoir computing, which remains a growing area of study.”

After this interview, the team explained why this type of reservoir computing could be beneficial for applications in society. “One of the main fields of research that we are working on in the Nano Sensing Unit is time series classification for internet-of-things (IoT) devices and edge computing, such as animal behavior classification [2]. A very important consideration for these devices is their battery lifetime, since it determines the barrier towards concrete adoption. What is exciting about physical reservoirs such as the one we have proposed is the possibility of operating, in future integrated realizations yet to be built, at a lower power than large digital neural networks. As the circuit represents one of the smallest known types of spike-generating oscillators, going beyond the present proof-of-concept stage, we expect researchers worldwide to explore its many possible variations for additional computational frameworks, such as neural networks,” commented Dr. hab. Eur-Ing. Ludovico Minati, who is the leading author of the study. The experiments that were undertaken, the design of the hardware, the results and their discussion are reported in a recent article published in the journal Chaos, Solitons & Fractals [3]. Furthermore, all design materials and experimental data have been made freely downloadable.

References:

[1] Sprott JC, Thio WJC, Elegant Circuits: Simple Chaotic Oscillators, Singapore: World Scientific 2022, DOI: 10.1142/12362

[2] Bartels J, Tokgoz KK, A S, Fukawa M, Otsubo S, Li C, Takeda K Rachi I, Minati L, Ito H, TinyCowNet: Memory- and Power-Minimized RNNs Implementable on Tiny Edge Devices for Lifelong Cow Behavior Distribution Estimation, IEEE Access, vol. 10, pp. 32706-32727, 2022, DOI: 10.1109/ACCESS.2022.3156278

[3] Minati L, Bartels J, Li C, Frasca M, Ito H, Synchronization phenomena in dual-transistor spiking oscillators realized experimentally towards physical reservoirs, Chaos, Solitons and Fractals, vol. 162, art. 112415, 2022, DOI: 10.1016/j.chaos.2022.112415

###

About Tokyo Institute of Technology

Tokyo Tech stands at the forefront of research and higher education as the leading university for science and technology in Japan. Tokyo Tech researchers excel in fields ranging from materials science to biology, computer science, and physics. Founded in 1881, Tokyo Tech hosts over 10,000 undergraduate and graduate students per year, who develop into scientific leaders and some of the most sought-after engineers in industry. Embodying the Japanese philosophy of “monotsukuri,” meaning “technical ingenuity and innovation,” the Tokyo Tech community strives to contribute to society through high-impact research.

https://www.titech.ac.jp/english/