A View of Criticality in the Ising Model Through the Relevance Index

Abstract

The Relevance Index has been introduced to detect key features of the organisation of complex dynamical systems. It is based upon Shannon entropies and can be used to identify groups of variables that change in a coordinated fashion, while they are less integrated with the rest of the system. In previous work, we have shown that the average Relevance Index attains its maximum at the phase transition in both Ising model and random Boolean networks. In this contribution we present a further study on the Ising model, showing that the relevance index is maximised for large groups of variables at criticality. These results provide further evidence to the hypothesis that this index is a powerful measure for capturing criticality.

1 Introduction

The Relevance Index (RI) had been originally introduced to identify key features of the organisation of complex dynamical systems, and it has proven able to provide useful results in various kinds of models, including e.g. those of gene regulatory networks and protein-protein interactions. The method can be applied directly to data and does not need to resort to models, possibly helping to uncover some non-trivial features of the underlying dynamical organisation. The RI is based upon Shannon entropies and can be used to identify groups of variables that change in a coordinated fashion, while they are less integrated with the rest of the system. These groups of integrated variables make it possible to provide an aggregate description of the system, at levels higher than that of the single variables and it can be applied also to networks, that are widespread in complex biological and social systems. In previous work, we have found that the RI can also be used to identify critical states in complex systems (Roli et al. 2017). We showed that the average RI, computed across random samples of cells of a given size in the Ising lattice, attains its maximum at the critical temperature.

In this contribution we present an in-depth analysis of the RI values across all subset sizes. Results show that a parameter defined as a function of subset size and RI is strictly correlated to the susceptibility of the system, which in turn assumes its maximum at the critical temperature. These results provide further evidence to the hypothesis that the RI is a powerful measure for capturing criticality and they also suggest that a possible explanation for this is that larger subsets are more correlated at criticality, as a consequence of all-range correlations typical of critical points in phase transitions.

After a brief introduction to the RI in Sect. 2 and a summary of previous results in Sect. 3, we will presents and discuss the new results on the Ising model in Sect. 4. In Sect. 5 we conclude the contribution and outline lines for future work.

2 The Relevance Index

The main concepts related to RI had been conceived in the work on biological neural networks by Tononi et al. (1998), who introduced several measures, among them the Cluster Index. The RI is an extension of this latter measure, that can be applied to dynamical systems (Filisetti et al. 2015; Roli et al. 2016; Villani et al. 2014, 2015). The purpose of the RI is to identify sets of variables that behave in a coordinated way in a dynamical system; the variables that belong to the set are integrated with the other variables of the set, much more than with the others. Since these subsets are possible candidates as higher-level entities, to be used to describe the system organisation, they will be called relevant subsets (omitting for brevity the specification that they are candidates). A quantitative measure, well suited for identifying them, is defined as follows—the presentation below follows the one given in Villani et al. (2014).

Let U be a system whose elements are discrete variables that change in time, and suppose that the time series of their values are known. The Relevance Index r(S) of S ⊂ U is defined as the ratio between the integration of S and the mutual information between S and the rest of the system:

$$\displaystyle \begin{aligned} r(S) = \frac{I(S)}{M(S;U\setminus S)} = \frac{\sum_{x \in S} H(x)-H(S)}{H(S) + H(S|U\setminus S)} \end{aligned} $$

(1)

where H(x) is the Shannon entropy of x and H(S) is joint entropy of the set of variables in S.

When the RI is applied to identify relevant subsets, it is necessary to compare sets of different sizes. However, entropies scale with system size, so this requires considerable ingenuity. Following the original work of Tononi, a “RI method” has been developed for this purpose, where a statistical index is computed that allows meaningful comparisons of sets of different sizes:

$$\displaystyle \begin{aligned} T_c(S_{k}) ~= \frac{r(S_{k}) - \left\langle r_{h} \right\rangle}{\sigma(r_{h})} \end{aligned} $$

(2)

where $\left \langle r_{h} \right \rangle$ and σ(r _h) are respectively the average and the standard deviation of the RI of a sample of subsets of size k extracted from a reference system U _h randomly generated according to the frequency of each single state in U. It is worth noting that the aim of the reference system is that of quantify the finite size effects affecting the information theoretical measures on a random instance of a system with finite size.

In principle, a list of candidate relevant sets can be obtained by computing the RI and the of every possible subset of variables in U and ranking the subsets by T _c values. The subsets occupying the first positions are most likely to play a relevant role in system dynamics. For large-size systems, exhaustive enumeration is computationally impractical as it requires to enumerate the power set of U. In this case, we resort to sampling or to heuristic algorithms.

3 RI and Criticality

Criticality usually refers to the existence of two qualitatively different behaviours that a system can show, depending upon the values of some parameters and it is then associated to parameter values that separate these qualitatively different behaviours. However, slightly different meanings of the word can be found in the literature, two major cases being (a) the one related to phase transitions and (b) dynamical criticality, sometimes called the “edge of chaos”. In the former case, the different behaviours refer to equilibrium states that can be observed by varying the value of a macroscopic external parameter. In the latter case, the different behaviours are characterised by their dynamical properties: the attractors that describe the asymptotic behaviour of the system can be ordered states, like fixed points or limit cycles, or chaotic states. These two meanings are related but not identical.¹

In a previous work, we have shown that the RI can be used to locate critical regions in complex systems (Roli et al. 2017). In our experiments we considered two different kinds of systems: the Ising model for phase transitions, and the Random Boolean Network model for dynamical criticality. For both the models, we computed the RI of randomly sampled groups of variables of varying size. Our main finding is that the RI is able to satisfactorily locate the critical points in both cases. An excerpt of the results on the Ising model is shown in Fig. 1.

Fig. 1

Left: plot of RI for 10 × 10 Ising lattice. The median of the average RI values for groups of size 2 to 10 is plotted against T. The curves shift up with group size. The peaks of RI correspond to the susceptibility peak, which in turn corresponds to the critical value of the control parameter. Right: plot of median susceptibility values for 10 × 10 Ising lattice. The peak of susceptibility empirically identifies the critical T value

4 RI of the Ising Model

The Ising model is a notable example of a system that can undergo a phase transition as a function of a control parameter (Binney et al. 1992; Brush 1967; Stanley 1971). Let us consider a d-dimensional lattice of N atoms characterised by a spin, which can be either up (+ 1) or down (− 1). The atoms exert short-range forces on each other and each atom tends to align its spin according to the values of its first neighbours. An external field may also be considered, which biases the orientation of the atoms. The energy of the system is defined as follows:

$$\displaystyle \begin{aligned} E = - \frac{1}{2}\sum_{\langle i, j \rangle} J \; s_i \; s_j \; + B \sum_{i} s_i \end{aligned} $$

(3)

where s _i is the spin of atom i, J > 0 is a parameter accounting for the coupling between atoms, 〈i, j〉 denotes the set of all neighbouring pairs and B is a parameter playing the role of an external field. The system can be studied by means of usual statistical mechanics methods and it can be assessed whether it undergoes a phase transition; Onsager (1944) proved that the d = 2 model can undergo a phase transition under the hypothesis that B = 0.

In this work, we consider the two-dimensional model, with B = 0. We performed Monte Carlo simulations at constant temperature T. The Monte Carlo algorithm used is a classical Metropolis algorithm with Boltzmann distribution (see Algorithm 1).

The temperature is the control parameter, while the order parameter is the so-called magnetisation:

$$\displaystyle \begin{aligned} \mu = \frac{1}{N} \sum_{i} s_i \end{aligned} $$

(4)

Algorithm 1 Monte Carlo simulation of a 2d Ising model. Adapted from Solé (2011)

For low values of T, the steady state of the system will be composed of atoms mostly frozen at the same spin and the time average of the magnetisation 〈μ(T)〉 will be close either to 1 or − 1; for high values of T the spins will randomly flip and it will be 〈μ(T)〉≈ 0. For values close to the critical temperature T _c, a phase transition occurs: the system magnetisation undergoes a change in its possible steady state values.

In our experiments, we considered Ising lattices of L ² spins, with L ∈{10, 12}, arranged on a torus. We set k _B = 2, hence we expect a phase transition around the value T _c ≈ 1.13. For each lattice size we run simulations with values of T spanning the range [1, 2] at steps of 0.05. In finite size Ising models, the critical value of temperature is expected to deviate from the theoretical value. Therefore, the actual critical temperature value was estimated by computing the susceptibility (Christensen and Moloney 2005), defined as:

$$\displaystyle \begin{aligned} \chi = \frac{1}{TN} \big(\langle \mu^2 \rangle - \langle \mu \rangle^2 \big) \end{aligned} $$

(5)

where T is the temperature, N the number of atoms, μ the magnetisation of the system at a given time step and angular brackets denote the time average. The peak of χ may be used to identify the actual critical temperature value for finite instances. In Fig. 2 the susceptibility is plotted against the temperature value. As we can observe, the critical values are around T = 1.25 for both the lattice sizes considered, which is slightly higher than the theoretical one. This discrepancy is due to the finite size of the systems. This specific value will be taken as the critical one in the Ising models of our experiments.

Fig. 2

Plots of susceptibility values vs. temperature

Simulations were run until the transient was expired and we recorded 10⁴ lattice configurations every L ² steps. We computed RI and T _c for 10³ randomly sampled subsets for each size between two and L ² and kept the best 10³, i.e. those with the highest values of T _c.

The most relevant result we observed is that larger clusters with high T _c seems to be correlated to high susceptibility values. To assess this informal observation we introduce an index ρ(S) := 〈|S|× T _c(S)〉, where |S| is the size of the subset, T _c(S) its RI statistical significance and 〈⋅〉 denotes the average. The correlation between ρ and χ is striking, as shown in Fig. 3.

Fig. 3

Plots of susceptibility values and ρ as a function of T

A further evidence for supporting this correlation is provided by considering separately the average size and average T _c of the best 1000 subsets. In Fig. 4 we show a diagram in which each circle has a centre at coordinates (〈size〉, 〈T _c〉) and radius proportional to χ. We observe that the largest circles are characterised by both high size and T _c.

Fig. 4

Diagrams representing the susceptibility (proportional to the radius of the circle) at coordinates (〈size〉, 〈T _c〉)

5 Conclusion and Future Work

The results we observed by computing the RI on subsets of atoms of every size show that the most significant dynamically relevant sets in the Ising lattice tend to be larger and characterised by higher significance in correspondence to the critical temperature. Furthermore, an index defined as the product of size and T _c is shown to be highly correlated to the susceptibility, making it possible to locate the phase transition.

The relation between T _c and large subsets may be a consequence of the all-range correlations appearing at criticality in the Ising model. In future work we plan to further investigate this relation and assess to what extent the RI method can be used to detect early signals of criticality and dynamical change in complex systems.