image: (a) The effective information(EI) is denoted as J in this paper. (b) A case demonstrating CE in a discrete Markov chain. The micro-dynamics consist of eight micro-states. During the coarse-graining process, the first seven states are grouped together as one macro-state, while the eighth micro-state corresponds to the second macro-state. As a result, a transition probability matrix is formed at the macro-scale, where the effective information J(fM)=1, which is greater than J(fm)=0.55. This difference, ΔJ=0.45, indicates the occurrence of CE, as ΔJ>0. Figure credit: Jiang Zhang and Mingzhe Yang.
Credit: Figure credit: Jiang Zhang and Mingzhe Yang.
This study is led by Prof. Jiang Zhang (School of Systems Science, Beijing Normal University and Swarma Research). He has been pursuing the essence behind the emergent phenomena in complex systems. "How do ants aggregate into colonies? How do a multitude of molecules collide to form the first cell? How does the chaotic firing of neurons constitute the thinking self? These fascinating questions all lead me to the same keyword—emergence," said Zhang.
One day, he read an article in PNAS titled "Quantifying causal emergence shows that macro can beat micro" which greatly excited him. The causal emergence theory proposed in the article transformed emergence from philosophical discussion to a quantifiable description. Generally, the dynamics of a system are assumed to be describable by a Markov probability transition matrix. After a coarse-graining strategy is given, by comparing the causal effect strength (i.e., the amount of effective information) of macroscopic and microscopic dynamics, an observer can determine whether causal emergence has occurred. "This paper turned the vague thoughts about emergence, causality, observers, and so on that have been lingering in my mind for over a decade into a very specific quantitative theory. I felt a tremendous shock," said Zhang.
However, there has been considerable criticism of the theory of causal emergence. One argument is that it does not specify how to perform coarse-graining, as different coarse-graining methods can yield vastly different outcomes. Additionally, it requires knowledge of the dynamics to calculate effective information. Jiang Zhang's research group has long been dedicated to the automatic modeling of complex systems. Thus, he pondered: Can an algorithm be devised to automatically learn coarse-graining functions and dynamics from data while ascertaining the emergence of causality by maximizing effective information? Combining artificial intelligence technologies such as reversible neural networks, Jiang Zhang quickly constructed the original Neural Information Squeezer (NIS) framework, which can model multi-scale data-driven fitting of macroscopic and microscopic dynamics. However, regrettably, this framework could not maximize the effective information of macroscopic dynamics; it could only perform an ordinary prediction task and could not ensure the full identification of causal emergence. Zhang was determined to solve this problem.
In September 2022, Mingzhe Yang, Zhipeng Wang, and Kaiwei Liu, among other students, established the "Machine Learning for Causal Emergence" research group under the guidance of Jiang Zhang. After some discussion, the team decided to develop a machine learning framework based on the NIS framework that could theoretically and experimentally address the problem of identifying causal emergence, named Neural Information Squeezer Plus (NIS+). At that time, the team drew on the sample reweighting techniques from the field of causal inference and discovered that a variational lower bound of mutual information could be found, and that optimizing this lower bound could maximize mutual information. Inspired by Zhang, Mingzhe Yang completed the mathematical proof, transforming the maximization of effective information into an equivalent machine-learnable prediction problem, and conducted experiments on a simple SIR infection dynamics model with observed noise. Zhipeng Wang experimented with the Game of Life, and Kaiwei Liu experimented with a flocking model. During this period, Bing Yuan from Swarma Research also joined the team. With his rich practical experience in coding, he provided many useful suggestions for the code implementation of the project.
In addition to simulation experiments, the team also needed to experiment with real data to complete a satisfactory work. The research team also recruited a scientific research volunteer, Yingqi Rong, a master's student from Johns Hopkins University. Familiar with fMRI data of the brain, he modeled NIS+ on the fMRI time series of subjects watching movies. Surprisingly, the macroscopic dynamics with the strongest causal effect was a one-dimensional model. By employing the integral gradient method, the team discovered that the visual areas in the brain contribute the most to this one-dimensional macroscopic state. Furthermore, in the resting-state data where the subjects did nothing, the brain's dynamics require at least 3 to 7 macroscopic dimensions for description. Do these macroscopic dynamics reveal the essence of consciousness? On which macroscopic scales does consciousness actually occur? Perhaps in future research, these mysteries will one by one have their answers revealed.
"In fact, NIS+ is an ideal 'machine observer'. Complex system properties such as emergence and consciousness are not solely intrinsic to the system but also exist within the observer's subjective world. By establishing an objective machine observer, we can study these 'higher-order' attributes more objectively. This represents a new paradigm for research." said Zhang. This research can be considered a milestone in the study of the essence of complex systems using artificial intelligence.
See the article:
Finding emergence in data by maximizing effective information
https://doi.org/10.1093/nsr/nwae279
Journal
National Science Review