CC BY
3Physics of Complex Systems, 2024, vol. 5, no. 3 _www.physcomsys.ru
R Check for updates
Theoretical physics. Theory of condensed matter
UDC 538.9
EDN WTEGLX
https://www.doi.org/10.33910/2687-153X-2024-5-3-146-153
Information in complex physical systems: Kolmogorov complexity plane of interacting amplitudes
D. T. Mihailovic V. P. Singh 1
1 University of Novi Sad, 3 Dositeja Obradovica Sq., Novi Sad 21000, Serbia 2 Texas A&M University, 400 Bizzell Str., Texas 77843, USA
Dragutin T. Mihailovic, ORCID: 0000-0002-8380-1844, e-mail: [email protected]
Vijay P. Singh, ORCID: 0000-0003-1299-1457, e-mail: [email protected]
For citation: Mihailovic, D. T., Singh, V. P. (2024) Information in complex physical systems:
Kolmogorov complexity plane of interacting amplitudes. Physics of Complex Systems, 5 (3), 146-153.
https://www.doi.org/10.33910/2687-153X-2024-5-3-146-153 EDN WTEGLX
Received 23 May 2024; reviewed 6 July 2024; accepted 6 July 2024. Funding: The study did not receive any external funding.
Copyright: © D. T. Mihailovic, V. P. Singh (2024) Published by Herzen State Pedagogical University of Russia. Open access under CC BY-NC License 4.0.
Abstract. One of the most challenging tasks in studying complex physical systems is determining the contributions of complexities of individual components to the complexity of the entire system. To investigate these contributions, this paper proposed the Kolmogorov complexity plane (the KC plane) based on the Kolmogorov complexity. We applied both the Kolmogorov complexity plane and the Kolmogorov complexity to daily (measured) and monthly (simulated by a climate model) values of precipitation representing a complex physical system, and temperature and water vapor pressure as individual components of that system. From the KC plane, we determined the intervals of interacting amplitudes in which the contributions of the complexities of individual components to the overall complexity could be observed.
Keywords: physical complex systems, Kolmogorov complexity, Kolmogorov complexity spectrum, Kolmogorov complexity plane, overall complexity, complexity of components
It took almost half a century after the publication of Nobel Laureate Philip Anderson's pioneering and visionary work 'More is different: Broken symmetry and the nature of the hierarchical structure of science' (Anderson 1972) for complex systems, including the climate system, to receive formal recognition from the scientific community. In 2021, three scientists were awarded the Nobel Prize 'for groundbreaking contributions to our understanding of complex systems': Syukuro Manabe and Klaus Hasselmann 'for the physical modelling of Earth's climate, quantifying the variability and reliably predicting global warming' and Giorgio Parisi 'for the discovery of the interplay of disorder and fluctuations in physical systems from atomic to planetary scales' (The Nobel Prize in Physics 2021... 2021).
When it comes to a complex system, there is no clear formal definition that describes it in terms of its properties. Many papers have been written on this topic (for example, (Estrada 2023)); however, in this paper, we do not deal with this issue but discuss only the two most discriminating features of complex systems — emergence and complexity. Emergence is loosely considered as a behavior of a system that is more than the sum of its parts (De Wolf, Holvoet 2005). The behavior of a complex system is referred to as weak emergence, or simply emergence, when interactions between components at lower levels
Authors
Introduction
create new properties at higher levels. Weak emerging properties are scale-dependent. On the other hand, the behavior when high-level characteristics cannot be inferred from low-level properties is called strong emergence. A system exhibits complexity when we cannot explain its behavior by examining its components. We can only use various complexity measures to compute complexity as the only way to obtain information about the complex systems' nature through measured time series. Emergence and complexity cannot be modeled due to various reasons, such as the boundaries of our knowledge and computational power. That is why one resorts to 'model' complexity and emergence with more or less sophisticated models (Mihailovic et al. 2023).
In physics, emergence describes a property, law or phenomenon occurring spatially or temporally at macroscopic scales, but not at microscopic scales, although a macroscopic system can be viewed as a very large ensemble of microscopic systems (Girvin, Yang 2019). Complex physical systems follow fixed physical laws, usually described by differential equations, and exhibit properties such as self-organized criticality, self-similarity, scaling and power laws. Also, the majority of complex systems in physics are complicated, while the opposite is not true (Gell-Mann, Lloyd 1996). What do emergent phenomena look like in complex physical systems? Spontaneously broken symmetries that characterize distinct phases of matter are weakly emergent. Thus, according to (Bedau 1997), some phase transitions can be classified as weak emergence. Quantum mechanics is one promising area of physics that can be considered a candidate for strong emergence (Mihailovic et al. 2023). Except for the mentioned phase transition, the superconductivity, granular materials are typical physical systems characterized by a weak emergency. Physical complex systems consist of many interconnected components whose interactions produce emergent behavior that cannot be easily predicted from the behavior of individual components. Here are several examples across different domains: weather systems, ecosystems, wildfires, oil spills, human brain, financial markets, traffic systems, social networks, cellular systems, power grids, internet and communication networks, supply chains, galaxies and stellar systems and biochemical pathways.
What is the situation regarding information in the physics of complex systems? It is more complicated to extract information from other complex systems in nature than from systems in physics. A favorable circumstance for physical systems is that they possess some characteristics that are not inherent in other systems (Hanel, Thurner 2013; Thurner et al. 2018). For physical systems that are described algorithmically, in general, we can obtain information about how their internal states (interactions) and the states of components evolve over time. There are several approaches to studying complex physical systems. Some of those include physical approaches, computational approaches, network approaches, statistical approaches (statistical methods, statistical physics, information theory and nonlinear dynamics) and interdisciplinary approaches. In general, methods for obtaining information from complex systems can be divided into methods for (1) analyzing data, (2) constructing and evaluating models and (3) computing complexity. Here we focus only on algorithmic information theory.
Perhaps one of the most challenging tasks in studying complex physical systems is determining the contributions of the complexities of individual components to the complexity of the entire system. Some of the methods attempting to meet this challenge include: (1) analyzing interconnectedness; (2) assessing feedback loops; (3) mapping out dependencies; (4) conducting sensitivity analysis; (5) using network theory; and (6) simulating scenarios. In this paper, we propose a method for determining the information about the interaction of components in a complex physical system by the Kolmogorov complexity spectra (hereafter, KC spectrum/spectra) amplitudes. This method, which we call the Kolmogorov complexity plane (KC plane), is based on the use of the Kolmogorov complexity (KC) (Kolmogorov 1965) and the KC spectrum (Mihailovic et al. 2015). More specifically, the KC plane is imagined to have the following axes: (i) interactive amplitudes of the system components' complexity (x-axis) and (ii) interactive amplitudes of the system's overall complexity (y-axis).
Kolmogorov complexity
Kolmogorov complexity K(x) is a well-known concept in algorithmic information theory (Kolmogorov 1965). It is generally incomputable, and the most famous and most widely used algorithm (LZA) for its approximation was developed by Lempel and Ziv (Lempel, Ziv 1976) and improved (known as the LZW algorithm) by (Welch 1984). For a given time series, LZA determines the minimal number of different patterns (Kaspar, Schuster 1987). When applied to a time series {x.}, i = 1,2,3, ... , N, LZA includes the following steps: 1. Creating a sequence {5.}, i = 1,2,3, ... , N of the characters 0 and 1 by applying the rule S. = 0 if x. < xt or 1 if x. > x, where x is a threshold. The mean value of a time series is usually chosen as the
i tr i tr tr J
threshold. 2. Computing the minimum number of different patterns c(N) in x.. c(N) is called the complexity counter that approaches an ultimate value as N gets closer to infinity, i. e. c(N) = O(b(N)) and b(N) = log2N. 3. Computing the normalized information measure Ck(N) = c(N)/b(N) = c(N)/log2N. Ck(N) ranges between 0 and 1 for a nonlinear time series, but it can also be much greater than 1 (Kovalsky et al. 2018).
Kolmogorov complexity spectrum
The complexity of a system is hidden in its dynamics. The only information available about a physical state is found in time series; therefore, a time series is the only source for establishing the level of complex physicality. This term usually refers to the level of complexity and interconnectivity within a complex physical system. Physical complexity refers to the intricate and multifaceted nature of an object's or a system's physical characteristics. It encompasses various physical attributes and behaviors that contribute to the overall complexity. Two methods are available for obtaining the time series: measurement and simulation. Regarding the former, the exact states of an observed physical system are translated into a sequence of symbols. This process is described by a parameterized partition of the state space Me , consisting of cells of size e that are sampled at each time sample point t. A measurement sequence consists of the successive elements Me visited over time by the system's state. Using the instrument {Me , e}, we get information as a sequence of states {x.}. Here, we consider a possible way to calculate the level of complex physicality of the system, i. e. the complexity of the time series that represents that the system is passing through different states. Connecting complex physicality with Kolmogorov complexity, we can explore how the intricate, multifaceted nature of complex physical systems might be analyzed or described in terms of their informational content and computational complexity.
Def. 1 We call the time series {x.}, i = 1,2,3, ... , N a normalized one (or a time series with normalized amplitude) after the transformation x. = (X. - X . )/(X - X .), where {X}, i = 1,2,3, ... , N is a time
r ' i^i mm' v max mm' K i /// /
series obtained either by a measurement procedure or as an output of a physical model, Xmin = min{X} and X = max{X}.
max i
Remark. From Def. 1 it follows that all elements of the time series {x.} are in the interval [0,1].
Def. 2. If the LZA algorithm is applied N times to a time series {x.}, using all the elements of {x.} as thresholds {xtri} forming the sequence {c.}, i = 1,2,3, ... , N then we will call the sequence {c.} the Kolmogorov complexity spectrum of a time series {x}.
Remark. The time series {x.} is transformed into a string of finite symbols by comparison with a series of thresholds {xtri}, i = 1,2,3, ... , N, where each element is equal to the corresponding element in the time series , applying the LZA algorithm. The original time series samples are converted into a set of 0-1 sequence {S/k)}, i = 1,2,3, ... , N, k = 1,2,3, ... , N defined by comparison with a threshold xtrk ,
{0 x. < xt ,
i trk . (1) 0 x. > x„,
i tr,k
After applying the LZA algorithm to each element of the series {S/k)}, we get the KC complexity spectrum {c.}, i = 1,2,3, ..., N (fig. 1). This spectrum was introduced by (Mihailovic et al. 2015) to especially examine complex systems with high complexity, i. e. those that have many stochastic components. It can provide new insights into the complexity of physical and other complex systems, their time evolution and predictability.
To clarify the meaning of the KC spectrum {ci}, we use the following example: the time series {xi} is obtained by {Me, e} = e-wa, where a is a random number uniformly distributed in the interval [0, 1], w is the amplitude, taking values in the interval [0, 1], and {x.} is sampled at each time sample point t = 1. Fig. 1a shows the KC spectra for w = 1.0, 0.75, 0.50 and 0.25, respectively. All of the spectra have a shape that is similar to the shape of the curve in Fig. 1b, which is merely one of many possibilities due to the fact that different systems have different complexity versus randomness plots, since there is no 'universal' complexity-entropy relationship (Feldman, Crutchfield 1998). Random numbers were generated with the intrinsic subroutines CALL SEED and CALL RANDOM NUMBER (arg) from the Microsoft Fortran Developer Studio library.
Fig. 1. (a) The normalized KC spectra {c.} of time series obtained by the instrument obtained {M, e} with {Me, e} = e-wa, where w is the amplitude factor, a is the random number uniformly distributed in the interval [0,1], and sampling t = 1 time unit; (b) Complexity versus randomness plotted following physical intuition (Grassberger 2012)
Kolmogorov complexity plane of interacting amplitudes
A comparison of the two KC spectra is shown in Fig. 2b, which represents two merged two-dimensional graphs (with different meaning of axes): (1) KC master vs. individual components (both amplitudes are on x-axis and both complexities are on y-axis) and (2) aM (the master amplitude) vs. at (individual amplitude), i. e. the KC plane. The black and red squares represent master (KM) and individual spectra (K), which are calculated from the time series generated by a random generator (see the previous subsection), while the crosses indicate points in the KC plane. The quantities in these two planes were compared in the two-dimensional system (0, 1) using normalized time series. Since these quantities are on the same scale after normalization, we can assess relationships and patterns between the compared quantities without the influence of varying scales (Asesh 2022).
When we look at Fig. 2b, we see that the crosses in the (ap aM) system are scattered. This is due to the dynamic relationship and interdependence between the 'master' variant and individual ones. Here, the 'master' variant characterizes the primary, overarching complexity that defines the core complexity. The greater overlapping of the KC spectra (K{, KM) determines the reduced scattering of points in the KC plane, approaching a linear distribution of the set of points, as shown in Fig. 2a. It can also be seen that there are two groups of points: one that belongs to the area under the KM spectrum curve (the set Sm) and another that belongs to the area under the K{ spectrum curve (the set S. ). However, only points from
the set S defined as
p
S = S n S. (2)
p m i v '
are candidates for comparison of the complexity of different sequences or systems (in this case, the entire complex physical system and its components). The amplitudes that belong to this set will conditionally
Fig. 2. (a) Towards the Kolmogrov complexity plane (KC plane); (b) The values of amplitudes of the master and individual KC spectra that are set along aM and a} axes (crosses), respectively. KC spectra of the entire system (KM , black squares) and its component (K , red squares) were calculated from the time series generated by a random generator (see the previous subsection)
be called interactive amplitudes. Analyzing Fig. 2b, we can draw a crucial conclusion as follows: interplay-ing master and individual interactive components are most pronounced in the domains (0.05, 0.8) for master amplitudes and (0.05, 0.6) for the individual components. Perhaps more precisely stated, individual interactive amplitudes in the domain (0.17, 0.32) contribute most to the complexity of the overall system across the entire interval (0.05, 0.8) of master interactive amplitudes.
Numerical examples
Precipitation is a complex physical system par excellence. Precipitation and its formation are influenced by many factors and processes: temperature, humidity, pressure, atmospheric conditions, cloud formation, condensation of water vapor into droplets or ice crystals, falling of these particles to the ground as rain, snow, sleet, or hail, geography, topography, wind and air masses. So, the prediction of precipitation events by models of different scales is a hard task because of the difficulties in understanding these interconnected influencing factors. We use the KC plane of interacting amplitudes and two time series of precipitation: monthly precipitation simulated by a climate model and daily participation measured over twenty years (Fig. 3).
In the analysis of monthly time series, we use precipitation (the master amplitude) and temperature (individual amplitudes) for Novi Sad (45°15' N, 19°50' E) in Serbia, which were obtained from the EBU-POM model simulation under the A1B scenario during the period 2071-2100 (Djurdjevic, Rajkovic 2012). Specifically, for this integration, the center of the atmospheric Eta model was at 41.5°N, 15°E, with ±19.9° boundaries in the east-west direction, ±13.0° boundaries in the north-south direction, 0.25° horizontal resolution and 32 vertical levels (with the first level at 20 m and the top level at 10 hPa). The ocean model featured 0.2 x 0.2° of horizontal resolution and 21 vertical levels. The size of the time series was N = 1560; the highest values of precipitation amount and temperature were 248.7 mm and 29.2°C,
respectively. In Fig. 3, the KC spectra of precipitation (black squares) and temperature (red squares), as well as points in the KC plane (crosses), are shown. The number of these points is 821 in the area that belongs to the area below both spectra and 739 points outside this area.
Fig. 3. KC spectra of the entire system (black squares) and its component (red squares). The values of amplitudes of the precipitation and temperature KC spectra are set along aM and a1 axes (crosses), respectively. Monthly values of precipitation and temperature for Novi Sad (45°15' N, 19°50' E) were obtained from the EBU-POM model simulation under the A1B scenario during the period 2071-2100 (Djurdjevic, Rajkovic, 2012)
Looking at the distribution of points in this area within the temperature amplitude interval (0.2, 0.6), it can be seen that the complexities are relatively low (0, 0.4). We can ask ourselves what quantitative conclusion we can draw from the facts presented in Fig. 3. We cannot say anything else except that the influence of interactive temperature amplitudes on precipitation complexity is observed (1) throughout the domain of these amplitudes and (2) in a narrow range of complexity (0, 0.4). Moreover, these findings seem to point towards a solution to the question of how elements of a complex physical system affect its overall complexity, which will be elaborated on later. Let us note that if the KC spectra of time series partially overlap, it could indicate that there are similarities or patterns present in the data at different levels of complexity. This could suggest that certain aspects of the data have a consistent level of complexity across different scales or resolutions. It may also indicate that there are recurring structures in the data that contribute to its overall complexity. There exist some possible interpretations of partially overlapping KC spectra for two or more time series, such as shared patterns, similar complexity profiles, overlapping generative mechanisms and non-random similarities. Some potential applications of this concept include time series clustering, anomaly detection and feature extraction (Fig. 4).
In this paper, we make feature extraction of the interactive amplitudes in the KC plane by detecting the points (aj, aM) that belong to the area(s) below the overlapping KC spectra in that plane. Note that all compared time series were normalized with their highest values. This procedure allowed us to establish how the complexity of individual components of a complex physical system contributed to the overall complexity. This will be demonstrated using precipitation as a complex physical system (the overall complexity) and temperature and water vapor pressure (the complexity of components). Note that this comparison is only possible if all-time series have the same length. Daily values of precipitation, temperature and water vapor pressure for Banatski Karlovac (45°2' N, 21°1'E), averaged over the period 1986-2005, were taken from the Daily Reports of the Hydrometeorological Service of Serbia. Due to the length of the time series (N = 14610) in Fig. 4, points are not shown as in Fig. 3, which would make the graph unclear.
Fig. 4 shows the areas of influence of the complexity of components on the overall complexity: (1) red spectrum-green line-aj axis-green line; (2) green line-blue spectrum-black spectrum; (3) blue spectrum-ajaxis-green line; (4) green line-aj axis-black spectrum; and (5) entire area above the curve of the black spectrum. Translating the position of points in these areas into the statement influence
Fig. 4. KC spectra of the entire physical complex system (black squares) and its components (red and blue squares). The values of amplitudes of precipitation, temperature and KC spectra are set along the aM and a I axes. Areas of influence of complexity of components on the overall complexity: red spectrum-green line-a7 axis-green line (1); green line-blue spectrum-black spectrum (2); blue spectrum-a7 axis-green line (3); green line-a7 axis-black spectrum (4); and the entire area above the curve of the black spectrum (5) (for more explanation see the text). Daily values of precipitation, temperature and water vapor pressure for Banatski Karlovac (45°2' N, 21°1'E), averaged over the period 1986-2005, were taken from the Daily Reports of the Hydrometeorological Service of Serbia
of the complexity of individual components on the overall complexity, we can see (1) only the influence of temperature (0, 0.05); (2) only the influence of water vapor pressure; (3) no influence of temperature or water vapor pressure; (4) the influence of both components (temperature and water vapor pressure (0.12, 1)); and (5) no influence of the complexity of any individual component. It can be said that the KC plane allows us to determine the intervals of interacting amplitudes in which the contribution of the complexity of individual components to the overall complexity can be established. In other words, the contribution of component complexity to overall system complexity is not constant but varies with its interactive amplitudes. This means the interactivity of components with the system changes with component amplitude. This approach has important implications for (i) theory and modelling of complex systems (particularly in the segment that is related to interconnectivity of system components) and (ii) environmental and other physical complex system models having a huge practical application.
Conclusions
We considered the influence of complexity of individual components of a physical complex system on the overall complexity through the following steps: (i) we proposed Kolmogorov complexity plane of interactive amplitudes whose point coordinates (KC plane) were derived based on Kolmogorov complexity spectrum (KC spectrum); and (ii) we selected two time series encompassing the monthly time series of precipitation and temperature simulated by the EBU-POM climate model over the 1971-2100 period (Novi Sad, Serbia), and the measured daily time series of precipitation, temperature and humidity, averaged for the 1986-2005 period), for Banatski Karlovac (Serbia). All-time series were normalized using their highest values.
We calculated: (i) normalized KC spectra for all-time series; (ii) positions of points in the KC plane (interactive master amplitudes vs. interactive individual amplitudes); we made feature extraction of the interactive amplitudes in the KC plane by detecting the points lying in the areas which are below the overlapping KC spectra in that plane.
From the overlapping areas in the Kolmogorov plane, we determined the intervals of interacting amplitudes in which the contribution of complexity of individual components to overall complexity can be observed.
Conflict of Interest
The authors declare that there is no conflict of interest, either existing or potential.
Acknowledgment
The authors are grateful to Professor Darko Kapor and Master of Science Anja Mihailovic for carefully reading the paper and providing useful comments.
References
Anderson, P. W. (1972) More is different: Broken symmetry and the nature of the hierarchical structure of science.
Science, 177 (4047), 393-396. https://doi.org/10.1126/science.177.4047.393 (In English) Asesh, A. (2022) Normalization and bias in time series data. In: C. Biele, J. Kacprzyk, W. Kopec et al. (eds.). Digital interaction and machine intelligence. Cham: Springer Publ., pp. 88-97. https://doi.org/10.1007/978-3-031-11432-8 8 (In English) Bedau, M. A. (1997) Weak emergence. Noüs, 31, 375-399. (In English)
De Wolf, T., Holvoet, T. (2005) Emergence versus self-organization: Different concepts but promising when combined. In: S. A. Brueckner, G. Di Marzo Serugendo, A. Karageorgos, R. Nagpal (eds.). Engineering self-organising systems: Methodologies and applications. Berlin; Heidelberg: Springer Publ., pp. 1-15. (In English) Djurdjevic, V., Rajkovic, B. (2012) Development of the EBU-POM coupled regional climate model and results from climate change experiments. In: D. T. Mihailovic, B. Lalic (eds.). Advances in environmental modeling and measurements. New York: Nova Science Publ., pp. 23-32. (In English) Estrada, E. (2023) What is a complex system, after all? Foundations of Science. https://doi.org/10.1007/s10699-023-09917-w (In English)
Feldman, D. P., Crutchfield, J. P. (1998) Measures of statistical complexity: Why? Physics Letters A, 238 (4-5),
244-252. https://doi.org/10.1016/S0375-9601(97)00855-4 (In English) Gell-Mann, M., Lloyd, S. (1996) Information measures, effective complexity, and total information. Complexity, 2 (1), 44-52. https://doi.org/10.1002/(SICI)1099-0526(199609/10)2:1%3C44::AID-CPLX10%3E3.0.C0;2-X (In English) Girvin, S. M., Yang, K. (2019) Modern Condensed Matter Physics. Cambridge: Cambridge University Press, 714 p. (In English)
Grassberger, P. (2012) Randomness, information, and complexity. Available at: https://arxiv.org/abs/1208.3459
(accessed 10.03.2024). (In English) Hanel, R., Thurner, S. (2013) Generalized (c, d)-entropy and aging random walks. Entropy, 15 (12), 5324-5337.
https://doi.org/10.3390/e15125324 (In English) Kaspar, F., Schuster, H. G. (1987) Easily calculable measure for the complexity of spatiotemporal patterns. Physical
Review A, 36 (2), 842-848. https://doi.org/10.1103/PhysRevA.36.842 (In English) Kolmogorov, A. N. (1965) Tri podkhoda k opredeleniyu ponyatiya "kolichestvo informatsii" [Three approaches
to the quantitative definition of information]. Problemy Peredachi Informatsii, 1 (1), 3-11. (In Russian) Kovalsky, M. G., Hnilo, A. A., Agüero, M. B. (2018) Kolmogorov complexity of sequences of random numbers generated in Bell's experiments. Physical Review A, 98 (4), article 042131. https://doi.org/10.1103/PhysRevA.98.042131 (In English)
Lempel, A., Ziv, J. (1976) On the complexity of finite sequences. IEEE Transactions on Information Theory, 22 (1),
75-81. https://doi.org/10.1109/TIT.1976.1055501 (In English) Mihailovic, D. T., Mimic, G., Nikolic-Boric, E., Arsenic, I. (2015) Novel measures based on the Kolmogorov complexity for use in complex system behavior studies and time series analysis. Open Physics, 13 (1), 1-14. https://doi.org/10.1515/phys-2015-0001 (In English) Mihailovic, D., Kapor, D., Crvenkovic, S., Mihailovic, A. (2023) Physics of complex systems: Discovery in the age
of Godel. Boca Raton: CRC Press, 200 p. (In English) The Nobel Prize in Physics 2021. (2021) The Nobel Prize. [Online]. Available at: https://www.nobelprize.org/prizes/
physics/2021/summary/ (accessed 10.03.2024). (In English) Thurner, S., Hanel, R., Klimek, P. (2018) Introduction to the theory of complex systems. Oxford: Oxford University Press, 448 p. (In English)
Welch, T. A. (1984) A technique for high-performance data compression. Computer, 17 (6), 8-19. https://doi. org/10.1109/MC.1984.1659158 (In English)
