CC BY
31
Mathematical Structures and Modeling 2016. N. 1(37). PP. 18-24
UDC 510.5
conditional dimension in metric spaces: a natural metric-space counterpart of kolmogorov-complexity-based mutual
dimension
V. KREiNovicH
Ph.D. (Phys.-Math.), Professor, e-mail: [email protected] L. LongprE
Ph.D. (Computer Science), Associate Professor, e-mail: [email protected]
O. KosHElEva
Ph.D. (Phys.-Math.), Associate Professor, e-mail: [email protected] University of Texas at El Paso, El Paso, TX 79968, USA
AbstRact. It is known that dimension of a set in a metric space can be characterized in information-related terms - in particular, in terms of Kolmogorov complexity of different points from this set. The notion of Kolmogorov complexity K(x) - the shortest length of a program that generates a sequence x -can be naturally generalized to conditional Kolmogorov complexity K(x : y) - the shortest length of a program that generates x by using y as an input. It is therefore reasonable to use conditional Kolmogorov complexity to formulate a conditional analogue of dimension. Such a generalization has indeed been proposed, under the name of mutual dimension. However, somewhat surprisingly, this notion was formulated in pure Kolmogorov-complexity terms, without any analysis of possible metric-space meaning. In this paper, we describe the corresponding metric-space notion of conditional dimension - a natural metric-space counterpart of the Kolmogorov-complexity-based mutual dimension.
KEywoRds: metric dimension, Kolmogorov complexity, conditional Kol-
mogorov complexity, conditional dimension, mutual dimension.
1. NEEd foR a MEtRic AnaloguE of Mutual DimEnsion: FoRmulation of a PRoblEm
What is dimEnsion: an informal idEa. A straight line segment Si is
a 1-dimensional set, meaning that to select a point on this segment, it is sufficient to describe the value of a single real-valued quantity.
Similarly, a planar area S2 is a 2-dimensional set meaning that to select a point in this area, we need to describe the values of two real-valued quantities: namely, two coordinates of this point.
Mathematical Structures and Modeling, 2016. N, 1(37)
19
A spatial area S3 is a 3-dimensional set meaning that to select a point in this area, we need to describe the values of three real-valued quantities: namely, three coordinates of this point, etc.
Metric dimension as a formalization of this informal idea. In practice, we can only describe a real number with some accuracy, and thus, we can only describe a point with some accuracy e > 0.
Let us start with a straight line segment Si of length L. On this segment, two points which are e-close are, within this accuracy, indistinguishable. So, if we start with a point on this segment, the next e-distinguishable point has to be at a distance > e. Thus, the overall number of e-distinguishable points (or, equivalently, e-distinguishable real numbers) is
Ne(Si) - L.
e
In a 2-D domain S2 of area A, we can place
-
Ne(S2) —
e2
e-distinguishable points: e.g., we can place such points on a rectangular grid, with
c
— - distinguishable values along each dimension. This number is asymptotically
equal to the number (Ne(S1))2 of pairs of e-distinguishable real numbers - which is in perfect accordance with the fact that we need two real numbers to describe a point in the 2-D domain S2.
Similarly, in a 3-D domain S3 of volume V, we can place
N£(S3) — V e3
e-distinguishable points: e.g., we can place such points on a rectangular grid. This number is asymptotically equal to the number (Ns(S1))3 of triples of e-distinguishable real numbers - which is also in perfect accordance with the fact that we need three real numbers to describe a point in the 3-D domain S2.
For a general metric space, we arrive at the following natural definition.
Definition 1.
• Let e > 0 be a real number. We say that a finite set F is an e-net far the metric space S if every paints fram S is e-clase ta ane af the paints fram the set F.
• Far each set S in a metric space M, let Ne(S) denote the smallest passible number af paints in an e-net af S. We say that the set S has dimension a
if Ne(S) — e~a as e ^ 0.
Comment. This definition is the main case of the so-called Hausdorff (metric) dimension. This definition goes beyond the usual 1-D, 2-D, and 3-D spaces, since
20 V, Kreinovich, L, Longpre, O, Kosheleva, Conditional Dimension in Metric,,,
it is also applicable to irregular sets called fractals [3]: e,g,, a trajectory of a Brownian motion has dimension a = 1.5.
Dimension and information. Dimension can be also described in information terms, namely, in terms of the number of bits (0s or Is) which are needed to uniquely determine a point in a metric space S with a given accuracy e,
Specifically, to describe a point with a given accuracy, we need to pinpoint one of the Ne(S) e-close points. If we use b-bit binary strings, then we can identify no more than 2b different points. Thus, the smallest number of bits b which is needed to identify Ne(S) different points is the smallest integer for which 2n ^ Ne(S), i.e., the value b = |~log2(Ne(S))]. This logarithm is usually denoted by He(S) and is called an e-entropy of the metric space.
In terms of the e-entropy He(S) = log2(Ne(S)), the definition Ne(S) ~ e-a takes the form
He(S)-----a ■ log2(e).
Comment, This asymptotic relation is used as an alternative definition of metric dimension.
Relation with Kolmogorov complexity. For subsets of a real line or subsets of an Euclidean space, we can have yet another reformulation of metric dimension: in terms of so-called Kolmogorov complexity K(x). Kolmogorov complexity K(ж) of a string x is defined as the shortest length of a program (in some fixed programming language) that is needed to generate the string x; see, e.g., [2].
To generate all infinitely many bits of a well-defined sequence of bits such as
00.. . or 0101... , we can use a program of finite length. However, to generate n bits of a truly random sequence x, bits of which do not follow any law, we need to actually list all these n bits in the description of the generating program, so we have K(ж) ^ n (to be more precise, K(ж) ^ n — c for some constant c). This is exactly why Kolmogorov complexity was invented in the first place: to formally describe the meaning of a random sequence.
In the 1-D case, if we select a random infinite binary sequence
x = XiX2... xn. . .
that describes a random point in a segment, then for its initial fragments x1... xn that describe this point with accuracy 2-n, we get K (x1x2 ...xn) ~ n, i.e.,
lim K(xi ...xn)
n——<^ n
1.
For non-random points, we need fewer bits, so dimension 1 can be defined as the largest possible value of the above limit over all points
x = x1x2...xn...
from this segment.
Mathematical Structures and Modeling, 2016. N. 1(37)
21
In the 2-D case, if we select a random point, i.e,, a random pair of sequences
(X\,X2) = (X11X12 . . . x1 n . . . , x21x22 . . . x2n . . .),
then, to describe this point with accuracy 2-n, we need
K(x11x12... xnx21x22... x2n) ~ 2n, i.e., we have
lim K (x 11 x12 . . . xnx21x22 . . . x2n) = 2
n——<^ n
For non-random points, we need fewer bits, so dimension 2 can be defined as the largest possible value of the above limit over all points
(xb x2) = (x11 x12 . . . x 1n . . . , x21x22 . . . x2n . . .)
from the corresponding 2-D domain.
In general, for any d-dimensional point
x (x1, . . . , xd) (x 11 x12 . . . x1n ... , ... , xd1xd2 . . . xdn . . 0
in an Euclidean space, we can define its dimension dim(x) as the limit
K(x 11x 12 . . . x1nx21x22 . . . x2n . . . xd1xd2 . . . xdn)
dim(x) = lim
n
Then, for many reasonable sets S c IRd, the metric dimension is equal to the largest dimension of the corresponding points:
dim(S) = maxdim(x).
Conditional Kolmogorov complexity and mutual dimension. The notion of Kolmogorov complexity K(x) has been naturally extended to the notion of conditional Kolmogorov complexity K(x : y) as the shortest length of a program that, given y as an input, generates x.
In [1], this notion was used to produce the corresponding analog of dimension - which the authors called mutual dimension. Specifically, let us consider a Euclidean space M = IRd which is represented as a Cartesian product M = X x Y, where X = IRdx and Y = IRdy with dx + dy = d. Then, for every point (x,y) e M,
we can define mutual dimension dim(x : y) as
/ s def K(x11 . . . x1nx21 . . . xdx1 . . . xdxn : y11 . . . ydy 1 . . . xdyn)
dim(x : y) = lim ----------------------------------------------—.
n—ro П
Challenge. The problem with the above definition is that it is described in purely Kolmogorov-complexity terms, it is not clear what is its metric analogue of this definition. Such a metric analogue is described in this paper.
22 V. Kreinovich, L, Longpre, O, Kosheleva, Conditional Dimension in Metric,,,
2. Conditional Dimension in Metric Spaces: A Natural Definition and its Relation to Kolmogorov-Complexity-Based Mutual Dimension
What is conditional dimension: an intuitive idea. As we have mentioned, the usual metric-space dimension describes how many bits of information we need to describe a point in the given set S C M with a given accuracy e > 0.
In this case, a natural interpretation is that points from the set S (or, more generally, from the metric space M) represent physical objects: a point can be an actual point in space, or it can be a point that characterizes a physical object. From the practical viewpoint, this interpretation makes perfect sense:
• all we know about each object is the results of measuring different quantities related to this object,
• so it is natural to represent the object as a tuple consisting of the values of these measurement results.
For example, the state of a point-wise mechanical object can be characterized, at each moment of time, by a tuple consisting of 6 real numbers: 3 spatial coordinates and three components of the velocity vector.
To describe the state of a possibly rotating solid body (e.g., a planet or an asteroid), we need to supplement these 6 numbers with 2 angles describing this body’s current orientation and 3 numbers describing its angular velocity.
In many practical situations, we have a system consisting of two interacting subsystems. In such situations, to describe the state of a system, we need to describe the pair (x,y) of states: the state x of the first subsystem and the state y of the second subsystem. In mathematical terms, the set S of all possible states of the system as a whole is thus a subset of the space M = X x Y of all possible pairs of states, where:
• X is the set of all possible states of the first subsystem and
• Y is the set of all possible states of the second subsystem.
It is natural to ask a question: if we know the state y of the second subsystem, how many bits do we need to describe the state x of the first subsystem?
If the states x and y were unrelated, then of course, the knowledge of y would be of no help. But when they are related, we expect that the knowledge of y can help us find x.
Towards a formal definition. Once we know y, we thus know that the state of possible values of x is limited to the set {x : (x,y) e S}. Let us denote this set by Sy. For each y, it is thus reasonable to describe the corresponding number of bits as dim(Sy).
This number may depend on the choice of y. We want to make sure that the corresponding bound on the number of bits holds for all possible y, so we should
Mathematical Structures and Modeling, 2016. N. 1(37)
23
consider the largest of the corresponding values dim(Sy) as the proper description of the “conditional” dimension.
Thus, we arrive at the following definition,
Definition 2. Let X and Y be metric spaces, and let S be a subset of the set X x Y of all pairs (x,y). By the conditional dimension dimX:Y(S), we mean
dimX.Y (S) == maxdim(Sy),
yeY
where Sy == {x : (x,y) e S}.
Examples. Let us consider examples in which X and Y are straight line segments and S is a straight line in the rectangle X x Y,
If this straight line is neither parallel to X nor to Y, then the value x is uniquely determined by the value y, i,e,, S is a graph of a linear function x = f (y): S = {(f(y),y) : y e Y}, In this case, as expected, the conditional dimension is equal to 0: dimX:Y(S) = 0 < dim(nX(S)) = 1, where (S) = {x : (x,y) e S} is
the set of all possible x-values, i,e,, in mathematical terms, a projection of the set S onto X,
If X and Y are straight line segments and S is a straight line in the rectangle X x Y which is parallel to Y, then knowing y does not provide us any information about x, so in this case, dimX:Y(S) = dim(nX(S)) = 1,
If S is the graph of the Brownian motion, i,e,, y is time and x is the value of the Brownian motion at time y, then:
• knowing time y, we can uniquely determine the value x, so dimX:Y(S) = 0;
• on the other hand, when we know x, we can only determine y with uncertainty; the corresponding set has dimension 0,5, so dimY:X(S) = 0.5,
In many of these cases, we have
dim(nX (S)) = dimX:Y (S) + dim(nY (S)),
where nY(S) == {y : (x,y) e S} is the projection of the set S on Y, However, sometimes,
dim(nX(S)) < dimX:Y(S) + dim(nY(S)).
For example, if S consists of two straight-line segments, one parallel to X and one parallel to Y, then dimX:Y(S) = 1 and dim(nY(S)) = 1, but
dim(nX(S)) = 1 < dimX:Y(S) + dim(nY(S)) = 1 + 1 = 2.
Relation to Kolmogorov complexity. In many of the above examples, we have dimX:Y(S) = max dim(x : y).
(x,y)es
24 V. Kreinovich, L, Longpre, O, Kosheleva, Conditional Dimension in Metric,,,
Acknowledgments. This work was supported in part by the National Science Foundation grants HRD-0734825 and HRD-1242122 (Cyber-ShARE Center of Excellence) and DUE-0926721.
The authors are thankful to all the participants of the 2015 North American Annual Meeting of the Association for Symbolic Logic (Urbana, Illinois, March 2528, 2015), especially to Jack Lutz, for valuable discussions.
References
1. Case A., Lutz J.H. Mutual dimension. Electronic Colloquium on Computational Complexity. 2014. Report № 133.
2. Li M., Vitanyi P.M.B. An Introduction to Kolmogorov Complexity. Springer-Verlag, Berlin, Heidelberg, New York, 2008.
3. Mandelbrot B. The Fractal Geometry of Nature. Freeman, San Francisco, California, 1983.
условная размерность в метрических пространствах: естественный аналог взаимной размерности, основанной на колмагоровской сложности, для метрического
пространства
В. Крейнович
к.Ф.-м.н., профессор, e-mail: [email protected] Л. Лонгпре
к.т.н., доцент, e-mail: [email protected]
О. Кошелева
к.Ф.-м.н., доцент, e-mail: [email protected]
Техасский университет в Эль Пасо, США
Аннотация. Известно, что размерность множества в метрическом пространстве можно охарактеризовать в терминах, связанных с информацией — в частности, в терминах колмогоровской сложности различных точек множества. Понятие колмогоровской сложности K(х) — наименьшая длина программы, которая генерирует последовательность x — может быть естественным образом обобщено в условную колмогоровскую сложность K(х : у) — наименьшая длина программы, которая генерирует х, используя у в качестве входных данных. Поэтому целесообразно использовать условную колмогоровскую сложность, чтобы сформулировать условный аналог размерности. Такое обобщение действительно было предложено под названием взаимной размерности. Однако это понятие несколько неожиданно было сформулировано в терминах чистой колмогоровской сложности, без какого-либо анализа возможного значения для метрического пространства. В этой статье мы описываем соответствующее понятие условной размерности для метрического пространства — естественный аналог взаимной размерности, основанной на колмагоровской сложности.
Ключевые слова: метрическая размерность, колмогоровская сложность, условная колмогоровская сложность, условная размерность, взаимная размерность.
