4
0 0.2 0.4 0.6
0
0.1
(c)
COUPLING STRENGTH ε
i(X→Y), i(Y→X)
0 0.2 0.4 0.6
0
0.1
0
0.1
0.2
(a)
P(X→Y), P(Y→X)
0
0.1
0.2
0.1
0.2
0.3
0.4
(b)
S(X|Y), S(Y|X)
0.1
0.2
0.3
0.4
FIG. 2: (a) Cross-predictability P (X → Y ) (solid line) and
P (Y → X) (dashed line), (b) relative average distance of
the mutual nearest neighbours S
(k)
(Y|X) (solid line) and
S
(k)
(X|Y) (dashed line), and (c) coarse-grained transinfor-
mation rate i(X → Y ) (solid line) and i(Y → X) (dashed
line) for the unidirectionally coupled Henon system (3),(4),
as functions of the coupling strength ².
with that of Ref. [19]. Quian Quiroga et al. [19] ex-
plain that the higher-dimensional system (obtained by
the reconstruction from the time series {y
i
} which bears
information about both the coupled systems) is “more
active” than the lower-dimensional (autonomous, driv-
ing) system. Only the CTIR gives the same relation as
in the previous case: i(X → Y ) > i(Y → X) (Fig. 2c)
suggesting the fact that {X} influences {Y }, while {X}
evolves autonomously.
Figure 3 presents the analysis of the unidirectionally
coupled R¨ossler systems (5),(6). We can see that the
results are qualitatively the same as in the case of the
coupled Henon systems (Fig. 2), although these sys-
tems are more similar to the first example of the cou-
pled R¨ossler-Lorenz systems. We can se that neither the
cross-predictability, nor the mutual nearest neighbours
statistics give consistent results when using three differ-
ent examples of unidirectionally coupled systems. Only
the coarse-grained transinformation rate correctly iden-
tifies the direction of the causal influence in the above
three examples as well as in many other systems of dif-
ferent origins tested by the authors.
In the above examples of unidirectionally coupled sys-
tems we could see that the used measures are generally
non-zero in both directions even before the systems be-
come synchronized and comparison of the values of such
measures does not always reflect the true causality given
by the unidirectional coupling of the studied systems.
The intuitively understandable implication lower predic-
0 0.05 0.1
0
0.05
0.1
(c)
COUPLING STRENGTH ε
i(X→Y), i(Y→X)
0 0.05 0.1
0
0.05
0.1
0
0.2
0.4
0.6
(a)
P(X→Y), P(Y→X)
0
0.2
0.4
0.6
0
0.01
0.02
0.03
(b)
S(X|Y), S(Y|X)
0
0.01
0.02
0.03
FIG. 3: (a) Cross-predictability P (X → Y ) (solid line) and
P (Y → X) (dashed line), (b) relative average distance of
the mutual nearest neighbours S
(k)
(Y|X) (solid line) and
S
(k)
(X|Y) (dashed line), and (c) coarse-grained transinfor-
mation rate i(X → Y ) (solid line) and i(Y → X) (dashed
line) for the unidirectionally coupled R¨ossler systems (5),(6),
as functions of the coupling strength ².
tion error (better predictability) => stronger dependence
cannot generally be applied for nonlinear systems. When
the coupling of the systems is weaker than that neces-
sary for the emergence of synchronization, as used in the
above examples, any smooth deterministic function be-
tween the states of the systems does not exist yet. How-
ever, there is already some statistical relation valid on
the coarse-grained description level. Although the deter-
ministic quantities are based on the existence of a smooth
functional relation, when estimated with finite precision
they usually give nonzero values influenced not only by
the existing statistical dependence but also by proper-
ties of the systems other then the coupling. It is there-
fore necessary to use quantities proposed for measur-
ing statistical dependence such as information-theoretic
measures which have solid mathematical background and
their properties have thoroughly been studied since their
introduction in 1948 [35].
III. DEPENDENCE MEASURES FROM
INFORMATION THEORY
In this Section we review basic measures from informa-
tion theory which we will need in further considerations.
More details can be found, e.g., in Refs. [35, 36]. Then
we will describe how these measures can help in inference
of causal relations or directionality of coupling.
Consider discrete random variables X and Y with sets