GRAPHICAL MODELLING OF MULTIVARIATE TIME SERIES Michael Eichler Department of Quantitative Economics, University of Maastricht 6 P.O. Box 616, 6200 MD Maastricht, The Netherlands 0 0 February 2, 2008 2 t c Abstract. Weintroducegraphicaltimeseriesmodelsfortheanalysisofdynamic O relationshipsamongvariablesinmultivariatetimeseries. Themodellingapproach 2 isbasedonthenotionofstrongGrangercausalityandcanbeappliedtotimeseries 2 with non-linear dependencies. The models are derived from ordinary time series modelsbyimposingconstraintsthatareencodedbymixedgraphs. Inthesegraphs ] T eachcomponentseriesisrepresentedbyasinglevertexanddirectededgesindicate S possibleGranger-causalrelationshipsbetweenvariableswhileundirectededgesare . used to map the contemporaneous dependence structure. We introduce various h notions of Granger-causalMarkovproperties and discuss the relationships among t a them and to other Markov properties that can be applied in this context. m [ Keywords: Graphical models, multivariate time series, Granger causality, global Markov property 1 v 4 5 6 1. Introduction 0 1 6 Graphical models have become an important tool for the statistical analysis of 0 complex multivariate data sets, which are now increasingly available in many scien- / h tific fields. The key feature of these models is to merge the probabilistic concept of t a conditional independence with graph theory by representing possible dependencies m among the variables of a multivariate distribution in a graph. This has led to simple : graphical criteria for identifying the conditional independence relations that are im- v i plied byamodelassociated withagiven graph. Further important advantagesofthe X graphical modelling approach are statistical efficiency due to parsimonious parame- r a terizations of the joint distribution of the variables and the visualization of complex dependence structures, which allows an intuitive understanding of the interrelations among the variables and, thus, facilitates the communication of statistical results. For an introduction to graphical models we refer to the monographs by Whittaker (1990), Edwards (2000), and Cox and Wermuth (1996); a mathematically more rig- orous treatment can be found in Lauritzen (1996). While graphical models originally have been developed for variables that are sam- pled with independent replications, they have been applied more recently also to the analysisoftimedependent data. Somefirstgeneralremarksconcerning thepotential The paper was written while the author was working at the Institut fu¨r Angewandte Mathe- matik, Universit¨at Heidelberg, Germany. E-mail address: [email protected] (M. Eichler). 1 2 MICHAEL EICHLER useofgraphicalmodelsintimeseries analysiscanbefoundinBrillinger(1996); since then there has been an increasing interest in the use of graphical modelling tech- niques for analyzing multivariate time series (e.g., Stanghellini and Whittaker 1999, Dahlhaus2000,Reale and Tunnicliffe Wilson2001,Dahlhaus and Eichler2003,Oxley et al. 2004, Moneta and Spirtes 2005, Eichler 2006a,b). However, all these works have been restricted to the analysis of linear interdependencies among the variables whereas the recent trend in time series analysis has shifted towards non-linear para- metric and non-parametric models (e.g., Tong 1993, Rothman 1999, Fan and Yao 2003). Moreover, in most of these approaches, the variables at different time points are represented by separate nodes, which leads to graphs with theoretically infinitely many vertices for which no rigorous theory exists so far. In this paper, we present a general approach for graphical modelling of multi- variate stationary time series, which is based on simple graphical representations of the dynamic dependencies of a process. To this end, we utilize the concept of strong Granger causality (e.g., Florens and Mouchart 1982), which is formulated in terms of conditional independencies and, thus, can be applied to model arbitrary non-linear relationships among the variables. The concept of Granger causality orig- inally has been introduced by Granger (1969) and is commonly used for studying dynamic relationships among the variables in multivariate time series. For the graphical representations, we consider mixed graphs in which each vari- able as a complete time series is represented by a single vertex and directed edges indicate possible Granger-causal relationships among the variables while undirected edges are used to map the contemporaneous dependence structure. We note that similar graphs have been used in Eichler (2006a) as path diagrams for the autore- gressive structure of weakly stationary processes. Formally, the graphical encoding of the dynamic structure of a time series is achieved by a new type of Markov properties, which we call Granger-causal Markov properties. We introduce various levels, namely the pairwise, the local, the block-recursive, and the global Granger- causalMarkovproperty, anddiscuss therelationshipsamongthem. Inparticular,we give sufficient conditions under which the various Granger-causal Markov properties are equivalent; such conditions allow formulating models based on a simple Markov property while interpreting the associated graph by use of the global Granger-causal Markov property. The paper is organized as follows. In Section 2, we introduce the concepts of Granger-causal Markov properties and graphical time series models; some examples of graphical time series models are presented in Section 3. In Section 4, we discuss global Markov properties, which relate certain separation properties of the graph to conditional independence or Granger noncausality relations among the variables of the process. Finally in Section 5, we compare the presented graphical modelling approach with other approaches in the literature and discuss possible extensions. The proofs are technical and put into the appendix. 2. Graphical time series models In graphical modelling, the focus is on multivariate statistical models for which the possible dependencies between the studied variables can be represented by a graph. In multivariate time series analysis, statistical models for a time series X = V GRAPHICAL MODELLING OF MULTIVARIATE TIME SERIES 3 X (t) are usually specified in terms of the conditional distribution of X (t+1) V t∈ V g(cid:0)iven it(cid:1)s pZast X (t) = X (s) in order to study the dynamic relationships over V V s≤t time among the series.(cid:0)Thus,(cid:1)a time series model may be described formally as a family of probability kernels P from V× to V, and we write X ∼ P if P is a N V R R version of the conditional probability of X (t+1) given X (t). V V For modelling specific dependence structures, we utilize the concept of Granger (non-)causality, which has been introduced by Granger (1969) and has proved to be particularly useful for studying dynamic relationships in multivariate time series. This probabilisticconcept ofnoncausality fromaprocess X toanother processX is a b based on studying whether at time t the next value of X can be better predicted by b using the entire information up to time t than by using the same information apart from the former series X . In practice, not all relevant variables may be available a and, thus, the notion of Granger causality clearly depends on the used informa- tion set. In the sequel, we use the concept of strong Granger noncausality (e.g., Florens and Mouchart 1982), which is defined in terms of conditional independence and σ-algebras and, thus, can be used also for non-linear time series models. LetX = X (t) withX (t) = (X (t))′ ∈ V beamultivariatestationary V V t∈ V v v∈V R stochastic pr(cid:0)ocess o(cid:1)n Za probability space (Ω,F, ). For A ⊆ V, we denote by P X = (X (t)) the multivariate subprocess with components X , a ∈ A. The A A t∈ a Z information provided by the past and present values of X at time t ∈ can A be represented by the sub-σ-algebra X (t) of F that is generated by X Z(t) = A A X (s) . We write X = (X (t),t ∈ ) for the filtration induced by X . This A s≤t A A Z A l(cid:0)eads to(cid:1) the following definition of (strong) Granger noncausality in multivariate time series. Definition 2.1. Let A and B be disjoint subsets of V. (i) X is strongly Granger-noncausal for X with respect to the filtration X if A B V X (t+1)⊥⊥X (t)|X (t) B A V\A for all t ∈ . This will be denoted by X 9 X [X ]. A B V Z (ii) X and X are contemporaneously conditionally independent with respect to A B the filtration X if V X (t+1)⊥⊥X (t+1)|X (t)∨X (t+1) A B V V\(A∪B) for all t ∈ . This will be denoted by X ≁ X [X ]. A B V Z In the following, we will speak simply of Granger (non-)causality in the sense of the above definition. Intuitively, the dynamic relationships of a stationary multivariate time series X V can be visualized by a mixed graph G = (V,E) in which each vertex v ∈ V rep- resents one component X and two vertices a and b are joined by a directed edge v a b whenever X is Granger-causal for X or by an undirected edge a b a b (cid:3) (cid:0) whenever X and X are contemporaneously conditionally dependent. Conversely, a b for formulating models with specific dynamic dependencies, a mixed graph G can be associated with a set of Granger noncausality and contemporaneous conditional independence constraints that are imposed on a time series model for X . Such a V set of conditional independence relations encoded by a graph G is generally known 4 MICHAEL EICHLER 2 4 2 4 2 4 1 3 5 1 3 5 1 3 5 Figure 2.1. EncodingofrelationsXA 9XB [XX]bythe(a)pairwise,(b)local, and (c) block-recursive Granger-causal Markov property (A and B are indicated by grey and black nodes, respectively). as Markov property with respect to G. In the context of multivariate time series, graphs may encode different types of conditional independence relations, and we therefore speak of Granger-causal Markov properties when dealing with Granger noncausality and contemporaneous conditional independence relations. In the fol- lowing definition, pa(a) = {v ∈ V|v a ∈ E} denotes the set of parents of a vertex (cid:3) a, while ne(a) = {v ∈ V|v a ∈ E} is the set of neighbours of a; furthermore, for (cid:0) A ⊆ V, we define pa(A) = ∪ pa(a)\A and ne(A) = ∪ ne(a)\A . a∈A a∈A Definition 2.2 (Granger-causal Markov properties). Let G = (V,E) be a mixed graph. Then the stochastic process X satisfies V (PC) the pairwise Granger-causal Markov property with respect to G if for all a,b ∈ V with a 6= b (i) a b ∈/ E ⇒ X 9 X [X ], a b V (cid:3) (ii) a b ∈/ E ⇒ X ≁ X [X ]; a b V (cid:0) (LC) the local Granger-causal Markov property with respect to G if for all a ∈ V (i) X 9 X [X ], V\(pa(a)∪{a}) a V (ii) X ≁ X [X ]; V\(ne(a)∪{a}) a V (BC) the block-recursive Granger-causal Markov property with respect to G if for all subsets A of V (i) X 9 X [X ], V\(pa(A)∪A) A V (ii) X ≁ X [X ]. V\(ne(A)∪A) A V Similarly, if P is a probability kernel from V× to V, we say that P satisfies N R R the pairwise, the local, or the block-recursive Granger-causal Markov property with respect to a graph G whenever the same is true for every stationary process X V with X ∼ P. V Example 2.3. To illustrate the various Granger-causal Markov properties, we con- sider the graph G in Figure 2.1. Suppose that a stationary process X satisfies the V pairwise Granger-causal Markov property with respect to this graph G. Then the absence of the edge 1 4 in G implies that X is Granger-noncausal for X with 1 4 respect to X . Next, (cid:3)in the case of the local Granger-causal Markov property, we V find that the bivariate subprocess X is Granger-noncausal for X with respect {1,2} 4 to X since vertex 4 has parents 3 and 5. Similarly, if X obeys the block-recursive V V Granger-causalMarkovproperty,thegraphencodesthatX isGranger-noncausal {1,2} for X with respect to X since pa(4,5) = {3}. {4,5} V The block-recursive Granger-causal Markov property obviously implies the other twoGranger-causalMarkovpropertiesand,thus,isthestrongestofthethreeMarkov GRAPHICAL MODELLING OF MULTIVARIATE TIME SERIES 5 properties; similarly, the pairwise Granger-causal Markov property clearly is the weakest of the three properties. The question arises whether and under which con- ditions the three Granger-causal Markov properties are equivalent. In the case of random vectors (Y ) with values in V, the various levels of Markov properties v v∈V R for graphical interaction models are equivalent if the distribution of Y satisfies V Y ⊥⊥Y |Y ∧Y ⊥⊥Y |Y ⇒ Y ⊥⊥Y |Y A B C∪D A C B∪D A B∪C D for all disjoints subsets A, B, C, and D of V (Pearl and Paz 1987). A sufficient con- dition for this intersection property is that the joint distribution of Y is absolutely V continuous with respect to some product measure and has a positive and continuous density (e.g., Lauritzen 1996, Prop. 3.1). The following result establishes similar conditions for the time series case. Proposition 2.4. Suppose that the following two conditions hold: (M) X = X (t) is a stationary, strongly mixing stochastic process on some V V t∈ probabi(cid:0)lity spa(cid:1)ceZ(Ω,F, ) taking values in V; P R (P) the conditional distribution XV(t+1)|XV(t), t ∈ , has a regular version that P Z is almost surely absolutely continuous with respect to some product measure ν on |V| with ν-a.e. positive and continuous density. R Then, for every F-measurable random variable Y and every t ∈ , Z Y ⊥⊥X (t)|X (t)∧Y ⊥⊥X (t)|X (t) ⇔ Y ⊥⊥X (t)|X (t). A B∪C B A∪C A∪B C With this intersection property, we obtain the following relations among the three Granger-causal Markov properties. Theorem 2.5. Suppose that X satisfies conditions (M) and (P). Then the three V Granger-causal Markov properties (BC), (LC), and (PC) are related by the following implications: (BC) ⇒ (LC) ⇔ (PC). Furthermore, if X additionally satisfies V X 9 X [X ] ⇔ X 9 X [X ] ∀b ∈ B, (2.1) A B V A b V then the three Granger-causal Markov properties (BC), (LC), and (PC) are equiva- lent. The theorem shows that, similarly as in the case of chain graph models with the Andersson-Madigan-Perlman (AMP) Markov property (Andersson et al. 2001), the pairwise andthelocalGranger-causalMarkov propertyareingeneralnotsufficiently strong to encode all Granger-causal relationships that hold among the components of a multivariate time series with respect to full information X . This suggests to V specify graphical time series models in terms of the block-recursive Granger-causal Markov property. Definition 2.6 (Graphical time series model). Let G be a mixed graph and let P be a statistical time series model given by a family of probability kernels G P ∈ P from V× to V. Then P is said to be a graphical time series model G N G associated withRthe grapRh G if, for all P ∈ P , the distribution P satisfies the G block-recursive Granger-causal Markov property with respect to G. 6 MICHAEL EICHLER ThethreeGranger-causalMarkovpropertiesconsideredsofarencodeonlyGranger noncausality relations with respect to the complete information X . The discussion V of phenomena such as spurious causality (e.g., Hsiao 1982, Eichler 2005a), how- ever, requires also the consideration of Granger-causal relationships with respect to partial information sets, that is, with respect to filtrations X for subsets S of V. S To this end, we introduce in Section 4 a global Granger-causal Markov property that more generally relates pathways in a graph to Granger-causal relations among the variables, and we establish, under conditions (M) and (P), its equivalence to the block-recursive Granger-causalMarkov property; this shows that the block-recursive Granger-causal Markov property is indeed sufficiently rich to describe the dynamic dependence structure in multivariate time series. Before we continue our discussion of Markov properties in Section 4, we illustrate the introduced concept of graphical time series models by a few examples. 3. Examples In the previous section, graphical time series models have been defined in terms of the block-recursive Granger-causal Markov property. For many time series models, however, condition (2.1) in Proposition 2.5 holds, and, hence, the pairwise and the block-recursive Granger-causal Markov property are equivalent. This enables us to derive the constraints on the parameters from the pairwise Markov property. There are no simple conditions known that are both necessary and sufficient for (2.1). The following proposition lists some sufficient conditions that cover many examples, as will be shown subsequently. A counter-example that demonstrates that the pairwise and the block-recursive Granger-causal Markov property are in general not equivalent will be provided in Example 3.5. Proposition 3.1. Suppose that X satisfies conditions (M) and (P) and one of the V following conditions: (i) X is a Gaussian process; V (ii) X (t + 1), v ∈ V, are mutually contemporaneously independent, that is, the v joint conditional distribution factorizes as XV(t+1)|XV(t) = ⊗ Xv(t+1)|XV(t) ∀t ∈ ; v∈V P P Z (iii) X (t+1) depends on its past only in its conditional mean, that is, V X(t+1)− X(t+1)|X (t) ⊥⊥X (t) ∀t ∈ . V V E Z (cid:2) (cid:3) Then the three Granger-causal Markov properties (BC), (LC), and (PC) are equiv- alent. Example 3.2 (Vector autoregressive processes). LetX beastationaryvector V autoregressive process of order p, p iid X (t) = Φ(u)X (t−u)+ε(t), ε(t) ∼ N(0,Σ), (3.1) V V u=1 P where Φ(u) are V × V matrices and the variance matrix Σ is non-singular. Then X is Granger-noncausal for X with respect to X if and only if the corresponding A B V entries Φ (u) vanish for all u = 1,...,p (e.g., Boudjellaba et al. 1992). Further- BA more, X and X are contemporaneously conditionally independent if and only if A B GRAPHICAL MODELLING OF MULTIVARIATE TIME SERIES 7 the corresponding error components ε (t) and ε (t) are conditionally independent A B given all remaining components ε (t). For Gaussian errors, conditional inde- V\(A∪B) pendencies are given by zeros in the concentration matrix K = Σ−1 (e.g., Lauritzen 1996). Consequently, the process X satisfies the pairwise—and by Proposition V 3.1(i) also the block-recursive—Granger-causal Markov property with respect to a graph G = (V,E) if the following two conditions hold: (i) a b ∈/ E ⇒ Φ (u) = 0 ∀u = 1,...,p; ba (cid:3) (ii) a b ∈/ E ⇒ K = K = 0. ab ba (cid:0) Thus, the graphical vector autoregressive model of orderp associated with the graph G, denoted by VAR(p,G), is given by the set of all stationary VAR(p) processes whose parameters are constrained to zero according to the conditions (i) and (ii). Example 3.3 (Nonparametric additive models). More generally, we may also consider nonlinear autoregressive time series models. For example, let X be a V strongly mixing stationary process given by p (u) X (t) = m X (t−u) +ε (t), b ∈ V, t ∈ , b ba a b Z aP∈V uP=1 (cid:0) (cid:1) (u) wherem aremeasurablereal-valuedfunctionsandtheerrorsε (t)areindependent ba V and identically distributed. Furthermore, we assume that the distribution of the errors ε (t) has a positive and continuous density f on V. Then a nonparametric V V additive model of order p is given by the set P = {PR } of regular conditional m,f distributions P of such processes X . For a mixed graph G = (V,E), we now m,f V define P as the subset of all P ∈ P such that G m,f (u) (i) m (·), u = 1,...,p, are constant whenever a b ∈/ E and ba (cid:3) (ii) f factorizes as f (z ) = g(z )h(z ) whenever a b ∈/ E. V V V V\{b} V\{a} (cid:0) The second condition implies ε (t)⊥⊥ε (t)|ε (t) and ensures that the distri- a b V\{a,b} bution of the errors ε(t) obeys the so-called pairwise Markov property with respect to the undirected subgraph Gu of G (i.e., the subgraph Gu obtained from G by removing all directed edges). Consequently, X and X are contemporaneously con- a b ditionally independent with respect to X whenever the corresponding vertices a V andbarenotjoinedby anundirected edge inG. Similarly, the first conditionimplies that, for any process with X ∼ P , the conditional distribution of X (t) given V m,f b the past X (t−1) satisfies V Xb(t)|XV(t−1) = Xb(t)|XV\{a}(t−1) P P whenever the graph G does not contain the edge a b. It follows that any process X with X ∼ P ∈ P obeys the pairwise G(cid:3)ranger-causal Markov property V V m,f G with respect to G. Furthermore, since X (t) depends on its past X (t − 1) only V V in its conditional mean, it follows from Proposition 3.1(iii) that the pairwise and the block-recursive Granger-causal Markov properties are equivalent. Thus, P is G indeed the graphical nonparametric additive model of order p associated with the graph G. Example 3.4 (Binary time series). As an example with categorical data, we consider a binary time series model that has been used for the identification of neural interactions from neural spike train data (Brillinger 1988a,b). Suppose that 8 MICHAEL EICHLER the data consist of the recorded spike trains for a set of neurons, that is, of the sequences of firing times (τ ) for neurons v ∈ V, and let X be the binary time v,n n∈ v N series obtained by setting X (t) = 1 if neuron v has fired in the interval [t,t+1) and v X (t) = 0 otherwise. Then, the interactions between the neurons can be modelled v by the conditional probabilities X (t) = 1 X (t−1) = Φ U (t)−θ , (3.2) b V ba P(cid:0) (cid:12) (cid:1) (cid:16)a∈Ppa(b) (cid:17) (cid:12) where Φ(x) denotes the normal cumulative function, γb(t) U (t) = g (u)X (t−u) (3.3) ba ba a u=1 P measures the influence of process a on process b, and γ (t) = min u ∈ X (t−u) = 1 b b N (cid:8) (cid:12) (cid:9) is the time elapsed since the last event of pr(cid:12)ocess Xb. Furthermore, we assume that thetimeunithasbeenchosensmallenoughsuchthattherearenointeractionsamong the neurons within one time interval, and that, consequently, the joint conditional probability factorizes as X (t) = x X (t−1) = X (t) = x X (t−1) V V V v v V P P (cid:0) (cid:12) (cid:1) vQ∈V (cid:0) (cid:12) (cid:1) (cid:12) (cid:12) for all x ∈ {0,1}V. Then the pairwise and the block-recursive Granger-causal V Markov property are equivalent by Proposition 3.1(ii) and, thus, we can use the former for modelling dependencies between the processes. From (3.2) and (3.3), it follows that X is Granger-noncausal for X if and only if g (u) = 0 for all u ∈ . a b ba N Example 3.5. In general, the pairwise and the block-recursive Granger-causal Markov property are not equivalent. To demonstrate this, we consider a simple stationary Markov process X with conditional distributions V X (t)|X (t−1) ∼ N 0,Σ(t) V V (cid:0) (cid:1) and conditional covariance matrix 1 ρ(t) 0 Σ(t) = ρ(t) 1 0. 0 0 1   We assume that the conditional correlation between X (t) and X (t) given X (t−1) 1 2 V depends on X (t−1) by 3 ρ if |X (t−1)| > c ρ(t) = 3 (cid:26) 0 otherwise for some constants ρ with 0 < |ρ| < 1 and c > 0. In other words, the variables X 1 and X start becoming contemporaneously dependent once the most recent value of 2 variable X exceeds a certain threshold. For this model, we find that, on the one 3 hand, the marginal conditional distributions of X (t) given X (t−1) are standard v V normal and, thus, do not depend on X (t−1). This implies that the process X V V satisfies the pairwise Granger-causal Markov property with respect to the graph (a) in Figure 3.1. On the other hand, the bivariate conditional distribution of GRAPHICAL MODELLING OF MULTIVARIATE TIME SERIES 9 3 3 (a) (b) 1 2 1 2 Figure 3.1. Illustration of non-equivalence of pairwise and block-recursive Granger-causal Markov properties: the process in Example 3.5 satisfies the pair- wise Granger-causal Markov property with respect to the graphs in (a) and (b) whereasitsatisfiesthe block-recursiveGranger-causalMarkovpropertyonlywith respect to the graph in (b). X (t),X (t) depends on the value of X (t−1) by the conditional correlation ρ(t). 1 2 3 (cid:0)Hence, XV ob(cid:1)eys the block-recursive Granger-causal Markov property with respect to the graph (b) in Figure 3.1, but not with respect to the graph (a). 4. Global Markov properties The interpretation of graphs describing the dependence structure of graphical models in general is enhanced by global Markov properties that merge the notion of conditional independence with a purely graph theoretical concept of separation allowing one to state whether two subsets of vertices are separated by a third sub- set of vertices. In this section, we show that the concept of p-separation intro- duced by Levitz et al. (2001) for chain graph models with the AMP Markov prop- erty (Andersson et al. 2001) can be used to obtain global Markov properties in the present context of graphical time series models. Throughout this section we assume that conditions (M) and (P) in Proposition 2.4 hold. 4.1. The global AMP Markov property Westartwithsomefurthergraphicalterminology. LetG = (V,E) beamixedgraph. Then a path π between two vertices a and b in G is a sequence π = he ,...,e i of 1 n edges e ∈ E such that e is anedge between v andv forsome sequence of vertices i i i−1 i v = a,v ,...,v = b. The vertices a and b are the endpoints of the path, while 0 1 n v ,...,v are the intermediate vertices on the path. Notice that paths may be 1 n−1 self-intersecting since we do not require that the vertices v are distinct. j An intermediate vertex c on a path π is said to be a p-collider on the path if the edges preceding and suceeding c on the path either have both an arrowhead at c or one has an arrowhead at c and the other is a line, i.e. c , c , c ; (cid:3) (cid:2) (cid:3) (cid:0) (cid:0) (cid:2) otherwise the vertex c is said to be a p-noncollider on the path. A path π between vertices a and b is said to be p-connecting given a set S if (i) every p-noncollider on the path is not in S, and (ii) every p-collider on the path is in S, otherwise we say the path is p-blocked given S. Definition 4.1 (p-separation). Two vertices a and b in a mixed graph G are p- separated given a set S if all paths between a and b are p-blocked given C. Similarly, two sets A and B in G are said to be p-separated given S if, for every pair a ∈ A and b ∈ B, a and b are p-separated given S. This will be denoted by A ⋊⋉ B|S. p 10 MICHAEL EICHLER (a) 2 4 (b) 2 4 (c) 2 4 1 3 5 1 3 5 1 3 5 Figure 4.1. Illustration of global AMP Markov property: (a) path between 1 and 4 that is p-connecting given S ⊆ {2,5}; (b) path between 1 and 4 that is p-connecting given S = {2,3} (or {2,3,5}); (c) path between 1 and 4 that is p-connecting given S ={3,5} (or {3}). We note that the above conditions for p-separation are simpler than those in Levitz et al. (2001) due to the fact that we consider the larger class of all possibly self-intersecting paths. The equivalence of the two notions of p-separation is shown in Appendix D. The following results show that the concept of p-separation can be applied to graphs encoding dynamic relationships in multivariate time series and allows reading off conditional independencies among the stochastic processes that are represented by the vertices in the graph. Lemma 4.2. Suppose that X satisfies the block-recursive Granger-causal Markov V property with respect to the graph G. Then, for any disjoint subsets A, B, and S of V, we have A ⋊⋉ B|S ⇒ X (t)⊥⊥X (t)|X (t) ∀t ∈ . p A B S Z Letting t tend to infinity, we can translate p-separation in the graph into con- ditional independence statements for complete subprocesses. For this, we define X (∞) = ∨ X (t) as the σ-algebra generated by the subprocess X . S t∈ S S Z Theorem 4.3. Suppose X satisfies the block-recursive Granger-causal Markov V property with respect to the graph G. Then, for any disjoint subsets A, B, and S of V, we have A ⋊⋉ B|S ⇒ X (∞)⊥⊥X (∞)|X (∞). p A B S We say that X satisfies the global AMP Markov property (GA) with respect to G. Example 4.4. For an illustration of the global AMP Markov property, we consider again the graph G in Figure 2.1. In this graph, vertices 1 and 4 are not adjacent. Nevertheless, it can be shown that the two vertices cannot be p-separated by any set S ⊆ {2,3,5}: firstly, the path 1 3 4 is p-connecting given a set S unless (cid:2) (cid:3) the set S contains the vertex 3 (Fig. 4.1 a). Secondly, the path 1 3 2 4 (cid:3) (cid:0) (cid:2) is p-connecting given S whenever both intermediate vertices 2 and 3 belong to S (Fig. 4.1 b). Finally, the path 1 3 2 4 is p-connecting given S if S (cid:3) (cid:2) (cid:2) contains vertex 3 but not 2 (Fig. 4.1 c). Thus, if X is a stationary process that V obeys the block-recursive Granger-causal Markov property with respect to G, then the graph G does not encode that X and X are conditionally independent given 1 4 X regardless of the choice of S ⊆ {2,3,5}. S Similarly, it can be shown that vertices 1 and 5 are p-separated given S = {3,4}: every path between 1 and 5 that contains the edge 3 5 or the subpath 3 4 (cid:3) (cid:3) (cid:2) 5 is p-blocked by vertex 3. All other paths between 1 and 5 contain the subpath 2 4 5 and, thus, are blocked by vertex 4. It follows that for every process (cid:2) (cid:2)