This article is a brief overview of conditional independence in graphical models, and the related d-separation. Let us begin with a definition.
For three random variables , and , we say is conditionally independent of given iff
We can use a shorthand notation
Before we can define d-separation, let us first show three different types of graphs. Consider the same three variables as before, we’ll be interested in conditional independence based on whether we observe .
The first case is called the tail-tail.
We can factor the joint distribution to get
and conditioning on the value of we get (using the Bayes’ theorem)
From this we can immediately see that conditioning on in the tail-tail case makes and independent, that is .
The second case is called the head-tail and looks as the following.
We can again write the joint distribution for the graph
and again conditioning on we get (using rules of conditional probability)
and so again, and are conditionally independent given , that is .
Checking marginal independence
For completeness, we can also check if and are marginally independent, which they shouldn’t be, since we just showed they’re conditionally independent.
which gives us the following when marginalizing over
from which we can immediately see it does not factorize into in the general case, and thus and are not marginally independent.
The last case is called the head-head and is a little bit tricky
We can again write out the joint distribution
but this does not immediately help us when we try to condition on , we would want
which does not hold in general. For example, consider and if , and otherwise. In this case if we know and observe , it immediately tells us the value of , hence and are not conditionally independent given .
We can however do a little trick and write the as a marginalization over , that is
since . As a result, in the head-head case we have marginal independence between and , that is .
Having shown the three cases, we can finally define d-separation. Let be a DAG, and let be disjoint subsets of vertices.
A path between two vertices is blocked if it passes through a vertex , such that either:
- the edges are head-tail or tail-tail, and , or
- the edges are head-head, and , and neither are any of its descendants.
We say that and are d-separated by if all paths from a vertex of to a vertex of are blocked w.r.t. . And now comes the important part, if and are d-separated by , then .
Thig might all look very complicated, but this property of directed graphical models is actually extremely useful, and very easy to do quickly after seeing just a few examples.
To get a feel for d-separation, let us look at the following example ( is observed).
We can immediately see that since this is the head-tail case. We can also see that (not conditionally independent), because while the path through is blocked, the path through is not.
Share on Twitter and Facebook
Discussion of "Graphical Models: D-Separation"
If you have any questions, feedback, or suggestions, please do share them in the comments! I'll try to answer each and every one. If something in the article wasn't clear don't be afraid to mention it. The goal of these articles is to be as informative as possible.
If you'd prefer to reach out to me via email, my address is loading ..