Visualizing expected value

Let \(X\) be a random variable taking values in (some subset of) \(\mathbb{R}\), write its law as \(P_X\), and its cumulative distribution function as \(F_X(x) = P_X\big((-\infty, x]\big) = P(X \le x)\).If we want to be really formal to define all the notation, \(X : \Omega \to \mathbb{R}\) is a measurable map on a probability space \((\Omega, \mathcal{F}, P)\), and its law is the pushforward \(P_X \defeq X_* P\), so that \(P_X(A) = P(X \in A)\) for every Borel set \(A\).

The expected value of a function \(g(X)\) is given by

\[ \E[g(X)] \defeq \int_{\mathbb{R}} g \dee{P_X} \]

Writing \(p_X\) for the probability density function (or probability mass function), this general definition simplifies to a sum or integral of \(g\) weighted by \(p_X\):In what follows, we’ll consider both discrete and continuous cases, and write \(p_X\) for either pdf or pmf. This same notation reflects the fact that either can be defined formally as \(p_X = \dee{P_X} / \dee{\mu}\), the density of the random variable’s law, \(P_X\), with respect to some reference measure \(\mu\) on \(\mathbb{R}\), which is the counting measure for the discrete case, or the Lebesgue measure for the continuous case, so either the integral or the sum is the same: \(\E[g(X)] = \int_{\mathbb{R}} g \, p_X \dee{\mu}\).

\[ \E[g(X)] = \begin{cases} \displaystyle\int_{-\infty}^{\infty} g(x)\,p_X(x) \dee{x} & \text{if } X \text{ is continuous,}\\[1.2ex] \displaystyle\sum_{x \in \supp{X}} g(x)\,p_X(x) & \text{if } X \text{ is discrete.} \end{cases} \]

We can equivalently write the expectation as an integral over the unit interval, using a substitution by \(F_X^{-1}(u) = \inf\{x : F_X(x) \ge u\}\), the inverse CDF (aka quantile function).If \(X\) is discrete, \(F_X\) is a staircase that jumps by \(p_X(x)\) at each outcome \(x\). The infimum simply picks the first \(x\) at which the staircase reaches \(u\), so \(F_X^{-1}(u) = x\) for every \(u\) in the interval \(\big(P(X < x),\, P(X \le x)\big]\). These intervals partition \([0, 1]\) into blocks, one per outcome, each as wide as its probability, in order of \(x\); \(F_X^{-1}\) is the staircase that takes the value \(x\) on the block for \(x\).

\[ \E[g(X)] = \int_0^1 g\big(F_X^{-1}(u)\big) \dee{u} \]

The fact that we’ve expressed the expectation as an integral over the unit interval of the function \(g\), suggests a nice visual interpretation of the expectation as an area. We use the map \(x \mapsto u = F_X(x)\) to rescale the real line into \([0, 1]\) so that each outcome takes up as much room as its probability. Then evaluate \(g\) over the rescaled axis, and the area under \(g \circ F_X^{-1}\) is \(\E[g(X)]\).

• If \(X\) is discrete, \(g \circ F_X^{-1}\) is a step function: one block of height \(g(x)\) and width \(p_X(x)\) for each outcome \(x\), side by side in order of \(x\), so the area is the sum.
• If \(X\) is continuous, the block widths shrink to infinitesimal \(p_X(x) \dee{x}\), and the area is the integral.

Below, play around with visualizing the expectation as an area under a curve. Set the density or mass function \(p_X\), and the function \(g\) to integrate, and scrub the \(x\) or \(u\) axis to see the area build up.