Monday, October 23, 2017

probability theory - Expected value of visits in a state of a discrete Markov chain





Let




  • $X=(X_n)_{n\in\mathbb N_0}$ be a Markov chain with values in a at most countable Polish space $E$ and $\mathcal E$ be the Borel $\sigma$-algebra on $E$

  • $(\operatorname P_x)_{x\in E}$ be the distributions of $X$

  • $N(y)=\sum_{n\in\mathbb N_0}1_{\left\{X_n=y\right\}}$ be the number of visits of $X$ in $y\in E$



Clearly, $$\operatorname E_x[N(y)]=\sum_{n\in\mathbb N_0}\operatorname P_x[X_n=y]\;.$$




I've read that it holds $$\operatorname E_x[N(y)]=\sum_{k\in\mathbb N}\operatorname P_x[N(y)\ge k]\;,$$ but I don't understand why this is true. Is it a typo and what's really meant is "=" instead of "\ge"?


Answer



Here's a formula with uses in lots of places: If $N$ is a non-negative integer-valued random variable, then $E[N] =\sum_{k=1}^\infty P[N\ge k]$. To see this write
$$
\sum_{k=1}^\infty P[N\ge k]=\sum_{k=1}^\infty \sum_{j=k}^\infty P[N=j]=\sum_{j=1}^\infty \sum_{k=1}^j P[N=j]=\sum_{j=1}^\infty j\cdot P[N=j]=E[N]
$$


No comments:

Post a Comment

analysis - Injection, making bijection

I have injection $f \colon A \rightarrow B$ and I want to get bijection. Can I just resting codomain to $f(A)$? I know that every function i...