>[!tldr] Markov chain >A Markov chain is specified by a number of states $N$ and a transition [[Stochastic matrix|probability matrix]] $P \in M_{N \times N}(\mathbb{R})$. Intuitively think of this as a state machine which at state $i$ has probability $p_{i,j}$ of transitioning to state $j$.