Bellman equation

The Bellman equation is used to determine the optimum value function for a given Markov decision process. It defines this value function recursively as follows: