Irreducible error
In any form of modelling, inherent irreducible error arises due to limitations in our knowledge, unobservable hidden variables, and measurement constraints. For a function
, irreducible error is typically represented as an random variable in the target space , with —expressing that, on average, the irreducible error lacks preference in . For example suppose we have some perfectly random sample that is either 0 or 1 with a half chance. No further data about the world will help us decide if it is either 0 or 1, it is all irreducible error.