In probability theory, a Markov kernel (also known as a stochastic kernel or probability kernel) is a map that in the general theory of Markov processes plays the role that the transition matrix does in the theory of Markov processes with a finite state space.

Formal definition

be measurable spaces. A Markov kernel with the following properties:

1. -measurable

2. is a probability measure

a probability measure is measurable with respect to the \sigma\mathcal A.

Examples

Simple random walkSimple random walk on the integers

(the power set:

1. .

is defined by

is the Kronecker delta for the random walk are equivalent to the Markov kernel.

General Markov processMarkov processes with countable state space

1. ,

define a (countable) stochastic matrix i.e.

We then define

1. .

Again the transition probability, the stochastic matrix and the Markov kernel are equivalent reformulations.

Markov kernel defined by a kernel function and a measure

be a measure a measurable function with respect to the \sigma-algebra such that

1. ,

i.e. the mapping

defines a Markov kernel. was the counting measure. Moreover it encompasses other important examples such as the convolution kernels, in particular the Markov kernels defined by the heat equation. The latter example includes the Gaussian kernel standard Lebesgue measure and

Measurable functions

i.e.

1. .

is measurable.

This example allows us to think of a Markov kernel as a generalised function with a (in general) random rather than certain value. That is, it is a multivalued function where the values are not equally weighted.

Galton–Watson processGalton–Watson process

with the standard sigma algebra of Borel sets. Then

are i.i.d. random variables is the indicator function. For the simple case of coin flips this models the different levels of a Galton board.

Composition of Markov Kernels

by the Chapman-Kolmogorov equation

1. .

The composition is associative by the Monotone Convergence Theorem and the identity function) is the unit for this composition.

This composition defines the structure of a category on the measurable spaces with Markov kernels as morphisms, first defined by Lawvere, the category of Markov kernels.

Probability Space defined by Probability Distribution and a Markov Kernel

A composition of a probability space, where the probability measure is given by

Properties

Semidirect product

, such that:

Regular conditional distribution

be a Borel space is a version of the conditional expectation, i.e.

and is not uniquely defined.

Generalizations

Transition kernels, the map

can be any type of (non negative) measure, not necessarily a probability measure.

External links

• Markov kernel in nLab.

References

1. §36. Kernels and semigroups of kernels

See also

• Category of Markov kernels