In statistics and numerical analysis, isotonic regression or monotonic regression is the technique of fitting a free-form line to a sequence of observations such that the fitted line is non-decreasing (or non-increasing) everywhere, and lies as close to the observations as possible.

Applications

Isotonic regression has applications in statistical inference. For example, one might use it to fit an isotonic curve to the means of some set of experimental results when an increase in those means according to some particular ordering is expected. A benefit of isotonic regression is that it is not constrained by any functional form, such as the linearity imposed by linear regression, as long as the function is monotonic increasing.

Another application is nonmetric multidimensional scaling, where a low-dimensional embedding for data points is sought such that order of distances between points in the embedding matches order of dissimilarity between points. Isotonic regression is used iteratively to fit ideal distances to preserve relative dissimilarity order.

Isotonic regression is also used in probabilistic classification to calibrate the predicted probabilities of supervised machine learning models.

Estimation of the complete dose-response curve without any additional assumptions is usually done via linear interpolation between the point estimates.

Software for computing isotone (monotonic) regression has been developed for R, Stata, and Python.

Problem statement and algorithms

fall in some partially ordered set.

metric seeks a weighted least-squares. This gives the following quadratic program:

(and may be regarded as the set of edges of some directed acyclic graph). Problems of this form may be solved by generic quadratic programming techniques.

values fall in a totally ordered set, we may assume WLOG. In this case, a simple and efficient iterative algorithm metric) is the pool adjacent violators algorithm. Conversely, Best and Chakravarti studied the problem as an active set identification problem, and proposed a primal algorithm. These two algorithms can be seen as each other's dual, and both have a computational complexity on already sorted data.

for all i. Any such function obviously solves

1. being nondecreasing

, as illustrated in the figure, yielding a continuous piecewise linear function:

. fit has a least absolute deviation fit with its least squares criterion.

Another generalisation is to unimodal regressions, i.e. to fits with a non-decreasing segment followed by a non-increasing segment (or vice versa).

Centered isotonic regression

is not only monotone but also smooth's assumed shape, and can be shown to be biased. A simple improvement for such applications, named centered isotonic regression (CIR), was developed by Oron and Flournoy and shown to substantially reduce estimation error for both dose-response and dose-finding applications. Both CIR and the standard isotonic regression for the univariate, simply ordered case, are implemented in the R package "cir". This package also provides analytical confidence-interval estimates.

References

Further reading