Nonlinear regression
Part of a series on |
Regression analysis |
---|
Models |
Estimation |
|
Background |
|
In statistics, nonlinear regression is a form of regression analysis in which observational data are modeled by a function which is a nonlinear combination of the model parameters and depends on one or more independent variables. The data are fitted by a method of successive approximations (iterations).
General
In nonlinear regression, a statistical model of the form,
relates a vector of
This function, which is a rectangular hyperbola, is nonlinear because it cannot be expressed as a linear combination of the two s.
Other examples of nonlinear functions include exponential functions, logarithmic functions, trigonometric functions, power functions, Gaussian function, and Lorentz distributions. Some functions, such as the exponential or logarithmic functions, can be transformed so that they are linear. When so transformed, standard linear regression can be performed but must be applied with caution. See Linearization§Transformation, below, for more details.
In general, there is no closed-form expression for the best-fitting parameters, as there is in
For details concerning nonlinear data modeling see least squares and non-linear least squares.
Regression statistics
The assumption underlying this procedure is that the model can be approximated by a linear function, namely a first-order Taylor series:
where are Jacobian matrix elements. It follows from this that the least squares estimators are given by
compare generalized least squares with covariance matrix proportional to the unit matrix. The nonlinear regression statistics are computed and used as in linear regression statistics, but using J in place of X in the formulas.
When the function itself is not known analytically, but needs to be linearly approximated from , or more, known values (where is the number of estimators), the best estimator is obtained directly from the Linear Template Fit as [1]
The linear approximation introduces bias into the statistics. Therefore, more caution than usual is required in interpreting statistics derived from a nonlinear model.
Ordinary and weighted least squares
The best-fit curve is often assumed to be that which minimizes the sum of squared
Linearization
Transformation
Some nonlinear regression problems can be moved to a linear domain by a suitable transformation of the model formulation.
For example, consider the nonlinear regression problem
with parameters a and b and with multiplicative error term U. If we take the logarithm of both sides, this becomes
where u = ln(U), suggesting estimation of the unknown parameters by a linear regression of ln(y) on x, a computation that does not require iterative optimization. However, use of a nonlinear transformation requires caution. The influences of the data values will change, as will the error structure of the model and the interpretation of any inferential results. These may not be desired effects. On the other hand, depending on what the largest source of error is, a nonlinear transformation may distribute the errors in a Gaussian fashion, so the choice to perform a nonlinear transformation must be informed by modeling considerations.
For Michaelis–Menten kinetics, the linear Lineweaver–Burk plot
of 1/v against 1/[S] has been much used. However, since it is very sensitive to data error and is strongly biased toward fitting the data in a particular range of the independent variable, [S], its use is strongly discouraged.
For error distributions that belong to the exponential family, a link function may be used to transform the parameters under the Generalized linear model framework.
Segmentation
The
The figure shows that the soil salinity (X) initially exerts no influence on the crop yield (Y) of mustard, until a critical or threshold value (breakpoint), after which the yield is affected negatively.[4]
See also
- Non-linear least squares
- Curve fitting
- Generalized linear model
- Local regression
- Response modeling methodology
- Genetic Programming
- Multi expression programming
- Linear or quadratic template fit
References
- .
- PMID 3315805.
- ISBN 90-70754-33-9 . Download as PDF : [1]
- ^ R.J.Oosterbaan, 2002. Drainage research in farmers' fields: analysis of data. Part of project “Liquid Gold” of the International Institute for Land Reclamation and Improvement (ILRI), Wageningen, The Netherlands. Download as PDF : [2]. The figure was made with the SegReg program, which can be downloaded freely from [3]
Notes
- ^ This model can also be expressed in the conventional biological notation:
Further reading
- Bethea, R. M.; Duran, B. S.; Boullion, T. L. (1985). Statistical Methods for Engineers and Scientists. New York: Marcel Dekker. ISBN 0-8247-7227-X.
- Meade, N.; Islam, T. (1995). "Prediction Intervals for Growth Curve Forecasts". Journal of Forecasting. 14 (5): 413–430. .
- Schittkowski, K. (2002). Data Fitting in Dynamical Systems. Boston: Kluwer. ISBN 1402010796.
- Seber, G. A. F.; Wild, C. J. (1989). Nonlinear Regression. New York: John Wiley and Sons. ISBN 0471617601.