Logo

statsmodels.regression.linear_model.WLS.fit_regularized

WLS.fit_regularized(method='coord_descent', maxiter=1000, alpha=0.0, L1_wt=1.0, start_params=None, cnvrg_tol=1e-08, zero_tol=1e-08, **kwargs)

Return a regularized fit to a linear regression model.

Parameters:

method : string

Only the coordinate descent algorithm is implemented.

maxiter : integer

The maximum number of iteration cycles (an iteration cycle involves running coordinate descent on all variables).

alpha : scalar or array-like

The penalty weight. If a scalar, the same penalty weight applies to all variables in the model. If a vector, it must have the same length as params, and contains a penalty weight for each coefficient.

L1_wt : scalar

The fraction of the penalty given to the L1 penalty term. Must be between 0 and 1 (inclusive). If 0, the fit is ridge regression. If 1, the fit is the lasso.

start_params : array-like

Starting values for params.

cnvrg_tol : scalar

If params changes by less than this amount (in sup-norm) in once iteration cycle, the algorithm terminates with convergence.

zero_tol : scalar

Any estimated coefficient smaller than this value is replaced with zero.

Returns:

A RegressionResults object, of the same type returned by :

``fit``. :

Notes

The approach closely follows that implemented in the glmnet package in R. The penalty is the “elastic net” penalty, which is a convex combination of L1 and L2 penalties.

The function that is minimized is: ..math:

0.5*RSS/n + alpha*((1-L1_wt)*|params|_2^2/2 + L1_wt*|params|_1)

where RSS is the usual regression sum of squares, n is the sample size, and

System Message: WARNING/2 (|*|_1)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.15 (TeX Live 2014) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2011/06/27> Babel <3.9k> and hyphenation patterns for 2 languages loaded. (/usr/share/texlive/texmf-dist/tex/latex/base/article.cls Document Class: article 2007/10/19 v1.4h Standard LaTeX document class (/usr/share/texlive/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texlive/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def’ not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.131 \endinput ^^M No pages of output. Transcript written on math.log.
and

System Message: WARNING/2 (|*|_2)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.15 (TeX Live 2014) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2011/06/27> Babel <3.9k> and hyphenation patterns for 2 languages loaded. (/usr/share/texlive/texmf-dist/tex/latex/base/article.cls Document Class: article 2007/10/19 v1.4h Standard LaTeX document class (/usr/share/texlive/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texlive/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def’ not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.131 \endinput ^^M No pages of output. Transcript written on math.log.
are the L1 and L2 norms.

Post-estimation results are based on the same data used to select variables, hence may be subject to overfitting biases.

References

Friedman, Hastie, Tibshirani (2008). Regularization paths for generalized linear models via coordinate descent. Journal of Statistical Software 33(1), 1-22 Feb 2010.

Previous topic

statsmodels.regression.linear_model.WLS.fit

Next topic

statsmodels.regression.linear_model.WLS.from_formula

This Page