You want an example for solving least absolute deviation by linear programming. I will show you an simple implementation in R. Quantile regression is a generalization of least absolute deviation, which is the case of the quantile 0.5, so I will show a solution for quantile regression. Then you can check the results with the R quantreg package:
rq_LP <- function(x, Y, r=0.5, intercept=TRUE) {
require("lpSolve")
if (intercept) X <- cbind(1, x) else X <- cbind(x)
N <- length(Y)
n <- nrow(X)
stopifnot(n == N)
p <- ncol(X)
c <- c(rep(r, n), rep(1-r, n), rep(0, 2*p))
# cost coefficient vector
A <- cbind(diag(n), -diag(n), X, -X)
res <- lp("min", c, A, "=", Y, compute.sens=1)
### Desempaquetar los coefs:
sol <- res$solution
coef1 <- sol[(2*n+1):(2*n+2*p)]
coef <- numeric(length=p)
for (i in seq(along=coef)) {
coef[i] <- (if(coef1[i]<=0)-1 else +1) *
max(coef1[i], coef1[i+p])
}
return(coef)
}
Then we use it in a simple example:
library(robustbase)
data(starsCYG)
Y <- starsCYG[, 2]
x <- starsCYG[, 1]
rq_LP(x, Y)
[1] 8.1492045 -0.6931818
then you yourself can do the check with quantreg.
Linear Programming can be generalized with convex optimization, where in addition to simplex, many more reliable algorithms are available.
I would suggest you to check The Convex Optimization Book and the CVX toolbox they provided. Where you can easily formulate least absolute deviation with regularization.
https://web.stanford.edu/~boyd/cvxbook/bv_cvxbook.pdf
http://cvxr.com/cvx/
Videos
I have been making a program to analyse sets of data. I have implemented linear regression via least squares, but I was hoping to implement Least Absolute Deviation. For clarity, LAD wishes to minimise the sum of | y_i - a - b x_i |. To set up the problem as a set of linear constraints, a new variable, e_i, is used. Setting e_i = | y_i - a - b x_i |, we can form the inequalities e_i >= y_i - a - b x_i and -e_i <= y_i - a - b x_i . I have implemented simplex method in my program, so I wished to get it into a form usable by the simplex method. As such, I have converted each to [variables] <= value form, being -y_i = -a - b x_i - e_i + s_{2i} and y_i = a + b x_i - e_i + s_{2i +1}. The objective function is z = sum_{i=1}^{n} e_i. Since we wish to minimise this, I have negated everything to give -z + sum_{i=1}^{n} e_i = 0. Is this the correct setup for use of simplex? That is to ask: if I were to form a tableau from the following and apply simplex, would I get the desired a, b such that y = b x + a is the LAD line?
Any help would be appreciated.
EDIT: This formulation doesn't seem to function since the only selectable pivot column is the "z" column which is all zeros. How can this be avoided?
