Boundary problem and overfitting reduction in convex regression

Abstract

Convex regression is a nonparametric approach for estimating a convex or concave function from observed data. It is widely used in operations research, economics, machine learning, and related fields. However, empirical evidence has shown that convex regression can yield excessively large subgradients on the boundary. In this paper, we provide theoretical evidence of this boundary problem. To address such a problem, we propose two new estimators by placing a bound on the subgradients of the convex function. We further prove that they converge to the underlying true convex function and that their subgradients converge to the gradient of the underlying function, both uniformly over the domain with probability one as the sample size increases to infinity. The proposed methods also help to reduce overfitting in finite samples: Monte Carlo simulations and empirical illustrations with large-scale datasets confirm the superior performance of the proposed estimators in predictive power over the existing methods.