Many classification and regression problems are solved in practice by regularized empirical risk minimizers (RERM). The risk is measured
via a loss function. The quadratic loss function is the most popular function for
regression. It has been extensively studied (cf. [23, 31] among others). Still many other loss functions are popular among practitioners and are indeed extremely useful in specific situations
ALQUIER, P., COTTET, V. et LECUE, G. (2019). Estimation bounds and sharp oracle inequalities of regularized procedures with Lipschitz loss functions. Annals of Statistics, 47(4), pp. 2117-2144.