Package 'scoringfunctions'

Overview of the functions in the scoringfunctions package

Description

The scoringfunctions package implements consistent scoring (loss) functions and identification functions.

Details

The table below lists a selection of predictive functionals alongside their pointwise scoring functions (_sf) / realised average scores (_rs), and identification functions (_if). The complete listing of the package functions is given in the numbered sections below the table:

Target Functional	Loss Functions (_sf / _rs)	Identification (_if)
Mean	serr_sf / mse, bregman1_sf / bregman1_rs	mean_if
	bregman2_sf / bregman2_rs, bregman3_sf / qlike
	bregman4_sf / bregman4_rs
Expectile ($p$)	expectile_sf / expectile_rs	expectile_if
Median	aerr_sf / mae, maelog_sf / maelog_rs	quantile_if ($p=0.5$)
	maesd_sf / maesd_rs
$\beta$-Median	bmedian_sf / bmedian_rs	---
Quantile ($p$)	quantile_sf / quantile_rs, gpl1_sf / gpl1_rs	quantile_if
	gpl2_sf / gpl2_rs
Huber Mean	huber_sf / huber_rs	hubermean_if
Huber Quantile	ghuber_sf / ghuber_rs	huberquantile_if
$L_q$-Mean	lqmean_sf / lqmean_rs	---
$L_q$-Quantile	lqquantile_sf / lqquantile_rs	---
Interval ($p$)	interval_sf	---
Mean - Variance	mv_sf	mv_if
Error - Spread	errorspread_sf	---
Relative Error	relerr_sf / mre	---
Percentage Error	aperr_sf / mape	---

The package functions are categorised into six classes, each of which has its own section below:

1. Scoring functions

2. Realised (average) score functions

3. Skill score functions

4. Identification functions

5. Functions for sample levels

6. Supporting functions

1. Scoring functions

1.1. Consistent scoring functions for one-dimensional functionals

1.1.1. Consistent scoring functions for the mean

bregman1_sf: Bregman scoring function (type 1)

bregman2_sf: Bregman scoring function (type 2, Patton scoring function)

bregman3_sf: Bregman scoring function (type 3, QLIKE scoring function)

bregman4_sf: Bregman scoring function (type 4, Patton scoring function)

serr_sf: Squared error scoring function

1.1.2. Consistent scoring functions for expectiles

expectile_sf: Asymmetric piecewise quadratic scoring function (expectile scoring function, expectile loss function)

1.1.3. Consistent scoring functions for the median

aerr_sf: Absolute error scoring function

maelog_sf: MAE-LOG scoring function

maesd_sf: MAE-SD scoring function

1.1.4. Consistent scoring functions for quantiles

gpl1_sf: Generalized piecewise linear power scoring function (type 1)

gpl2_sf: Generalized piecewise linear power scoring function (type 2)

quantile_sf: Asymmetric piecewise linear scoring function (quantile scoring function, quantile loss function)

1.1.5. Consistent scoring functions for Huber functionals

ghuber_sf: Generalized Huber scoring function

huber_sf: Huber scoring function

1.1.6. Consistent scoring functions for other functionals

aperr_sf: Absolute percentage error scoring function

bmedian_sf: $\beta$-median scoring function

linex_sf: LINEX scoring function

lqmean_sf: $L_q$-mean scoring function

lqquantile_sf: $L_q$-quantile scoring function

nmoment_sf: $n$-th moment scoring function

obsweighted_sf: Observation-weighted scoring function

powerweighted_sf: Power-weighted squared error scoring function

relerr_sf: Relative error scoring function (MAE-PROP scoring function)

serrexp_sf: Squared error exp scoring function

serrlog_sf: Squared error log scoring function

serrpower_sf: Squared error of power transformations scoring function

serrsq_sf: Squared error of squares scoring function

sperr_sf: Squared percentage error scoring function

srelerr_sf: Squared relative error scoring function

1.2. Consistent scoring functions for two-dimensional functionals

interval_sf: Interval scoring function (Winkler scoring function)

mv_sf: Mean - variance scoring function

1.3. Consistent scoring functions for multi-dimensional functionals

errorspread_sf: Error - spread scoring function

2. Realised (average) score functions

2.1. Realised (average) score functions for one-dimensional functionals

2.1.1. Realised (average) score functions for the mean

bregman1_rs: Realised Bregman score (type 1)

bregman2_rs: Realised Bregman score (type 2, Patton score)

bregman4_rs: Realised Bregman score (type 4, Patton score)

mse: Mean squared error (MSE)

qlike: QLIKE

2.1.2. Realised (average) score functions for expectiles

expectile_rs: Realised expectile score

2.1.3. Realised (average) score functions for the median

mae: Mean absolute error (MAE)

maelog_rs: Realised MAE-LOG score

maesd_rs: Realised MAE-SD score

2.1.4. Realised (average) score functions for quantiles

gpl1_rs: Realised generalized piecewise linear power score (type 1)

gpl2_rs: Realised generalized piecewise linear power score (type 2)

quantile_rs: Realised quantile score

2.1.5. Realised (average) score functions for Huber functionals

ghuber_rs: Realised generalized Huber score

huber_rs: Realised Huber score

2.1.6. Realised (average) score functions for other functionals

bmedian_rs: Realised $\beta$-median score

linex_rs: Realised LINEX score

lqmean_rs: Realised $L_q$-mean score

lqquantile_rs: Realised $L_q$-quantile score

mape: Mean absolute percentage error (MAPE)

mre: Mean relative error (MRE)

mspe: Mean squared percentage error (MSPE)

msre: Mean squared relative error (MSRE)

nmoment_rs: Realised $n$-th moment score

obsweighted_rs: Realised observation-weighted score

serrexp_rs: Realised squared error exp score

serrlog_rs: Realised squared error log score

serrpower_rs: Realised squared error of power transformations score

serrsq_rs: Realised squared error of squares score

3. Skill score functions

3.1. Skill score functions for one-dimensional functionals

3.1.1. Skill score functions for the mean

nse: Nash-Sutcliffe efficiency (NSE)

4. Identification functions

4.1. Identification functions for one-dimensional functionals

expectile_if: Expectile identification function

hubermean_if: Huber mean identification function

huberquantile_if: Huber quantile identification function

mean_if: Mean identification function

meanexp_if: Exp-transformed identification function

meanlog_if: Log-transformed identification function

meanpower_if: Power-transformed identification function

nmoment_if: $n$-th moment identification function

powerweighted_if: Power-weighted identification function

quantile_if: Quantile identification function

4.2. Identification functions for two-dimensional functionals

mv_if: Mean - variance identification function

5. Functions for sample levels

quantile_level: Sample quantile level function

6. Supporting functions

capping_function: Capping function

References

Banerjee A, Guo X, Wang H (2005) On the optimality of conditional expectation as a Bregman predictor. IEEE Transactions on Information Theory 51(7):2664–2669. doi:10.1109/TIT.2005.850145.

Bellini F, Klar B, Muller A, Rosazza Gianin E (2014) Generalized quantiles as risk measures. Insurance: Mathematics and Economics 54:41–48. doi:10.1016/j.insmatheco.2013.10.015.

Brehmer JR, Gneiting T (2021) Scoring interval forecasts: Equal-tailed, shortest, and modal interval. Bernoulli 27(3):1993–2010. doi:10.3150/20-BEJ1298.

Chen Z (1996) Conditional $L_p$-quantiles and their application to the testing of symmetry in non-parametric regression. Statistics and Probability Letters 29(2):107–115. doi:10.1016/0167-7152(95)00163-8.

Christensen HM, Moroz IM, Palmer TN (2015) Evaluation of ensemble forecast uncertainty using a new proper score: Application to medium-range and seasonal forecasts. Quarterly Journal of the Royal Meteorological Society 141(687)(Part B):538–549. doi:10.1002/qj.2375.

Dimitriadis T, Fissler T, Ziegel JF (2024) Osband's principle for identification functions. Statistical Papers 65:1125–1132. doi:10.1007/s00362-023-01428-x.

Dunsmore IR (1968) A Bayesian approach to calibration. Journal of the Royal Statistical Society, Series B (Methodological) 30(2):396–405. doi:10.1111/j.2517-6161.1968.tb00740.x.

Ferguson TS (1967) Mathematical Statistics: A Decision-Theoretic Approach. Academic Press, New York.

Fissler T, Pesenti SM (2023) Sensitivity measures based on scoring functions. European Journal of Operational Research 307(3):1408–1423. doi:10.1016/j.ejor.2022.10.002.

Fissler T, Ziegel JF (2016) Higher order elicitability and Osband's principle. The Annals of Statistics 44(4):1680–1707. doi:10.1214/16-AOS1439.

Fissler T, Ziegel JF (2019) Order-sensitivity and equivariance of scoring functions. Electronic Journal of Statistics 13(1):1166–1211. doi:10.1214/19-EJS1552.

Gerber HU (1974) On additive premium calculation principles. ASTIN Bulletin: The Journal of the IAA 7(3):215–222. doi:10.1017/S0515036100006061.

Gneiting T (2011) Making and evaluating point forecasts. Journal of the American Statistical Association 106(494):746–762. doi:10.1198/jasa.2011.r10138.

Gneiting T, Raftery AE (2007) Strictly proper scoring rules, prediction, and estimation. Journal of the American Statistical Association 102(477):359–378. doi:10.1198/016214506000001437.

Houghton-Carr HA (1999) Assessment criteria for simple conceptual daily rainfall-runoff models. Hydrological Sciences Journal 44(2):237–261. doi:10.1080/02626669909492220.

Huber PJ (1964) Robust estimation of a location parameter. Annals of Mathematical Statistics 35(1):73–101. doi:10.1214/aoms/1177703732.

Koenker R, Bassett Jr G (1978) Regression quantiles. Econometrica 46(1):33–50. doi:10.2307/1913643.

Nash JE, Sutcliffe JV (1970) River flow forecasting through conceptual models Part I - A discussion of principles. Journal of Hydrology 10(3):282–290. doi:10.1016/0022-1694(70)90255-6.

Newey WK, Powell JL (1987) Asymmetric least squares estimation and testing. Econometrica 55(4):819–847. doi:10.2307/1911031.

Park H, Stefanski LA (1998) Relative-error prediction. Statistics and Probability Letters 40(3):227–236. doi:10.1016/S0167-7152(98)00088-1.

Patton AJ (2011) Volatility forecast comparison using imperfect volatility proxies. Journal of Econometrics 160(1):246–256. doi:10.1016/j.jeconom.2010.03.034.

Raiffa H, Schlaifer R (1961) Applied Statistical Decision Theory. Colonial Press, Clinton.

Saerens M (2000) Building cost functions minimizing to some summary statistics. IEEE Transactions on Neural Networks 11(6):1263–1271. doi:10.1109/72.883416.

Savage LJ (1971) Elicitation of personal probabilities and expectations. Journal of the American Statistical Association 66(336):783–801. doi:10.1080/01621459.1971.10482346.

Taggart RJ (2022) Point forecasting and forecast evaluation with generalized Huber loss. Electronic Journal of Statistics 16(1):201–231. doi:10.1214/21-EJS1957.

Thirel G, Santos L, Delaigue O, Perrin C (2024) On the use of streamflow transformations for hydrological model calibration. Hydrology and Earth System Sciences 28(21):4837–4860. doi:10.5194/hess-28-4837-2024.

Thomson W (1979) Eliciting production possibilities from a well-informed manager. Journal of Economic Theory 20(3):360–380. doi:10.1016/0022-0531(79)90042-5.

Tyralis H, Papacharalampous G (2026) Variable transformations in consistent loss functions. Knowledge-Based Systems 336:115202. doi:10.1016/j.knosys.2025.115202.

Varian HR (1975) A Bayesian approach to real estate assessment. In: Fienberg SE, Zellner A (eds) Studies in Bayesian Econometrics and Statistics in Honor of Leonard J. Savage. Amsterdam: North-Holland, pp 195–208.

Winkler RL (1972) A decision-theoretic approach to interval estimation. Journal of the American Statistical Association 67(337):187–191. doi:10.1080/01621459.1972.10481224.

Winkler RL, Murphy AH (1979) The use of probabilities in forecasts of maximum and minimum temperatures. Meteorological Magazine 108(1288):317–329.

Yeh C-C, Yeh H-W, Chan W (2008) Some equivalent forms of the arithematic-geometric mean inequality in probability: A survey. Journal of Inequalities and Applications 2008:386715. doi:10.1155/2008/386715.

Zellner A (1986) Bayesian estimation and prediction using asymmetric loss functions. Journal of the American Statistical Association 81(394):446–451. doi:10.1080/01621459.1986.10478289.

---

Absolute error scoring function

Description

The function aerr_sf computes the absolute error scoring function when $y$ materialises and $x$ is the predictive median functional.

The absolute error scoring function is defined in Table 1 in Gneiting (2011).

Usage

aerr_sf(x, y)

Arguments

x	Predictive median functional (prediction). It can be a vector of length $n$ (must have the same length as $y$).
y	Realisation (true value) of process. It can be a vector of length $n$ (must have the same length as $x$).

Details

The absolute error scoring function is defined by:

$S(x, y) := |x - y|$

Domain of function:

$x \in \mathbb{R}$

$y \in \mathbb{R}$

Range of function:

$S(x, y) \geq 0, \forall x, y \in \mathbb{R}$

Value

Vector of absolute errors.

Note

For details on the absolute error scoring function, see Gneiting (2011).

The median functional is the median of the probability distribution $F$ of $Y$ (Gneiting 2011).

The absolute error scoring function is negatively oriented (i.e. the smaller, the better).

The absolute error scoring function is strictly $\mathbb{F}$-consistent for the median functional. $\mathbb{F}$ is the family of probability distributions $F$ for which $\textnormal{E}_F[Y]$ exists and is finite (Raiffa and Schlaifer 1961, p.196; Ferguson 1967, p.51; Thomson 1979; Saerens 2000; Gneiting 2011).

References

Ferguson TS (1967) Mathematical Statistics: A Decision-Theoretic Approach. Academic Press, New York.

Gneiting T (2011) Making and evaluating point forecasts. Journal of the American Statistical Association 106(494):746–762. doi:10.1198/jasa.2011.r10138.

Raiffa H, Schlaifer R (1961) Applied Statistical Decision Theory. Colonial Press, Clinton.

Saerens M (2000) Building cost functions minimizing to some summary statistics. IEEE Transactions on Neural Networks 11(6):1263–1271. doi:10.1109/72.883416.

Thomson W (1979) Eliciting production possibilities from a well-informed manager. Journal of Economic Theory 20(3):360–380. doi:10.1016/0022-0531(79)90042-5.

See Also

mae, quantile_if

Examples

# Compute the absolute error scoring function.

df <- data.frame(
    y = rep(x = 0, times = 5),
    x = -2:2
)

df$absolute_error <- aerr_sf(x = df$x, y = df$y)

print(df)

---

Absolute percentage error scoring function

Description

The function aperr_sf computes the absolute percentage error scoring function when $y$ materialises and $x$ is the predictive $\textnormal{med}^{(-1)}(F)$ functional.

The absolute percentage error scoring function is defined in Table 1 in Gneiting (2011).

Usage

aperr_sf(x, y)

Arguments

x	Predictive $\textnormal{med}^{(-1)}(F)$ functional (prediction). It can be a vector of length $n$ (must have the same length as $y$).
y	Realisation (true value) of process. It can be a vector of length $n$ (must have the same length as $x$).

Details

The absolute percentage error scoring function is defined by:

$S(x, y) := |(x - y)/y|$

Domain of function:

$x > 0$

$y > 0$

Range of function:

$S(x, y) \geq 0, \forall x, y > 0$

Value

Vector of absolute percentage errors.

Note

For details on the absolute percentage error scoring function, see Gneiting (2011).

The $\beta$-median functional, $\textnormal{med}^{(\beta)}(F)$ is the median of a probability distribution whose density is proportional to $y^\beta f(y)$, where $f$ is the density of the probability distribution $F$ of $Y$ (Gneiting 2011).

The functional $\textnormal{med}^{(-1)}(F)$ elicited by the absolute percentage error scoring function is the $\beta$-median at $\beta = -1$, whose density is proportional to $f(y)/y$ (p. 752 in Gneiting 2011). The $\beta$-median functional at general $\beta$ is elicited by bmedian_sf.

The absolute percentage error scoring function is negatively oriented (i.e. the smaller, the better).

The absolute percentage error scoring function is strictly $\mathbb{F}^{(w)}$-consistent for the $\textnormal{med}^{(-1)}(F)$ functional. $\mathbb{F}$ is the family of probability distributions for which $\textnormal{E}_F[Y]$ exists and is finite. $\mathbb{F}^{(w)}$ is the subclass of probability distributions in $\mathbb{F}$, which are such that $w(y) f(y)$, $w(y) = 1/y$ has finite integral over $(0, \infty)$, and the probability distribution $F^{(w)}$ with density proportional to $w(y) f(y)$ belongs to $\mathbb{F}$ (see Theorems 5 and 9 in Gneiting 2011).

References

Gneiting T (2011) Making and evaluating point forecasts. Journal of the American Statistical Association 106(494):746–762. doi:10.1198/jasa.2011.r10138.

See Also

mape

Examples

# Compute the absolute percentage error scoring function.

df <- data.frame(
    y = rep(x = 2, times = 3),
    x = 1:3
)

df$absolute_percentage_error <- aperr_sf(x = df$x, y = df$y)

print(df)

---

Realised $\beta$-median score

Description

The function bmedian_rs computes the realised $\beta$-median score with parameter $b$, when $\textbf{\textit{y}}$ materialises and $\textbf{\textit{x}}$ is the prediction.

Realised $\beta$-median score is a realised score corresponding to the $\beta$-median scoring function bmedian_sf.

Usage

bmedian_rs(x, y, b)

Arguments

x	Prediction. It can be a vector of length $n$ (must have the same length as $\textbf{\textit{y}}$).
y	Realisation (true value) of process. It can be a vector of length $n$ (must have the same length as $\textbf{\textit{x}}$).
b	It can be a scalar.

Details

The realised $\beta$-median score is defined by:

$S(\textbf{\textit{x}}, \textbf{\textit{y}}, b) := (1/n) \sum_{i = 1}^{n} L(x_i, y_i, b)$

where

$\textbf{\textit{x}} = (x_1, ..., x_n)^\mathsf{T}$

$\textbf{\textit{y}} = (y_1, ..., y_n)^\mathsf{T}$

and

$L(x, y, b) := |1 - (y/x)^b|$

Domain of function:

$\textbf{\textit{x}} > \textbf{0}$

$\textbf{\textit{y}} > \textbf{0}$

$b \neq 0$

where

$\textbf{0} = (0, ..., 0)^\mathsf{T}$

is the zero vector of length $n$ and the symbol $>$ indicates pairwise inequality.

Range of function:

$S(\textbf{\textit{x}}, \textbf{\textit{y}}, b) \geq 0, \forall \textbf{\textit{x}}, \textbf{\textit{y}} > \textbf{0}, b \neq 0$

Value

Value of the realised $\beta$-median score.

Note

For details on the $\beta$-median scoring function, see bmedian_sf.

The concept of realised (average) scores is defined by Gneiting (2011) and Fissler and Ziegel (2019).

The realised $\beta$-median score is the realised (average) score corresponding to the $\beta$-median scoring function.

References

Fissler T, Ziegel JF (2019) Order-sensitivity and equivariance of scoring functions. Electronic Journal of Statistics 13(1):1166–1211. doi:10.1214/19-EJS1552.

Gneiting T (2011) Making and evaluating point forecasts. Journal of the American Statistical Association 106(494):746–762. doi:10.1198/jasa.2011.r10138.

See Also

bmedian_sf

Examples

# Compute the realised beta-median score.

set.seed(12345)

b <- 2

x <- 0.5

y <- rlnorm(n = 100, meanlog = 0, sdlog = 1)

print(bmedian_rs(x = x, y = y, b = b))

print(bmedian_rs(x = rep(x = x, times = 100), y = y, b = b))

---

$\beta$-median scoring function

Description

The function bmedian_sf computes the $\beta$-median scoring function when $y$ materialises and $x$ is the predictive $\textnormal{med}^{(\beta)}(F)$ functional.

The $\beta$-median scoring function is defined in eq. (4) in Gneiting (2011).

Usage

bmedian_sf(x, y, b)

Arguments

x	Predictive $\textnormal{med}^{(\beta)}(F)$ functional (prediction). It can be a vector of length $n$ (must have the same length as $y$).
y	Realisation (true value) of process. It can be a vector of length $n$ (must have the same length as $x$).
b	The parameter $\beta$ of the $\beta$-median functional. It can be a vector of length $n$ (must have the same length as $y$).

Details

The $\beta$-median scoring function is defined by, with $b = \beta$:

$S(x, y, b) := |1 - (y/x)^b|$

Domain of function:

$x > 0$

$y > 0$

$b \neq 0$

Range of function:

$S(x, y, b) \geq 0, \forall x, y > 0, b \neq 0$

Value

Vector of $\beta$-median losses.

Note

For details on the $\beta$-median scoring function, see Gneiting (2011).

The $\beta$-median functional, $\textnormal{med}^{(\beta)}(F)$ is the median of a probability distribution whose density is proportional to $y^\beta f(y)$, where $f$ is the density of the probability distribution $F$ of $Y$ (Gneiting 2011).

The $\beta$-median scoring function is negatively oriented (i.e. the smaller, the better).

The $\beta$-median scoring function is strictly $\mathbb{F}^{(w)}$-consistent for the $\textnormal{med}^{(\beta)}(F)$ functional. $\mathbb{F}$ is the family of probability distributions for which $\textnormal{E}_F[Y]$ exists and is finite. $\mathbb{F}^{(w)}$ is the subclass of probability distributions in $\mathbb{F}$, which are such that $w(y) f(y)$, $w(y) = y^\beta$ has finite integral over $(0, \infty)$, and the probability distribution $F^{(w)}$ with density proportional to $w(y) f(y)$ belongs to $\mathbb{F}$ (see Theorems 5 and 9 in Gneiting 2011).

References

Gneiting T (2011) Making and evaluating point forecasts. Journal of the American Statistical Association 106(494):746–762. doi:10.1198/jasa.2011.r10138.

See Also

bmedian_rs

Examples

# Compute the bmedian scoring function.

df <- data.frame(
    y = rep(x = 2, times = 3),
    x = 1:3,
    b = c(-1, 1, 2)
)

df$bmedian_error <- bmedian_sf(x = df$x, y = df$y, b = df$b)

print(df)

---

Realised Bregman score (type 1)

Description

The function bregman1_rs computes the realised Bregman score (type 1) with parameter $a$, when $\textbf{\textit{y}}$ materialises and $\textbf{\textit{x}}$ is the prediction.

Realised Bregman score (type 1) is a realised score corresponding to the Bregman scoring function (type 1) bregman1_sf.

Usage

bregman1_rs(x, y, a)

Arguments

x	Prediction. It can be a vector of length $n$ (must have the same length as $\textbf{\textit{y}}$).
y	Realisation (true value) of process. It can be a vector of length $n$ (must have the same length as $\textbf{\textit{x}}$).
a	It can be a scalar.

Details

The realised Bregman score (type 1) is defined by:

$S(\textbf{\textit{x}}, \textbf{\textit{y}}, a) := (1/n) \sum_{i = 1}^{n} L(x_i, y_i, a)$

where

$\textbf{\textit{x}} = (x_1, ..., x_n)^\mathsf{T}$

$\textbf{\textit{y}} = (y_1, ..., y_n)^\mathsf{T}$

and

$L(x, y, a) := |y|^a - |x|^a - a \textnormal{sign}(x) |x|^{a - 1} (y - x)$

Domain of function:

$\textbf{\textit{x}} \in \mathbb{R}^n$

$\textbf{\textit{y}} \in \mathbb{R}^n$

$a > 1$

Range of function:

$S(\textbf{\textit{x}}, \textbf{\textit{y}}, a) \geq 0, \forall \textbf{\textit{x}}, \textbf{\textit{y}} \in \mathbb{R}^n, a > 1$

Value

Value of the realised Bregman score (type 1).

Note

For details on the Bregman scoring function (type 1), see bregman1_sf.

The concept of realised (average) scores is defined by Gneiting (2011) and Fissler and Ziegel (2019).

The realised Bregman score (type 1) is the realised (average) score corresponding to the Bregman scoring function (type 1).

References

Fissler T, Ziegel JF (2019) Order-sensitivity and equivariance of scoring functions. Electronic Journal of Statistics 13(1):1166–1211. doi:10.1214/19-EJS1552.

Gneiting T (2011) Making and evaluating point forecasts. Journal of the American Statistical Association 106(494):746–762. doi:10.1198/jasa.2011.r10138.

See Also

bregman1_sf, mean_if

Examples

# Compute the realised Bregman score (type 1).

set.seed(12345)

a <- 3

x <- 0

y <- rnorm(n = 100, mean = 0, sd = 1)

print(bregman1_rs(x = x, y = y, a = a))

print(bregman1_rs(x = rep(x = x, times = 100), y = y, a = a))

---

Bregman scoring function (type 1)

Description

The function bregman1_sf computes the Bregman scoring function when $y$ materialises and $x$ is the predictive mean functional.

The Bregman scoring function is defined by eq. (18) in Gneiting (2011) and the form implemented here for $\phi(x) = |x|^a$ is defined by eq. (19) in Gneiting (2011).

Usage

bregman1_sf(x, y, a)

Arguments

x	Predictive mean functional (prediction). It can be a vector of length $n$ (must have the same length as $y$).
y	Realisation (true value) of process. It can be a vector of length $n$ (must have the same length as $x$).
a	It can be a vector of length $n$ (must have the same length as $y$).

Details

The Bregman scoring function (type 1) is defined by:

$S(x, y, a) := |y|^a - |x|^a - a \textnormal{sign}(x) |x|^{a - 1} (y - x)$

Domain of function:

$x \in \mathbb{R}$

$y \in \mathbb{R}$

$a > 1$

Range of function:

$S(x, y, a) \geq 0, \forall x, y \in \mathbb{R}, a > 1$

Value

Vector of Bregman losses.

Note

The implemented function is named type 1 because it corresponds to a specific type of $\phi(x)$ of the general form of the Bregman scoring function defined by eq. (18) in Gneiting (2011).

For details on the Bregman scoring function, see Savage (1971), Banerjee et al. (2005) and Gneiting (2011).

The mean functional is the mean $\textnormal{E}_F[Y]$ of the probability distribution $F$ of $Y$ (Gneiting 2011).

The Bregman scoring function is negatively oriented (i.e. the smaller, the better).

The herein implemented Bregman scoring function is strictly $\mathbb{F}$-consistent for the mean functional. $\mathbb{F}$ is the family of probability distributions for which $\textnormal{E}_F[Y]$ and $\textnormal{E}_F[|Y|^a]$ exist and are finite (Savage 1971; Gneiting 2011).

References

Banerjee A, Guo X, Wang H (2005) On the optimality of conditional expectation as a Bregman predictor. IEEE Transactions on Information Theory 51(7):2664–2669. doi:10.1109/TIT.2005.850145.

Gneiting T (2011) Making and evaluating point forecasts. Journal of the American Statistical Association 106(494):746–762. doi:10.1198/jasa.2011.r10138.

Savage LJ (1971) Elicitation of personal probabilities and expectations. Journal of the American Statistical Association 66(336):783–801. doi:10.1080/01621459.1971.10482346.

See Also

bregman1_rs, mean_if

Examples

# Compute the Bregman scoring function (type 1).

df <- data.frame(
    y = rep(x = 0, times = 7),
    x = c(-3, -2, -1, 0, 1, 2, 3),
    a = rep(x = 3, times = 7)
)

df$bregman1_penalty <- bregman1_sf(x = df$x, y = df$y, a = df$a[1])

print(df)

# Equivalence of Bregman scoring function (type 1) and squared error scoring
# function, when a = 2.

set.seed(12345)

n <- 100

x <- runif(n = n, min = -20, max = 20)
y <- runif(n = n, min = -20, max = 20)
a <- rep(x = 2, times = n)

u <- bregman1_sf(x = x, y = y, a = a)

v <- serr_sf(x = x, y = y)

max(abs(u - v)) # values are slightly higher than 0 due to rounding error
min(abs(u - v))

---

Realised Bregman score (type 2, Patton score)

Description

The function bregman2_rs computes the realised Bregman score (type 2, Patton score) with parameter $b$, when $\textbf{\textit{y}}$ materialises and $\textbf{\textit{x}}$ is the prediction.

Realised Bregman score (type 2) is a realised score corresponding to the Bregman scoring function (type 2, Patton scoring function) bregman2_sf.

Usage

bregman2_rs(x, y, b)

Arguments

x	Prediction. It can be a vector of length $n$ (must have the same length as $\textbf{\textit{y}}$).
y	Realisation (true value) of process. It can be a vector of length $n$ (must have the same length as $\textbf{\textit{x}}$).
b	It can be a scalar.

Details

The realised Bregman score (type 2) is defined by:

$S(\textbf{\textit{x}}, \textbf{\textit{y}}, b) := (1/n) \sum_{i = 1}^{n} L(x_i, y_i, b)$

where

$\textbf{\textit{x}} = (x_1, ..., x_n)^\mathsf{T}$

$\textbf{\textit{y}} = (y_1, ..., y_n)^\mathsf{T}$

and

$L(x, y, b) := \dfrac{1}{b (b - 1)} (y^b - x^b) - \dfrac{1}{b - 1} x^{b - 1} (y - x)$

Domain of function:

$\textbf{\textit{x}} > \textbf{0}$

$\textbf{\textit{y}} > \textbf{0}$

$b \in \mathbb{R} \setminus \lbrace 0, 1 \rbrace$

where

$\textbf{0} = (0, ..., 0)^\mathsf{T}$

is the zero vector of length $n$ and the symbol $>$ indicates pairwise inequality.

Range of function:

$S(\textbf{\textit{x}}, \textbf{\textit{y}}, b) \geq 0, \forall \textbf{\textit{x}}, \textbf{\textit{y}} > \textbf{0}, b \in \mathbb{R} \setminus \lbrace 0, 1 \rbrace$

Value

Value of the realised Bregman score (type 2).

Note

For details on the Bregman scoring function (type 2), see bregman2_sf.

The concept of realised (average) scores is defined by Gneiting (2011) and Fissler and Ziegel (2019).

The realised Bregman score (type 2) is the realised (average) score corresponding to the Bregman scoring function (type 2).

References

Fissler T, Ziegel JF (2019) Order-sensitivity and equivariance of scoring functions. Electronic Journal of Statistics 13(1):1166–1211. doi:10.1214/19-EJS1552.

Gneiting T (2011) Making and evaluating point forecasts. Journal of the American Statistical Association 106(494):746–762. doi:10.1198/jasa.2011.r10138.

See Also

bregman2_sf, mean_if

Examples

# Compute the realised Bregman score (type 2).

set.seed(12345)

b <- 3

x <- 0.5

y <- rlnorm(n = 100, meanlog = 0, sdlog = 1)

print(bregman2_rs(x = x, y = y, b = b))

print(bregman2_rs(x = rep(x = x, times = 100), y = y, b = b))

---

Bregman scoring function (type 2, Patton scoring function)

Description

The function bregman2_sf computes the Bregman scoring function when $y$ materialises and $x$ is the predictive mean functional.

The Bregman scoring function is defined by eq. (18) in Gneiting (2011) and the form implemented here for $\phi(x) = \dfrac{1}{b (b - 1)} x^b$, $b \in \mathbb{R} \setminus \lbrace 0, 1 \rbrace$ is defined by eq. (20) in Gneiting (2011).

Usage

bregman2_sf(x, y, b)

Arguments

x	Predictive mean functional (prediction). It can be a vector of length $n$ (must have the same length as $y$).
y	Realisation (true value) of process. It can be a vector of length $n$ (must have the same length as $x$).
b	It can be a vector of length $n$ (must have the same length as $y$).

Details

The Bregman scoring function (type 2) is defined by:

$S(x, y, b) := \dfrac{1}{b (b - 1)} (y^b - x^b) - \dfrac{1}{b - 1} x^{b - 1} (y - x)$

Domain of function:

$x > 0$

$y > 0$

$b \in \mathbb{R} \setminus \lbrace 0, 1 \rbrace$

Range of function:

$S(x, y, b) \geq 0, \forall x, y > 0, b \in \mathbb{R} \setminus \lbrace 0, 1 \rbrace$

Value

Vector of Bregman losses.

Note

The implemented function is named type 2 because it corresponds to a specific type of $\phi(x)$ of the general form of the Bregman scoring function defined by eq. (18) in Gneiting (2011).

For details on the Bregman scoring function, see Savage (1971), Banerjee et al. (2005) and Gneiting (2011). For details on the specific form implemented here, see Patton (2011).

The mean functional is the mean $\textnormal{E}_F[Y]$ of the probability distribution $F$ of $Y$ (Gneiting 2011).

The Bregman scoring function is negatively oriented (i.e. the smaller, the better).

The herein implemented Bregman scoring function is strictly $\mathbb{F}$-consistent for the mean functional. $\mathbb{F}$ is the family of probability distributions $F$ for which $\textnormal{E}_F[Y]$ and $\textnormal{E}_F[\dfrac{1}{b (b - 1)} Y^b]$ exist and are finite (Savage 1971; Gneiting 2011).

References

Banerjee A, Guo X, Wang H (2005) On the optimality of conditional expectation as a Bregman predictor. IEEE Transactions on Information Theory 51(7):2664–2669. doi:10.1109/TIT.2005.850145.

Gneiting T (2011) Making and evaluating point forecasts. Journal of the American Statistical Association 106(494):746–762. doi:10.1198/jasa.2011.r10138.

Patton AJ (2011) Volatility forecast comparison using imperfect volatility proxies. Journal of Econometrics 160(1):246–256. doi:10.1016/j.jeconom.2010.03.034.

Savage LJ (1971) Elicitation of personal probabilities and expectations. Journal of the American Statistical Association 66(336):783–801. doi:10.1080/01621459.1971.10482346.

See Also

bregman2_rs, mean_if

Examples

# Compute the Bregman scoring function (type 2).

df <- data.frame(
    y = rep(x = 2, times = 6),
    x = rep(x = 1:3, times = 2),
    b = rep(x = c(-3, 3), each = 3)
)

df$bregman2_penalty <- bregman2_sf(x = df$x, y = df$y, b = df$b)

print(df)

# The Bregman scoring function (type 2) is half the squared error scoring
# function, when b = 2.

df <- data.frame(
    y = rep(x = 5.5, times = 10),
    x = 1:10,
    b = rep(x = 2, times = 10)
)

df$bregman2_penalty <- bregman2_sf(x = df$x, y = df$y, b = df$b)

df$squared_error <- serr_sf(x = df$x, y = df$y)

df$ratio <- df$bregman2_penalty/df$squared_error

print(df)

# When a = b > 1 the Bregman scoring function (type 2) equals the
# Bregman scoring function (type 1) up to a multiplicative constant.

df <- data.frame(
    y = rep(x = 5.5, times = 10),
    x = 1:10,
    b = rep(x = c(3, 4), each = 5)
)

df$bregman2_penalty <- bregman2_sf(x = df$x, y = df$y, b = df$b)

df$bregman1_penalty <- bregman1_sf(x = df$x, y = df$y, a = df$b)

df$ratio <- df$bregman2_penalty/df$bregman1_penalty

print(df)

---

Bregman scoring function (type 3, QLIKE scoring function)

Description

The function bregman3_sf computes the Bregman scoring function when $y$ materialises and $x$ is the predictive mean functional.

The Bregman scoring function is defined by eq. (18) in Gneiting (2011) and the form implemented here for $\phi(x) = -\log(x)$ is defined by eq. (20) in Gneiting (2011).

Usage

bregman3_sf(x, y)

Arguments

x	Predictive mean functional (prediction). It can be a vector of length $n$ (must have the same length as $y$).
y	Realisation (true value) of process. It can be a vector of length $n$ (must have the same length as $x$).

Details

The Bregman scoring function (type 3) is defined by:

$S(x, y) := (y/x) - \log(y/x) - 1$

Domain of function:

$x > 0$

$y > 0$

Range of function:

$S(x, y) \geq 0, \forall x, y > 0$

Value

Vector of Bregman losses.

Note

The implemented function is named type 3 because it corresponds to a specific type of $\phi(x)$ of the general form of the Bregman scoring function defined by eq. (18) in Gneiting (2011).

For details on the Bregman scoring function, see Savage (1971), Banerjee et al. (2005) and Gneiting (2011). For details on the specific form implemented here, see the QLIKE scoring function in Patton (2011).

The mean functional is the mean $\textnormal{E}_F[Y]$ of the probability distribution $F$ of $Y$ (Gneiting 2011).

The Bregman scoring function is negatively oriented (i.e. the smaller, the better).

The herein implemented Bregman scoring function is strictly $\mathbb{F}$-consistent for the mean functional. $\mathbb{F}$ is the family of probability distributions $F$ for which $\textnormal{E}_F[Y]$ and $\textnormal{E}_F[\log(Y)]$ exist and are finite (Savage 1971; Gneiting 2011).

References

Banerjee A, Guo X, Wang H (2005) On the optimality of conditional expectation as a Bregman predictor. IEEE Transactions on Information Theory 51(7):2664–2669. doi:10.1109/TIT.2005.850145.

Gneiting T (2011) Making and evaluating point forecasts. Journal of the American Statistical Association 106(494):746–762. doi:10.1198/jasa.2011.r10138.

Patton AJ (2011) Volatility forecast comparison using imperfect volatility proxies. Journal of Econometrics 160(1):246–256. doi:10.1016/j.jeconom.2010.03.034.

Savage LJ (1971) Elicitation of personal probabilities and expectations. Journal of the American Statistical Association 66(336):783–801. doi:10.1080/01621459.1971.10482346.

See Also

qlike, mean_if

Examples

# Compute the Bregman scoring function (type 3, QLIKE scoring function).

df <- data.frame(
    y = rep(x = 2, times = 3),
    x = 1:3
)

df$bregman3_penalty <- bregman3_sf(x = df$x, y = df$y)

print(df)

---

Realised Bregman score (type 4, Patton score)

Description

The function bregman4_rs computes the realised Bregman score (type 4, Patton score) when $\textbf{\textit{y}}$ materialises and $\textbf{\textit{x}}$ is the prediction.

Realised Bregman score (type 4) is a realised score corresponding to the Bregman scoring function (type 4, Patton scoring function) bregman4_sf.

Usage

bregman4_rs(x, y)

Arguments

x	Prediction. It can be a vector of length $n$ (must have the same length as $\textbf{\textit{y}}$).
y	Realisation (true value) of process. It can be a vector of length $n$ (must have the same length as $\textbf{\textit{x}}$).

Details

The realised Bregman score (type 4) is defined by:

$S(\textbf{\textit{x}}, \textbf{\textit{y}}) := (1/n) \sum_{i = 1}^{n} L(x_i, y_i)$

where

$\textbf{\textit{x}} = (x_1, ..., x_n)^\mathsf{T}$

$\textbf{\textit{y}} = (y_1, ..., y_n)^\mathsf{T}$

and

$L(x, y) := y \log(y/x) - y + x$

Domain of function:

$\textbf{\textit{x}} > \textbf{0}$

$\textbf{\textit{y}} > \textbf{0}$

where

$\textbf{0} = (0, ..., 0)^\mathsf{T}$

is the zero vector of length $n$ and the symbol $>$ indicates pairwise inequality.

Range of function:

$S(\textbf{\textit{x}}, \textbf{\textit{y}}) \geq 0, \forall \textbf{\textit{x}}, \textbf{\textit{y}} > \textbf{0}$

Value

Value of the realised Bregman score (type 4).

Note

For details on the Bregman scoring function (type 4), see bregman4_sf.

The concept of realised (average) scores is defined by Gneiting (2011) and Fissler and Ziegel (2019).

The realised Bregman score (type 4) is the realised (average) score corresponding to the Bregman scoring function (type 4).

References

Fissler T, Ziegel JF (2019) Order-sensitivity and equivariance of scoring functions. Electronic Journal of Statistics 13(1):1166–1211. doi:10.1214/19-EJS1552.

Gneiting T (2011) Making and evaluating point forecasts. Journal of the American Statistical Association 106(494):746–762. doi:10.1198/jasa.2011.r10138.

See Also

bregman4_sf, mean_if

Examples

# Compute the realised Bregman score (type 4).

set.seed(12345)

x <- 0.5

y <- rlnorm(n = 100, meanlog = 0, sdlog = 1)

print(bregman4_rs(x = x, y = y))

print(bregman4_rs(x = rep(x = x, times = 100), y = y))

---

Bregman scoring function (type 4, Patton scoring function)

Description

The function bregman4_sf computes the Bregman scoring function when $y$ materialises and $x$ is the predictive mean functional.

The Bregman scoring function is defined by eq. (18) in Gneiting (2011) and the form implemented here for $\phi(x) = x \log(x)$ is defined by eq. (20) in Gneiting (2011).

Usage

bregman4_sf(x, y)

Arguments

x	Predictive mean functional (prediction). It can be a vector of length $n$ (must have the same length as $y$).
y	Realisation (true value) of process. It can be a vector of length $n$ (must have the same length as $x$).

Details

The Bregman scoring function (type 4) is defined by:

$S(x, y) := y \log(y/x) - y + x$

Domain of function:

$x > 0$

$y > 0$

Range of function:

$S(x, y) \geq 0, \forall x, y > 0$

Value

Vector of Bregman losses.

Note

The implemented function is named type 4 because it corresponds to a specific type of $\phi(x)$ of the general form of the Bregman scoring function defined by eq. (18) in Gneiting (2011).

For details on the Bregman scoring function, see Savage (1971), Banerjee et al. (2005) and Gneiting (2011). For details on the specific form implemented here, see Patton (2011).

The mean functional is the mean $\textnormal{E}_F[Y]$ of the probability distribution $F$ of $Y$ (Gneiting 2011).

The Bregman scoring function is negatively oriented (i.e. the smaller, the better).

The herein implemented Bregman scoring function is strictly $\mathbb{F}$-consistent for the mean functional. $\mathbb{F}$ is the family of probability distributions $F$ for which $\textnormal{E}_F[Y]$ and $\textnormal{E}_F[Y \log(Y)]$ exist and are finite (Savage 1971; Gneiting 2011).

References

Banerjee A, Guo X, Wang H (2005) On the optimality of conditional expectation as a Bregman predictor. IEEE Transactions on Information Theory 51(7):2664–2669. doi:10.1109/TIT.2005.850145.

Gneiting T (2011) Making and evaluating point forecasts. Journal of the American Statistical Association 106(494):746–762. doi:10.1198/jasa.2011.r10138.

Patton AJ (2011) Volatility forecast comparison using imperfect volatility proxies. Journal of Econometrics 160(1):246–256. doi:10.1016/j.jeconom.2010.03.034.

Savage LJ (1971) Elicitation of personal probabilities and expectations. Journal of the American Statistical Association 66(336):783–801. doi:10.1080/01621459.1971.10482346.

See Also

bregman4_rs, mean_if

Examples

# Compute the Bregman scoring function (type 4).

df <- data.frame(
    y = rep(x = 2, times = 3),
    x = 1:3
)

df$bregman4_penalty <- bregman4_sf(x = df$x, y = df$y)

print(df)

---

Capping function

Description

The function capping_function computes the value of the capping function, defined in Taggart (2022), p.205.

It is used by the generalized Huber loss function among others (see Taggart 2022).

Usage

capping_function(t, a, b)

Arguments

t	It can be a vector of length $n$.
a	It can be a vector of length $n$ (must have the same length as $t$).
b	It can be a vector of length $n$ (must have the same length as $t$).

Details

The capping function $\kappa_{a, b}(t)$ is defined by:

$\kappa_{a, b}(t) := \max \lbrace \min \lbrace t, b \rbrace, -a \rbrace$

or equivalently,

$\kappa_{a, b}(t) := \left\lbrace \begin{array}{ll} -a, & t \leq -a\\ t, & -a < t \leq b\\ b, & t > b \end{array} \right.$

Domain of function:

$t \in \mathbb{R}$

$a \in [0, \infty]$

$b \in [0, \infty]$

Range of function:

$-a \leq \kappa_{a, b}(t) \leq b, \forall t \in \mathbb{R}, a, b \in [0, \infty]$

Value

Vector of values of the capping function.

Note

For the definition of the capping function, see Taggart (2022), p.205.

Taggart (2022), p.205 admits the endpoints $a = \infty$ and $b = \infty$, at which the corresponding cap is removed; the examples below include those cases.

References

Taggart RJ (2022) Point forecasting and forecast evaluation with generalized Huber loss. Electronic Journal of Statistics 16(1):201–231. doi:10.1214/21-EJS1957.

See Also

ghuber_sf, huber_sf

Examples

# Compute the capping function.

df <- data.frame(
    t = c(1, -1, 1, -1, 1, -1, 1, -1, 1, 1, 2.5, 2.5, 3.5, 3.5),
    a = c(0, 0, 0, 0, Inf, Inf, Inf, Inf, 2, 3, 2, 3, 2, 3),
    b = c(0, 0, Inf, Inf, 0, 0, Inf, Inf, 3, 2, 3, 2, 3, 2)
)

df$cf <- capping_function(t = df$t, a = df$a, b = df$b)

print(df)

---

Error - spread scoring function

Description

The function errorspread_sf computes the error - spread scoring function, when $y$ materialises, $x_1$ is the predictive mean, $x_2$ is the predictive variance and $x_3$ is the predictive skewness.

The error - spread scoring function is defined by eq. (14) in Christensen et al. (2015).

Usage

errorspread_sf(x1, x2, x3, y)

Arguments

x1	Predictive mean (prediction). It can be a vector of length $n$ (must have the same length as $y$).
x2	Predictive variance (prediction). It can be a vector of length $n$ (must have the same length as $y$).
x3	Predictive skewness (prediction). It can be a vector of length $n$ (must have the same length as $y$).
y	Realisation (true value) of process. It can be a vector of length $n$ (must have the same length as $x_1$, $x_2$ and $x_3$).

Details

The error - spread scoring function is defined by:

$S(x_1, x_2, x_3, y) := (x_2 - (x_1 - y)^2 - (x_1 - y) x_2^{1/2} x_3)^2$

Domain of function:

$x_1 \in \mathbb{R}$

$x_2 > 0$

$x_3 \in \mathbb{R}$

$y \in \mathbb{R}$

Range of function:

$S(x_1, x_2, x_3, y) \geq 0, \forall x_1, x_3, y \in \mathbb{R}, x_2 > 0$

Value

Vector of error - spread losses.

Note

The mean functional is the mean $\textnormal{E}_F[Y]$ of the probability distribution $F$ of $Y$ (Christensen et al. 2015).

The variance functional is the variance $\textnormal{Var}_F[Y] := \textnormal{E}_F[Y^2] - (\textnormal{E}_F[Y])^{2}$ of the probability distribution $F$ of $Y$ (Christensen et al. 2015).

The skewness functional is the skewness $\textnormal{Sk}_F[Y] := \textnormal{E}_F[((Y - \textnormal{E}_F[Y])/(\textnormal{Var}_F[Y])^{1/2})^3]$ (Christensen et al. 2015).

The error - spread scoring function is negatively oriented (i.e. the smaller, the better).

The error - spread scoring function is consistent for the triple (mean, variance, skewness) functional. $\mathbb{F}$ is the family of probability distributions $F$ for which $\textnormal{E}_F[Y^4]$ exists and is finite (Christensen et al. 2015).

References

Christensen HM, Moroz IM, Palmer TN (2015) Evaluation of ensemble forecast uncertainty using a new proper score: Application to medium-range and seasonal forecasts. Quarterly Journal of the Royal Meteorological Society 141(687)(Part B):538–549. doi:10.1002/qj.2375.

Examples

# Compute the error - spread scoring function.

df <- data.frame(
    y = rep(x = 0, times = 6),
    x1 = c(2, 2, -2, -2, 0, 0),
    x2 = c(1, 2, 1, 2, 1, 2),
    x3 = c(3, 3, -3, -3, 0, 0)
)

df$errorspread_penalty <- errorspread_sf(x1 = df$x1, x2 = df$x2, x3 = df$x3,
    y = df$y)

print(df)

---

Expectile identification function

Description

The function expectile_if computes the expectile identification function at a specific level $p$, when $y$ materialises and $x$ is the predictive expectile at level $p$.

The expectile identification function is defined in Table 9 in Gneiting (2011).

Usage

expectile_if(x, y, p)

Arguments

x	Predictive expectile (prediction) at level $p$. It can be a vector of length $n$ (must have the same length as $y$).
y	Realisation (true value) of process. It can be a vector of length $n$ (must have the same length as $x$).
p	It can be a vector of length $n$ (must have the same length as $y$).

Details

The expectile identification function is defined by:

$V(x, y, p) := 2 |\textbf{1} \lbrace x \geq y \rbrace - p| (x - y)$

Domain of function:

$x \in \mathbb{R}$

$y \in \mathbb{R}$

$0 < p < 1$

Range of function:

$V(x, y, p) \in \mathbb{R}$

Value

Vector of values of the expectile identification function.

Note

For the definition of expectiles, see Newey and Powell (1987).

The expectile identification function is a strict $\mathbb{F}$-identification function for the $p$-expectile functional (Gneiting 2011; Fissler and Ziegel 2016; Dimitriadis et al. 2024).

$\mathbb{F}$ is the family of probability distributions $F$ for which $\textnormal{E}_F[Y]$ exists and is finite (Gneiting 2011; Fissler and Ziegel 2016; Dimitriadis et al. 2024).

References

Dimitriadis T, Fissler T, Ziegel JF (2024) Osband's principle for identification functions. Statistical Papers 65:1125–1132. doi:10.1007/s00362-023-01428-x.

Fissler T, Ziegel JF (2016) Higher order elicitability and Osband's principle. The Annals of Statistics 44(4):1680–1707. doi:10.1214/16-AOS1439.

Gneiting T (2011) Making and evaluating point forecasts. Journal of the American Statistical Association 106(494):746–762. doi:10.1198/jasa.2011.r10138.

Newey WK, Powell JL (1987) Asymmetric least squares estimation and testing. Econometrica 55(4):819–847. doi:10.2307/1911031.

See Also

expectile_sf, expectile_rs

Examples

# Compute the expectile identification function.

df <- data.frame(
    y = rep(x = 0, times = 6),
    x = c(2, 2, -2, -2, 0, 0),
    p = rep(x = c(0.05, 0.95), times = 3)
)

df$expectile_if <- expectile_if(x = df$x, y = df$y, p = df$p)

print(df)

---

Realised expectile score

Description

The function expectile_rs computes the realised expectile score at a specific level $p$ when $\textbf{\textit{y}}$ materialises and $\textbf{\textit{x}}$ is the prediction.

Realised expectile score is a realised score corresponding to the expectile scoring function expectile_sf.

Usage

expectile_rs(x, y, p)

Arguments

x	Prediction. It can be a vector of length $n$ (must have the same length as $\textbf{\textit{y}}$).
y	Realisation (true value) of process. It can be a vector of length $n$ (must have the same length as $\textbf{\textit{x}}$).
p	It can be a scalar.

Details

The realised expectile score is defined by:

$S(\textbf{\textit{x}}, \textbf{\textit{y}}, p) := (1/n) \sum_{i = 1}^{n} L(x_i, y_i, p)$

where

$\textbf{\textit{x}} = (x_1, ..., x_n)^\mathsf{T}$

$\textbf{\textit{y}} = (y_1, ..., y_n)^\mathsf{T}$

and

$L(x, y, p) := |\textbf{1} \lbrace x \geq y \rbrace - p| (x - y)^2$

Domain of function:

$\textbf{\textit{x}} \in \mathbb{R}^n$

$\textbf{\textit{y}} \in \mathbb{R}^n$

$0 < p < 1$

Range of function:

$S(\textbf{\textit{x}}, \textbf{\textit{y}}, p) \geq 0, \forall \textbf{\textit{x}}, \textbf{\textit{y}} \in \mathbb{R}^n, p \in (0, 1)$

Value

Value of the realised expectile score.

Note

For details on the expectile scoring function, see expectile_sf.

The concept of realised (average) scores is defined by Gneiting (2011) and Fissler and Ziegel (2019).

The realised expectile score is the realised (average) score corresponding to the expectile scoring function.

References

Fissler T, Ziegel JF (2019) Order-sensitivity and equivariance of scoring functions. Electronic Journal of Statistics 13(1):1166–1211. doi:10.1214/19-EJS1552.

Gneiting T (2011) Making and evaluating point forecasts. Journal of the American Statistical Association 106(494):746–762. doi:10.1198/jasa.2011.r10138.

See Also

expectile_sf, expectile_if

Examples

# Compute the realised expectile score.

set.seed(12345)

x <- 0.5

y <- rnorm(n = 100, mean = 0, sd = 1)

print(expectile_rs(x = x, y = y, p = 0.7))

print(expectile_rs(x = rep(x = x, times = 100), y = y, p = 0.7))

---

Asymmetric piecewise quadratic scoring function (expectile scoring function, expectile loss function)

Description

The function expectile_sf computes the asymmetric piecewise quadratic scoring function (expectile scoring function) at a specific level $p$, when $y$ materialises and $x$ is the predictive expectile at level $p$.

The asymmetric piecewise quadratic scoring function is defined by eq. (27) in Gneiting (2011).

Usage

expectile_sf(x, y, p)

Arguments

x	Predictive expectile (prediction) at level $p$. It can be a vector of length $n$ (must have the same length as $y$).
y	Realisation (true value) of process. It can be a vector of length $n$ (must have the same length as $x$).
p	It can be a vector of length $n$ (must have the same length as $y$).

Details

The asymmetric piecewise quadratic scoring function is defined by:

$S(x, y, p) := |\textbf{1} \lbrace x \geq y \rbrace - p| (x - y)^2$

or equivalently,

$S(x, y, p) := p (\max \lbrace -(x - y), 0 \rbrace)^2 + (1 - p) (\max \lbrace x - y, 0 \rbrace)^2$

Domain of function:

$x \in \mathbb{R}$

$y \in \mathbb{R}$

$0 < p < 1$

Range of function:

$S(x, y, p) \geq 0, \forall x, y \in \mathbb{R}, p \in (0, 1)$

Value

Vector of expectile losses.

Note

For the definition of expectiles, see Newey and Powell (1987).

The asymmetric piecewise quadratic scoring function is negatively oriented (i.e. the smaller, the better).

The asymmetric piecewise quadratic scoring function is strictly $\mathbb{F}$-consistent for the $p$-expectile functional. $\mathbb{F}$ is the family of probability distributions $F$ for which $\textnormal{E}_F[Y]$ and $\textnormal{E}_F[Y^2]$ exist and are finite (Gneiting 2011).

References

Gneiting T (2011) Making and evaluating point forecasts. Journal of the American Statistical Association 106(494):746–762. doi:10.1198/jasa.2011.r10138.

Newey WK, Powell JL (1987) Asymmetric least squares estimation and testing. Econometrica 55(4):819–847. doi:10.2307/1911031.

See Also

expectile_rs, expectile_if

Examples

# Compute the asymmetric piecewise quadratic scoring function (expectile scoring
# function).

df <- data.frame(
    y = rep(x = 0, times = 6),
    x = c(2, 2, -2, -2, 0, 0),
    p = rep(x = c(0.05, 0.95), times = 3)
)

df$expectile_penalty <- expectile_sf(x = df$x, y = df$y, p = df$p)

print(df)

# The asymmetric piecewise quadratic scoring function (expectile scoring
# function) at level p = 0.5 is half the squared error scoring function.

df <- data.frame(
    y = rep(x = 0, times = 3),
    x = c(-2, 0, 2),
    p = rep(x = c(0.5), times = 3)
)

df$expectile_penalty <- expectile_sf(x = df$x, y = df$y, p = df$p)

df$squared_error <- serr_sf(x = df$x, y = df$y)

print(df)

---

Realised generalized Huber score

Description

The function ghuber_rs computes the realised generalized Huber score at a specific level $p$ and parameters $a$ and $b$, when $\textbf{\textit{y}}$ materialises and $\textbf{\textit{x}}$ is the prediction.

Realised generalized Huber score is a realised score corresponding to the generalized Huber scoring function ghuber_sf.

Usage

ghuber_rs(x, y, p, a, b)

Arguments

x	Prediction. It can be a vector of length $n$ (must have the same length as $\textbf{\textit{y}}$).
y	Realisation (true value) of process. It can be a vector of length $n$ (must have the same length as $\textbf{\textit{x}}$).
p	It can be a scalar.
a	It can be a value.
b	It can be a value.

Details

The realised generalized Huber score is defined by:

$S(\textbf{\textit{x}}, \textbf{\textit{y}}, p, a, b) := (1/n) \sum_{i = 1}^{n} L(x_i, y_i, p, a, b)$

where

$\textbf{\textit{x}} = (x_1, ..., x_n)^\mathsf{T}$

$\textbf{\textit{y}} = (y_1, ..., y_n)^\mathsf{T}$

and

$L(x, y, p, a, b) := |\textbf{1} \lbrace x \geq y \rbrace - p| f_{a, b}(x - y)$

where

$f_{a, b}(t) := \kappa_{a, b}(t) (2 t - \kappa_{a, b}(t))$

and $\kappa_{a, b}(t)$ is the capping function defined by:

$\kappa_{a, b}(t) := \max \lbrace \min \lbrace t, b \rbrace, -a \rbrace$

Domain of function:

$\textbf{\textit{x}} \in \mathbb{R}^n$

$\textbf{\textit{y}} \in \mathbb{R}^n$

$0 < p < 1$

$a > 0$

$b > 0$

Range of function:

$S(\textbf{\textit{x}}, \textbf{\textit{y}}, p, a, b) \geq 0, \forall \textbf{\textit{x}}, \textbf{\textit{y}} \in \mathbb{R}^n, p \in (0, 1), a, b > 0$

Value

Value of the realised generalized Huber score.

Note

For details on the generalized Huber scoring function, see ghuber_sf.

The concept of realised (average) scores is defined by Gneiting (2011) and Fissler and Ziegel (2019).

The realised generalized Huber score is the realised (average) score corresponding to the generalized Huber scoring function.

References

Fissler T, Ziegel JF (2019) Order-sensitivity and equivariance of scoring functions. Electronic Journal of Statistics 13(1):1166–1211. doi:10.1214/19-EJS1552.

Gneiting T (2011) Making and evaluating point forecasts. Journal of the American Statistical Association 106(494):746–762. doi:10.1198/jasa.2011.r10138.

See Also

ghuber_sf, huberquantile_if

Examples

# Compute the realised generalized Huber score.

set.seed(12345)

p <- 0.7

a <- 0.5

b <- 0.5

x <- 0

y <- rnorm(n = 100, mean = 0, sd = 1)

print(ghuber_rs(x = x, y = y, p = p, a = a, b = b))

print(ghuber_rs(x = rep(x = x, times = 100), y = y, p = p, a = a, b = b))

---