Appendix A Data fitting

Orthogonal distance regression, also called generalized least squares regression, errors-in-variables models or measurement error models, attempts to tries to find the best fit taking into account errors in both x- and y- values. Assuming the relationship

$$y^{*}=f(x^{*};\beta )$$

(60)

where $\beta$ are parameters and $x^{*}$ and $y^{*}$ are the “true” values, without error, this leads to a minimization of the sum

$$\underset{\beta ,\delta }{\min }\sum _{i=1}^n\left[{\left(y_i-f(x_i+\delta ;\beta )\right)}^2+{\delta }_i^2\right]$$

(61)

which can be interpreted as the sum of orthogonal distances from the data points $(x_i,y_i)$ to the curve $y=f(x,\beta )$. It can be rewritten as

$$\underset{\beta ,\delta ,\epsilon }{\min }\sum _{i=1}^n\left[{\epsilon }_i^2+{\delta }_i^2\right]$$

(62)

subject to

$$y_i+{\epsilon }_i=f(x_i+{\delta }_i;\beta ).$$

(63)

This can be generalized to accomodate different weights for the datapoints and to higher dimensions

$$\underset{\beta ,\delta ,\epsilon }{\min }\sum _{i=1}^n\left[{\epsilon }_i^T{w}_{\epsilon }^2{\epsilon }_i+{\delta }_i^T{w}_{\delta }^2{\delta }_i\right],$$

where $\epsilon$ and $\delta$ are $m$ and $n$ dimensional vectors and $w_{\epsilon }$ and $w_{\delta }$ are symmetric, positive diagonal matrices. Usually the inverse uncertainties of the data points are chosen as weights. We use the implementation ODRPACK [2].

There are different estimates of the covariance matrix of the fitted parameters $\beta$. Most of them are based on the linearization method which assumes that the nonlinear function can be adequately approximated at the solution by a linear model. Here, we use an approximation where the covariance matrix associated with the parameter estimates is based ${\left(J^{T}J\right)}^{-1}$, where $J$ is the Jacobian matrix of the x and y residuals, weighted by the triangular matrix of the Cholesky factorization of the covariance matrix associated with the experimental data. ODRPACK uses the following implementation [1]

$$\widehat{V}={\widehat{\sigma }}^2\left[\sum _{i=1}^n\frac{\partial f(x_i+{\delta }_i;\beta )}{\partial {\beta }^T}{w}_{{\epsilon }_i}^2\frac{\partial f(x_i+{\delta }_i;\beta )}{\partial \beta }+\frac{\partial f(x_i+{\delta }_i;\beta )}{\partial {\delta }^T}{w}_{{\delta }_i}^2\frac{\partial f(x_i+{\delta }_i;\beta )}{\partial \delta }\right]$$

(64)

The residual variance ${\widehat{\sigma }}^2$ is estimated as

$${\widehat{\sigma }}^2=\frac{1}{n-p}\sum _{i=1}^n\left[{\left(y_i-f(x_i+\delta ;\beta )\right)}^T{w}_{{\epsilon }_i}^2\left(y_i-f(x_i+\delta ;\beta )\right)+{\delta }_i^T{w}_{{\delta }_i}^2{\delta }_i\right]$$

(65)

where $\beta \in {ℝ}^p$ and ${\delta }_i\in {ℝ}^m,i=1,\mathrm{\dots },n$ are the optimized parameters,

The Deming fit is a special case of orthogonal regression which can be solved analytically. It seeks the best fit to a linear relationship between the x- and y-values

$$y^{*}=a{x}^{*}+b,$$

(66)

by minimizing the weighted sum of (orthogonal) distances of datapoints from the curve

$$S=\sum _{i=1}^n\frac{1}{{\sigma }_{ϵ}^2}{(y_i-a{x}_i^{*}-b)}^2+\frac{1}{{\sigma }_{\eta }^2}{(x_i-x_i^{*})}^2,$$

with respect to the parameters $a$, $b$, and $x_i^{*}$. The weights are the variances of the errors in the x-variable (${\sigma }_{\eta }^2$) and the y-variable (${\sigma }_{ϵ}^2$). It is not necessary to know the variances themselves, it is sufficient to know their ratio

$$\delta =\frac{{\sigma }_{ϵ}^2}{{\sigma }_{\eta }^2}.$$

(67)

The solution is

	$a$	$=$	${\displaystyle \frac{1}{2s_{xy}}}\left[s_{yy}-\delta s_{xx}\pm \sqrt{{(s_{yy}-\delta s_{xx})}^2+4\delta s_{xy}^2}\right]$		(68)
	$b$	$=$	$\overline{y}-a\overline{x}$		(69)
	$x_i^{*}$	$=$	$\mathrm{ }{x}_i+{\displaystyle \frac{a}{\delta +a^2}}\left(y_i-b-a{x}_i\right),$		(70)

where

	$\overline{x}$	$=$	${\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}{x}_i$		(71)
	$\overline{y}$	$=$	${\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}{y}_i$		(72)
	$s_{xx}$	$=$	${\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}{(x_i-\overline{x})}^2$		(73)
	$s_{yy}$	$=$	${\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}{(y_i-\overline{y})}^2$		(74)
	$s_{xy}$	$=$	${\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}(x_i-\overline{x})(y_i-\overline{y}).$		(75)

We seek the best fit

$$y=a{x}^{3/2}+b,$$

(76)

by minimizing the sum of (vertical) distances of datapoints from the curve

$$S=\sum _{i=1}^n{(y_i-a{x}_i^{3/2}-b)}^2,$$

with respect to the parameters $a$, $b$. The solution is

	$a$	$=$	${\displaystyle \frac{\overline{x^{3/2}y}-\overline{x^{3/2}}\overline{y}}{\overline{x^3}-{\left(\overline{x^{3/2}}\right)}^2}}$		(77)
	$b$	$=$	$\overline{y}-a\overline{x^{3/2}}$		(78)

where

	$\overline{x^{3/2}y}$	$=$	${\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}{x}_i^{3/2}{y}_i$		(79)
	$\overline{x^{3/2}}$	$=$	${\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}{x}_i^{3/2}$		(80)
	$\overline{x^3}$	$=$	${\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}{x}_i^3$		(81)
	$\overline{y}$	$=$	${\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}{y}_i$		(82)