Deming regression

MethComp package
May 2007

Anders Christian Jensen Steno Diabetes Center, Gentofte, Denmark

[email protected]
 Contents
1 Introduction 1

2 Deming regression 1

3 The likelihood function 1

4 Solving for ξi 2

5 Solving for α 2

6 Solving for β 3

7 Solving for ξi - again 6

8 Solving for σ 2 6

9 Summing up 7

10 The Deming function 8

Deming regression 1

1 Introduction
This document is related to the Deming function in the package MethComp and contains the
derivation of the maximum likelihood estimates related to the Deming regression model. It
is based on the book ’Models in regression and related topics’ (chapter three), from 1969
by Peter Sprent, but with more detailed calculations included.

2 Deming regression
The mathematical model η = α + βξ describes a linear relationship between two variables ξ
and η. Observations x and y of two variables are usually desribed by a regression of y on x
where x is assumed to be observed without error (or, equivantly using the conditional
distribution of y given x). In linear regression with observations subject to additive
random variation on both x and y and observed values for individuals (xi , yi ), i = 1, . . . , n,
a model may be written

xi = ξi + exi ,

yi = ηi + eyi = α + βξi + eyi ,

where exi and eyi denotes the random part of the model. This is known as a functional
relationship because the ξi ’s are assumed to be fixed parameters, as oppposed to a
structural relationship where some distribution for the ξi ’s is assumed. In the following it is
assumed that the exi s are iid with eyi ∼ N (0, σ 2 ), and that the eyi s are iid with
eyi ∼ N (0, λσ 2 ), for some λ > 0. Furthermore exi is assumed to be independent of eyi .
The aim of this document is to derive the maximum likelihood estimates for α, β, ξi and
2
σ in the functional model stated above.

3 The likelihood function

The likelihood function fx1 ,x2 ,...,xn ,y1 ,y2 ,...,yn (α, β, ξ1 , ξ2 , . . . , ξn , σ 2 ) denoted f is
n
(xi − ξi )2 (yi − α − βξi )2
   
Y
1
2 −2

1
2 −2
f = 2πσ exp − 2πλσ exp −
i=1
2σ 2 2λσ 2

and the loglikelihood, denoted L, is

n
X 1
(xi − ξi )2 1
(yi − α − βξi )2
L = − log 2πσ 2 − − log 2πλσ 2
−
i=1
2 2σ 2 2 2λσ 2
Pn 2
Pn
n
n i=1 (xi − ξi ) i=1 (yi− α − βξi )2
= − log 4π 2 − log λσ 4 −


− .
2 2 2σ 2 2λσ 2
It follows that the likelihood function is not bounded from above when σ 2 goes to 0, so in
the following it is assumed that σ 2 > 0.
2 4 Solving for ξi

4 Solving for ξi
Differentiation of L with respect to ξi gives

 Pn 2
Pn 2

∂L ∂ i=1 (xi − ξi ) i=1 (yi − α − βξi )
= − −
∂ξi ∂ξi 2σ 2 2λσ 2

(xi − ξi ) β(yi − α − βξi )

= + .
σ2 λσ 2

∂L
Setting ∂ξi
equal to zero yields

∂L λσ 2 xi + βσ 2 yi − βασ 2 λxi + β(yi − α)

= 0 ⇒ ξi = 2 2 2
= . (1)
∂ξi λσ + β σ λ + β2

So to estimate ξi , estimates for β and α are needed. Therefore focus is turned to the
derivation of α̂.

5 Solving for α
Differentiation of L with respect to α gives

 Pn 2

∂L ∂ i=1 (yi − α − βξi )
= −
∂α ∂α 2λσ 2
Pn
i=1 (yi− α − βξi )
= ,
λσ 2

∂L
and putting ∂α
equal to zero yields

n
∂L 1X
=0 ⇒ α= (yi − βξi ).
∂α n i=1
Deming regression 3

Now one can use (1) to dispense with ξi

n
1X
α = (yi − βξi )
n i=1

n  
1X λxi + β(yi − α)
= yi − β
n i=1 λ + β2

n 
β 2α

1X λxi + βyi
= yi − β +
n i=1 λ + β2 λ + β2

m
n 
β2
  
1X λxi + βyi
α 1− = yi − β
λ + β2 n i=1 λ + β2

n  
β2
 
1X βλ
= yi 1 − − xi
n i=1 λ + β2 λ + β2

m
n 
βλ λ + β 2

1X
α = y i − xi
n i=1 λ + β2 λ

n
1X
= (yi − xi β)
n i=1

= y − xβ.
Hence the estimate for α becomes
α̂ = y − xβ̂.

6 Solving for β
Differentiation of L with respect to β gives
 Pn 2
 Pn
∂L ∂ (yi − α − βξ i ) (yi − α − βξi )ξi
= − i=1 2
= i=1 .
∂β ∂β 2λσ λσ 2
∂L
Setting ∂β
equal to zero yields
n
∂L X
=0⇔ (yi − α − βξi )ξi = 0,
∂β i=1
4 6 Solving for β

and using (1)

n
X
0 = (yi − α − βξi )ξi
i=1

n  
X λxi + β(yi − α) λxi + β(yi − α)
= yi − α − β .
i=1
λ + β2 λ + β2

This implies that

n 
X  
0 = (yi − α)(λ + β 2 ) − βλxi − β 2 (yi − α) λxi + β(yi − α)
i=1

n
X
= λ2 xi (yi − α) + β 2 λxi (yi − α) − βλ2 x2i − β 2 λxi (yi − α) +
i=1

n
X
βλ(yi − α)2 + β 3 λ(yi − α)2 − β 2 λxi (yi − α) − β 3 (yi − α)2
i=1

n
X  n
X  n
X 
2 2
= −β λ xi yi − βλ x2i +λ 2
xi y i
i=1 i=1 i=1

n
X  n
X  n
X 
+β 2 λα xi + βλ (yi − α)2 − λ2 α xi .
i=1 i=1 i=1

Dividing with λ and using the fact that α = y. − x.β it is seen that
n
X  n
X  n
X  n
X 
0 = −β 2 xi yi − βλ x2i + λ xi yi + β 2 (y. − x.β) xi
i=1 i=1 i=1 i=1

n
X n

2  X 
+β yi − (y. − x.β) − λ(y. − x.β) xi
i=1 i=1

n
X  n
X  n
X  n
X 
= −β 2 xi yi − βλ x2i + λ xi yi + β 2 y. xi
i=1 i=1 i=1 i=1

n
X  n
X  n
X 
3
−β x.β xi + β yi2 +β (y. − x.β)2
i=1 i=1 i=1

n
X  n
X  n
X 
−2β yi (y. − x.β) − λy. xi + λx.β xi .
i=1 i=1 i=1
Deming regression 5

Splitting up the sums even more gives

n
X  n
X  n
X  n
X  n
X 
2
0 = −β xi yi − βλ x2i +λ 2
xi yi + β y. 3
xi − β x.β xi
i=1 i=1 i=1 i=1 i=1

n
X  n
X  n
X  n
X  n
X 
+β yi2 + β y.2 + β (x.β)2 − 2β y.x.β − 2β yi y.
i=1 i=1 i=1 i=1 i=1

n
X  n
X  n
X 
+2β yi x.β − λy. xi + λx.β xi .
i=1 i=1 i=1

Finally the terms are sorted and collected according to powers of β:

n n
!
X X
0 = β3 x.2 − x. xi
i=1 i=1

n n n n
!
X X X X
+β 2 y. xi − xi yi − 2 y.x. + 2 yi x.
i=1 i=1 i=1 i=1

n n n n n
!
X X X X X
+β yi2 − λ x2i + y.2 − 2 yi y. + λx. xi
i=1 i=1 i=1 i=1 i=1

n n
!
X X
+λ xi yi − y. xi .
i=1 i=1

Since
Pn
x.2 − x. ni=1 xi = 0
P
• i=1

• y. ni=1 xi − ni=1 xi yi − 2 ni=1 y.x. + 2 ni=1 yi x. = −SPDxy

P P P P
Pn 2 Pn 2 Pn 2
Pn Pn
• i=1 yi − λ i=1 xi + i=1 y. − 2 i=1 yi y. + λx. i=1 xi = SSDy − λSSDx
Pn Pn
• i=1 xi yi − y. i=1 xi = SPDxy

it is clear that the derivation of β comes down to solve

−β 2 SPDxy + β(SSDy − λSSDx ) + λSPDxy = 0. (2)
For SPDxy 6= 0 this implies that
p
−(SSDy − λSSDx ) ± (SSDy − λSSDx )2 − 4(−SPDxy )λSPDxy
β =
−2SPDxy
q
SSDy − λSSDx ± (SSDy − λSSDx )2 + 4λSPD2xy
= .
2SPDxy
6 7 Solving for ξi - again

q
Since SSDy − λSSDx ≤ (SSDy − λSSDx )2 + 4λSPD2xy there is always a positive and a
negative solution to (2). The desired solution should always have the same sign as SPDxy ,
hence the solution with the positive numerator is selected. Therefore
q
SSDy − λSSDx + (SSDy − λSSDx )2 + 4λSPD2xy
β̂ = .
2SPDxy

7 Solving for ξi - again

With estimates for β and α it is now possible to estimate ξi using (1):

λxi + β̂(yi − α̂)

ξˆi = .
λ + β̂ 2

8 Solving for σ 2
Differentiation of L with respect to σ 2 gives
Pn Pn
− xii )2 − α − βξi )2
 
∂L ∂ n i=1 (xi i=1 (yi
2
= − log(λσ 4 ) − −
∂σ ∂σ 2 2 2σ 2 2λσ 2
Pn 2
Pn 2
−nσ 2 i=1 (x i − xi i ) i=1 (yi − α − βξi )
= + +
σ4 2σ 4 2λσ 4

−2λnσ 2 + λ ni=1 (xi − xii )2 + ni=1 (yi − α − βξi )2

P P
= ,
2λσ 4
∂L
and setting ∂σ 2
equal to zero yields

n n
∂L 2
X
2
X
= 0 ⇒ −2λnσ + λ (x i − xi i ) + (yi − α − βξi )2 = 0
∂σ 2 i=1 i=1

Pn
− ξi )2 + ni=1 (yi − α − βξi )2
P
2 λ i=1 (xi
⇒ σ = .
2λn

To get a central estimate of σ 2 one must divide by n − 2 instead of 2n since there are n + 2
parameters to be estimated, namely ξ1 , ξ2 , . . . , ξn , α and β. Hence the degrees of freedom
are 2n − (n + 2) = n − 2. Therefore

− ξˆi )2 + ni=1 (yi − α̂ − β̂ ξˆi )2

Pn P
2 λ i=1 (xi
σ̂ = .
2λ(n − 2)
Deming regression 7

9 Summing up

α̂ = y − xβ̂
q
SSDy − λSSDx + (SSDy − λSSDx )2 + 4λSPD2xy
β̂ =
2SPDxy
s P
λ ni=1 (xi − ξˆi )2 + ni=1 (yi − α̂ − β̂ ξˆi )2
P
σ̂ =
2λ(n − 2)
λxi + β̂(yi − α̂)
ξˆi =
λ + β̂ 2
These formula are implemented in the Deming function in the MethComp package.
8 10 The Deming function

10 The Deming function

Deming <-
function( x, y, vr=sdr^2, sdr=sqrt(vr), boot=FALSE, keep.boot=FALSE, alpha=0.05 )
{
if( missing( vr ) & missing( sdr ) ) var.ratio <- 1
else var.ratio <- vr
vn <- c( deparse( substitute( x ) ),
deparse( substitute( y ) ) )
pn <- c( "Intercept", "Slope", paste( "sigma", vn, sep="." ) )
alfa <- alpha
dfr <- data.frame( x=x, y=y )
dfr <- dfr[complete.cases(dfr),]
x <- dfr$x
y <- dfr$y
n <- nrow( dfr )
SSDy <- var( y )*(n-1)
SSDx <- var( x )*(n-1)
SPDxy <- cov( x, y )*(n-1)
beta <- ( SSDy - var.ratio*SSDx +
sqrt( ( SSDy - var.ratio*SSDx )^2 +
4*var.ratio*SPDxy^2 ) ) / ( 2*SPDxy)
alpha <- mean( y ) - mean( x ) * beta
ksi <- ( var.ratio*x + beta*(y-alpha) )/(var.ratio+beta^2)
sigma.x <- ( var.ratio*sum( (x-ksi)^2 ) +
sum( (y-alpha-beta*ksi)^2 ) ) /
# The ML-estiamtes requires 2*n at this point bu we do not think we have that
# many observations so we stick to (n-2). Any corroboation from litterature?
( (n-2)*var.ratio )
sigma.y <- var.ratio*sigma.x
sigma.x <- sqrt( sigma.x )
sigma.y <- sqrt( sigma.y )
if( !boot ){
res <- c( alpha, beta, sigma.x, sigma.y )
names( res ) <- pn
res
}
else
{
if( is.numeric( boot ) ) N <- boot else N <- 1000
res <- matrix( NA, N, 4 )
for( i in 1:N )
{
wh <- sample( 1:n, n, replace=TRUE )
res[i,] <- Deming( x[wh], y[wh], vr=var.ratio, boot=FALSE )
}
ests <- cbind( c(alpha,beta,sigma.x, sigma.y),
se <- sqrt( diag( cov( res ) ) ),
t( apply( res, 2, quantile, probs=c(0.5,alfa/2,1-alfa/2 ), na.rm=T ) ) )
colnames( res ) <- rownames( ests ) <- pn
colnames( ests )<- c("Estimate", "S.e.(boot)", colnames(ests)[3:5] )
if(keep.boot)
{
print( ests )
invisible( res )
}
else
{
cat( vn[2], " = alpha + beta*", vn[1], "\n" )
ests
}
}
}