Sufficiency .

Sufficient statistic
outline
{ Properties of sufficient
Exponential family .
statistic

Part 1 :
sufficient statistic
• Recall that a statistic -1=-11×4×4 Nn ) is form
' -

of data reduction e-
g. An estimator § uses a single number
.

to reflect all information contained in the data ( ✗ ii., xn )

about the parameter 0 .

( K dim
-
estimator § k many )
uses
Question : Is there
of information by
any loss data reduction ?
Cant preserve all information about 0 via a
few number
of statistics ?
1¥ Not .

always .
 ✗, Xz , ✗n Ed Berto ) O Coin
e.
g. c-
-
-

, ,
,
.

"
( i -

e .

pmf of Xi is fix :o) = 0×4-05 ,

✗ c- 9913 )
consider statistic Y =
¥
,
Xi .

We know that Y ~ 13inch , 0 ) ( Binomial disks ) .

Let's compute the joint conditional disks of (✗ " KIND

given Y y
=
.

1PM
HIED
R¥,¥-

MH-mxj-EIEE.EE?Ii--Y#pmfvfBincmO
=
RCAI B) =

) at
y .
If y t Fen Ki ,
then BLAND -

-
O and RCA 1B ) -0 .

If y =
Fg Xi ,
then
=MH h-iXn=x#
RC Al B) =
RYtpY =
Mptp, "" B) .

Chindits II .

oxictogtxi
=
Hasnt
Fiona
-

B) Hetchy
=

Ty, .

if y 't Enixi
f Ty
0
PCI Ely y)
-
-
-
.

-
-

,
if y
-

ixi .

This conditional dish does not

depend on O !
 Implications For any other
:
① statistic U UCHI Nn )
-

• =
,

the conditional dish of U given Y y -

,
also does
not depend on it .
i. e. there is no extra
information
about 0 in U if we are
given the knowledge of Y .

knowing Y sufficientis .

⑦ . Given the knowledge of Y ,

we can generate another
'
data set Xi Xi by sampling ¥
✗n from the
'

;
-

, ,

conditional distln
of ☒ I ' Y=Y ( since this .
is a known
distln ) Now.

☒ and I , are two data sets from

distln with same site
the same
sample ,
they should
contain the same information about 0 .

knowing Y is sufficient .
 Def ( sufficient statistic ) Let Yuki Xn denote a
. -

• i
.

random sample
of size h from a parametric distribution
family fix :o) d C- Let Y uchi Xn ) be i
-

=
.
,

statistic with
a
pdflpmf fycy ) Y is called .
a

sufficient statistic for 0 if the conditional distribution

of ☒ given Y does not
depend on 0 ,
or ,

=
Haim

does 0 and
not
depend on
for all Of ,

✗ ink ,
-
"

/ Rn C- DX / .

space of Xi 's )
iidr.is with
e-
g- Let Xi , K ,
-
-

, Xn be pdf
" -0 )
fix )= e- Leo , a) Cx )
.
 " -0 )

.io:*:: it
.io
"
-

fax ) e Leo ik )

¥÷
-

,
-

is
,
.

.
.
in man .

:* =

f
1
O
if
if
2170
XE O
k
O

show that Y =

Min Xi is a
sufficient statistic .

" -

t
C statistic )
f- y ly ) = n
-

fly ) [ I -

Fly )] order

It is to obtain that
easy
if
fi's
gogod
xeo
Fact -

fluid w =

'w
Mdw if
-

e- X > O
O)
)
H
[I Ico she )
-
-

e
=
-
-

,
 -
n
'T "
fyiy) =
ne
-

2,0 ,
is > ly )

=¥i:÷÷÷¥÷÷→÷× ,

=é"I¥:¥¥÷÷¥÷
Xi 0 Fi

(
is 7

)
numerator >
since non zero ,
-

miin Xi > 0 Ico ,

a) Cmiinxi ) =3

É ✗ it
nm.in xi
=
tn e ,

does not depend on d. '

f- Min Xi is sufficient
for 0 .
Questions Is any general way find
there to such a

sufficient statistic ?
•
Theorem ( Neyman -1hm 7.2.1 ) , .

Let Xi Xz , Xn be a random
sample from fix ;D )
- -

i
, ,

The statistic Y ,Xn ) is

Of uchi sufficient for
= .

0 if and if we
only can find two functions g and h ,

such
that
( lo ) = f- Hi :O) =
GIUCXI ,
- "

Nn ) ; 0 ] .
hcxnki.im ,

where he does not

depend d. on

That is ,
the likelihood function depends on I only
transformed statistic nah , Xy Hn )
through the
"

, .

Let Xi , Xy Xn be iid with

pdf
g.
e- i
-

" -0 )
f-a) =
e- 2 co , a) CX ) .
 Find a
sufficient statistic for a .

II fcxiiot II '
"" -
o
)
-

Lue ) -

,
e scans , cxi
,

"'
- Z Xi
=
e 2am cnn.in Xi ) e
- -

gcmiinxi ; ol hike )
where
gcu ; ) em Icu
v =
,
mln )

Xn )
-

7. Xi
hcxii -

i.
=
e .

From Neyman 's theorem .

Min Xi is sufficient fence .

Part 2 :

properties of sufficient statistic .

I .

Any one -
to -
one transformation of a
sufficient
 statistic is also a
sufficient statistic ( no loss of information
.

2. If Y =
UCI ) is a
sufficient statistic for 0 ,

then the MLE Ñ for 0 is a function of Y .

To see this :
Llo ) =

GI Yi 0 ] .

HII )

Imre =
argmax LIU ) =
atgmaxg-y.io ]
Of Of

e-
g. U the
previous example ,

Hi
40 ) =
end 210 ,
is ) ( my:b Xi ) e-

1 µ z
ÑM 4- =
Y -
- min ✗ i
i
?⃝
 *
3. ( Rao -
Blackwell izahm ) Theorem 7.3.1

Any estimator can be improved by conditioning

on a
sufficient statistic .

Part 3- .

Exponential family .
C Chapter 7- 5)
•

Def : An exponential distln

family is a
family of
distributions whose pdtlpmf takes the
following
form :

-117101kW -151×1+810 )]
folio )
{ exp Dx
=
.
✗ c-

0 otherwise .

range DX is independent of d.
where the

Most commonly used parametric families belong to the

 exponential family .

g. Normal distln

N.NL/MiF1.pctRfcxiM- #-g.expl-4-I )- expfM-ozX-

e- with known

¥2 -

1¥.
-

tzlug ears )
¥0
war w

pyulkcx) scx )

DX =
C -
p , is )

Binomial dish . Bin chip .

PEND
(1)
" ""
f- cxip> =

p Ctp
 =

exp ( 105¥
in
-

an
X + lug ( %)
~
+
nlogctp
w
)
Pcp) K c) say ftp.
DX =
{ 0,112 ' -

in } .

Other examples :
Bernoulli , negative Binomial ,

Poisson ,
Gamma ,
exponential , normal .

•
Theorem : Let Xi , Xi, - -

, ✗n be a random
sample
from the above exponential family .
Then
Y =
IÉ,
Klxi ) is sufficient force .

Prof :
LW ) =
¥ ,
exp ( Plo ) Kcxi ) + Slxi ) -18101 )
=
exp ( plo ) IKCXijtnfloD.expl-E.SK/iD
in my
☐
 '
e-
g. normal distln Nyu ,r4 with known r

É
KCX)=K .

EY Kai )=
it
✗i is sufficient
for µ
( one-to-one map preserves off )
.

'
E. ✗ i I is sufficient
-

n =

Binomial distln 13inch /

p) pt 10117 ,

Kat .
x ?_? ,
Kalil -

¥,
Xi is sufficient ferp .