3/5/2015

OrderstatisticWikipedia,thefreeencyclopedia

Orderstatistic
FromWikipedia,thefreeencyclopedia

Instatistics,thekthorderstatisticofastatisticalsampleisequaltoitskthsmallestvalue.[1]
Togetherwithrankstatistics,orderstatisticsareamongthemostfundamentaltoolsinnon
parametricstatisticsandinference.
Importantspecialcasesoftheorderstatisticsaretheminimumandmaximumvalueofasample,
and(withsomequalificationsdiscussedbelow)thesamplemedianandothersamplequantiles.
Whenusingprobabilitytheorytoanalyzeorderstatisticsofrandomsamplesfromacontinuous
distribution,thecumulativedistributionfunctionisusedtoreducetheanalysistothecaseoforder
statisticsoftheuniformdistribution.

Contents

Probabilitydensityfunctionsofthe
orderstatisticsforasampleofsize
n=5fromanexponentialdistribution
withunitscaleparameter.

1Notationandexamples
2Probabilisticanalysis
2.1Probabilitydistributionsoforderstatistics
2.1.1Orderstatisticssampledfromauniformdistribution
2.1.2Thejointdistributionoftheorderstatisticsoftheuniformdistribution
2.1.3OrderstatisticssampledfromanErlangdistribution
2.1.4Thejointdistributionoftheorderstatisticsofanabsolutelycontinuous
distribution
3Application:confidenceintervalsforquantiles
3.1Asmallsamplesizeexample
3.2Largesamplesizes
3.2.1Proof
4Dealingwithdiscretevariables
5Computingorderstatistics
6Seealso
6.1Examplesoforderstatistics
7References
8Externallinks

Notationandexamples
Forexample,supposethatfournumbersareobservedorrecorded,resultinginasampleofsize4.ifthesamplevaluesare
6,9,3,8,
theywillusuallybedenoted

wherethesubscriptiin indicatessimplytheorderinwhichtheobservationswererecordedandisusuallyassumednottobesignificant.
Acasewhentheorderissignificantiswhentheobservationsarepartofatimeseries.
Theorderstatisticswouldbedenoted

wherethesubscript(i)enclosedinparenthesesindicatesthei thorderstatisticofthesample.
Thefirstorderstatistic(orsmallestorderstatistic)isalwaystheminimumofthesample,thatis,

http://en.wikipedia.org/w/index.php?title=Order_statistic&printable=yes

1/6

3/5/2015

OrderstatisticWikipedia,thefreeencyclopedia

where,followingacommonconvention,weuseuppercaseletterstorefertorandomvariables,andlowercaseletters(asabove)torefer
totheiractualobservedvalues.
Similarly,forasampleofsizen,then thorderstatistic(orlargestorderstatistic)isthemaximum,thatis,

Thesamplerangeisthedifferencebetweenthemaximumandminimum.Itisclearlyafunctionoftheorderstatistics:

Asimilarimportantstatisticinexploratorydataanalysisthatissimplyrelatedtotheorderstatisticsisthesampleinterquartilerange.
Thesamplemedianmayormaynotbeanorderstatistic,sincethereisasinglemiddlevalueonlywhenthenumbernofobservationsis
odd.Moreprecisely,ifn=2m+1forsomem,thenthesamplemedianis
andsoisanorderstatistic.Ontheotherhand,whenn
iseven,n=2mandtherearetwomiddlevalues,
and
,andthesamplemedianissomefunctionofthetwo(usuallythe
average)andhencenotanorderstatistic.Similarremarksapplytoallsamplequantiles.

Probabilisticanalysis
GivenanyrandomvariablesX1,X2...,Xn,theorderstatisticsX(1),X(2),...,X(n)arealsorandomvariables,definedbysortingthevalues
(realizations)ofX1,...,Xninincreasingorder.
WhentherandomvariablesX1,X2...,Xnformasampletheyareindependentandidenticallydistributed.Thisisthecasetreatedbelow.In
general,therandomvariablesX1,...,Xncanarisebysamplingfrommorethanonepopulation.Thentheyareindependent,butnot
necessarilyidenticallydistributed,andtheirjointprobabilitydistributionisgivenbytheBapatBegtheorem.
Fromnowon,wewillassumethattherandomvariablesunderconsiderationarecontinuousand,whereconvenient,wewillalsoassume
thattheyhaveaprobabilitydensityfunction(thatis,theyareabsolutelycontinuous).Thepeculiaritiesoftheanalysisofdistributions
assigningmasstopoints(inparticular,discretedistributions)arediscussedattheend.

Probabilitydistributionsoforderstatistics
Inthissectionweshowthattheorderstatisticsoftheuniformdistributionontheunitintervalhavemarginaldistributionsbelongingtothe
Betadistributionfamily.Wealsogiveasimplemethodtoderivethejointdistributionofanynumberoforderstatistics,andfinally
translatetheseresultstoarbitrarycontinuousdistributionsusingthecdf.
Weassumethroughoutthissectionthat
isarandomsampledrawnfromacontinuousdistributionwithcdf
.
Denoting
weobtainthecorrespondingrandomsample
fromthestandarduniformdistribution.Notethatthe
orderstatisticsalsosatisfy
.
Orderstatisticssampledfromauniformdistribution
Theprobabilityoftheorderstatistic

fallingintheinterval

isequalto[2]

thatis,thekthorderstatisticoftheuniformdistributionisaBetarandomvariable.[2][3]

Theproofofthesestatementsisasfollows.For
tobebetweenuandu+du,itisnecessarythatexactlyk1elementsofthesample
aresmallerthanu,andthatatleastoneisbetweenuandu+du.Theprobabilitythatmorethanoneisinthislatterintervalisalready
,sowehavetocalculatetheprobabilitythatexactlyk1,1andnkobservationsfallintheintervals
,
and
respectively.Thisequals(refertomultinomialdistributionfordetails)

andtheresultfollows.
Themeanofthisdistributionisk/(n+1).

http://en.wikipedia.org/w/index.php?title=Order_statistic&printable=yes

2/6

3/5/2015

OrderstatisticWikipedia,thefreeencyclopedia

Thejointdistributionoftheorderstatisticsoftheuniformdistribution
Similarly,fori<j,thejointprobabilitydensityfunctionofthetwoorderstatisticsU(i)<U(j)canbeshowntobe

whichis(uptotermsofhigherorderthan
intervals
,
,

)theprobabilitythati1,1,j1i,1andnjsampleelementsfallinthe
,
respectively.

Onereasonsinanentirelyanalogouswaytoderivethehigherorderjointdistributions.Perhapssurprisingly,thejointdensityofthen
orderstatisticsturnsouttobeconstant:

Onewaytounderstandthisisthattheunorderedsampledoeshaveconstantdensityequalto1,andthattherearen!differentpermutations
ofthesamplecorrespondingtothesamesequenceoforderstatistics.Thisisrelatedtothefactthat1/n!isthevolumeoftheregion
.
OrderstatisticssampledfromanErlangdistribution
TheLaplacetransformoforderstatisticssampledfromanErlangdistributionviaapathcountingmethod.[4]
Thejointdistributionoftheorderstatisticsofanabsolutelycontinuousdistribution
IfFXisabsolutelycontinuous,ithasadensitysuchthat

,andwecanusethesubstitutions

and

toderivethefollowingprobabilitydensityfunctions(pdfs)fortheorderstatisticsofasampleofsizendrawnfromthedistributionofX:

where
where

Application:confidenceintervalsforquantiles
Aninterestingquestionishowwelltheorderstatisticsperformasestimatorsofthequantilesoftheunderlyingdistribution.

Asmallsamplesizeexample
Thesimplestcasetoconsiderishowwellthesamplemedianestimatesthepopulationmedian.
Asanexample,considerarandomsampleofsize6.Inthatcase,thesamplemedianisusuallydefinedasthemidpointoftheinterval
delimitedbythe3rdand4thorderstatistics.However,weknowfromtheprecedingdiscussionthattheprobabilitythatthisinterval
actuallycontainsthepopulationmedianis

Althoughthesamplemedianisprobablyamongthebestdistributionindependentpointestimatesofthepopulationmedian,whatthis
exampleillustratesisthatitisnotaparticularlygoodoneinabsoluteterms.Inthisparticularcase,abetterconfidenceintervalforthe
medianistheonedelimitedbythe2ndand5thorderstatistics,whichcontainsthepopulationmedianwithprobability

http://en.wikipedia.org/w/index.php?title=Order_statistic&printable=yes

3/6

3/5/2015

OrderstatisticWikipedia,thefreeencyclopedia

Withsuchasmallsamplesize,ifonewantsatleast95%confidence,oneisreducedtosayingthatthemedianisbetweentheminimumand
themaximumofthe6observationswithprobability31/32orapproximately97%.Size6is,infact,thesmallestsamplesizesuchthatthe
intervaldeterminedbytheminimumandthemaximumisatleasta95%confidenceintervalforthepopulationmedian.

Largesamplesizes
Fortheuniformdistribution,asntendstoinfinity,thepthsamplequantileisasymptoticallynormallydistributed,sinceitisapproximated
by

ForageneraldistributionFwithacontinuousnonzerodensityatF1(p),asimilarasymptoticnormalityapplies:

wherefisthedensityfunction,andF1isthequantilefunctionassociatedwithF.Oneofthefirstpeopletomentionandprovethisresult
wasFrederickMostellerinhisseminalpaperin1946.[5]Furtherresearchleadinthe1960stotheBahadurrepresentationwhichprovides
informationabouttheerrorbounds.
Aninterestingobservationcanbemadeinthecasewherethedistributionissymmetric,andthepopulationmedianequalsthepopulation
mean.Inthiscase,thesamplemean,bythecentrallimittheorem,isalsoasymptoticallynormallydistributed,butwithvariance2/n
instead.Thisasymptoticanalysissuggeststhatthemeanoutperformsthemedianincasesoflowkurtosis,andviceversa.Forexample,the
medianachievesbetterconfidenceintervalsfortheLaplacedistribution,whilethemeanperformsbetterforXthatarenormally
distributed.
Proof
Itcanbeshownthat

where

withZibeingindependentidenticallydistributedexponentialrandomvariableswithrate1.SinceX/nandY/nareasymptoticallynormally
distributedbytheCLT,ourresultsfollowbyapplicationofthedeltamethod.

Dealingwithdiscretevariables
Suppose
arei.i.d.randomvariablesfromadiscretedistributionwithcumulativedistributionfunction
probabilitymassfunction
.Tofindtheprobabilitiesofthe orderstatistics,threevaluesarefirstneeded,namely

Thecumulativedistributionfunctionofthe

Similarly,

and

orderstatisticcanbecomputedbynotingthat

isgivenby

http://en.wikipedia.org/w/index.php?title=Order_statistic&printable=yes

4/6

3/5/2015

Notethattheprobabilitymassfunctionof

OrderstatisticWikipedia,thefreeencyclopedia

isjustthedifferenceofthesevalues,thatistosay

Computingorderstatistics
Theproblemofcomputingthekthsmallest(orlargest)elementofalistiscalledtheselectionproblemandissolvedbyaselection
algorithm.Althoughthisproblemisdifficultforverylargelists,sophisticatedselectionalgorithmshavebeencreatedthatcansolvethis
problemintimeproportionaltothenumberofelementsinthelist,evenifthelististotallyunordered.Ifthedataisstoredincertain
specializeddatastructures,thistimecanbebroughtdowntoO(logn).Inmanyapplicationsallorderstatisticsarerequired,inwhichcase
asortingalgorithmcanbeusedandthetimetakenisO(nlogn).MoresophisticatedmethodscanreducethetimetoO(n).

Seealso
Rankit
Boxplot
Concomitant(statistics)
FisherTippettdistribution
BapatBegtheoremfortheorderstatisticsofindependentbutnotnecessarilyidenticallydistributedrandomvariables
Bernsteinpolynomial
Lestimatorlinearcombinationsoforderstatistics

Examplesoforderstatistics
Samplemaximumandminimum
Quantile
Percentile
Decile
Quartile
Median

References
1. ^David,H.A.Nagaraja,H.N.(2003)."OrderStatistics".WileySeriesinProbabilityandStatistics.doi:10.1002/0471722162
(https://dx.doi.org/10.1002%2F0471722162).ISBN9780471722168.
2. ^abGentle,JamesE.(2009),ComputationalStatistics(http://books.google.com/books?id=mQ5KAAAAQBAJ&pg=PA63),Springer,p.63,
ISBN9780387981444.
3. ^Jones,M.C.(2009),"Kumaraswamysdistribution:Abetatypedistributionwithsometractabilityadvantages",StatisticalMethodology6(1):
7081,doi:10.1016/j.stamet.2008.04.001(https://dx.doi.org/10.1016%2Fj.stamet.2008.04.001),"Asiswellknown,thebetadistributionisthe
distributionofthemthorderstatisticfromarandomsampleofsizenfromtheuniformdistribution(on(0,1))."
4. ^Hlynka,M.Brill,P.H.Horn,W.(2010)."AmethodforobtainingLaplacetransformsoforderstatisticsofErlangrandomvariables".Statistics
&ProbabilityLetters80:9.doi:10.1016/j.spl.2009.09.006(https://dx.doi.org/10.1016%2Fj.spl.2009.09.006).
5. ^Mosteller,Frederick(1946)."OnSomeUseful"Inefficient"Statistics"(http://projecteuclid.org/euclid.aoms/1177730881).Annalsof
MathematicalStatistics(InstituteofMathematicalStatistics)17(4):377408.doi:10.1214/aoms/1177730881
(https://dx.doi.org/10.1214%2Faoms%2F1177730881).RetrievedFebruary26,2015.

Sefling,R.J.(1980).ApproximationTheoremsofMathematicalStatistics.NewYork:Wiley.ISBN0471024031.
http://en.wikipedia.org/w/index.php?title=Order_statistic&printable=yes

5/6

3/5/2015

OrderstatisticWikipedia,thefreeencyclopedia

Externallinks
Orderstatistics(http://planetmath.org/OrderStatistics)atPlanetMath.org.RetrievedFeb02,2005
Weisstein,EricW.,"OrderStatistic"(http://mathworld.wolfram.com/OrderStatistic.html),MathWorld.RetrievedFeb02,2005
Dr.SusanHolmesOrderStatistics(http://wwwstat.stanford.edu/~susan/courses/s116/node79.html)RetrievedFeb02,2005
C++sourceDynamicOrderStatistics(https://github.com/xtaci/algorithms/blob/master/include/dos_tree.h)
Retrievedfrom"http://en.wikipedia.org/w/index.php?title=Order_statistic&oldid=648978881"
Categories: Nonparametricstatistics Summarystatistics Permutations
Thispagewaslastmodifiedon26February2015,at19:33.
TextisavailableundertheCreativeCommonsAttributionShareAlikeLicenseadditionaltermsmayapply.Byusingthissite,you
agreetotheTermsofUseandPrivacyPolicy.WikipediaisaregisteredtrademarkoftheWikimediaFoundation,Inc.,anonprofit
organization.

http://en.wikipedia.org/w/index.php?title=Order_statistic&printable=yes

6/6