Scribd
Upload
601 views216 pages

Polak E. Optimization Algorithms and Consistent Approximations

A book on optimization

Uploaded by

Jimmy Pham Hoang Nhat
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF or read online on Scribd
601 views216 pages

Polak E. Optimization Algorithms and Consistent Approximations

A book on optimization

Uploaded by

Jimmy Pham Hoang Nhat
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF or read online on Scribd
You are on page 1/ 216
‘Tis bak cone alpothn ad deowtzston posed lt th eon ot aie progr, aii clio an epi cer pier: ‘ra mrt sn el alt ed roams wt ard wiht eran hoo fcr approximant ‘sang wine cowergercs lapping pols ames agotre ‘Stn sacar renirmrrograraing arin ssbb heh pe Moov a tenon er eatin eet nite oiiafe, opal cre Und ohapeepimizaten peters with very gral corsa ar ha com ‘neous ihc art rear Chapter Scones fees acted nopinzton fon age essere of eoutos ally condoned epttns lrrinmex clone at wel en ere hon ceencaet ppimaton anh ema mse ‘maton red opti eat plein anno This ook recent he fe eu wane inant pinaion {psn rots era ples wih stator and canal con is nd uty acco tae opoxnaons a ora mus axa ‘pony ttre tb cna tere sprite oe Severna ter end sets anoy ocean option acon nated ‘nana engneeing ard ecaroncs wert bok uel ae ‘lll 4e10d uonez}windo suopewijxosddy quays|suog pue sub 4H 6896 Applied Mathematical ‘Sciences Elijah Polak Optimization Algorithms and Consistent Approximations 6 Springer Applied Mathematical Sciences Volume 124 Springer Applied Mathematical Sciences Elijah Polak Optimization Algorithms and Consistent Approximations & Springer yo Pas Dip ole Eang comes erates Aterit npc ar Hat pina Tomy grandee: Preface ‘This tok del wath pti condos, and craton ch ges fo ola ropanig ot inne ote, ad opi ‘ol poblens The ulying read in te prs cme of an shat ‘eon. itn wich optimality condone expend int frm of eof pil ft, apse hrc by pot ost {a ale mamealspproimatns gure te slim of sine ‘pieson an opal ool rte sed within te et se ‘Sst spyreinations and sign inplenentioa ech. ‘Tron, necessary optimality conn fr opinion problens speed in Lagrage, Fol, o¢ Kana RanTocker pes forma ‘hacen wed for ama robime ansbgrai fo oon eo Tem We resent cal opty codons td show tha he ‘subd a pont antl fh poit ro nope etna Srna fmcton. ‘The ws of optimality nce at sve aes a pili etn a eed in an bt say of pine tims Second, many epiicsoe sgorihs canbe shown we seach tection dt oe oad in erating pinay fants, ths esablsing $s ckrtlatontip twee pal eos edge, Tid eta Ting opimaiy conan forty complex probens, sch optima cn to pote wh sna nd jt cons, sich ee ne pty fcions an he clea maar Inne, reatohp ‘rven pili onions oredr pens som se pinion and optimal cancel problems tecmss tar aay palit Tanton an smpoant at of ou tery of conten peo ons ht we tse fo cnc een. pment algeria se ‘nf pninn an epi st Bef atemping ta the pat vty of opinion sgridns ‘curate prset my src thoy of opinion alg. The ‘ep of his ory cea wit sly small mer of lgpetin el reed nemo oe eon mage for ich tps xe ence theorems. These horns hihi he i opens hat an pina ‘on algriton ast have wo be converge Ase rear wil sou 30 ‘ihm moses an convergence teerens St most easing epiisaton {hh leita ent provides perl ii Famevoi re say of teconepac: poet ‘nai cognitive es ey me fain te ease pie gn ere) oe pt Ses erm tenis oe art a “he eon ra er igi eay dl wih ie inne Sn, Wie mine popu igen fray rsa ‘sie ef nite popammig. oil te pe spinon otis eres copa om ce ns an open ‘Nes tas of fess pat, cbc ee) {matte spouted Oe asm npn ey pve “ehck ier er buing he pcon te apes, hat Sereling plone goes one “he i pat fee arti hey el wit coc ein Sos td i med som te epoca, te he ‘tineton pte ncn te ph dnc agen ‘Spleen, fe nt poms spc sh pea Sonepat bt nes, qos atm na ne ‘hoc etm on ye es farsa rte, ole by nants The ey roves con Sey eos eg th oa mney el rs Story pen fe gposintns poten veg pal mar ‘minis nt sana po pei, egal pen Sire te ncn oto by ard vn Rope ig teh ot points sppach a sgn acl vg clinic med fw ein tan. {ton renal ests conpuat propane te ait, te hay of cnt spproinnns joie ane ap ‘ita meal omega ent Te me ale ‘Sniet case potnaon, ete pois. ‘inn hs erst, debe opin we ot nase ‘Ah a a econ spit ct, al ewe Ces penny at ed om Be Cat prone 22 am Soe if be me alps ko iis nt ee pce fe nl pgs ‘Nain ii nd cones os fr ened opin Sone psd Sen 2 ve ter coed seas ‘ed in Sta 23, Fey the tery of esse appreniatons pinay condos napster on progr eben ren ins into cape whan opr on ona grins tounecans of welsa”ar say ie oso sense {emerging ad er prong spin, ch ney meds tel iodiel bur tc mets otal we er gris, Smwotst npn mend dg th ooo doe op i tc only tl sheesh Tie sion os yr grt = Chay snh eet ely ton, seh gus regan ‘ing mbes wa malo ne tsb of tres nib whch Nem ped {iiccay wine tansot opin toe ‘Optimal conn ad sgt or en option and ioctl potions eed be id orth per, A et {talon urs ia prt econ of pinay cot {2'cxpen opi, coion, ad pry bene spss {Enctped wine funevr fcoea opesion: Ass it Xt enfin pons opi ois poset en Stoo oft mer apt Gat como preci fronton nd ‘St nines opening softer a wou i pee Sesntpttintans apo fu coe nvsge oe c- ‘ia uzntm implemen, ates he meng gp eee {ers tan te tk of ting compte oe fr penn Sl copa alga nc iol, he th cpr sets a tl calcton of mate seu ne nang of lath Bonk Tee le ‘ou fon fea mu covet may oy vad pe ‘osmond an uly sd es ey. “he bony ening with sa i i ok enon, nd sont no nese fe ek hep, ‘cit tom ecnlepforma wach tote cy te gery of Gifs WELV¥C®sbroy noose MELEE SPEC? oan he ‘Stliy he vanaf bop ta ae nd ae ened ‘Setnkrineeed crying oot cane sah, “is wo pr cao os of cn ect sk. Te itt ova atm it wo Cape fs book see fo Shh nda cd or ny ge eye, oe tose, aan Cone fate Snes sink eon ct wer one ‘tv! onsen poring ado oa owl ame ot ‘Ser stand maa pound mh hg tr sk i westn ascot yar, oe ener pode come, Hee tk ile aa ex fr soosmerer pce nes oo pain, {nbc of dep of vege, pve ey wa owcinas ‘Te anor sgn in Bue, Jc Denis, Assn Dench, Jeph Deen Oe Mangan or Mord Rest vee. Anew Pip, Stepan Robinson Ten Rockall, Ane Ti, Riche Viner Vert War, Roger Wes, nl Jian Za, wp hve feed omnes on hit de of enkeravie on clr op Pal thse de Bley Bala Stephen Wright forte lp 'sclesing op he heey of ose appoint, o Daniel Ral forbs help declping x prsentaton of SOP mst he omy recor, 0 Darel Sermon fr aince ithe relon tl, fo Cae Kier, Dad Maye, Adan Schwan and Thoma Yan. nese ny ‘The ent of his Book eft he ato’ elihrton with David Mayne sd Clove Gooagn at well a with his forme te Tod Baer [Linin He Joe gs, Caos jr, Rote Klip Ml, Olas Prom, Adan Scar, Ane Tis, Yrs War, Thomas Yan aly. Lam pal my fl, Gln, fr her uot a ecung met ini oct swell ork noose mane ‘Te preparation of ls volume ele a gest smn of recarh which ssutd have en poe thot th pp ese ho he Natal Sr {hee Foe, hr Ar Face Ofte of Soatie Resend th Unies ‘of Caifomin a Beil. Tis sop spat sednow ged Contents Protace Conventions and Symbols 1 Unconstained Optimization 11 Optima Conans TInt Ta and Sand Onc Nees Coats U2. Stic omfons 113. The Comer Ce 12. Algrthn Malsand Convergence Coons 121 Geometry of Dest Metts 122 Base Agri Modes 123. The Wot snd Polk Suge Scan Theorems 124 ATasttepoe Mess 125. Aton inplomentica Tey 126 Rte of Convergence of eqns 127 Alpin ey 12. Gradient Motos TSI Metotf Stepet Desc 132 Amip Greet Metad 133. Rojed Get Metbod LA Newt's Method Til Thea New Meta 142. Glial Newion Meta or Conver Facto 143 Dace Newen Metod 144 Glta Newon Met foe Gene! Factons 185 ‘Thettaod Newton Meios 15 Methods of Conese Directions 153.” Decompotion of Que Fctons 152 Metudr f Coie Grant 153. Forma Extension Genel ants Si ThePobb ber Cougs Gent Alin USS. TheFitcher Rese Cojete Grad Netad 156, Parl Cnjrt Grado Mens 16 Qua Newton Maths 161 The Varn Mec Cnet 162 Scat Meth 163. Symmes Rai-One Updates 68 Symmes Ren Two Ups 165. he Comergace on Qua Fecons 1186. Ghtal Cavern on Ginx Pinos 17 One-Dimensional Opinion 171 Sect Meted Basel on Cai terpltion E72. The olden Section Seah 13. Mette int nets 1 Nevon's Met for Equations nd Ines 181" Mangan “rms Contin Quaicaon 2 Th Lc Newton Agen 13. Ghtal Newen Method 2 Pinte min tax and Constrained Optimization 21 Optinaity Contin or Min Max ZL” Fie Order Condoms 212 Opinay ance 213 Sion Onder Cinfans 22. Optimality Conditions or Contained Optinizaton 22:1" Psst Onker Opin Codon ort 222 AnOpinaiy Panton orlCP 223. Secont Order Conor CP 224. Fist Or Optmnity Condos fr ECP 2s ty Conlon for ECP 23. Algom Medel and Convergence Coons 6 im is 3 7 231 Alpen Mose fort 232 Alps Mosel or IEC 24. Fist Onder Min Max Alerts TEL The PPP Minha Algo 242 Raoof Cmerpne ote PPP Alger 2143 Algor for ere Deco Compuaton 244 Quatre Comergene 4 Har Pot 245 Box-Contesd Minx Agi AG. Aare Pncon Met 25. Nenton's Method for Min-Mae Problems 26. Phve = Phase H Methods of Centers 261 MinMax-Type Phe Phase IE Mehoss 26a Abani Function Mato 2.7 Pennty Function Algerie 211 se Thar of ely Fnetons 212, Braet Poly intr 2.73, Etat Ply Psion Alain 274 Nowe 28, Augmented Lagrangian Methods {RI Prbins wh Egat Cosas 282 Problems ih Med Comte 283 Nove 219, Sequential Quadratic Programming 292 Fang's Meta 3103 The Lacal Marte Mayoe ok Medd fr) 204 Gloal MP Algrdin f 295 The Manto: Mayne Polk Pate Method for) 5 Semiinite Optimization ‘1 Optinaity Conditions or Smite Mina SI Fis Order Opal Contos fe SMMP 12 AnOpdmaiy Fenton for SMIMP as 2 Sia New 22 Optimty Condens or Constant ‘Staite Opiation 321 Pinder Oia Conon fr SIC 322 Aa Optima Fncton fr SCP 323. ‘Seine Condon fr SIC 5:24 Fine Ordena Coen fr STECE 3.3 Theory of Consent Appa 531” Fpicomerrce nd Optimally Feces 533 Mie Alm Mass 34 Somat MMs Aerts SA Ages Ba on Agra Mods 33.2988 3317 343. Fkac mcrae tn as és tod of Oar Aprotios 35 Alerts for Iety-Cansraned ‘Stovngnte Opin 531 Comin Aryosinatons 4132 Alpin ed on Arn Models 331413320 353, Nato apo 36-bit fo Sen-intate Opinion Asa Nende owe Ape 363. Anat Pe Posto Alpin 44 optimal conet “41 Cannl Fors of Opin Cont Problems ‘AIT Pere of Detsing Fonction ‘412 Trseptio oCeona Fors ‘413 Niet ineron 2 Optinaiy Connor Opin Cot 421” Unoarae Opi Cl 422, Mimic Optimal Cara 423 Opial Conway Coins 424 Opens Cota wi Egy Coes {25 fama a i ny Coie ‘pwn or Uso Opin Cont SST Canin Apna “633, Bad oo ane Algo M3312 {134 ‘Alpes don Mavs Alsi Me 33.09 435 Alon Base Man Alan Model 3320 436 Ipeneatono Newton's te rts Opi Cntr 22 Sooke soon cee” 4243. Anon er on ase Agri Model 9.3.22 {Rt ‘Aon ad on ce lio Md 33.07 ‘45 ‘ns doo ws Alen M2320 46 eet psi Aiton for robes wh State Crate nega Constraints {ST Content Apyroinatons {152 rem Reema oo RO R™ 453. roms ba on Mane Algetbm Mae 33.16 {84 pontine don Me Aled 3337 455, Mahode Oar Agoxinaions ‘Alps fo roms ith Sate Comer yealty Contra {81 Combat Aeroinatons 462. Anu Peay Pino lst ‘Artin robes th State Contras "qty a Iga Constants {G11 Cote goin 4472. Anat Pom Fin Agritn 413 Nae us 3s SS «0 5 Mathematien Background 51 Rests rom Patil Anas SI RealNomed Sts S12 Popes of maa Factions 513, erates and Expansion Forme {S14 Diwan! Dea tn Sebi S15 Tyelmplt incon Therm S18 Nace 52 Comer Ser and Conner Functons 53 Proper of Vale Fanctons SSSI" Oner nd ae Senet 532 Nae S41" Miwon Teens $42 Dial Dean Shans 53. Minimax Trem 551" Dusty snd Dee Mina Teens 552 Tuevoe Nema Teen 56 Diteenst Easton S61" iene, Uniaes nl Bounded of Stsins 562 Lip Cony ol Dery of Soon 583. Deere Tine Aptninatns $4 Reunion protic Ere B8R8EE s os 6 ELEL ES 13 76 Conventions and Symbols 1. Numbering and Cross-Referencing System “Teflon sem of mri aed rosters in his ok ete, lems, propo, rene, ctl, areas ae uated in on of ete, ing ve bet ie oe) we er ithe con es, oc te be. Tis [Ste eater oo ombring ys ot igus betwen fins, hoes es, Win ech sein, outs mast cama sing ig ‘umber sem nate td 4 sg notre om eh (Soho ne fe ong He same Sse (2), whew ‘hapten aber, anc fam nome. When we ‘sere Thome af) ne mem prt Teen 3.2 2 Conventions 1. deo he sen sac of tpl of el sumer. B= mento RY de ower ee ett. The ects Ra aves ‘Seno wn can ey wel tex = (Fa) dee Be ‘Sen vos witout tenon sl. The me pointes gue ited by (and dfed by (ey) #3 8 The am Re ede by Handi ceed by 1 (22% 2. LAO.) decir petite pce coming of eee uses of Catenge mew snly bonded fanebns fom 0,110 RE, vis with tne rie Cayay),8[ (ws) and wars Hy "(G1 dene te Hier see consisting of ane cases of Lebesgue sega fines om 1) Ry wih mer pro (eyanh twinge edn ety a= (09 4. 14,1 dense he anche consis fps cases ebetgee men sentially founded foes fom (0,1) © Be, ne erm db aes oy CO 5. (0) dns tec, wid edo ning forte andxgmed aie 0) dens be ye oJ) te pox. Wevwie fea +B 1 Indes at hedaman of J) ie ace Aan ht ng ein he space. Wen B= B72) = (FAG) fo) eco ee. 6. Gives finsion FR", we dens Jacob msi by tz) ov by Byers Tsien mats howe it sent Be (OO Silay. when gy Rs we eo Hen yg) This aa ‘whose lone (2 "2 Spec ec te avr of mato tn peter. A" 8 Sopenci T does hers of» mache aif on 9. When we sy tata eon f() spit coatieslaifies ‘lon tonnded se ee tS continu reba as ‘eae Lge conc on unde ts 3 symbols ‘Spaces balan nape Bo Gem isav:y TEy(011 GEILILCeymay Pana my Reecfion By Roatan ty” freien bap Bo Ree th Rityeta mh wealy i serz00) Bb ed eb v wemeiey Ww W=Gneiie MME ety Few (eB 1W -nlgsp) sical leet eto) emia of el srof 0) ‘tog pe (daa? tondcnay Geaisiey=wny GEatre20) {161210 1B S Pe) (2 eL2510) been) (4 eU Tu) eU CBO). VF e031) Rew len of Rev, Hoty Wa iy Sloat of ween (12) wn 2012) Givers Sy 91) AGP) eR WERE Bho w = 1) roa (h ena) ‘al ott inte space 96 Semin of matic A Fewer (0r) FER a) BaF = omen Sle Bye) sbgaie ar yeyat YO. amie tiete Netov(e) spin (EAE © av) > Avirrh) dito Sve {QP Cey:h) acon ave wih eet cd eget 0%)” pinay fncon fr annie contin ote 8.6) apiaiy fonction fr net’ pode 856) Gpiaity ncn fr contre bem ‘Sequences a Gphe Ceidfe sapere FE Coren © Gy igcomeasies R aineRt yet) otek new” rH Es, lnkaer ates sent os) nto ofc eur Dien! and DiterenceEzuatons 230) et epeien Se Stmetoner ane eancn KO inarieroan ho Fey fi) BGAN Yon sero fea) FEMgeND Gai) Fea ine) Fuss in Fay) ‘Abbreviations cx arenas ike! ermine ety stmt scotoneous sly detain mayen S ipmatpet Sather poet Babette cal, nd tna Chapter 1 Unconstrained Optimization We dv is her pry 6 apimsiy canis ad gare for ‘Shing cotinine optiiaton ale fe iy ef), ma 10084 coninnuay cies cot fenton ne He When an ene we wil ext eo ess © pebews of te fom mi «xf, ‘tee (9 oxo etal ot intone on a ec Re an“ conven conn Weegee ee to ‘fami wih mates! besos comin he et oe 9 {orotate 44 Optimality Conditions et pone ol nett of yl meta sgl wot abi Sng ity phys and hoa ot. Tegal nose el wheter Yosor sono an epinisaic pote wih checking i ace ermal covdtns. As ne wllstrly Se, ts fe ewe wel go {Gan s slat to optiniaton polenta pruned cone sete eta hem ae ich ep ‘We wl rent de fp conn, Nees conto sth ha ast eae ny al mia Sie condons ‘tse iping hat pit aol ins. Throat i tok, we lk ect meer opal cantons ice ferns Tae St het ‘yan expres the al ha ost er sod re he st a cree Ine vical mine. The sed fom sa ose of So and ens fon eqtion voling grins or uate ea inolvingsoamd derivates, snd pouty so ps When oe tes Aton meted fr veg wheter ncesy coms ie of he ‘ove two fms ae Sted x pln, ee Te othe hid om wih 2 12 Opty Cosi carci ll minnie sr of a ely ve pina fa Tin AS we Wil Sel, ur favorite optinayfctons ae eo a cy eae, neal mel fran opinion ple ed ave {we utp advantages () tt eaicn yes coon cs deco ‘ecton and (mo can be eda cmt pr end awe ‘feninoam vale eng oe [AS we Biv mentzaed above, we wl consider to probleme in ie oo mse oo tr fs an myo. a hn Rs a ce bo (9) Wma at eng ma om reese). vaeee a0 (2) Wo wi 97 at 8 oem eo (6) thar ent D>0 woneat {Df vr eB6.0) eo BG, Pye (eR 1ke-815p), on) (©) Wo wt ny at? eae oc ia er ns 2 >0 Such at /() flr ae HEE DO ths 28 (6) ovat any a iba mnie ope 1 FO)sple), vex a (6) Wow ay at 2a iin ren 8 teenie 2 D> 0 meni JO)S/0). V4 88GB OX @ (We way at 2 tet mie ton (10 a is ed>0 swhiat/B)0, ath Oxon 1.13, than FE20, vaew ° Prog. Toobin a cnttcn, pote tre exe a RY sich ‘df @ 8) <0; ley, #0. Se by dion of te ene sed ee ga[ {2222 -era.n] =o, Se ea 6 (DAN echt +268 BF, Pen [seespatt0 a] S-Ke'G sy comet £0 99h) Fs wRA4F eH) <0 eh emia the opi o a oro 1.18. Suppose hat) I (a) 8 conta eerentaie ac a aati (a) Then vay oy Pron By Trem 1.12, f(2:4) (V4), H) 20, forall eR Shee v =0 she nly ect" wis he opr ht (4) 20 oral WER, edesied et follows a ‘ 1 Oyaaty Cote verse 1A. Conse th prim (8) and spore hit XE He ‘anette an ht fall eh e Rh deca dative de ‘las Show taf? @X Seas minnie (1, en 7 -2)20, veer. ° a ‘Theorem 1.15. Suppose a) (Tat to commcesy erent stig nd tfc) 8FO)aE (4) 18 ect mamea leprae (9, ton fH 20, Va eRe o (0) 12 act mer praion (0, on (8 fate 89120, « forats eX su vat (9/@).2~#) «0 Proof (a) Let BCP) be he al weit with 2 Sats by Coatiny 113, 9/08) =O, ows om te optimality 08 and. bt foray 1 eR and > Ocha LUM, hte xa, (0, och hat {G40 -FE)= WAH fuk +A) =D, Bd ich ingles that fuG8 +524) 2, Laing -¥0, cee at Oh fal) 20 ow ‘Sic (tole rah © Rh dso es olows (0) Let B02) be te bl aso wth 2, and et x be ach hat (WSR) nF) =0 Lath a2. Th frat €(0,1L8 038 eX, ant for ny (1) sch tat 219, (a) mk Bl, or we 61,1) ‘ing 20 wedi he eid eh a tis yt exes) sel (9) aerate aye. We tein with) ‘Ths, pone ha fre fonton fin te poles (Ia) need econ erate sh) ext for all-tsh eR stn ae) mina and coven Let = Rb dei by Ye ay iin en ge dic ive ee sh a at Sr 668 in ah) a ‘he, cise the all (0,1) compet floes fom Coralia 5.25 hat (0 wel eed. Fama flows by spect at ls ad coy € 9) 0. Th, terion of 1) provides w conse way of ‘demining whee) sated or ot. Whan (3 nts ten (93) <0, vecwe Nyt, i OH @ in spe dest” dein fore ot funtion f(t 2. Nee at te Seaman renal, UE) = -W/ EVIRE CD ‘ext cme). Lt RY >Re ded by 21109, lige ny POEM. a “The follows fo Cen) 51.25 tar 6) is mel dio flows ty asin tit 5) de and oly (2) =0. Tus, the eatin of 8) pve acre way of taining whe (ssi at When (iat sed at BP)

You might also like