0% found this document useful (0 votes)
12 views17 pages

07 - Separation Theorems

Uploaded by

mahe.velay
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
12 views17 pages

07 - Separation Theorems

Uploaded by

mahe.velay
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 17

MGT-418 Convex Optimization:

Separation Theorems
Daniel Kuhn
([email protected])

136
137

Closure, Interior, Boundary


Definition: A set C is called closed if it contains the limits of all
converging sequences in C. We denote by cl(C) the closure of C,
i.e., the smallest closed set containing C.
Note: Intersections of closed sets are closed; Rn and ; are closed <latexit sha1_base64="wK4afuXscDbL+6ciln3RiCS6oy8=">AAADvHicbVJdb9MwFHUWYKN8bfDIS6BCGlJVJQg0eKIIEDzwUBBlk9pS2c5NZ9V2Itspq6L+I34Nbwh+DNdpNNF0jmIfn3NsX/teVkhhXRz/DvbCa9dv7B/c7Ny6fefuvcOj+99sXhoOI57L3JwxakEKDSMnnISzwgBVTMIpW7z1+ukSjBW5/upWBUwVnWuRCU4dUrOj4P1Eww+eK0V1Wk2ydTVhuUztSuGw7myJiqOqqDvndEdijcRYSyklKqVOwfgQW2K+XPvuSm0IpkC1wGHyqP48bJmYLAFN9TtU9aRlMJBe6h635LkB0JeGzQwtKWQY0IZUq8Zk5njJuJ/0orj/wncJOhWLvnzXndlhN+7HdYt2QdKALmnaEB9+NklzXirQjktq7TiJCzetqHGCS/8WpYWC8gWdwxihpgrstKpDWkdPkEmjLDf4axfV7P8rKqa2dqiosj6luNInybY1T16ljUuXvZxWQhelA803B2eljFwe+WqKUmGAO7lCQLkRGHvEz6mh3GHNbZ1S710A347rotSC5ym0WOkunKFIWnCKCu1vWb0xoq67msPNPHn8TsyFs71PWM669wETtXjaGDElSTsBu2D0rP+qH39+3h0MmtwckIfkMTkmCTkhA/KRDMmI8OBn8Cv4E/wNX4cQLkK1se4FzZoHZKuFy38Tm0SO</latexit>

<latexit sha1_base64="Uic/owy+KmyjpQhmVzkA+rU/rTE=">AAADvnicbVJLj9MwEHY2PJby2oUjl0CFtEhVlSAQcOsKpOXAoUiUXamtKseZZK36EdnO0irqX+LXcOEAv4VxGq1ouo5ij7/vG3s8M2kpuHVx/Ds4CG/dvnP38F7v/oOHjx4fHT/5bnVlGEyYFtpcpNSC4AomjjsBF6UBKlMB5+nyo+fPr8BYrtU3ty5hLmmheM4ZdQgtjoOzmYIfTEtJVVbP8k09S7XI7FrisuntkJIhK6m7ZHSPSlsqTTtMJZCpVAbGh9gh9dXGTzdyYzAlsiUus+fN582OKBUVoKjJQ91sOgID2TXv7Q5dGAB1LdjuUJJBjgFtQbluRabAR8bDZBDFw7d+SlAJsnRrC663OOrHw7gZ0b6RtEaftGOMqV/MMs0qCcoxQa2dJnHp5jU1jjPhs1FZKClb0gKmaCoqwc7rJqhN9BKRLMq1wV+5qEH/96hTuXNCTaX1RUVPXybb5Tx4EzetXP5+XnNVVg4U216cVyJyOvL9FGXcAHNijQZlhmPsEbukhjKHXbdzS3N2CWw3rlWlONMZdFDhVs5QBDG3knLlX1mfGt50XoPhYR48+cQL7uzgCza0GpxhqZavWiGWJOkWYN+YvB5+GMZf3/RHo7Y2h+QZeUFOSELekRH5TMZkQljwM/gV/An+hqdhEcpQb6UHQevzlOyMcPUPWvNGEA==</latexit>
<latexit

n
R
Definition: A set C is called open if its complement \C is <latexit sha1_base64="sVKPNB7b/BIqc3Xct2kzV6sYoa8=">AAADyHicbVJNj9MwEHU2fCzlaxeOXAIV0iJVVYJAwK1okUCCQxdRdqWmVLYzba3aTrCd3a2iXvhV/BUuXOFnME6jFc2uo9jj996Mx55hhRTWxfGvYCe8dv3Gzd1bndt37t67v7f/4KvNS8NhxHOZmxNGLUihYeSEk3BSGKCKSThmy0PPH5+CsSLXX9yqgImicy1mglOH0HQ/OEo1nPFcKaqzKp2tq5TlMrMrhcu6s0UqjqyibsHpJYo1FGMtppTIlDoD41Nskfnp2k9XckMwBbIFLunj+vNmS8RkCSiq36GqNy2BgeyC93aLnhsAfSHY7FCSwQwT2oBq1YjMHC8Z95NeFPdf+ilBpWLR5286ZZQvraR2ER12pnvduB/XI7psJI3RJc0YYhGmaZbzUoF2HIPYcRIXblJR4wSX/l1KCwWeQOcwRlNTBXZS1emto6eIZNEsN/hrF9Xo/x4VU1sRKqqsLy96+oLZNufBq7hx6WavJ5XQRelA883Bs1JGLo98Z0WZMMCdXKFBuRGYe8QX1FDusP+2TqljF8C38zovteB5Bi1UunNnKIIWnKJC+1tWb42oe7DGMJgHD96JuXC29wlbW/feY9GWzxohliRpF+CyMXref9OPj150B4OmNrvkEXlCDkhCXpEB+UCGZER48DP4HfwJ/oYfw+/hWbjaSHeCxuch2Rrhj38rXUlx</latexit>

closed. We denote by int(C) the interior of C, i.e., the largest open


set contained in C.
Note: The interior is the complement of the closure of the
complement. Unions of open sets are open.

Definition: We call bd(C) = cl(C)\int(C) the boundary of C.


<latexit sha1_base64="hJhXo9CUzuClBEUbFXpaYv2TWFY=">AAAD6XicbVJNj9MwEHUbPrrlaxeOXAIVUleqqgSBgANS0SLBgUORKLtSU1WOM22t2k5kO0urqH+CE+LKr9pfA+MkWtF0HcWeee95PPZMnAlubBBctdrerdt37naOuvfuP3j46Pjk8XeT5prBhKUi1RcxNSC4gonlVsBFpoHKWMB5vD5z/PklaMNT9c1uM5hJulR8wRm1CM1PWjpS8IOlUlKVFNFiV0RxKhKzlbjsunukZMhKaleMHlBxTcVxg8kFMrlKQLsUG2R6uXPTjdwYdIZshkv0rPyc2RDFIgcUle9QlE5DoCG55p3doJcaQF0LKg8lCSwwoQqU21qkl3jJYBgO/GD42k0hKi1sbBEnu/6Zf/q+8pgovSimbG0ENSu/wrmyJdGdH/eCYVAO/9AIa6NH6jHGKs2jJGW5BGUZRjTTMMjsrKDacibcw+UGMjyOLmGKpqISzKwo89/5LxBJ/EWq8VfWL9H/dxSx3ItQUGlc/XGnq6hpcg68iZvmdvF2htfMcguKVQcvcuHb1Het5ydcA7NiiwZlmmPuPltRTZnFBt07pYydAdvPa5MrztIEGqiwG6spggaspFy5WxYfNC+btMQwmAP7H/mSWzP4gr2vBp+wquvTWoglCZsFODQmL4fvhsHXV73RqK5Nhzwlz0mfhOQNGZHPZEwmhLWuWn/bnfaRJ7yf3i/vdyVtt+o9T8je8P78A0x5VJU=</latexit>

C = (a, b] int(C) = (a, b)


a b a b
cl(C) = [a, b] bd(C) = {a, b}
a b a b
138

Closed Functions
Definition: A function f is called closed if its epigraph is a closed
set. We denote by cl(f) the closure of f, i.e., the largest closed
function not larger than f.

Note: The epigraph of cl(f) is given by the closure of epi(f). A


function is closed iff all of its sublevel sets are closed (not easy to
prove).

epi(cl(f))
epi(f)
= cl(epi(f))

x x
a b a b
1 if a < x b
1 if a x b
f(x) = 3 if x = a cl(f)(x) =
else
else
139

Strict Separating Hyperplane Theorem


Theorem: Consider a closed convex set C and a point x0 2
/ C. <latexit sha1_base64="AM3RfJfcV6lWlaXnMOLuzWWmXSc=">AAADwXicbVJdb9MwFHUWPkb52uCRl0CFNKSqShAIeCvaJEDioUiUTWqq4ji3nak/ItsZraL+J34NEk/wU7hOw0TTOYp9fM6xfe17s0Jw6+L4V7AXXrt+4+b+rc7tO3fv3T84fPDF6tIwGDEttDnLqAXBFYwcdwLOCgNUZgJOs8Wx108vwFiu1We3KmAi6VzxGWfUITU9DD6kCr4zLSVVeZXO1lWaaZHblcRh3dkSJUNVUnfO6I6UNVKWtZRSoFKqHIwPsSXqi7XvrtSGYApUCxzSx/XnYcuUiRLQVL9DVU9aBgP5pe5xS54bAHVp2MzQksMMA9qQctWYzBwvGfeTXhT3X/ouWXeW0zhV2nEVHXemB924H9ct2gVJA7qkaUN8/K9prlkpQTkmqLXjJC7cpKLGcSb8e5QWCsoWdA5jhIpKsJOqDmsdPUUmj2ba4K9cVLP/r6gyubVDRaX1acWVPlG2rXnyKm1cutnrScVVUTpQbHPwrBSR05GvqCjnBpgTKwSUGY6xR+ycGsoc1t3WKfXeBbDtuJal4kzn0GKFWzpDkbTgJOXK37J6a3hdezWHm3ny6ITPubO9j1jSqvcOk7V49s+IOUnaGdgFo+f9N/3404vuYNAkZ588Ik/IEUnIKzIg78mQjAgLfgQ/g9/Bn/Ak/BYWodlY94JmzUOy1cLqLxaoRks=</latexit>

Then, there exist a 6= 0 and b with <latexit sha1_base64="Ms0ifWhN4U31rN9lNYRQncNsgOc=">AAADvXicbVJbb9MwFHYWLqPcNnjkJVAhDamqkgkEvBUBggceikTZpLYajnOSWfUl2M62Kuo/4tfwhuDHcJyGiaZzFPv4+z4fH59z0lJw6+L4V7ATXrt+4+burd7tO3fv3d/bf/DV6sowmDAttDlOqQXBFUwcdwKOSwNUpgKO0sVbzx+dgbFcqy9uWcJc0kLxnDPqEDrZD97PFJwzLSVVWT3LV/Us1SKzS4nLqrdBSoaspO6U0S0qbak07TCVQKZSGRgfYofUZys/XcmNwZTIlrjMHjefNzuiVFSAoiYPdbPpCAxkl7y3O3RhANSlYL1DSQY5BrQG5bIVmQIfGQ+TQRQPX/gpWfUoevsexb2TvX48jJsRbRtJa/RJO8aY+G+zTLNKgnJMUGunSVy6eU2N40z4XFQWSsoWtIApmopKsPO6CWkVPUUki3Jt8FcuatD/T9Sp3PBQU2l9SfGkL5Ltch68iptWLn81r7kqKweKrS/OKxE5HfluijJugDmxRIMywzH2iJ1SQ5nDntu4pfFdAtuM66JSnOkMOqhwF85QBC04Sbnyr6zfGN70XYOhMw8evOMFd3bwCdtZDT5goRbP/gmxJkm3AtvG5HD4ehh/ft4fjdri7JJH5Ak5IAl5SUbkIxmTCWHBj+Bn8Dv4E47CPBShWkt3gvbMQ7IxwvO/HW1Ehg==</latexit>

a> x < b 8x 2 C
<latexit sha1_base64="nd7WT+866R5Zbn6HkKAs17nw1Ds=">AAAD9HicbVNLbxMxEN4kPNrw6uPIxRAhFSmKdisQICEUVCQ4cAgSoZWyIXi9k9SKH4vtbRMt4Z9wQlz5P5z5I4w3aUU29Wrt8fd9Y4894yQT3Low/FOrN65dv3Fza7t56/adu/d2dvc+WZ0bBn2mhTYnCbUguIK+407ASWaAykTAcTI98vzxGRjLtfro5hkMJZ0oPuaMOoRGu7VvsYJzpqWkKi3i8aKIEy1SO5c4LJprpGTISupOGd2gkhWVJBUmF8jkKgXjQ6yQ+mzhuyu5HpgM2QyH+EH5ebMiSkQOKCrvoSgnFYGB9JL3doWeGAB1KVjOUJLCGANagnK+EpkJHjLsRG0Sdp76Llo06efY6YzMXpLkezzWhgpBZjFX5Cj+mtOU4MXoWYFbLcgSuHAYha9I0hnttMJOWDayaUQroxWsWg/z9SVONcslKMcEtXYQhZkbFtQ4zoS/wtxCRtmUTmCApqIS7LAoT7IgjxBJCYaJv3KkRP/3KBK5tkJBpfWVgJ4+t7bKefAqbpC78fNhwVWWO1BsufE4F8Rp4ouQpNwAc2KOBmWGY+yEnVJDmcNSXdulXDsDth7XLFec6RQqqHAzZyiCFpykXPlTFq8NL8u1xHAxDx684RPubPs9vgLVfov5nT6+EGJOomoGNo3+YedFJ/zwpNXtrpKzFdwPHgYHQRQ8C7rBu6AX9ANW+1vfru/V9xvnjR+Nn41fS2m9tvLZD9Za4/c/f/NXzg==</latexit>
and a> x0 > b.

Example: a

<latexit sha1_base64="VQ3zlHG7iX07YacPnSr7ipLZSHI=">AAADuHicbVJdb9MwFHUWPkb52uCRl0CFNKSqSiYQIF6KQIIHHopE2aS2Ko5zk1n1R2Q7o1XU38Ov4RXEv+E6DRPN5ij28TnH9rXvTUvBrYvjP8FeeO36jZv7t3q379y9d//g8MFXqyvDYMK00OY0pRYEVzBx3Ak4LQ1QmQo4SZfvvH5yDsZyrb64dQlzSQvFc86oQ2pxGIxmCr4zLSVVWT3LN/Us1SKza4nDprcjSoaqpO6M0UtS2kpp2lEqgUqlMjA+xI6ozze+u1IbgylRLXGYPW4+DzumVFSApuYd6mbSMRjILnSPO3JhANSFYTtDSwY5BrQl5bo1mQIvGQ+TQRQPX/gu2fRWi3hx0I+HcdOiyyBpQZ+0bYxv/m2WaVZJUI4Jau00iUs3r6lxnAn/DJWFkrIlLWCKUFEJdl430Wyip8hkUa4N/spFDfv/ijqVOzvUVFqfTVzp82O7miev0qaVy1/Na67KyoFi24PzSkROR76QoowbYE6sEVBmOMYesTNqKHNYbjunNHuXwHbjWlWKM51BhxVu5QxF0oKTlCt/y/qt4U3JNRxu5smj97zgzg4+YSWrwQfM0fLZPyPmJOlm4DKYHA9fD+PPz/ujUZucffKIPCFHJCEvyYh8JGMyISz4EfwMfgW/wzchDYuQb617QbvmIdlpofkLHB5DAA==</latexit>
x0
C

a x<b a x>b
a x=b
140

Strict Separating Hyperplane Theorem*


Proof: Define the distance of C to x0 as <latexit sha1_base64="VQ3zlHG7iX07YacPnSr7ipLZSHI=">AAADuHicbVJdb9MwFHUWPkb52uCRl0CFNKSqSiYQIF6KQIIHHopE2aS2Ko5zk1n1R2Q7o1XU38Ov4RXEv+E6DRPN5ij28TnH9rXvTUvBrYvjP8FeeO36jZv7t3q379y9d//g8MFXqyvDYMK00OY0pRYEVzBx3Ak4LQ1QmQo4SZfvvH5yDsZyrb64dQlzSQvFc86oQ2pxGIxmCr4zLSVVWT3LN/Us1SKza4nDprcjSoaqpO6M0UtS2kpp2lEqgUqlMjA+xI6ozze+u1IbgylRLXGYPW4+DzumVFSApuYd6mbSMRjILnSPO3JhANSFYTtDSwY5BrQl5bo1mQIvGQ+TQRQPX/gu2fRWi3hx0I+HcdOiyyBpQZ+0bYxv/m2WaVZJUI4Jau00iUs3r6lxnAn/DJWFkrIlLWCKUFEJdl430Wyip8hkUa4N/spFDfv/ijqVOzvUVFqfTVzp82O7miev0qaVy1/Na67KyoFi24PzSkROR76QoowbYE6sEVBmOMYesTNqKHNYbjunNHuXwHbjWlWKM51BhxVu5QxF0oKTlCt/y/qt4U3JNRxu5smj97zgzg4+YSWrwQfM0fLZPyPmJOlm4DKYHA9fD+PPz/ujUZucffKIPCFHJCEvyYh8JGMyISz4EfwMfgW/wzchDYuQb617QbvmIdlpofkLHB5DAA==</latexit>

dist(C, D) = inf{kx x0 k2 : x 2 C}. <latexit sha1_base64="UFpt9CqMjzh36x9/ieP5goTwFfg=">AAAD4HicbVLLjtMwFE0mPIbymBlYsglUSB2pVMkIxENCKupIsGBRJMqM1FTFcW5aq7YT2c7QKpMPYIXY8lOs+RC2wE0aRjQdR7Gvzzm2r+9xmHKmjef9tHecK1evXd+90bp56/advf2Dux91kikKI5rwRJ2GRANnEkaGGQ6nqQIiQg4n4WJQ8idnoDRL5AezSmEiyEyymFFiEJoe2PNAwmeaCEFklAdxkQdhwiO9EjgUrQ1SUGQFMXNKtqiwpsKwwWQcmUxGoMoUG2RyVpTdpdwQVIpsikPwoPrKsCEKeQYoquqQV5OGQEF0wZdxg54pAHkhWM9QEkGMCa1BsapFaoaX9Hp+1/V6T8vOL1p4gHAjtKnoDLrHh68CJuMgD86Xj5dTLzifHr1cIuQOgqI33W97Pa9q7nbg10HbqtsQvfkURAnNBEhDOdF67HupmeREGUZ5Wa5MQ0rogsxgjKEkAvQkr7Iu3EeIRG6cKPylcSv0/xV5KDZ2yInQpeu4svRRN7kSvIwbZyZ+PsmZTDMDkq4PjjPumsQtHxxWRwE1fIUBoYph7i6dE0WowWe5cUq1dwp0M69lJhlNImig3CyNIghqMIJg2fGW+WvFqqdZYbhZCXaO2YwZ3X2HL15236CXi8N/QvTEbzqwHYyOei963vsn7X6/NmfXum89tDqWbz2z+tZba2iNLGr/sH/Zv+0/DnW+OF+db2vpjl2vuWdtNOf7X/iaUn0=</latexit>

As C is closed, there is c 2 C with kc x0 k2 = dist(C, D). Next, set


a = x0 c and b = 12 (kx0 k22 kck22 ). Assume that there is x 2 C
<latexit sha1_base64="Mps2V/xr0lNFt65HRkNY2sQIWeQ=">AAAD5HicbVLLjtMwFE0bHkN5zAws2QQqpI5UqmQEAhZIRR0JFiyKRJmRmqo4zm1r1XYi25lplcknsEJs+SlWfAo7rtMwouk4in19zrF97XOjlDNtfP93o+neuHnr9t6d1t179x/sHxw+/KKTTFEY0YQn6iwiGjiTMDLMcDhLFRARcTiNlgPLn56D0iyRn806hYkgc8lmjBKD0PSwwUMJFzQRgsg4D2dFHkYJj/Va4FC0tkhBkRXELCjZoaKKiqIak3FkMhmDsinWyOS8sN213BBUimyKQ/ik/GxYE0U8AxSV75CXk5pAQXzF27hGzxWAvBJsZiiJYYYJbUCxrkRqjpf0e0HX83svbRcULRoy6Q3w+skq9y6YWXhFeEmfr6Z+eDk9fovnCy9GF4vOoHty1JoetP2eXzZvNwiqoO1UbYj+fA3jhGYCpKGcaD0O/NRMcqIMo9w+WaYhJXRJ5jDGUBIBepKXmRfeM0Rib5Yo/KXxSvT/FXkktnbIidDWeVxpvdR1zoLXcePMzF5PcibTzICkm4NnGfdM4tmiwydQQA1fY0CoYpi7RxdEEWqwNLdOKfdOgW7ntcoko0kMNZSblVEEQQ1GECbtLfN3ipXlWWK4mQU7J2zOjO5+xKqX3ffo5/LonxA9CeoO7Aaj496bnv/pRbvfr8zZcx47T52OEzivnL7zwRk6I4c2fjX+NJtN152739zv7o+NtNmo1jxytpr78y8BpVHc</latexit>

<latexit sha1_base64="SJhurEhjJ3BLAmHueo7pcolyM1Y=">AAADvHicbVJdb9MwFHUWPkb52uCRl0CFNKSqSiYQ8ETRJsEDD0WibFJTDce5yazaTmQ7pVXUX8Sv4RH4M1ynYaLpHMU+PufYvva9SSm4sWH4y9vzb9y8dXv/Tu/uvfsPHh4cPvpqikozmLBCFPo8oQYEVzCx3Ao4LzVQmQg4S+YnTj9bgDa8UF/sqoSZpLniGWfUInVx6J3GCr6zQkqq0jrO1nWcFCI1K4nDurclSoaqpPaS0R0paaUk6SiVQKVSKWgXYkcsFmvXXauNQZeoljjET5vPwY4pERWgqXmHupl0DBrSK93hjpxrAHVl2MzQkkKGAW1IuWpNOsdLhsNoEITDV66L1r1lzFVwElwc9MNh2LRgF0Qt6JO2jfHdv8VpwSoJyjJBjZlGYWlnNdWWM+GeojJQUjanOUwRKirBzOomonXwHJk0yAqNv7JBw/6/ok7k1g41lcZlFFe6HJmu5sjrtGllszezmquysqDY5uCsEoEtAldMQco1MCtWCCjTHGMP2CXVlFksua1Tmr1LYNtxLSvFWZFChxV2aTVF0oCVlCt3y/q95k3ZNRxu5sijU55zawafsJrV4APmaf7inxFzEnUzsAsmx8O3w/Dzy/5o1CZnnzwhz8gRichrMiIfyZhMCPN+eD+9394f/50P/tyXG+ue1655TLaav/gLGSFETw==</latexit>

with a> x b. Then,


<latexit sha1_base64="mDBV9lHJ5HQ8BOIIoGuOiuoXu5s=">AAAD/HicbVPNjtMwEE5bfpbytws39mJYIXWlbpVUIOCw0iKQ4MBhkSi7UlOK40y6Vm0nsp2lVRpx4VU4Ia68C8/ASzBOy4qm6yj25Ps+j8czkygT3Fjf/91otq5cvXZ960b75q3bd+5u79z7aNJcMxiwVKT6NKIGBFcwsNwKOM00UBkJOImmrxx/cg7a8FR9sPMMRpJOFE84oxah8U7jW6jgC0ulpCouwqQswigVsZlLXMr2GikZspLaM0Y3qGhFRVGNyQUyuYpBuxBrZHpeuulS7hh0hmyGS/iwepxZE0UiBxRVeSiqj5pAQ3zBO7tGTzSAuhAsv1ASQ4IBLUE5X4n0BC/p94Iu8XtP3RSg0sLMGjsXQOjhbOwfMIK5SGcFQfekJNFhmGjKiqAs+mUnXKAkXIz7n/oHJFywpbnfa4+39/yeXw2yaQQrY89bjWOs2+cwTlkuQVkmqDHDwM/sqKDaciZcKnMDGWVTOoEhmopKMKOiulFJHiMSkyTV+CpLKvT/HUUk1zwUVBrXEbjT1djUOQdexg1zmzwfFVxluQXFlgcnuSA2Ja4ZScw1MCvmaFCmOcZO2BnFhFls2bVTKt8ZsPW4ZrniLI2hhgo7s5oiaMBKypW7ZfFS86ptKwydObDzmk+4Nd13+Deo7hus83T/nxBrEtQrsGkM+r0XPf/9k72jo1Vxtrxd75HX8QLvmXfkvfWOvYHHGn+aO80Hzd3W19b31o/Wz6W02Vjtue+tjdavv+jKWdQ=</latexit>

9
<latexit sha1_base64="fYkuU+f0bafUcR1fALZrOOVelvo=">AAADxHicbVJdb9MwFHUWPkb52uCRl0CFNKQqSiYQ8FYE0njYQ5Eom9SU4Tg3mVV/BNsZraLwp/g1e4U/gp2GiaZzFPv4nGP72vemJaPaRNGlt+PfuHnr9u6dwd179x883Nt/9EXLShGYEsmkOk2xBkYFTA01DE5LBZinDE7SxXunn1yA0lSKz2ZVwpzjQtCcEmwsdbbvHScCfhDJORZZneRNnaSSZXrF7dAMNkROrMqxOSd4S0o7KU17SsWsUokMlAuxJ8qLxnXXahNQpVVLOyRP28/BnillFVhT+w51O+kZFGRXusM9uVAA4sqwnllLBrkNaE3yVWdShb1kFMajIApfuS5uBvhrYmQZLJMCvgdpeLY3jMKobcE2iDswRF2b2Pf/lmSSVByEIQxrPYuj0sxrrAwlzD1JpaHEZIELmFkoMAc9r9vImuC5ZbIgl8r+wgQt+/+KOuUbO9SYa5dZu9LlSvc1R16nzSqTv5nXVJSVAUHWB+cVC4wMXFEFGVVADFtZgImiNvaAnGOFibGlt3FKu3cJZDOuZSUokRn0WGaWRmFLajAcU+FuWb9TtC2/lrObOfLgAy2o0aNjW9VidGTztXjxz2hzEvczsA2mh+HbMPr0cjged8nZRU/QM3SAYvQajdFHNEFTRLxf3qX32/vjH/nc1361tu543ZrHaKP5P/8C2oVHwA==</latexit>

1
0  (x0 c)> (x 2 (x0 + c)) =
1 >
>
= (x0 c) (x c 2 (x0 c)) =) (x 0 c) (x c) > 0
> 1 2 ;
= (x0 c) (x c) 2 kx0 ck2 <latexit sha1_base64="/IzcLnsQpuXPk/nuhpSfgtDHktE=">AAAEi3icfVNdb9MwFM1KgVFgbOyRl8DEtIqtSqohPgRoCBA88DAkyibNXXGcm8ya7QTbGStZ/guv8I/4N1ynZVrTCUexr885vr6+vo5ywY0Ngj8LrSvtq9euL97o3Lx1e+nO8srdLyYrNIMBy0Sm9yNqQHAFA8utgP1cA5WRgL3o+I3j905AG56pz3acw1DSVPGEM2oRGq20VomC7yyTkqq4JElVkigTsRlLHKrODCkZspLaI0bnqGhKRVGDKQQyhYpBuxAbZHZSue5Sbhd0jmyOA7lff85siCJRAIrqPJT1pCHQEJ/zzm7QqQZQ54LJDCUxJBjQBJTjqUineMigF276Qe+x60JUCkhsj0SQclVSrekYdQyzE/jryH3z1/2N01GwxbqHxGY5TrZIoikrw6rsV456xLpdQjrrL+eVrKnd+o+2e1FMzmqanI36h/0OATzqJDiieXpkSeUTLrG8wHQuOqr9vAo6o+W1oBfUzZ83wqmx5k3b7mhl4SuJM1ZIUJYJasxBGOR2iJtazoS72cJATtkxTeEATUUlmGFZJ7jyHyIS+0mm8VfWr9GLK8pIzngoqTSuQHGlKznT5Bx4GXdQ2OTpsOQqLywoNtk4KYRvM9+9DT/mGpgVYzQo0xxj99kRxaxafEEzu9S+c2CzcZ0WirMshgYq7KnVFEEDVlKu3CnL15rXr6jG0JkDN97ylFuz+REfp9p8j2V33P0nxDsJmzcwbwz6vWe94NP22s7O9HIWvXveA2/DC70n3o73wdv1Bh5r/Wj9bP1q/W4vtbfbz9svJtLWwnTNqjfT2u/+AvOEhh8=</latexit>

a Thus,
k(1 t)c + tx x0 k22
= kc x0 k22 2t(x0 c)> (x c) + O(t2 )
x0
< kc x0 k22 for small t > 0.
C
<latexit sha1_base64="fwYtY3QyzYEgC2qR2q4rbMYWxDY=">AAAET3icbVNdb9MwFM3aAVv52uCRF8PG1GprlVQgQIJpCCR4QGJIlE1auuI4N50124lsZ7TKyo/kBfFPeEJcp2Gi2RwpPj7n2L6+vo4ywY31/V9Ljebytes3VlZbN2/dvnN3bf3eF5PmmsGApSLVhxE1ILiCgeVWwGGmgcpIwEF0+sbpB2egDU/VZzvNYCjpWPGEM2qRGq0v/QwVfGOplFTFRZjMijBKRWymErtZa0GUDFVJ7Qmjl6SokqKopuQClVzFoF2INTE9m7nfldo+6AzVDLvwYfk5WDNFIgc0lXkoykHNoCG+0B2uyWMNoC4M8xFaYkgwoDkpp5VJj/GQfi/YIX7vqfsF6IxgzFVBtaZTdDDMy9YWCc/bQdd22LadkC6ZjPzwfNQ/7odha+uVU1n3guv2bRsHXdY5Dm2atSeItjHT5GPbHvc7bsrL2pTvBNOdTgqSpJoYSYUgm3bX35z1WiHgoebBjNY2/J5fNnIZBBXY8Kq2j6XwNYxTlktQlglqzFHgZ3aIy1nOhLud3EBG2SkdwxFCRSWYYVEmaUYeIxOXESWpsqRk/59RRHJhhYJK44oMZ7qyMXXNkVdpR7lNng8LrrLcgmLzjZNcEJsSV98k5hqYFVMElGmOsRN2QjVlFl/Bwi7l2hmwxbgmueIsjaHGCjuxmiJpwErKlTtl8Vrz8iWUHC7myPZbPubW7HzAB6Z23mHpnHb+GfFOgvoNXAaDfu9Fz//0ZGNvr7qcFe+B98hre4H3zNvz3nv73sBjjd1G3JAN1fzR/N38s1xZG0sVuO8ttOXVv6dPc0M=</latexit>

c
<latexit sha1_base64="VQ3zlHG7iX07YacPnSr7ipLZSHI=">AAADuHicbVJdb9MwFHUWPkb52uCRl0CFNKSqSiYQIF6KQIIHHopE2aS2Ko5zk1n1R2Q7o1XU38Ov4RXEv+E6DRPN5ij28TnH9rXvTUvBrYvjP8FeeO36jZv7t3q379y9d//g8MFXqyvDYMK00OY0pRYEVzBx3Ak4LQ1QmQo4SZfvvH5yDsZyrb64dQlzSQvFc86oQ2pxGIxmCr4zLSVVWT3LN/Us1SKza4nDprcjSoaqpO6M0UtS2kpp2lEqgUqlMjA+xI6ozze+u1IbgylRLXGYPW4+DzumVFSApuYd6mbSMRjILnSPO3JhANSFYTtDSwY5BrQl5bo1mQIvGQ+TQRQPX/gu2fRWi3hx0I+HcdOiyyBpQZ+0bYxv/m2WaVZJUI4Jau00iUs3r6lxnAn/DJWFkrIlLWCKUFEJdl430Wyip8hkUa4N/spFDfv/ijqVOzvUVFqfTVzp82O7miev0qaVy1/Na67KyoFi24PzSkROR76QoowbYE6sEVBmOMYesTNqKHNYbjunNHuXwHbjWlWKM51BhxVu5QxF0oKTlCt/y/qt4U3JNRxu5smj97zgzg4+YSWrwQfM0fLZPyPmJOlm4DKYHA9fD+PPz/ujUZucffKIPCFHJCEvyYh8JGMyISz4EfwMfgW/wzchDYuQb617QbvmIdlpofkLHB5DAA==</latexit>

However, this is impossible as c is the


closest point to x0 from within C. <latexit sha1_base64="VQ3zlHG7iX07YacPnSr7ipLZSHI=">AAADuHicbVJdb9MwFHUWPkb52uCRl0CFNKSqSiYQIF6KQIIHHopE2aS2Ko5zk1n1R2Q7o1XU38Ov4RXEv+E6DRPN5ij28TnH9rXvTUvBrYvjP8FeeO36jZv7t3q379y9d//g8MFXqyvDYMK00OY0pRYEVzBx3Ak4LQ1QmQo4SZfvvH5yDsZyrb64dQlzSQvFc86oQ2pxGIxmCr4zLSVVWT3LN/Us1SKza4nDprcjSoaqpO6M0UtS2kpp2lEqgUqlMjA+xI6ozze+u1IbgylRLXGYPW4+DzumVFSApuYd6mbSMRjILnSPO3JhANSFYTtDSwY5BrQl5bo1mQIvGQ+TQRQPX/gu2fRWi3hx0I+HcdOiyyBpQZ+0bYxv/m2WaVZJUI4Jau00iUs3r6lxnAn/DJWFkrIlLWCKUFEJdl430Wyip8hkUa4N/spFDfv/ijqVOzvUVFqfTVzp82O7miev0qaVy1/Na67KyoFi24PzSkROR76QoowbYE6sEVBmOMYesTNqKHNYbjunNHuXwHbjWlWKM51BhxVu5QxF0oKTlCt/y/qt4U3JNRxu5smj97zgzg4+YSWrwQfM0fLZPyPmJOlm4DKYHA9fD+PPz/ujUZucffKIPCFHJCEvyYh8JGMyISz4EfwMfgW/wzchDYuQb617QbvmIdlpofkLHB5DAA==</latexit>

1
+ c)
<latexit sha1_base64="Hugh4QLsT89s8wwOrzGusMcnrNs=">AAAD0nicbVLLjtMwFHUmPIbhMTOwZBOokDqiqpIRCNgVgQQLFkWizIimKo5z07FqO5HtDK2iLBBbforP4AvYwh9wnZYRTcdR7ONzju1r35sUghsbhj+9Hf/K1WvXd2/s3bx1+87+weHdjyYvNYMRy0WuTxNqQHAFI8utgNNCA5WJgJNk/srpJ+egDc/VB7ssYCLpTPGMM2qRmh56n2IFX1guJVVpFWd1FSe5SM1S4lDvbYiSoSqpPWN0S0rWUpK0lFKgUqoUtAuxJebntesu1YagC1QLHOIHzedgy5SIEtDUvEPVTFoGDemF7nBLnmkAdWFYzdCSQoYBrUi5XJv0DC8Z9qNeEPafui5Cp4WFNXYpIIgzTVkV1dVx3V1Mw8fsaHrQCfth04JtEK1Bh6zbELPxOU5zVkpQlglqzDgKCzupqLacCfdApYGCsjmdwRihohLMpGrirINHyKRBlmv8lQ0a9v8VVSI3dqioNC7PuNJlzrQ1R16mjUubPZ9UXBWlBcVWB2elCGweuBILUq6BWbFEQJnmGHvAzii+jsVC3Dil2bsAthnXolSc5Sm0WGEXVlMkDVhJuXK3rF5q3hRjw+Fmjuy+5jNuTe8d1rjqvcHszY/+GTEnUTsD22B03H/RD98/6QwG6+TskvvkIemSiDwjA/KWDMmIMO+H98v77f3xR37lf/W/raw73nrNPbLR/O9/AWJeTaA=</latexit>
2 (x0 =)
<latexit sha1_base64="945tQ+3E1rbK9ElRjWpBMyRt3jQ=">AAAD3XicbVJNb9MwGE4WPsb42uDIxVAhDamqEgQCJA5FQ4IDhyJRNqkpm+O86az6I9jOaBWVIyfElT/FH+BvcIUDr9My0WyOYr9+nsf2az9vVgpuXRz/DDeiCxcvXd68snX12vUbN7d3br23ujIMhkwLbQ4yakFwBUPHnYCD0gCVmYD9bLrn+f0TMJZr9c7NSxhLOlG84Iw6hA53QkgVfGJaSqryOi0WdZppkdu5xGGxtUZKhqyk7pjRM1S2orKsxVQCmUrlYHyKLVKfLHx3LjcAUyJb4pDebT4ftkSZqABFzTvUzaQlMJCf8j5u0RMDoE4FyxlKcigwoSUo5yuRmeAl417SJXHvse8SVHKJLoFNP1Y0J/RD6nRJZs9J9jkttKFCkFnKFdk73O7Evbhp5GyQrIJOsGoDNOYozTWrJCjHBLV2lMSlG9fUOM6Ef6vKQknZlE5ghKGiEuy4blJekPuI5AQzwF850qD/r6gzubZDTaX1luNKb6Jtcx48jxtVrng6rrkqKweKLQ8uKkGcJr7aSM4NMCfmGFBmOOZO2DE1lDmsybVTmr1LYOt5zSrFmc6hhQo3c4YiaMFJypW/Zf3C8KYuGww38+DuSz7hznbfYLmr7is0cvrgnxA9SdoOnA2GD3vPevHbR51+f2XOZnAnuBfsBknwJOgHr4NBMAxY+CP8Ff4O/0RH0Zfoa/RtKd0IV2tuB2st+v4Xi/hSLQ==</latexit>
a> x < b 8x 2 C
141

Strict Separating Hyperplane Theorem*


Proof (con’t):
9
a> x0 b = (x0 c)> x0 12 (kx0 k22 kck22 ) =
= 12 (x> x
0 0 2c >
x 0 + c>
c) =) a >
x0 > b
1 2 ;
= 2 kx0 ck2 > 0
<latexit sha1_base64="UFMxZqHiK7uyOJerGapQTXicwv4=">AAAEk3icbVNdb9MwFM1KgVG+NpB42YthAnVirZIKBEgwDZgEDyANibJJc1cc5yazFjuR7YxVWfbAr+EV/g3/hps0dGs2R0mOzzn3+vrLT2NhrOv+XWhdaV+9dn3xRufmrdt37i4t3/tmkkxzGPIkTvSuzwzEQsHQChvDbqqBST+GHf/wfanvHIE2IlFf7SSFkWSREqHgzCI1Xm6tUAU/eCIlU0FOwyKnfhIHZiLxV3TmRMlRlcwecHZB8mvJ9xtKFqOSqQB0WWJDTI6K8nOptg06RTXFH31YPSVsmPw4AzRV65BXnYZBQzDTS9yQIw2gZoZpDy0BhFjQlJST2qQjnKTb99aJ239efjx0xhDaPvUhEipnWrMJ+nBx2D61SUqOxy7p+U/IG9JF2ONrM7pHQ8147hX5oOjSE2ToyXiwP+jREz5Fa5R2MPK8D11nCQaEzzpPa8jrIHIuilTZe3XWDbdDAWc+rZVqER1YWpxSIfG0gTklZ5Vv+OOlVbfvVo1cBF4NVp26bY+XF77TIOGZBGV5zIzZ89zUjnA0K3hc7nBmIGX8kEWwh1AxCWaUVwtdkMfIBCRMNL7Kkoo9H5H7ci5DzqQpDypGlkfPNLWSvEzby2z4cpQLlWYWFJ8OHGYxsQkp7wgJhAZu4wkCxrXA2gk/YLiiFm/S3ChV7hT4fF3HmRI8CaDBxvbYaoakASuZUOUs87daVLep4jBZSXa3RCSsWf+El1Stf8Djd7j234h74jV34CIYDvqv+u6XZ6ubm/XmLDorziOn63jOC2fT+ehsO0OHt362frV+t/60H7Rft9+1t6bW1kIdc9+Za+3P/wC0FYzA</latexit>
142

Representation of Convex Closed Sets

<latexit sha1_base64="VQ3zlHG7iX07YacPnSr7ipLZSHI=">AAADuHicbVJdb9MwFHUWPkb52uCRl0CFNKSqSiYQIF6KQIIHHopE2aS2Ko5zk1n1R2Q7o1XU38Ov4RXEv+E6DRPN5ij28TnH9rXvTUvBrYvjP8FeeO36jZv7t3q379y9d//g8MFXqyvDYMK00OY0pRYEVzBx3Ak4LQ1QmQo4SZfvvH5yDsZyrb64dQlzSQvFc86oQ2pxGIxmCr4zLSVVWT3LN/Us1SKza4nDprcjSoaqpO6M0UtS2kpp2lEqgUqlMjA+xI6ozze+u1IbgylRLXGYPW4+DzumVFSApuYd6mbSMRjILnSPO3JhANSFYTtDSwY5BrQl5bo1mQIvGQ+TQRQPX/gu2fRWi3hx0I+HcdOiyyBpQZ+0bYxv/m2WaVZJUI4Jau00iUs3r6lxnAn/DJWFkrIlLWCKUFEJdl430Wyip8hkUa4N/spFDfv/ijqVOzvUVFqfTVzp82O7miev0qaVy1/Na67KyoFi24PzSkROR76QoowbYE6sEVBmOMYesTNqKHNYbjunNHuXwHbjWlWKM51BhxVu5QxF0oKTlCt/y/qt4U3JNRxu5smj97zgzg4+YSWrwQfM0fLZPyPmJOlm4DKYHA9fD+PPz/ujUZucffKIPCFHJCEvyYh8JGMyISz4EfwMfgW/wzchDYuQb617QbvmIdlpofkLHB5DAA==</latexit>
x0
C

Corollary: Any convex closed set C coincides with the intersection


C0 of all closed halfspaces that contain C.
<latexit sha1_base64="yEbTwnf7fwgXdYLufSWe+tGBNms=">AAADt3icbVJbb9MwFHYWLqPcNnjkJVAhhlRVCQJxeaFoSPDAQ5Eom9RWm+OctFZ9CbYzVkX9O/waXkHi33CchommcxT78/d9to99TloIbl0c/wl2witXr13fvdG5eev2nbt7+/e+Wl0aBiOmhTbHKbUguIKR407AcWGAylTAUbo49PrRGRjLtfrilgVMJZ0pnnNGHVIn+8HbiYLvTEtJVVZN8lU1SbXI7FLisOpsiJKhKqmbM7olpY2Upi2lFKiUKgPjQ2yJ+mzlu0u1IZgC1QKHycP687BlSkUJaKrfoaonLYOB7EL3uCXPDIC6MKxnaMkgx4DWpFw2JjPDS8b9pBfF/Re+S1adwycne924H9ct2gZJA7qkaUN88tNJplkpQTkmqLXjJC7ctKLGcSb8K5QWCsoWdAZjhIpKsNOqDmYVPUYmi3Jt8Fcuqtn/V1Sp3NihotL6ZOJKnx7b1jx5mTYuXf5qWnFVlA4UWx+clyJyOvJ1FGXcAHNiiYAywzH2iM2pocxhtW2cUu9dANuM67xUnOkMWqxw585QJC04Sbnyt6zeGV5XXM3hZp48eM9n3NneJyxk1fuAKVo8/WfEnCTtDGyD0bP+6378+Xl3MGiSs0sekEfkgCTkJRmQj2RIRoQFP4Kfwa/gd/gmPA3zcL627gTNmvtko4Xf/gIxIEJZ</latexit>

Proof: It is clear that C ✓ C0, and thus it remains to be shown that <latexit sha1_base64="4KMqxDXRB+XAAlsVUcmRuVfDK6w=">AAADwnicbVLfb9MwEHYWYKP82AaPvAQqxJCqKkFMwFtRQSDEQ5Eom9RUw3EunVXbyWxnrAr9o/hrEG/wn3BOw0TTOYp9/r7v7PPdJYXgxobhL2/Lv3b9xvbOzc6t23fu7u7t3/ti8lIzGLNc5Po4oQYEVzC23Ao4LjRQmQg4SuZDxx+dgzY8V5/tooCppDPFM86oRehk3/sQK/jGcimpSqs4W1ZxkovULCQuy84aKRmyktpTRjeopKGSpMWUAplSpaBdiC0yP1+66UpuBLpAtsAlflh/zmyJElECiuo8VPWmJdCQXvLObtEzDaAuBasdSlLIMKAVKBeNSM/wkWE/6gVh/9BN0bIzjE2ZGLBwFgyfnOx1w35Yj2DTiBqjS5oxwux/jdOclRKUZYIaM4nCwk4rqi1nwiWkNFBQNqczmKCpqAQzreq4lsFjRNIgyzX+ygY1+r9Hlci1EyoqjasrerpKmTbnwKu4SWmzl9OKq6K0oNjq4qwUgc0D11JByjUwKxZoUKY5xh6wU6ops9h4a7fUZxfA1uO6KBVneQotVNgLqymCmF9JuXKvrF5rXjdfjeFhDjx4w2fcmt5H7GnVe4fVmj/9J8SaRO0KbBrjZ/1X/fDT8+5g0BRnhzwgj8gBicgLMiDvyYiMCfN+eD+9394f/60/9898s5JueY3PfbI2/O9/AQXjRvI=</latexit>

0
C ✓ C. For any x0 2 / C there is a closed halfspace that contains C <latexit sha1_base64="rmDxYzdr6EXAw7VVmq+sQ03ytR8=">AAADw3icbVJdb9MwFHUWPkb52AaPvAQqxJCqKkEg4K2oCJDgoUiUTWqq4Tg3nVXbCbYzVkX5U/waeIRfwnUaJprOUezrc861r++9SSG4sWH4y9vxr1y9dn33Ru/mrdt39vYP7n4xeakZTFkucn2cUAOCK5habgUcFxqoTAQcJcux44/OQBueq892VcBc0oXiGWfUInRy4H2IFXxnuZRUpVWc1VWc5CI1K4lL3dsgJUNWUnvK6BaVtFSSdJhSIFOqFLQLsUPmZ7WbLuUmoAtkC1ziB83nzI4oESWgqMlD1Ww6Ag3pBe/sDr3QAOpCsN6hJIUMA1qDctWK9AIfGQ6jQRAOn7spqnvjx7EpEwMWvgXj4cl+PxyGzQi2jag1+qQdE0z/1zjNWSlBWSaoMbMoLOy8otpyJlxGSgMFZUu6gBmaikow86oJrA4eIZIGWa7xVzZo0P89qkRunFBRaVxh0dOVynQ5B17GzUqbvZxXXBWlBcXWF2elCGweuJ4KUq6BWbFCgzLNMfaAnVJNmcXO27ilObsAthnXeak4y1PooMKeW00RxPxKypV7ZfVa86b7GgwPc+DhG77g1gw+YlOrwTss1/LJPyHWJOpWYNuYPh2+GoafnvVHo7Y4u+Q+eUgOSURekBF5TyZkSpj3w/vp/fb++G994WvfrqU7Xutzj2wMv/4LDCVHKg==</latexit>

0
x
but not 0. Thus, x0 2 / C, which implies
<latexit sha1_base64="udWfAtvVsaeVxfXEBz53b1uH8MI=">AAADwHicbVLLjtMwFHUmPIbymoElm0CFNEhVlSAQsCsqErNgUSTKjNRUxXFuO1b9CLYztET9Jr6GBRv4Fa7TMKKZcRT7+Jxj+9r3ZoXg1sXxr2AvvHb9xs39W53bd+7eu39w+OCz1aVhMGZaaHOaUQuCKxg77gScFgaozAScZMuh10/OwViu1Se3LmAq6ULxOWfUITU7DI5TBd+YlpKqvErnmyrNtMjtWuKw6eyIkqEqqTtj9JKUNVKWtZRSoFKqHIwPsSXq843vrtRGYApUCxzSx/XnYcuUiRLQVL9DVU9aBgP5he5xS14YAHVh2M7QksMcA9qSct2YzAIvGfeTXhT3X/ou2XRWszhV2nEVDWcH3bgf1y26DJIGdEnTRvj2X9Jcs1KCckxQaydJXLhpRY3jTPjnKC0UlC3pAiYIFZVgp1Ud1SZ6ikwezbXBX7moZv9fUWVyZ4eKSuuziit9nmxb8+RV2qR089fTiquidKDY9uB5KSKnI19QUc4NMCfWCCgzHGOP2Bk1lDksu51T6r0LYLtxrUrFmc6hxQq3coYiacFJypW/ZfXW8Lr0ag438+TRO77gzvY+YEWr3nvM1fLZPyPmJGln4DIYP++/6ccfX3QHgyY5++QReUKOSEJekQE5JiMyJiz4EfwMfgd/wmHIQx1+3Vr3gmbNQ7LTwu9/AVkbRjc=</latexit>

<latexit sha1_base64="VQ3zlHG7iX07YacPnSr7ipLZSHI=">AAADuHicbVJdb9MwFHUWPkb52uCRl0CFNKSqSiYQIF6KQIIHHopE2aS2Ko5zk1n1R2Q7o1XU38Ov4RXEv+E6DRPN5ij28TnH9rXvTUvBrYvjP8FeeO36jZv7t3q379y9d//g8MFXqyvDYMK00OY0pRYEVzBx3Ak4LQ1QmQo4SZfvvH5yDsZyrb64dQlzSQvFc86oQ2pxGIxmCr4zLSVVWT3LN/Us1SKza4nDprcjSoaqpO6M0UtS2kpp2lEqgUqlMjA+xI6ozze+u1IbgylRLXGYPW4+DzumVFSApuYd6mbSMRjILnSPO3JhANSFYTtDSwY5BrQl5bo1mQIvGQ+TQRQPX/gu2fRWi3hx0I+HcdOiyyBpQZ+0bYxv/m2WaVZJUI4Jau00iUs3r6lxnAn/DJWFkrIlLWCKUFEJdl430Wyip8hkUa4N/spFDfv/ijqVOzvUVFqfTVzp82O7miev0qaVy1/Na67KyoFi24PzSkROR76QoowbYE6sEVBmOMYesTNqKHNYbjunNHuXwHbjWlWKM51BhxVu5QxF0oKTlCt/y/qt4U3JNRxu5smj97zgzg4+YSWrwQfM0fLZPyPmJOlm4DKYHA9fD+PPz/ujUZucffKIPCFHJCEvyYh8JGMyISz4EfwMfgW/wzchDYuQb617QbvmIdlpofkLHB5DAA==</latexit>

<latexit sha1_base64="9CYooAU+aCjWFKmFM97npZv+Lh0=">AAADwXicbVJdb9MwFHUWPkb52AaPvAQqxJCqKkEg4K1okwCJhyJRNqmpiuPcdKb+iGxntIr6n/g1SDzBT+E6LRPN5ij28TnH9rXvzUrBrYvjX8FOeO36jZu7tzq379y9t7d/cP+L1ZVhMGJaaHOaUQuCKxg57gSclgaozAScZPMjr5+cg7Fcq89uWcJE0pniBWfUITU9CD6kCr4zLSVVeZ0WqzrNtMjtUuKw6myJkqEqqTtj9JKUbaQsaymVQKVSORgfYkvU5yvfXakNwZSoljikj5rPw5YpExWgqXmHupm0DAbyC93jljwzAOrCsJ6hJYcCA1qTcrkxmRleMu4nvSjuv/RdsuospnGqtOMqOno63e/G/bhp0WWQbECXbNoQH/9rmmtWSVCOCWrtOIlLN6mpcZwJ/x6VhZKyOZ3BGKGiEuykbsJaRU+QyaNCG/yVixr2/xV1Jrd2qKm0Pq240ifKtjVPXqWNK1e8ntRclZUDxdYHF5WInI58RUU5N8CcWCKgzHCMPWJn1FDmsO62Tmn2LoFtx7WoFGc6hxYr3MIZiqQFJylX/pb1W8Ob2ms43MyTh8d8xp3tfcSSVr13mKz5s39GzEnSzsBlMHref9OPP73oDgab5OySh+QxOSQJeUUG5D0ZkhFhwY/gZ/A7+BMeh9/CMjRr606wWfOAbLWw/gtSskZo</latexit>

<latexit sha1_base64="Wav8lakxhq2JL0h5TKAVxoPlHA0=">AAAD/HicbVPLjtMwFE1bHkN5zBR2zCZQIQapilIEAnZFRYIFi4IoM1JTBtu5aa3aTrCdYaqow4ZfYYXY8i98Az/BdVoGmtZR7JNzjq8f94Zmghsbhr9q9caFi5cu71xpXr12/cbuXuvme5PmmsGQpSLVR5QYEFzB0HIr4CjTQCQVcEhnfacfnoA2PFXv7DyDsSQTxRPOiEXquFX7Gin4zFIpiYqLKFkUEU1FbOYSh0VzTZQMVUnslJENia4kSitKLlDJVQzabbEipicL123VBqAzVDMcojvl42DFREUOaCrvoSg/KgYN8bnucEWeaAB1blh+oSWGBDe0JOV8ZdITPGQYdDt+GDx2XRedkvpvP6iIEjYzgpip349MTg1Y+ORvEe+fRTxJzhD8s/WD5vFeOwzCsvmboLsCbW/VBpi3j1GcslyCsgxDm1E3zOy4INpyJtxV5gYyXJdMYIRQEQlmXJQnWvj3kIn9JNX4KuuX7P8zCirXIhREGlcRONPl2FQ1R27TRrlNno4LrrLcgmLLhZNc+Db1XTH6MdfArJgjIExz3LvPpkQTZrFk11YpY2fA1vd1mivO0hgqrLCnVhMk8YIl4cqdsniueVm2JYfBHHnwgk+4NZ3X+DeozkvM8+zBXyPmpFvNwCYYPgyeBeGbR+1eb5WcHW/fu+sdeF3vidfzXnkDb+ix2u96q367vt/40vjW+N74sbTWa6s5t7y11vj5BwrmWtU=</latexit>
Rn \C ✓ Rn \C0 () C0 ✓ C.
143

Convex Hulls Revisited

Recall that the convex hull of a set X, denoted by conv(X), is the


set of all convex combinations of points in X.

Convex hull of X
X (a car)

Corollary: The closure of conv(X) coincides with the intersection


of all (closed) halfspaces containing X.
144

Convex Hull of a Function


Definition: We denote by conv(f) the convex hull (or convex
envelope) of a function f, i.e., the largest convex function  f . <latexit sha1_base64="+W3XvIaUDKLjR5UToRi+JjL8QLw=">AAADunicbVJbb9MwFHYWLmPcNnjkJVAhDamqEgQCHpDKRYIHHopE2aS2Go5zkln1JbKdsSrqD+LX8Mr+DcdpmEg6R7E/f99n+9jnpKXg1sXxRbATXrt+4+burb3bd+7eu79/8OC71ZVhMGVaaHOcUguCK5g67gQclwaoTAUcpcsPXj86A2O5Vt/cqoSFpIXiOWfUIXVyELyfK/jJtJRUZfU8X9fzVIvMriQO672OKBmqkrpTRrektJXStKdUApVKZWB8iD1Rn619d6U2AVOiWuIwf9x8HvZMqagATc071M2kZzCQXeoe9+TCAKhLw2aGlgxyDGhDylVrMgVeMh4lwygevfRdgk4BUX6yP4hHcdOibZC0YEDaNsFX/zHPNKskKMcEtXaWxKVb1NQ4zoR/iMpCSdmSFjBDqKgEu6ibeNbRU2SyKNcGf+Wihv1/RZ3Kzg41ldbnE1f6DNm+5smrtFnl8teLmquycqDY5uC8EpHTkS+lKOMGmBMrBJQZjrFH7JQayhwWXOeUZu8SWDeu80pxpjPoscKdO0ORtOAk5crfsn5neFN0DYebefLwIy+4s8MvWMtq+AmztHz2z4g5SfoZ2AbT56M3o/jri8F43CZnlzwiT8ghScgrMiafyYRMCQt+Bb+DP8FF+DZkIQ+XG+tO0K55SDotdH8BzD1DwA==</latexit>

conv(f)

x x

Note: The epigraph of conv(f) is essentially given by the convex


hull of epi(f). We have
epi(conv(f)) = {(x, t) : (x, s) 2 conv(epi(f)) 8s > t}.
<latexit sha1_base64="AD4BF0BhLXiqBVg5byzwz8l0pRM=">AAAEC3icbVPLjtMwFE1bHsPwmoElG0OF1KKqShCIhwQqAgkWLIpEmZGaanCcm45V24lsp7SKwiew4VdYIbZ8BL/Ciuu0jNp0HMU+Oedc+9rXiTLBjfX9P41m68LFS5f3ruxfvXb9xs2Dw1ufTJprBiOWilQfR9SA4ApGllsBx5kGKiMBR9HstdOP5qANT9VHu8xgIulU8YQzapE6OWx8DxV8YamUVMVFmJRFGKUiNkuJQ7m/JUqGqqT2lNEdKVpLUVRTcoFKrmLQLsWamM5L152rDUFnqGY4hHerx8GaKRI5oKk6h6L6qBk0xGe6wzV5qgHUmWH1hZYYEkxoRcrl2qSnuEm/H/SI33/sugCdFha2gIyXnRVkqZqXnaTbfUHCorPo2e5z7E035IpsOjYC0fw1TFJNhSDmpQ3L/slB2+/7VSO7IFiDtrduQyzj5zBOWS5BWSaoMePAz+ykoNpyJtzJ5gYyymZ0CmOEikowk6LaYEnuIxMTzABfZUnFbkYUkdyaoaDSuAuCka7kpq458jxtnNvk6aTgKsstKLZaOMkFsSlxd5PEXAOzYomAMs0xd8JOqabM4g3eWqWaOwO2ndciV5ylMdRYYRdWUyQNWEm5crssXmle3eKKw8kc2XnDp9ya3nv8OVTvLZZ91v1vxJoE9QrsgtHD/rO+/+FRezBYF2fPu+Pd8zpe4D3xBt47b+iNPNb42yTNbvNB61vrR+tn69fK2mysY257W631+x+zY2HO</latexit>
145

Envelope Representations
Theorem: If f is proper, convex and closed, then it is the pointwise
supremum of all affine functions  f . <latexit sha1_base64="+W3XvIaUDKLjR5UToRi+JjL8QLw=">AAADunicbVJbb9MwFHYWLmPcNnjkJVAhDamqEgQCHpDKRYIHHopE2aS2Go5zkln1JbKdsSrqD+LX8Mr+DcdpmEg6R7E/f99n+9jnpKXg1sXxRbATXrt+4+burb3bd+7eu79/8OC71ZVhMGVaaHOcUguCK5g67gQclwaoTAUcpcsPXj86A2O5Vt/cqoSFpIXiOWfUIXVyELyfK/jJtJRUZfU8X9fzVIvMriQO672OKBmqkrpTRrektJXStKdUApVKZWB8iD1Rn619d6U2AVOiWuIwf9x8HvZMqagATc071M2kZzCQXeoe9+TCAKhLw2aGlgxyDGhDylVrMgVeMh4lwygevfRdgk4BUX6yP4hHcdOibZC0YEDaNsFX/zHPNKskKMcEtXaWxKVb1NQ4zoR/iMpCSdmSFjBDqKgEu6ibeNbRU2SyKNcGf+Wihv1/RZ3Kzg41ldbnE1f6DNm+5smrtFnl8teLmquycqDY5uC8EpHTkS+lKOMGmBMrBJQZjrFH7JQayhwWXOeUZu8SWDeu80pxpjPoscKdO0ORtOAk5crfsn5neFN0DYebefLwIy+4s8MvWMtq+AmztHz2z4g5SfoZ2AbT56M3o/jri8F43CZnlzwiT8ghScgrMiafyYRMCQt+Bb+DP8FF+DZkIQ+XG+tO0K55SDotdH8BzD1DwA==</latexit>

Proof idea: When f is proper, convex and closed, then epi(f) is a


convex closed set. By the separating hyperplane theorem, it is the
intersection of all closed halfspaces containing epi(f).

Every (non-vertical) halfspace ◆ epi(f)


is the epigraph of an affine function. Thus,
<latexit sha1_base64="sswA1lOTqNVxgo9rqJ1kUz5c8oY=">AAADzXicbVLLjtMwFHUmPIbymoElm0CF1JGqKhmBgF0RSLBgUSTKjNRUg+PcdKzaTrCd0iqULT/Fj7BlCx/BdRpGNB1HsY/PObavfW9SCG5sGP709vwrV69d37/RuXnr9p27B4f3Ppq81AzGLBe5Pk2oAcEVjC23Ak4LDVQmAk6S+SunnyxAG56rD3ZVwFTSmeIZZ9QidXbojWMFX1guJVVpFWfrKk5ykZqVxGHd2RIlQ1VSe87ojpQ0UpK0lFKgUqoUtAuxJeaLtesu1UagC1QLHOKH9edgy5SIEtBUv0NVT1oGDemF7nBLnmkAdWHYzNCSQoYBbUi5akx6hpcMB1E/CAdPXReh05SFAQufg9jC0lZQ8HUvO+qcHXTDQVi3YBdEDeiSpo0wEZ/iNGelBGWZoMZMorCw04pqy5lwb1MaKCib0xlMECoqwUyrOsR18BiZNMhyjb+yQc3+v6JK5NYOFZXGpRhXuqSZtubIy7RJabPn04qrorSg2ObgrBSBzQNXXUHKNTArVggo0xxjD9g51ZRZrMGtU+q9C2DbcS1LxVmeQosVdmk1dc8NVlKu3C2rl5rXdVhzuJkje6/5jFvTf4flrfpvMHHzo39GzEnUzsAuGB8PXgzC90+6w2GTnH3ygDwiPRKRZ2RI3pIRGRPm/fB+eb+9P/7IX/hf/W8b657XrLlPtpr//S+x1EvZ</latexit>

epi(f)
f(x) = sup y> x + t
y2Rn ,t2R

<latexit sha1_base64="VgWND6MLV4trArUqwLjJBrBUGZg=">AAAEcXicbVPPb9MwFM66AqP82uAGF7OJqdOqKEUg4IBUBBIcOBRE2aS52xznpbXmOMF2RqOoHLnCv8f/wYUbz2k2aLYXxX7v+75nP/8KMymMDYJfK63V9pWr19aud27cvHX7zvrG3c8mzTWHEU9lqvdDZkAKBSMrrIT9TANLQgl74clrx++dgjYiVZ9skcE4YRMlYsGZRehoY+UPVfCVp0nCVFTSeF7SMJWRKRLs5p0lMuHIJsxOObtAhTUVhg0ml8jkKgLtSmyQ6encNZdyQ9AZshl29GH1ObchCmUOKKr2oayChkBDdM47v0FPNIA6FywilEQQY0ELMClqkZ7gIgO/3yOB/9Q1fVSGMBGqZFqzAhUc9yXuznbI9sttGgmTSVYYW0gg1OTZUVlQoXCvyMdD1bNn/pwUh9SmGZntWko7243Mb2jUwsyWxrf+3IVnCXTKbJUl4QuJu4twB+VxqpmUNf9vTr9DAVe9qLZztL4V+EFl5KLTr50tr7YhXpZjGqU8T0BZLpkxB/0gs2Mczwou3fnlBjLGT9gEDtBVLAEzLqttnJNHiEQEK8NfWVKh/2eUYbI0QskS464hZrqLZZqcAy/jDnIbPx+XQmW5BcUXE8e5JDYl7gWQSGjgVhboMK4F1k74lGnGLb6TpVmqsTPgy3XNciV4GkEDlXZmNUPQgE2YUG6V5SstqrdSYTiYA7tvxERY03uPT1D13uLlOtk5E+KZ9JsncNEZPfZf+MGHJ1uDQX04a94Db9Pren3vmTfw3nlDb+Tx1nHre+tH6+fq7/b9NmlvLqStlTrnnrdk7d2/JL6GyA==</latexit>
s.t. y> x̂ + t  f(x̂) 8x̂ 2 Rn .

Corollary: If f is an arbitrary function with conv(f) proper, then


cl(conv(f)) is the pointwise supremum of all affine functions  f . <latexit sha1_base64="+W3XvIaUDKLjR5UToRi+JjL8QLw=">AAADunicbVJbb9MwFHYWLmPcNnjkJVAhDamqEgQCHpDKRYIHHopE2aS2Go5zkln1JbKdsSrqD+LX8Mr+DcdpmEg6R7E/f99n+9jnpKXg1sXxRbATXrt+4+burb3bd+7eu79/8OC71ZVhMGVaaHOcUguCK5g67gQclwaoTAUcpcsPXj86A2O5Vt/cqoSFpIXiOWfUIXVyELyfK/jJtJRUZfU8X9fzVIvMriQO672OKBmqkrpTRrektJXStKdUApVKZWB8iD1Rn619d6U2AVOiWuIwf9x8HvZMqagATc071M2kZzCQXeoe9+TCAKhLw2aGlgxyDGhDylVrMgVeMh4lwygevfRdgk4BUX6yP4hHcdOibZC0YEDaNsFX/zHPNKskKMcEtXaWxKVb1NQ4zoR/iMpCSdmSFjBDqKgEu6ibeNbRU2SyKNcGf+Wihv1/RZ3Kzg41ldbnE1f6DNm+5smrtFnl8teLmquycqDY5uC8EpHTkS+lKOMGmBMrBJQZjrFH7JQayhwWXOeUZu8SWDeu80pxpjPoscKdO0ORtOAk5crfsn5neFN0DYebefLwIy+4s8MvWMtq+AmztHz2z4g5SfoZ2AbT56M3o/jri8F43CZnlzwiT8ghScgrMiafyYRMCQt+Bb+DP8FF+DZkIQ+XG+tO0K55SDotdH8BzD1DwA==</latexit>
146

Weak Separating Hyperplane Theorem


Theorem: Consider any convex set C and a point x0 2
/ C. Then, <latexit sha1_base64="AM3RfJfcV6lWlaXnMOLuzWWmXSc=">AAADwXicbVJdb9MwFHUWPkb52uCRl0CFNKSqShAIeCvaJEDioUiUTWqq4ji3nak/ItsZraL+J34NEk/wU7hOw0TTOYp9fM6xfe17s0Jw6+L4V7AXXrt+4+b+rc7tO3fv3T84fPDF6tIwGDEttDnLqAXBFYwcdwLOCgNUZgJOs8Wx108vwFiu1We3KmAi6VzxGWfUITU9DD6kCr4zLSVVeZXO1lWaaZHblcRh3dkSJUNVUnfO6I6UNVKWtZRSoFKqHIwPsSXqi7XvrtSGYApUCxzSx/XnYcuUiRLQVL9DVU9aBgP5pe5xS54bAHVp2MzQksMMA9qQctWYzBwvGfeTXhT3X/ouWXeW0zhV2nEVHXemB924H9ct2gVJA7qkaUN8/K9prlkpQTkmqLXjJC7cpKLGcSb8e5QWCsoWdA5jhIpKsJOqDmsdPUUmj2ba4K9cVLP/r6gyubVDRaX1acWVPlG2rXnyKm1cutnrScVVUTpQbHPwrBSR05GvqCjnBpgTKwSUGY6xR+ycGsoc1t3WKfXeBbDtuJal4kzn0GKFWzpDkbTgJOXK37J6a3hdezWHm3ny6ITPubO9j1jSqvcOk7V49s+IOUnaGdgFo+f9N/3404vuYNAkZ588Ik/IEUnIKzIg78mQjAgLfgQ/g9/Bn/Ak/BYWodlY94JmzUOy1cLqLxaoRks=</latexit>

there exist a 6= 0 and b with


<latexit sha1_base64="Ms0ifWhN4U31rN9lNYRQncNsgOc=">AAADvXicbVJbb9MwFHYWLqPcNnjkJVAhDamqkgkEvBUBggceikTZpLYajnOSWfUl2M62Kuo/4tfwhuDHcJyGiaZzFPv4+z4fH59z0lJw6+L4V7ATXrt+4+burd7tO3fv3d/bf/DV6sowmDAttDlOqQXBFUwcdwKOSwNUpgKO0sVbzx+dgbFcqy9uWcJc0kLxnDPqEDrZD97PFJwzLSVVWT3LV/Us1SKzS4nLqrdBSoaspO6U0S0qbak07TCVQKZSGRgfYofUZys/XcmNwZTIlrjMHjefNzuiVFSAoiYPdbPpCAxkl7y3O3RhANSlYL1DSQY5BrQG5bIVmQIfGQ+TQRQPX/gpWfUoevsexb2TvX48jJsRbRtJa/RJO8aY+G+zTLNKgnJMUGunSVy6eU2N40z4XFQWSsoWtIApmopKsPO6CWkVPUUki3Jt8FcuatD/T9Sp3PBQU2l9SfGkL5Ltch68iptWLn81r7kqKweKrS/OKxE5HfluijJugDmxRIMywzH2iJ1SQ5nDntu4pfFdAtuM66JSnOkMOqhwF85QBC04Sbnyr6zfGN70XYOhMw8evOMFd3bwCdtZDT5goRbP/gmxJkm3AtvG5HD4ehh/ft4fjdri7JJH5Ak5IAl5SUbkIxmTCWHBj+Bn8Dv4E47CPBShWkt3gvbMQ7IxwvO/HW1Ehg==</latexit>

a> x  b 8x 2 C
<latexit sha1_base64="kh3Sq3/GDeiZrkwEcQ9ZT2LS4no=">AAAD+nicbVPNbhMxEN4k/JTw0xYuIC6GCKlI1WpTgYBbUJHgwCFIhFbKhuD1TlIr/tna3pJotRx4FU6IKy/DI/AWjDdpRTb1au3x931jjz3jJBPcuij602i2rly9dn3rRvvmrdt3tnd2736yOjcMBkwLbY4TakFwBQPHnYDjzACViYCjZHbo+aMzMJZr9dEtMhhJOlV8whl1CI13G99jBV+ZlpKqtIgnZREnWqR2IXEo22ukZMhK6k4Y3aCSFZUkNSYXyOQqBeNDrJH6rPTdpVwfTIZshkP8qPq8WRMlIgcUVfdQVJOawEB6wXu7Rk8NgLoQLGcoSWGCAS1BuViJzBQPGYXdfRKFz33XLdv0c+x0RuaxgFOSfIsn2lAhcM4VOYxPc5oSvBs9L3C3kiyBc59xFE+9Vzje6URhVDWyaXRXRidYtT5m7UucapZLUI4Jau2wG2VuVFDjOBP+InMLGWUzOoUhmopKsKOiOk9JniCSEowUf+VIhf7vUSRybYWCSuvrAT19hm2d8+Bl3DB3k5ejgqssd6DYcuNJLojTxJciSbkB5sQCDcoMx9gJO6GGMocFu7ZLtXYGbD2uea440ynUUOHmzlAELThJufKnLF4bXhVtheFiHtx7w6fc2f33+BbU/lvM8uzpuRBz0q1nYNMYHISvwujDs06vt0rOVvAweBzsBd3gRdAL3gX9YBCwxt/mdvN+80GrbP1o/Wz9WkqbjZXPvWCttX7/A6y6Wsc=</latexit>
and a> x0 b.

Example: a

x0
<latexit sha1_base64="VQ3zlHG7iX07YacPnSr7ipLZSHI=">AAADuHicbVJdb9MwFHUWPkb52uCRl0CFNKSqSiYQIF6KQIIHHopE2aS2Ko5zk1n1R2Q7o1XU38Ov4RXEv+E6DRPN5ij28TnH9rXvTUvBrYvjP8FeeO36jZv7t3q379y9d//g8MFXqyvDYMK00OY0pRYEVzBx3Ak4LQ1QmQo4SZfvvH5yDsZyrb64dQlzSQvFc86oQ2pxGIxmCr4zLSVVWT3LN/Us1SKza4nDprcjSoaqpO6M0UtS2kpp2lEqgUqlMjA+xI6ozze+u1IbgylRLXGYPW4+DzumVFSApuYd6mbSMRjILnSPO3JhANSFYTtDSwY5BrQl5bo1mQIvGQ+TQRQPX/gu2fRWi3hx0I+HcdOiyyBpQZ+0bYxv/m2WaVZJUI4Jau00iUs3r6lxnAn/DJWFkrIlLWCKUFEJdl430Wyip8hkUa4N/spFDfv/ijqVOzvUVFqfTVzp82O7miev0qaVy1/Na67KyoFi24PzSkROR76QoowbYE6sEVBmOMYesTNqKHNYbjunNHuXwHbjWlWKM51BhxVu5QxF0oKTlCt/y/qt4U3JNRxu5smj97zgzg4+YSWrwQfM0fLZPyPmJOlm4DKYHA9fD+PPz/ujUZucffKIPCFHJCEvyYh8JGMyISz4EfwMfgW/wzchDYuQb617QbvmIdlpofkLHB5DAA==</latexit>

x0 sits on the
C
<latexit sha1_base64="VQ3zlHG7iX07YacPnSr7ipLZSHI=">AAADuHicbVJdb9MwFHUWPkb52uCRl0CFNKSqSiYQIF6KQIIHHopE2aS2Ko5zk1n1R2Q7o1XU38Ov4RXEv+E6DRPN5ij28TnH9rXvTUvBrYvjP8FeeO36jZv7t3q379y9d//g8MFXqyvDYMK00OY0pRYEVzBx3Ak4LQ1QmQo4SZfvvH5yDsZyrb64dQlzSQvFc86oQ2pxGIxmCr4zLSVVWT3LN/Us1SKza4nDprcjSoaqpO6M0UtS2kpp2lEqgUqlMjA+xI6ozze+u1IbgylRLXGYPW4+DzumVFSApuYd6mbSMRjILnSPO3JhANSFYTtDSwY5BrQl5bo1mQIvGQ+TQRQPX/gu2fRWi3hx0I+HcdOiyyBpQZ+0bYxv/m2WaVZJUI4Jau00iUs3r6lxnAn/DJWFkrIlLWCKUFEJdl430Wyip8hkUa4N/spFDfv/ijqVOzvUVFqfTVzp82O7miev0qaVy1/Na67KyoFi24PzSkROR76QoowbYE6sEVBmOMYesTNqKHNYbjunNHuXwHbjWlWKM51BhxVu5QxF0oKTlCt/y/qt4U3JNRxu5smj97zgzg4+YSWrwQfM0fLZPyPmJOlm4DKYHA9fD+PPz/ujUZucffKIPCFHJCEvyYh8JGMyISz4EfwMfgW/wzchDYuQb617QbvmIdlpofkLHB5DAA==</latexit>

boundary of C
(open)
147

Weak Separating Hyperplane Theorem*


Proof: Assume wlog that x0 = 0. We first show that there exists a <latexit sha1_base64="LwvxctBlc8gDVleWR588+/1Av1M=">AAADu3icbVJdb9MwFHUWPkb52uCRl0CFNKQqShAIeAAVDQkeeCgSZZPaqjjOTWZqO5HtjFZR/xC/htfxa7hOw0TTOYp9fM6xfe17k1JwY6Powtvzr12/cXP/Vu/2nbv37h8cPvhmikozGLNCFPo0oQYEVzC23Ao4LTVQmQg4SRbHTj85B214ob7aVQkzSXPFM86oRWp+6B1PFfxkhZRUpfU0W9fTpBCpWUkc1r0tUTJUJbVnjO5ISSslSUepBCqVSkG7EDticb523ZXaCHSJaonD9HHzOdgxJaICNDXvUDeTjkFDeqk73JFzDaAuDZsZWlLIMKANKVetSed4ySiMB0EUvnRdvO4t59HbKJwf9KMwalqwC+IW9EnbRvjs36dpwSoJyjJBjZnEUWlnNdWWM+FeojJQUragOUwQKirBzOomoHXwFJk0yAqNv7JBw/6/ok7k1g41lcYlFFe6FJmu5sirtElls9ezmquysqDY5uCsEoEtAldLQco1MCtWCCjTHGMP2BnVlFmsuK1Tmr1LYNtxLSvFWZFChxV2aTVF0oCVlCt3y/q95k3VNRxu5sijDzzn1gw+YzGrwUdM0+LZPyPmJO5mYBeMn4dvwujLi/5w2CZnnzwiT8gRickrMiSfyIiMCfN+eb+9C++P/85P/R++2Fj3vHbNQ7LV/OovT1pDuQ==</latexit>

sequence of points xi 2
/ cl(C) with xi ! 0. <latexit sha1_base64="HqR+mDcCz+5AL4pce0sV3mb/pro=">AAADy3icbVJdb9MwFHUWPkb52uCRl0CF1ElVlUygjbeiIcEDQkVQNqmpiuPcdFb9EdnOaAl55E/xT3jjFX4Fdhomms5R7ONzju1r35vkjGoThj+9Hf/a9Rs3d291bt+5e+/+3v6DT1oWisCYSCbVWYI1MCpgbKhhcJYrwDxhcJosTpx+egFKUyk+mlUOU47ngmaUYGOp2b73IRbwhUjOsUjLOKvKOJEs1Stuh6qzIXJiVY7NOcFbUtJISdJSCmaVQqSgXIgtUV5UrrtSG4HKrZrbIX5cfw62TAkrwJrqdyjrScugIL3UHW7JcwUgLg3rmbWkkNmA1iRfNSY1t5cMB1E/CAfPXRdVneWMxkIaKoLYwNKUhFW9k4PZXjcchHULtkHUgC5q2sim4XOcSlJwEIYwrPUkCnMzLbEylDD3MoWGHJMFnsPEQoE56GlZB1gFTy2TBplU9hcmqNn/V5QJ39ihxFy7BNuVLmW6rTnyKm1SmOx4WlKRFwYEWR+cFSwwMnC1FaRUATFsZQEmitrYA3KOFSbGVuDGKfXeOZDNuJaFoESm0GKZWRqFLanBcEyFu2X5UtG6CmvObubI3is6p0b339riFv3XNm2Lg39Gm5OonYFtMD4cvBiE7591h8MmObvoEXqCeihCR2iI3qARGiPi/fB+eb+9P/473/hf/W9r647XrHmINpr//S/sBUsX</latexit>
<latexit sha1_base64="XTh/w4UEJacYr59P7dvfkmMUqrY=">AAADxnicbVJdi9NAFE22fqz1a1cffYkWYYVSElHUt4qCCiIVrLvQlDqZ3KRD5yPMTLYtoeCf8seIr/o/vJPGxaY7ITNnzjkzc2fuTQrOjA3Dn/5B58rVa9cPb3Rv3rp95+7R8b2vRpWawpgqrvRZQgxwJmFsmeVwVmggIuFwmizeOP30HLRhSn6x6wKmguSSZYwSi9Ts2P8US1hSJQSRaRVnmypOFE/NWuCw6e6IgqIqiJ1TsicljZQkLaXkqJQyBe1CbInqfOO6S7UR6ALVAof4Yf052DIlvAQ01e9Q1ZOWQUN6oTvcknMNIC8M2xlaUsgwoC0p1o1J53jJcBD1g3Dw3HXRpruasVizfG6J1mqJ3OyoFw7CugX7IGpAz2vaCDPwLU4VLQVISzkxZhKFhZ1WRFtGuXuU0kBB6ILkMEEoiQAzrerYNsFjZNIgUxp/aYOa/X9FlYidHSoijMstrnTZMm3NkZdpk9JmL6cVk0VpQdLtwVnJA6sCV1ZByjRQy9cICNUMYw/onGhCLRbfzin13gXQ3bhWpWRUpdBiuV1ZTZA0YAVh0t2yeq1ZXYA1h5s58uQty5k1/Y9Y17L/DjO2ePLPiDmJ2hnYB+Ong1eD8POz3nDYJOfQe+A98k68yHvhDb333sgbe9T/4f/yf/t/Oh86qlN2llvrgd+sue/ttM73v3ATSOg=</latexit>

Equivalent statement: 8ε > 0 : 9x 2 B(0, ε), x 2


/ cl(C) <latexit sha1_base64="LnJ467lgVFWkuFWmBYpLa7NgYKE=">AAAD/3icbZPLjtMwFIbTlstQbjPADoEMFVJHqqoUgbgsUGGQYMGiSJQZqakGxznpWPUlsp2hVdRZsOFVWCG2PAovwTNwnHZGNB1HcU7+77dz7OPEmeDWheGfWr1x4eKly1tXmlevXb9xc3vn1merc8NgyLTQ5iCmFgRXMHTcCTjIDFAZC9iPp3ue7x+DsVyrT26ewVjSieIpZ9ShdLhT+x4p+Mq0lFQlRZQuiijWIrFziY9Fcw1KhlRSd8ToBopXKI4rJBdIcpWA8SlWoD5e+O5cNgCTIc3wET0oLx9WTLHIAU3lPhTlS8VgIDnjPq7giQFQZ4blG1oSSDGhpSjnK5OZ4CLDbq9Dwu5T3/XQmWpDhSARZJYLrV6FL08imGHdLJlFXBHcNPKmHXZODbudEwRKO88czFzBxKK9t9s83G6F3bBsZDPorYJWsGoDLN2XKNEsl6AcE9TaUS/M3LigxnEm/G7mFjLKpnQCIwwVlWDHRbmoBXmESkIwe7yVI6X6/4gilmszFFRafyhwpC+zrTIvnsdGuUufjwuustyBYssPp7kgThN/HknCDTAn5hhQZjjmTtgRNZQ5PLVrXynnzoCt5zXLFWc6gYoq3MwZiqIFJylXfpXFa8PLk1tqOJkX22/5hDvb+YA/hOq8w1JPd0+NWJNetQKbwfBx90U3/Pik1e+virMV3A0eBu2gFzwL+sH7YBAMA1b7W79Tv1e/3/jW+NH42fi1tNZrqzG3g7XW+P0PtGpcHg==</latexit>
(1)

Contraposition: 9ε > 0 : x 2 cl(C) 8x 2 B(0, ε) (2)


<latexit sha1_base64="92J0OPCKGPdqkiWDpao1EbMXnog=">AAAEPnicbVNLb9QwEE43PEp5tXDkYqhAu9KyShCIhwQqFAkOHBbB0krNqjjOZGut7US2U3YVpf+NC/+BX8AJceXIOJuWblpHsSff983Y45nEueDGBsHPlY5/4eKly6tX1q5eu37j5vrGrS8mKzSDEctEpndjakBwBSPLrYDdXAOVsYCdeLrt+J1D0IZn6rOd5zCWdKJ4yhm1CO1vrPyIFHxjmZRUJWWUVmUUZyIxc4lLtbZESoaspPaA0TNU3FBx3GIKgUyhEtDuiC0yO6zcdC43BJ0jm+MS3a0fZ7ZEsSgARfU9lPVHS6AhOeGd3aInGkCdCBZfKEkgxQMtQDlvRHqCSQaDsE+CwRM3hdXaAxLBDKtkcM0NF5l6Fbw4IrOIKxJZmNmSiaq73TuK0kxTIRqmuUTyphv0jx17hESRC8glFh4MCf4H4cpW3dPxeuTlaWq7t7++GQyCepCzRtgYm14zhlj5r1GSsUKCskxQY/bCILfjkmrLmXDFKAzklE3pBPbQVFSCGZf1nVTkPiIJwZzwVZbU6GmPMpZLEUoqjesp9HS5mzbnwPO4vcKmz8aYZF5YUGyxcVoIYjPi2pkkXAOzYo4GZZrj2Qk7oJoyi02/tEsdOwe2fK5ZoTjLEmihws6spggasJJy5bIsX2teN36NYTAHdt/yCbem/wH/J9V/h50y7R0LsSZhuwJnjdGjwfNB8PHx5tZWU5xV7453z+t6offU2/Lee0Nv5LHOw86nTtQZ+9/9X/5v/89C2llpfG57S8P/+w8hH3M+</latexit>
=) 0 2 int(cl(C)) = int(C)
As (2) is false, (1) must be true, and thus the sequence exists!

/ cl(C) , there exist ai 6= 0 and bi with


As xi 2
<latexit sha1_base64="HqR+mDcCz+5AL4pce0sV3mb/pro=">AAADy3icbVJdb9MwFHUWPkb52uCRl0CF1ElVlUygjbeiIcEDQkVQNqmpiuPcdFb9EdnOaAl55E/xT3jjFX4Fdhomms5R7ONzju1r35vkjGoThj+9Hf/a9Rs3d291bt+5e+/+3v6DT1oWisCYSCbVWYI1MCpgbKhhcJYrwDxhcJosTpx+egFKUyk+mlUOU47ngmaUYGOp2b73IRbwhUjOsUjLOKvKOJEs1Stuh6qzIXJiVY7NOcFbUtJISdJSCmaVQqSgXIgtUV5UrrtSG4HKrZrbIX5cfw62TAkrwJrqdyjrScugIL3UHW7JcwUgLg3rmbWkkNmA1iRfNSY1t5cMB1E/CAfPXRdVneWMxkIaKoLYwNKUhFW9k4PZXjcchHULtkHUgC5q2sim4XOcSlJwEIYwrPUkCnMzLbEylDD3MoWGHJMFnsPEQoE56GlZB1gFTy2TBplU9hcmqNn/V5QJ39ihxFy7BNuVLmW6rTnyKm1SmOx4WlKRFwYEWR+cFSwwMnC1FaRUATFsZQEmitrYA3KOFSbGVuDGKfXeOZDNuJaFoESm0GKZWRqFLanBcEyFu2X5UtG6CmvObubI3is6p0b339riFv3XNm2Lg39Gm5OonYFtMD4cvBiE7591h8MmObvoEXqCeihCR2iI3qARGiPi/fB+eb+9P/473/hf/W9r647XrHmINpr//S/sBUsX</latexit>
<latexit sha1_base64="TUZyiPy4adREDLn6pz8B2b7vr6M=">AAADz3icbVJNj9MwEE02fCzlaxeOXAwV0iJVVboCAbcikODAoSBKV2qq4jiTrlXbCbaztIqCuPKn+B/cucJvYJyGFU3XUezxe2/GY8/EueDGhuFPfy+4dPnK1f1rnes3bt66fXB456PJCs1gzDKR6ZOYGhBcwdhyK+Ak10BlLGASL186fnIG2vBMfbDrHGaSLhRPOaMWofmhP4kUfGGZlFQlZZRWZRRnIjFriUvV2SIlQ1ZSe8roDhU3VBy3mEIgU6gEtEuxRWZnlZsu5Eagc2RzXKL79efMligWBaCofoey3rQEGpJz3tkteqEB1Llgs0NJAikmtAHluhHpBV4y7A96JOw/cdOg6tA5x3ifSUgiCytbEoxKKhLP+fygG/bDepBdY9AYXa8ZIyzFpyjJWCFBWSaoMdNBmNtZSbXlTLjXKQzklC3pAqZoKirBzMo6yYo8RCQhaabxV5bU6P8eZSy3IpRUGldk9HRlM23OgRdx08Kmz2YlV3lhQbHNwWkhiM2I6y+ScA3MijUalGmOuRN2SjVlFrtw65Q6dg5sO69VoTjLEmihwq6spggasJJy5W5ZvtC87sQaw2AOPHrFF9ya3ltscNV7jaVbPvonxJoM2hXYNcbH/ef98N3j7nDYFGffu+c98I68gffUG3pvvJE39pj/w//l//b/BO+DVfA1+LaR7vmNz11vawTf/wL/00vu</latexit>

> >
a i xi
<latexit sha1_base64="axJfK10qL9i6aeHAXq+uGs6NBxc=">AAAEC3icbVPLjtMwFE1bHkN5zcCSjaFCatGoShEIWCAVDRIsWBSJMiM1pTjObceqHxnbGVpF4RPY8CusEFs+gl9hxXVaBpqOo9gn5xzb1743cSq4dWH4q1ZvXLh46fLOlebVa9dv3Nzdu/Xe6swwGDIttDmKqQXBFQwddwKOUgNUxgIO4/mB1w9PwViu1Tu3TGEs6UzxKWfUITXZq32NFHxiWkqqkjyaFnkUa5HYpcShaG6IkqEqqTtmdEuK11IcV5RMoJKpBIwPsSLq08J352oDMCmqKQ7R3fLxsGKKRQZoKu8hLz8qBgPJme5xRZ4ZAHVmWH2hJYEpBrQi5XJtMjM8ZNjt7ZOw+9h3vaJJJ/xD5HRKFhP+nMQTHp1kNCF4H3qR4w4FWRH/fJGAE2/8HE21oUIgwxWJHCxczkTRPuh0m5PdVtgNy0a2QW8NWsG6DTCNH6NEs0yCckxQa0e9MHXjnBrHmfA3m1lIKZvTGYwQKirBjvPygAW5j0xCMBx8lSMl+/+MPJYbK+RUWl8gONOn3FY1T56njTI3fTrOuUozB4qtNp5mgjhNfG2ShBtgTiwRUGY4xk7YMTWUOazgjV3KtVNgm3EtMsWZTqDCCrdwhiJpwUnKlT9l/sLwsopLDhfzZPsln3Fn99/gz6H2X2Ha552/RsxJr5qBbTB82H3WDd8+avX76+TsBHeCe0E76AVPgn7wOhgEw4DVftdJvVN/0PjS+Nb43vixstZr6zm3g43W+PkHgEBhsQ==</latexit>
= bi and ai x  bi 8x 2 cl(C).
> >
We are free to choose a scaling where k(ai , bi ) k2 = 1. Thus, we
> >
ai ! a, bi ! b and k(a , b) k2 = 1.
<latexit sha1_base64="EnoiTT0YtmiAvfmhUMvCWEvf2WQ=">AAAD0nicbVJdb9MwFHUWPkb52uCRl0CF1ElVlUwg4AGpCCR44KFIlE00pTjOTWfVdiLbGauyPiBe+VP8DH4Br/APuE7DRLM5in18zrF97XuTQnBjw/Cnt+Vfunzl6va1zvUbN2/d3tm988HkpWYwZrnI9WFCDQiuYGy5FXBYaKAyEXCQLF46/eAYtOG5em+XBUwlnSuecUYtUrNd72Os4AvLpaQqreJsVcVJLlKzlDisOhuiZKhKao8YPScljZQkLaUUqJQqBe1CbIn58cp1F2oj0AWqBQ7x/fpzsGVKRAloqt+hqictg4b0THe4Jc81gDozrGdoSSHDgNakXDYmPcdLhoOoH4SDx66L0HnaozP+KbZ50U9mfK9G8els/3k06Mx2uuEgrFtwHkQN6JKmjTAbn+M0Z6UEZZmgxkyisLDTimrLmXAPVBooKFvQOUwQKirBTKs6zlXwEJk0yHKNv7JBzf6/okrkxg4VlcblGVe6zJm25siLtElps6fTiquitKDY+uCsFIHNA1diQco1MCuWCCjTHGMP2BHVlFksxI1T6r0LYJtxnZSKszyFFivsidUUSQNWUq7cLasXmtfFWHO4mSN7r/icW9N/izWu+q8xe4u9f0bMSdTOwHkw3h88G4TvHnWHwyY52+QeeUB6JCJPyJC8ISMyJsz74f3yfnt//LFf+V/9b2vrltesuUs2mv/9L6I0TM4=</latexit>

can assume wlog that <latexit sha1_base64="I1QewygNAd9w/YXyvV3IYqc9he8=">AAAD2HicbVLLjtMwFHUmPIbymoElm0CFNEhVlSAQsCsCCRYsikSZkdqq2M5NatWPyHZmpoqKxAqx5af4Bj6CLWyx0zAimXEU+/icY/va95KCM2Pj+GewE166fOXq7rXe9Rs3b93e27/z0ahSU5hQxZU+ItgAZxImllkOR4UGLAiHQ7J65fXDY9CGKfnBrguYC5xLljGKraMW+8GnmYQTqoTAMq1m2aaaEcVTsxZu2PRaoqBOFdguKT4nkUYipKOU3CmlTEH7EDuiOt747kJtDLpwauGG2f3687BjIrwEZ6rfoaonHYOG9Ez3uCPnGkCeGbYzZ0khcwFtSbFuTDp3l4yHySCKh099l2x6eMFmmuVLi7VWJxEeRJ9JmyKLvX48jOsWnQdJA/qoaeM6J6mipQBpKcfGTJO4sPMKa8so989UGigwXeEcpg5KLMDMqzraTfTQMWmUKe1+aaOa/X9FRURrhwoL47PtVvr8ma7myYu0aWmz5/OKyaK0IOn24KzkkVWRL7QoZRqo5WsHMNXMxR7RJdaYWleOrVPqvQug7bhOS8moSqHDcntqNXakASswk/6W1UvN6pKsObeZJw9es5xZM3jnKl0O3rgcrh79M7qcJN0MnAeTx8MXw/j9k/5o1CRnF91DD9ABStAzNEJv0RhNEA1+BL+C38GfcBp+Cb+G37bWnaBZcxe1Wvj9L0VSUKM=</latexit>

<latexit sha1_base64="XblcAWsE+MhYBjNOidUHVyLyAIo=">AAADznicbVJdb9MwFHUWPkb52AaPvAQqpE6qomQCAQ9IRSDBAw+dRLdJTSmOc9NZ9UdkO9uqrOKVP8UP4ZlX+A/YaZhoOkexr885tq99blowqk0U/fS2/Bs3b93evtO5e+/+g53dvYdHWpaKwIhIJtVJijUwKmBkqGFwUijAPGVwnM7fOf74DJSmUnw2iwImHM8EzSnBxkLTPe8oEXBOJOdYZFWSL6sklSzTC26HZWeN5MSyHJtTgjeotKHStMWUzDKlyEC5FFukPFu67lpuCKqwbGGH5En9ubAlSlkJVlS/Q1VPWgIF2RXv4hY9UwDiSrCaWUkGuU1oBfJFI1Ize8kojPtBFL5wXWyVlz38JTGy6Kf79ZhcTg/exGFnutuNwqhuwWYQN0EXNW1onfiaZJKUHIQhDGs9jqPCTCqsDCXMPU6pocBkjmcwtqHAHPSkqnNcBs8skgW5VPYXJqjR/1dUKV/bocJcO4/tSueabnMOvI4blyZ/NamoKEoDgqwOzksWGBm48goyqoAYtrABJora3ANyihUmxhbh2in13gWQ9bwuSkGJzKCFMnNhFLagBsMxFe6W1VtF60KsMbuZA3vv6Ywa3f9k61v0P1jn5vv/hNaTuO3AZjA6CF+H0eHz7mDQmLONHqOnqIdi9BIN0Ec0RCNEvB/eL++398c/9M/9pf9tJd3ymjWP0Frzv/8FqW5LFg==</latexit>
148

Weak Separating Hyperplane Theorem*


Proof (cont’): By construction, we have
b = lim bi = lim a> i xi = 0 =) a 6= 0.
<latexit sha1_base64="E9ccSI2vGWOejQi86V2ApsyQKkY=">AAAEE3icbVPLjtMwFE1THkN5zAws2RgqpCKNqmQEAhYjFYEECxZFosxITSmOc9Na9SPYzkyrqPMZbPgVVogtH8DfcJN2RjQdR7Fvzjm2r31u4kxw64Lgb8NvXrt+4+bOrdbtO3fv7e7t3/9sdW4YDJgW2pzE1ILgCgaOOwEnmQEqYwHH8exNyR+fgrFcq09ukcFI0oniKWfUITTeb/yIFJwxLSVVSRGlyyKKtUjsQuKwbG2QkiErqZsyukXFayqOa0wukMlVAqZMsUbq02XZXcn1wWTIZjhEj6qnDGuiWOSAouoeiuqjJjCQXPJlXKMnBkBdClZfKEkgxYRWoFysRWaChwy64QEJus/LLly24qNIcDkueGT4ZOqoMfqMRFylbrEk8ZgfkW3+gqZj/iVyOiNz1AXnEZdoOFhyTjHFb7h+a7zXDrpB1ch2EK6DtrdufbTza5RolktQjglq7TAMMjcqqHGcifKGcwsZZTM6gSGGikqwo6I66JI8QSQhqTb4Kkcq9P8ZRSw3ViiotGWh4MzSelvnSvAqbpi79OWo4CrLHSi22jjNBXGalDVKEm6AObHAgDLDMXfCptRQ5rCSN3ap1s6AbeY1zxVnOoEaKtzcGYqgBScpuoCnLF4bXlVzheFiJdh5yyfc2YMP+JOog3do/+zphRA9CesObAeDw+6rbvDxWbvXW5uz4z30HnsdL/ReeD3vvdf3Bh7zfb/jh/5h83vzZ/NX8/dK6jfWcx54G6355x9Nq2Ua</latexit>
i!1 i!1
Moreover, for each x 2 cl(C) we have <latexit sha1_base64="/kS+SlVoiKAaZzPVFL+4qkbawYw=">AAADx3icbVJdb9MwFHUWPkb52uCRl0CF1ElVlUwg4K1oSCCERJEom9RUw3FuO6u2E9lOaRXlgT/Ff0HiFX4H12mYaDpHsY/PObavfW+SC25sGP709vxr12/c3L/VuX3n7r37B4cPvpis0AzGLBOZPkuoAcEVjC23As5yDVQmAk6TxYnTT5egDc/UZ7vOYSrpXPEZZ9QidX7ofYwVfGOZlFSlZTyryjjJRGrWEoeqsyVKhqqk9oLRHSlppCRpKYVApVApaBdiS8yWleuu1Eagc1RzHOLH9edgy5SIAtBUv0NZT1oGDeml7nBLnmsAdWnYzNCSwgwD2pBy3Zj0HC8ZDqJ+EA6euy6qOquYqyC2sLIlE1Xv5KhzftANB2Hdgl0QNaBLmjbCFHyN04wVEpRlghozicLcTkuqLWfCvUphIKdsQecwQaioBDMt6+Cq4CkyaTDLNP7KBjX7/4oykVs7lFQal1xc6dJl2pojr9ImhZ29nJZc5YUFxTYHzwoR2CxwdRWkXAOzYo2AMs0x9oBdUE2ZxerbOqXeOwe2HdeqUJxlKbRYYVdWUyQNWEm5crcsX2teV2DN4WaO7L3hc25N/wMWtuq/xZQtjv4ZMSdROwO7YHw8eDUIPz3rDodNcvbJI/KE9EhEXpAheUdGZEyY98P75f32/vjv/dxf+quNdc9r1jwkW83//hfzdEjg</latexit>

> >
a x = lim ai x  lim bi = 0.
<latexit sha1_base64="rEbH/zo6VYBWJRY1mIpYNnMGe3w=">AAAECnicbVPNjtMwEE5bfpbys7tw5IChQupKqypBIOCwUhFIcOBQJMqu1JTiOJPWqn+C7ey2ivoIXHgVTogrL8GjcGOSZlc0XUexx9/3eTz2jKNUcOt8/0+j2bpy9dr1nRvtm7du39nd27/7yerMMBgyLbQ5iagFwRUMHXcCTlIDVEYCjqP564I/PgVjuVYf3TKFsaRTxRPOqENost/4Hio4Y1pKquI8TFZ5GGkR26XEYdXeICVDVlI3Y3SLiioqimpMJpDJVAymCLFG6tNV0V3KDcCkyKY4hA/LrzBrokhkgKLyHvJyUhMYiC/4wq7RUwOgLgTrGUpiSDCgNSiXlchM8ZB+Lzgkfu9Z0QWrNv0cOp2SxVEouJzkPDR8OnPUGH1GQq4St1wROuGVKhTwlWwrz4XRhB+h2/Zkr+P3/LKRbSOojI5XtQFm8UsYa5ZJUI4Jau0o8FM3zqlxnIniYjMLKWVzOoURmopKsOO8PN+KPEYkJok2+CtHSvT/FXkkNzzkVNqiPnBlkXFb5wrwMm6UueTFOOcqzRwott44yQRxmhSlSWJugDmxRIMywzF2wmbUUOawgDd2KX2nwDbjWmSKMx1DDRVu4QxF0IKTFO8aT5m/Mrws4hJDZwXYfcOn3NnD9/g21OFbzPr84FyIOQnqGdg2hk96L3v+h6edfr9Kzo5333vkdb3Ae+71vXfewBt6rPG3+aDZbR60vrV+tH62fq2lzUa15p630Vq//wF2D2Jg</latexit>
i!1 i!1
149

Supporting Hyperplanes
Definition: Consider C ✓ Rn and a boundary point x0 2 bd(C). If <latexit sha1_base64="BTa/XMM4bne9biTPeZfp91gB4/Q=">AAADyHicbVJNb9QwEHUaPsry1cKRS2CFVKTVKkEg4LaoSCDBYUEsrbRZiuNMUmttJ7WdtqtoL/wpfgsHrvA3GGdDxWbrKPb4vTf2eGaSUnBjw/Cnt+VfuXrt+vaN3s1bt+/c3dm998UUlWYwYYUo9GFCDQiuYGK5FXBYaqAyEXCQzPcdf3AK2vBCfbaLEmaS5opnnFGL0NGuN44VnLFCSqrSOs6WdZwUIjULicuyt0ZKhqyk9pjRDSppqSTpMJVAplIpaBdihyxOl266lBuDLpEtcYkfNp8zO6JEVICiJg91s+kINKQXvLM7dK4B1IVgtUNJChkGtALlohXpHB8ZDqNBEA6fuyla9vZjUyUGLJwEmIPg01fVO9rph8OwGcGmEbVGn7RjjDX4FqcFqyQoywQ1ZhqFpZ3VVFvOhEtLZaCkbE5zmKKpqAQzq5volsFjRNIgKzT+ygYN+r9Hnci1E2oqjasuerp6mS7nwMu4aWWzl7Oaq7KyoNjq4qwSgS0C11hByjUwKxZoUKY5xh6wY6ops9h+a7c0Z5fA1uM6rxRnRQodVNhzqymCmGVJuXKvrF9r3rRgg+FhDtx7w3NuzeADdrYavMWazZ/8E2JNom4FNo3J0+GrYfjxWX80aouzTR6QR2SPROQFGZF3ZEwmhHk/vF/eb++P/94/8c/8xUq65bU+98na8L//BTy2STc=</latexit>

a 6= 0 satisfies a> x  a> x0 8x 2 C, then {x : a> x = a> x0 } is called


<latexit sha1_base64="Ri301lblsTtD4399ZKpTxRClygc=">AAADyXicbVLLjtMwFHUmPIbymoElm0CF1JGqKEEgYFc0SLCYRUeizEhNVRznpmPVj8h2hpYoK36KX2HDFj4DOw0jmo6j2MfnHNvXvjctGNUmin56e/6Nm7du79/p3b13/8HDg8NHn7UsFYEJkUyq8xRrYFTAxFDD4LxQgHnK4CxdHjv97BKUplJ8MusCZhwvBM0pwcZS80PvNBHwlUjOsciqJK+rJJUs02tuh7q3JXJiVY7NBcE7UtpKadpRSmaVUmSgXIgdUV7WrrtWG4MqrFrYIXnafA52TCkrwZqad6iaScegILvSHe7ICwUgrgybmbVkkNuANiRftya1sJeMwngYROEr18V1bzWPEioSAytTpVk9OD4Ke/ODfhRGTQt2QdyCPmrb2CbhS5JJUnIQhjCs9TSOCjOrsDKUMPcupYYCkyVewNRCgTnoWdWEVwfPLZMFuVT2FyZo2P9XVCnf2qHCXLv02pUuYbqrOfI6bVqa/M2soqIoDQiyOTgvWWBk4CoryKgCYtjaAkwUtbEH5AIrTIytv61Tmr0LINtxrUpBicygwzKzMgpbUoPhmAp3y+qdok0NNpzdzJGD93RBjR6e2NIWww82acujf0abk7ibgV0weRG+DaPTl/3RqE3OPnqCnqEBitFrNEIf0RhNEPF+eL+8394f/8RX/sr/trHuee2ax2ir+d//An2ASYg=</latexit>

<latexit sha1_base64="ks+6uR9GYCkFFgzRO+awFh4zrLg=">AAAD3XicbVJNb9MwGE4WPkb52uDIxVAhDamqEgQCbkVDggOHIlE2qSmd47zprNpOsJ3RKipHTogrf4o/wN/gCgdep9lEszmK/fh5Htuv/b5JIbixYfjL3wouXb5ydfta5/qNm7du7+ze+WDyUjMYsVzk+jChBgRXMLLcCjgsNFCZCDhI5vtOPzgBbXiu3ttlARNJZ4pnnFGL1HTXh1jBZ5ZLSVVaxdmqipNcpGYpcVh1NkTJUJXUHjN6TkoaKUlaSilQKVUK2oXYEvOTlesu1IagC1QLHOL79edgy5SIEtBUv0NVT1oGDemZ7nBLnmkAdWZYz9CSQoYBrUm5bEx6hpcM+1GPhP2nrotWHfoxtnlBFrGAT+R0Mg2/xFmuqRAocEX2e53pTjfsh3Uj50HUgK7XtCEm5ihOc1ZKUJYJasw4Cgs7qai2nAn3VqWBgrI5ncEYoaISzKSqQ16Rh8ikBGPAX1lSs/+vqBK5sUNFpXEpx5UuiaatOfIibVza7Pmk4qooLSi2PjgrBbE5cdVGUq6BWbFEQJnmGDthx1RTZrEmN06p9y6Abca1KBVneQotVtiF1RRJA1ZSrtwtq5ea13VZc7iZI/de8Rm3pvcWy131XmMi549OjZiTqJ2B82D0uP+iH7570h0MmuRse/e8B96eF3nPvIH3xht6I4/5P/3f/h//b3AUfA2+Bd/X1i2/WXPX22jBj38hhVGC</latexit>

<latexit sha1_base64="UVCvV4+aArfX6zEGDoMj1YryOMo=">AAADvHicbVJbb9MwFHYWLqPcNnjkJVAhDamqkgk0eKKISfDAQ5Eom9RWw3FOOqu+ZLZTVkX9RfwaHoE/w3EaJprOUezj7/t8fHzOSQvBrYvjX8FOeOPmrdu7dzp3791/8HBv/9FXq0vDYMS00OY0pRYEVzBy3Ak4LQxQmQo4SefvPX+yAGO5Vl/csoCppDPFc86oQ+hsPzieKPjOtJRUZdUkX1WTVIvMLiUuq84GKRmykrpzRreotKHStMWUAplSZWB8iC1SL1Z+upYbgimQLXCZPK0/b7ZEqSgBRXUeqnrTEhjIrnhvt+iZAVBXgvUOJRnkGNAalMtGZGb4yLif9KK4/8pPyapD0dtFFJ/tdeN+XI9o20gao0uaMcS8f5tkmpUSlGOCWjtO4sJNK2ocZ8KnorRQUDanMxijqagEO63qiFbRc0SyKNcGf+WiGv3/RJXKDQ8VldZXFE/6Gtk258HruHHp8tfTiquidKDY+uK8FJHTkW+mKOMGmBNLNCgzHGOP2Dk1lDlsuY1bat8FsM24LkvFmc6ghQp36QxF0IKTlCv/yuqd4XXb1Rg68+DBMZ9xZ3ufsJtV7wPWaf7inxBrkrQrsG2MDvtv+vHnl93BoCnOLnlCnpEDkpAjMiAfyZCMCAt+BD+D38Gf8G0I4TyUa+lO0Jx5TDZGuPgLYYVEcg==</latexit>

a supporting hyperplane to C at x0 .
<latexit sha1_base64="HlwZNxuuIAFpCxSfNbNOsPExdEw=">AAAD0HicbVLLjtMwFE0mPIbymoElm0CFNEhVlSAQDwmpCCRYsCiCMpWaMjjOTceqH5HtDK2iCrHlp/gOPoAt/ALXaWZE03EU+/icY/va96YFZ8ZG0S9/J7hw8dLl3Sudq9eu37i5t3/rk1GlpjCiiis9TokBziSMLLMcxoUGIlIOh+n8ldMPT0AbpuRHuyxgKshMspxRYpE62vfHiYSvVAlBZFYl+apKUsUzsxQ4rDoboqCoCmKPKdmS0kZK05ZSclRKmYF2IbZEdbJy3bnaEHSBaoFDcrf+HGyZUl4Cmup3qOpJy6AhO9MdbskzDSDPDOsZWjLIMaA1KZaNSc/wklE/7oVR/7HrYnRWi+fkc2JVES5ehKfoKEpWnaO9btSP6hZug7gBXa9pQ8zFlyRTtBQgLeXEmEkcFXZaEW0Z5e55SgMFoXMygwlCSQSYaVVHuQrvI5OFudL4SxvW7P8rqlRs7FARYVyWcaXLm2lrjjxPm5Q2fzqtmCxKC5KuD85LHloVugILM6aBWr5EQKhmGHtIj4km1GIZbpxS710A3YxrUUpGVQYtltuF1QRJA1YQJt0tq5ea1aVYc7iZIw9esxmzpvcOK1z23mDu5g9OjZiTuJ2BbTB62H/Wj94/6g4GTXJ2vTvePe/Ai70n3sB76w29kUf9n/5v/4//N/gQLINvwfe1dcdv1tz2Nlrw4x9imExN</latexit>

<latexit sha1_base64="skFoEQKtuh2DA8n3OXWGhUAz5Lw=">AAADuXicbVJdb9MwFHUWPkb52uCRl0CFNKSpSiYQIPHQCSR44KFIlE1qq+I4N62pPyLbGa2i/h9+Da/Av+E6DRNN5yj28TnH9rXvTQvBrYvjP8FeeO36jZv7tzq379y9d//g8MEXq0vDYMi00OY8pRYEVzB03Ak4LwxQmQo4SxdvvX52AcZyrT67VQETSWeK55xRh9T0MDgdK/jOtJRUZdU4X1fjVIvMriQO686WKBmqkro5oztS2khp2lJKgUqpMjA+xJaoL9a+u1IbgClQLXAYP64/D1umVJSApvodqnrSMhjILnWPW/LMAKhLw2aGlgxyDGhDylVjMjO8ZNxLjqO498J3ybqznMad6UE37sV1i3ZB0oAuadoAH/3rONOslKAcE9TaURIXblJR4zgT/h1KCwVlCzqDEUJFJdhJVYezjp4ik0W5NvgrF9Xs/yuqVG7tUFFpfTpxpU+QbWuevEoblS5/Nam4KkoHim0OzksROR35SooyboA5sUJAmeEYe8Tm1FDmsN62Tqn3LoBtx7UsFWc6gxYr3NIZiqQFJylX/pbVqeF1zdUcbubJo3d8xp09/oilrI7fY5IWz/4ZMSdJOwO7YHjSe92LPz3v9vtNcvbJI/KEHJGEvCR98oEMyJCw4EfwM/gV/A7fhGk4D79trHtBs+Yh2Wqh/QvWpUMU</latexit>

a
x0
C
<latexit sha1_base64="skFoEQKtuh2DA8n3OXWGhUAz5Lw=">AAADuXicbVJdb9MwFHUWPkb52uCRl0CFNKSpSiYQIPHQCSR44KFIlE1qq+I4N62pPyLbGa2i/h9+Da/Av+E6DRNN5yj28TnH9rXvTQvBrYvjP8FeeO36jZv7tzq379y9d//g8MEXq0vDYMi00OY8pRYEVzB03Ak4LwxQmQo4SxdvvX52AcZyrT67VQETSWeK55xRh9T0MDgdK/jOtJRUZdU4X1fjVIvMriQO686WKBmqkro5oztS2khp2lJKgUqpMjA+xJaoL9a+u1IbgClQLXAYP64/D1umVJSApvodqnrSMhjILnWPW/LMAKhLw2aGlgxyDGhDylVjMjO8ZNxLjqO498J3ybqznMad6UE37sV1i3ZB0oAuadoAH/3rONOslKAcE9TaURIXblJR4zgT/h1KCwVlCzqDEUJFJdhJVYezjp4ik0W5NvgrF9Xs/yuqVG7tUFFpfTpxpU+QbWuevEoblS5/Nam4KkoHim0OzksROR35SooyboA5sUJAmeEYe8Tm1FDmsN62Tqn3LoBtx7UsFWc6gxYr3NIZiqQFJylX/pbVqeF1zdUcbubJo3d8xp09/oilrI7fY5IWz/4ZMSdJOwO7YHjSe92LPz3v9vtNcvbJI/KEHJGEvCR98oEMyJCw4EfwM/gV/A7fhGk4D79trHtBs+Yh2Wqh/QvWpUMU</latexit>

Theorem: For every convex set C and any x0 2 bd(C) there is a


supporting hyperplane to C at x0 .
<latexit sha1_base64="5yU5/txkgMI31V92gXS2ccz9UN4=">AAADyHicbVJdb9MwFHUWPkb52uCRl0CF1ElVlSIQ8FY0JJDgoUiUTWqq4jg3nVXbCbaztYrywp/it/DAK/wNrtMw0XSOYh+fc2xf+944F9zYMPzp7fnXrt+4uX+rc/vO3Xv3Dw4ffDFZoRlMWCYyfRpTA4IrmFhuBZzmGqiMBZzEy2Onn5yDNjxTn+06h5mkC8VTzqhFan7ojSMFFyyTkqqkjNKqjOJMJGYtcag6W6JkqEpqzxjdkeJGiuOWUghUCpWAdiG2xOy8ct2V2hh0jmqOQ/S4/hxsmWJRAJrqdyjrScugIbnUHW7JCw2gLg2bGVoSSDGgDSnXjUkv8JLhYNgPwsEL1w2rzmoeRlxFFla2jJOqd3zUmR90w0FYt2AXDBvQJU0bYw6+RknGCgnKMkGNmQ7D3M5Kqi1nwj1LYSCnbEkXMEWoqAQzK+voquApMkmQZhp/ZYOa/X9FGcutHUoqjcsurnT5Mm3NkVdp08Kmr2YlV3lhQbHNwWkhApsFrrCChGtgVqwRUKY5xh6wM6ops1h+W6fUe+fAtuNaFYqzLIEWK+zKaoqkASspV+6W5RvN6xKsOdzMkb23fMGt6X/Eylb9d5iz5dE/I+Zk2M7ALpg8G7wehJ+ed0ejJjn75BF5QnpkSF6SEXlPxmRCmPfD++X99v74H/xv/oW/3lj3vGbNQ7LV/O9/AXJuSVA=</latexit>

<latexit sha1_base64="skFoEQKtuh2DA8n3OXWGhUAz5Lw=">AAADuXicbVJdb9MwFHUWPkb52uCRl0CFNKSpSiYQIPHQCSR44KFIlE1qq+I4N62pPyLbGa2i/h9+Da/Av+E6DRNN5yj28TnH9rXvTQvBrYvjP8FeeO36jZv7tzq379y9d//g8MEXq0vDYMi00OY8pRYEVzB03Ak4LwxQmQo4SxdvvX52AcZyrT67VQETSWeK55xRh9T0MDgdK/jOtJRUZdU4X1fjVIvMriQO686WKBmqkro5oztS2khp2lJKgUqpMjA+xJaoL9a+u1IbgClQLXAYP64/D1umVJSApvodqnrSMhjILnWPW/LMAKhLw2aGlgxyDGhDylVjMjO8ZNxLjqO498J3ybqznMad6UE37sV1i3ZB0oAuadoAH/3rONOslKAcE9TaURIXblJR4zgT/h1KCwVlCzqDEUJFJdhJVYezjp4ik0W5NvgrF9Xs/yuqVG7tUFFpfTpxpU+QbWuevEoblS5/Nam4KkoHim0OzksROR35SooyboA5sUJAmeEYe8Tm1FDmsN62Tqn3LoBtx7UsFWc6gxYr3NIZiqQFJylX/pbVqeF1zdUcbubJo3d8xp09/oilrI7fY5IWz/4ZMSdJOwO7YHjSe92LPz3v9vtNcvbJI/KEHJGEvCR98oEMyJCw4EfwM/gV/A7fhGk4D79trHtBs+Yh2Wqh/QvWpUMU</latexit>

Proof: Apply the weak separating hyperplane thm to int(C) and x0. <latexit sha1_base64="skFoEQKtuh2DA8n3OXWGhUAz5Lw=">AAADuXicbVJdb9MwFHUWPkb52uCRl0CFNKSpSiYQIPHQCSR44KFIlE1qq+I4N62pPyLbGa2i/h9+Da/Av+E6DRNN5yj28TnH9rXvTQvBrYvjP8FeeO36jZv7tzq379y9d//g8MEXq0vDYMi00OY8pRYEVzB03Ak4LwxQmQo4SxdvvX52AcZyrT67VQETSWeK55xRh9T0MDgdK/jOtJRUZdU4X1fjVIvMriQO686WKBmqkro5oztS2khp2lJKgUqpMjA+xJaoL9a+u1IbgClQLXAYP64/D1umVJSApvodqnrSMhjILnWPW/LMAKhLw2aGlgxyDGhDylVjMjO8ZNxLjqO498J3ybqznMad6UE37sV1i3ZB0oAuadoAH/3rONOslKAcE9TaURIXblJR4zgT/h1KCwVlCzqDEUJFJdhJVYezjp4ik0W5NvgrF9Xs/yuqVG7tUFFpfTpxpU+QbWuevEoblS5/Nam4KkoHim0OzksROR35SooyboA5sUJAmeEYe8Tm1FDmsN62Tqn3LoBtx7UsFWc6gxYr3NIZiqQFJylX/pbVqeF1zdUcbubJo3d8xp09/oilrI7fY5IWz/4ZMSdJOwO7YHjSe92LPz3v9vtNcvbJI/KEHJGEvCR98oEMyJCw4EfwM/gV/A7fhGk4D79trHtBs+Yh2Wqh/QvWpUMU</latexit>
150

Subgradients
Definition: Assume that f is a proper convex function and
x0 2 dom(f). Then, y is called a subgradient of f at x0 if <latexit sha1_base64="ErgR3vWn2hp8RQbgd1TYuS4RDc0=">AAADynicbVJdb9MwFHUWYKN8bfDIS6BC6qQqShBo8FYEEjyAVBBlk5qqOM5NZ9Ufke2MVlHe+FP8FJ54hX+BnYaJpnMU+/icY/va96YFo9pE0U9vz792/cb+wc3erdt37t47PLr/RctSEZgQyaQ6S7EGRgVMDDUMzgoFmKcMTtPla6efXoDSVIrPZl3AjOOFoDkl2FhqfuR9SgR8I5JzLLIqyesqSSXL9Jrboe5tiZxYlWNzTvCOlLZSmnaUklmlFBkoF2JHlBe1667UxqAKqxZ2SB41n4MdU8pKsKbmHapm0jEoyC51hzvyQgGIS8NmZi0Z5DagDcnXrUkt7CWjMB4GUfjcdXHdW82jhIogMbAyVSZ5PciPw/lhPwqjpgW7IG5BH7VtbLPwNckkKTkIQxjWehpHhZlVWBlKmHuYUkOByRIvYGqhwBz0rGriq4MnlsmCXCr7CxM07P8rqpRv7VBhrl1+7UqXMd3VHHmVNi1N/mJWUVGUBgTZHJyXLDAycKUVZFQBMWxtASaK2tgDco4VJsYW4NYpzd4FkO24VqWgRGbQYZlZGYUtqcFwTIW7ZfVK0aYIG85u5sjBG7qgRg/f29oWw7c2a8vjf0abk7ibgV0weRq+DKOPz/qjUZucA/QQPUYDFKMTNELv0BhNEPF+eL+8394f/4Ov/bVfbax7XrvmAdpq/ve/mpRKRQ==</latexit>
<latexit sha1_base64="skFoEQKtuh2DA8n3OXWGhUAz5Lw=">AAADuXicbVJdb9MwFHUWPkb52uCRl0CFNKSpSiYQIPHQCSR44KFIlE1qq+I4N62pPyLbGa2i/h9+Da/Av+E6DRNN5yj28TnH9rXvTQvBrYvjP8FeeO36jZv7tzq379y9d//g8MEXq0vDYMi00OY8pRYEVzB03Ak4LwxQmQo4SxdvvX52AcZyrT67VQETSWeK55xRh9T0MDgdK/jOtJRUZdU4X1fjVIvMriQO686WKBmqkro5oztS2khp2lJKgUqpMjA+xJaoL9a+u1IbgClQLXAYP64/D1umVJSApvodqnrSMhjILnWPW/LMAKhLw2aGlgxyDGhDylVjMjO8ZNxLjqO498J3ybqznMad6UE37sV1i3ZB0oAuadoAH/3rONOslKAcE9TaURIXblJR4zgT/h1KCwVlCzqDEUJFJdhJVYezjp4ik0W5NvgrF9Xs/yuqVG7tUFFpfTpxpU+QbWuevEoblS5/Nam4KkoHim0OzksROR35SooyboA5sUJAmeEYe8Tm1FDmsN62Tqn3LoBtx7UsFWc6gxYr3NIZiqQFJylX/pbVqeF1zdUcbubJo3d8xp09/oilrI7fY5IWz/4ZMSdJOwO7YHjSe92LPz3v9vtNcvbJI/KEHJGEvCR98oEMyJCw4EfwM/gV/A7fhGk4D79trHtBs+Yh2Wqh/QvWpUMU</latexit>

f(x)<latexit sha1_base64="XzRtYastDjgw6yXw1ujsVQOJgmQ=">AAAD63icbVJdb9MwFHVbPrbysQ0eeTFUSK0YVYpAwFsRSPDAQ0GUTWq64jg3nVXbCbYzWkXdr+AJ8cqPgl/DdVomms5R7ONzju1r3xtlUlgXBL9r9caVq9eu7+w2b9y8dXtv/+DOZ5vmhsOQpzI1xxGzIIWGoRNOwnFmgKlIwlE0e+31ozMwVqT6k1tkMFZsqkUiOHNITQ5qNtTwjadKMR0XYbIswiiVsV0oHJbNDVFxVBVzp5xtSdFaiqKKkktUch2D8SFWxPRs6btLtQGYDNUMh/B++XlYMUUyBzSV71CUk4rBQHyhe1yRpwZAXxhWM7TEkGBAK1It1iYzxUsG3d4hDbrPfNdbNpP2vBNO4StFMAk6jxYnoUsz2p4/9lN6fh4mqWFS0nkoNL4S/Xiiu83JfivoBmWj26C3Bi2ybgNM05cwTnmuQDsumbWjXpC5ccGME1z6l8stZIzP2BRGCDVTYMdFeYElfYhMTDEQ/LWjJfv/iiJSGzsUTFlfALjSp9RWNU9epo1yl7wYF0JnuQPNVwcnuaQupb72aCwMcCcXCBg3AmOn/JQZxh1W6MYp5d4Z8M245rkWPI2hwko3d4YhacEpJrS/ZfHKiLJKSw4382T7jZgKZw/fY/Hrw7eY1lnnnxFz0qtmYBsMn3RfdoMPT1v9/jo5O+QeeUDapEeekz55RwZkSHjtT53Ud+vNhm58b/xo/FxZ67X1mrtkozV+/QUkQFLS</latexit>
f(x0 ) + y> (x x0 ) 8x 2 Rn .
The set @f(x0 ) of all subgradients is the subdifferential of f at x0. <latexit sha1_base64="TekehMLZOckAajqUiIPzrLkYE/I=">AAADxXicbVLbjtMwEHU2XJZy24VHXgIVUleqqgSBgLcikNgHkIpE2ZXaqjjOJGvVdiLbKa2iip/iZ+AVPoRxGlY0XUexx+ecGY89ExeCGxuGP70D/9r1GzcPb3Vu37l77/7R8YMvJi81gzHLRa7PY2pAcAVjy62A80IDlbGAs3jx1vFnS9CG5+qzXRcwkzRTPOWMWoTmx97HqYJvLJeSqqSapptqGuciMWuJy6azQ0qGrKT2gtE9Km6oOG4xpUCmVAlol2KLzJcbN13JjUAXyBa4TB/XnzNboliUgKL6Hap60xJoSC55Z7foTAOoS8F2h5IEUkxoC8p1I9IZXjIcRP0gHLxwU4TKgmrLqQjS3moennTmR91wENYj2DeixuiSZoywAF+nSc5KCcoyQY2ZRGFhZ5ULy4R7k9JAQdmCZjBBU1EJZlbVqW2Cp4gkQZpr/JUNavR/jyqWOxEqKo0rLXq6Ypk258CruElp01eziquitKDY9uC0FIHNA9dVQcI1MCvWaFCmOeYesAuqKbPYezun1LELYLt5rUrFWZ5ACxV2ZTVF0ICVlCt3y+qN5nX/1RgGc2DvHc+4Nf0P2Naq/x4Ltjj5J8SaRO0K7BvjZ4PXg/DT8+5w2BTnkDwiT0iPROQlGZJTMiJjwrwf3i/vt/fHP/WVb/3lVnrgNT4Pyc7wv/8FRvRHrA==</latexit>
<latexit sha1_base64="skFoEQKtuh2DA8n3OXWGhUAz5Lw=">AAADuXicbVJdb9MwFHUWPkb52uCRl0CFNKSpSiYQIPHQCSR44KFIlE1qq+I4N62pPyLbGa2i/h9+Da/Av+E6DRNN5yj28TnH9rXvTQvBrYvjP8FeeO36jZv7tzq379y9d//g8MEXq0vDYMi00OY8pRYEVzB03Ak4LwxQmQo4SxdvvX52AcZyrT67VQETSWeK55xRh9T0MDgdK/jOtJRUZdU4X1fjVIvMriQO686WKBmqkro5oztS2khp2lJKgUqpMjA+xJaoL9a+u1IbgClQLXAYP64/D1umVJSApvodqnrSMhjILnWPW/LMAKhLw2aGlgxyDGhDylVjMjO8ZNxLjqO498J3ybqznMad6UE37sV1i3ZB0oAuadoAH/3rONOslKAcE9TaURIXblJR4zgT/h1KCwVlCzqDEUJFJdhJVYezjp4ik0W5NvgrF9Xs/yuqVG7tUFFpfTpxpU+QbWuevEoblS5/Nam4KkoHim0OzksROR35SooyboA5sUJAmeEYe8Tm1FDmsN62Tqn3LoBtx7UsFWc6gxYr3NIZiqQFJylX/pbVqeF1zdUcbubJo3d8xp09/oilrI7fY5IWz/4ZMSdJOwO7YHjSe92LPz3v9vtNcvbJI/KEHJGEvCR98oEMyJCw4EfwM/gV/A7fhGk4D79trHtBs+Yh2Wqh/QvWpUMU</latexit>

Note: If f is differentiable at x0, then @f(x0 ) = {rf(x0 )}. Generally, <latexit sha1_base64="skFoEQKtuh2DA8n3OXWGhUAz5Lw=">AAADuXicbVJdb9MwFHUWPkb52uCRl0CFNKSpSiYQIPHQCSR44KFIlE1qq+I4N62pPyLbGa2i/h9+Da/Av+E6DRNN5yj28TnH9rXvTQvBrYvjP8FeeO36jZv7tzq379y9d//g8MEXq0vDYMi00OY8pRYEVzB03Ak4LwxQmQo4SxdvvX52AcZyrT67VQETSWeK55xRh9T0MDgdK/jOtJRUZdU4X1fjVIvMriQO686WKBmqkro5oztS2khp2lJKgUqpMjA+xJaoL9a+u1IbgClQLXAYP64/D1umVJSApvodqnrSMhjILnWPW/LMAKhLw2aGlgxyDGhDylVjMjO8ZNxLjqO498J3ybqznMad6UE37sV1i3ZB0oAuadoAH/3rONOslKAcE9TaURIXblJR4zgT/h1KCwVlCzqDEUJFJdhJVYezjp4ik0W5NvgrF9Xs/yuqVG7tUFFpfTpxpU+QbWuevEoblS5/Nam4KkoHim0OzksROR35SooyboA5sUJAmeEYe8Tm1FDmsN62Tqn3LoBtx7UsFWc6gxYr3NIZiqQFJylX/pbVqeF1zdUcbubJo3d8xp09/oilrI7fY5IWz/4ZMSdJOwO7YHjSe92LPz3v9vtNcvbJI/KEHJGEvCR98oEMyJCw4EfwM/gV/A7fhGk4D79trHtBs+Yh2Wqh/QvWpUMU</latexit>

@f(x0 ) can be empty or contain one or infinitely many subgradients.


<latexit sha1_base64="eKmuSA6qjco0Tjy0JbLU3bw0x+Y=">AAAD2HicbVLLjtMwFHUmPIbymoElm0CF1JGqKkEgYIFUBBIsWBSJMiM1VXGcm45V24lsZ2gVdcEKseWn+AY+gi1suU7DiKbjKPb1Ocf2tc9NCsGNDcOf3p5/6fKVq/vXOtdv3Lx1++DwzkeTl5rBmOUi1ycJNSC4grHlVsBJoYHKRMBxsnjl+OMz0Ibn6oNdFTCVdK54xhm1CM0OvU+xgs8sl5KqtIqzdRUnuUjNSuKw7myRkiErqT1ldIdKGipJWkwpkClVCtql2CLzs7XrLuRGoAtkCxzi+/XnwpYoESWgqH6Hqp60BBrSc97FLXquAdS5YDNDSQoZJrQB5aoR6TleMhxE/SAcPHFdhMqCasupCLLechYevYirWNFE0GYerwed2UE3HIR1C3aDqAm6pGmj2pM0Z6UEZZmgxkyisLDTyp3EhHum0kBB2YLOYYKhohLMtKqzXQcPEUmDLNf4KxvU6P8rqkRu7VBRaZzbuNL5Z9qcAy/iJqXNnk0rrorSgmKbg7NSBDYPXKEFKdfArFhhQJnmmHvATqmmzGI5bp1S710A285rWSrO8hRaqLBLqymCBqykXLlbVi81r0uyxnAzB/Ze8zm3pv8OK13136CHi6N/QvQkajuwG4wfDZ4PwvePu8NhY84+uUcekB6JyFMyJG/JiIwJ8354v7zf3h9/4n/xv/rfNtI9r1lzl2w1//tfQuROvQ==</latexit>

<latexit sha1_base64="TekehMLZOckAajqUiIPzrLkYE/I=">AAADxXicbVLbjtMwEHU2XJZy24VHXgIVUleqqgSBgLcikNgHkIpE2ZXaqjjOJGvVdiLbKa2iip/iZ+AVPoRxGlY0XUexx+ecGY89ExeCGxuGP70D/9r1GzcPb3Vu37l77/7R8YMvJi81gzHLRa7PY2pAcAVjy62A80IDlbGAs3jx1vFnS9CG5+qzXRcwkzRTPOWMWoTmx97HqYJvLJeSqqSapptqGuciMWuJy6azQ0qGrKT2gtE9Km6oOG4xpUCmVAlol2KLzJcbN13JjUAXyBa4TB/XnzNboliUgKL6Hap60xJoSC55Z7foTAOoS8F2h5IEUkxoC8p1I9IZXjIcRP0gHLxwU4TKgmrLqQjS3moennTmR91wENYj2DeixuiSZoywAF+nSc5KCcoyQY2ZRGFhZ5ULy4R7k9JAQdmCZjBBU1EJZlbVqW2Cp4gkQZpr/JUNavR/jyqWOxEqKo0rLXq6Ypk258CruElp01eziquitKDY9uC0FIHNA9dVQcI1MCvWaFCmOeYesAuqKbPYezun1LELYLt5rUrFWZ5ACxV2ZTVF0ICVlCt3y+qN5nX/1RgGc2DvHc+4Nf0P2Naq/x4Ltjj5J8SaRO0K7BvjZ4PXg/DT8+5w2BTnkDwiT0iPROQlGZJTMiJjwrwf3i/vt/fHP/WVb/3lVnrgNT4Pyc7wv/8FRvRHrA==</latexit>

x
f(x0 ) contains
f(x0 ) = f(x0 ) = { f(x0 )} many subgradients
151

Subgradients
Note: @f(x0 ) is a closed convex set. <latexit sha1_base64="TekehMLZOckAajqUiIPzrLkYE/I=">AAADxXicbVLbjtMwEHU2XJZy24VHXgIVUleqqgSBgLcikNgHkIpE2ZXaqjjOJGvVdiLbKa2iip/iZ+AVPoRxGlY0XUexx+ecGY89ExeCGxuGP70D/9r1GzcPb3Vu37l77/7R8YMvJi81gzHLRa7PY2pAcAVjy62A80IDlbGAs3jx1vFnS9CG5+qzXRcwkzRTPOWMWoTmx97HqYJvLJeSqqSapptqGuciMWuJy6azQ0qGrKT2gtE9Km6oOG4xpUCmVAlol2KLzJcbN13JjUAXyBa4TB/XnzNboliUgKL6Hap60xJoSC55Z7foTAOoS8F2h5IEUkxoC8p1I9IZXjIcRP0gHLxwU4TKgmrLqQjS3moennTmR91wENYj2DeixuiSZoywAF+nSc5KCcoyQY2ZRGFhZ5ULy4R7k9JAQdmCZjBBU1EJZlbVqW2Cp4gkQZpr/JUNavR/jyqWOxEqKo0rLXq6Ypk258CruElp01eziquitKDY9uC0FIHNA9dVQcI1MCvWaFCmOeYesAuqKbPYezun1LELYLt5rUrFWZ5ACxV2ZTVF0ICVlCt3y+qN5nX/1RgGc2DvHc+4Nf0P2Naq/x4Ltjj5J8SaRO0K7BvjZ4PXg/DT8+5w2BTnkDwiT0iPROQlGZJTMiJjwrwf3i/vt/fHP/WVb/3lVnrgNT4Pyc7wv/8FRvRHrA==</latexit>

Theorem: Assume that f is a proper convex function and


x0 2 int(dom(f)). Then, @f(x0 ) 6= ;. <latexit sha1_base64="6Ieq4thVLtR2BpbPz3E7qU/Pkgc=">AAAD1XicbVLLjtMwFHUmPIbymoElm0CF1EpVlYxAwK4IJFiwKBJlKjVVcZybjlU/ItsZWkVhhdjyU/wEv8AWPgA7DSOajqPYx+cc29e+N8kZ1SYMf3oH/pWr164f3ujcvHX7zt2j43sftSwUgQmRTKppgjUwKmBiqGEwzRVgnjA4TVavnH56DkpTKT6YTQ5zjpeCZpRgY6nFsTePBXwmknMs0jLOqjJOJEv1htuh6uyInFiVY3NG8J6UNFKStJSCWaUQKSgXYkuU55XrLtXGoHKr5naIH9afgy1TwgqwpvodynrSMihIL3SHW/JSAYgLw3ZmLSlkNqAtyTeNSS3tJcNhNAjC4VPXRVVnvQhjKmIDa1NSYareFqaSV72s3x8ujrrhMKxbsA+iBnRR08Y2IZ/iVJKCgzCEYa1nUZibeYmVoYS5Nyo05Jis8BJmFgrMQc/LOtQqeGyZNMiksr8wQc3+v6JM+M4OJebapdqudMnTbc2Rl2mzwmTP5/bGeWFAkO3BWcECIwNXZUFKFRDDNhZgoqiNPSBnWGFibC3unFLvnQPZjWtdCEpkCi2WmbVR2JIaDMdUuFuWLxWt67Hm7GaO7L2mS2r04J0tczF4YxO46v8z2pxE7Qzsg8nJ8MUwfP+kOxo1yTlED9Aj1EMReoZG6C0aowki3g/vl/fb++NP/S/+V//b1nrgNWvuo53mf/8LpoRPSA==</latexit>
<latexit sha1_base64="lXRdOUGK5Qjk3H/jTZ0pfeNcJaA=">AAAD03icbVLNjtMwEHY2/CzL3y4cuQQqpK5URekKBNyKQIIDhyLR3ZWaqjjOpGvVdoLtLK1CL4grL8Vb8AZc4QkYp2FF03UUe/x9n8fjmUkKwY2Nop/ejn/l6rXruzf2bt66fefu/sG9Y5OXmsGI5SLXpwk1ILiCkeVWwGmhgcpEwEkyf+X4k3PQhufqg10WMJF0pnjGGbUITQ+8cazgM8ulpCqt4mxVxUkuUrOUuKz2NkjJkJXUnjG6RSUNlSQtphTIlCoF7UJskfn5yk2XckPQBbIFLvHD+nNmS5SIElBU56GqNy2BhvSCd3aLnmkAdSFY71CSQoYBrUG5bER6ho+Mwn4viMKnbuqjsqDaciqCrLuYRofo+1MQgyzs0oANp/udKIzqEWwb/cbokGYMsRwf4zRnpQRlmaDGjPtRYSeVu4QJl6HSQEHZnM5gjKaiEsykqgNdBY8RSYMs1/grG9To/yeqRG54qKg0rtB40pXOtDkHXsaNS5s9n1RcFaUFxdYXZ6UIbB64HgtSroFZsUSDMs0x9oCdUU2ZxU7cuKX2XQDbjGtRKs7yFFqosAurKYKYW0m5cq+sXmped2ONoTMHdl/zGbem9w6bXPXeYPnmh/+EWJN+uwLbxugofBFG7590BoOmOLvkAXlEuqRPnpEBeUuGZESY98P75f32/vjH/hf/q/9tLd3xmjP3ycbwv/8FmvRN8w==</latexit>

Proof: Note that (x0 , f(x0 )) is on the boundary of epi(f). By the


n
supporting hyperplane theorem, there is b) R ⇥ R with
<latexit sha1_base64="5iS6fe9BcTrcSABXvKziPe6nVWg=">AAADwXicbVJdb9MwFHUWPkb52uCRl0CF1EpVlSAQ8FbEJEDioUiUTWqr4jg3mantRLYzWkX9T/waJJ7gp3CdhommcxT7+Jxj+9r3xoXgxobhL+/Av3b9xs3DW53bd+7eu390/OCLyUvNYMJykeuzmBoQXMHEcivgrNBAZSzgNF6+dfrpBWjDc/XZrguYS5opnnJGLVKLY+/DTMF3lktJVVLN0k01i3ORmLXEYdPZESVDVVJ7zuieFDdSHLeUUqBSqgS0C7El5hcb112pjUEXqBY4zB7Xn4MtUyxKQFP9DlU9aRk0JJe6wy050wDq0rCdoSWBFAPaknLdmHSGlwyH0SAIhy9cF206vdUiHKSu7/cXR91wGNYt2AdRA7qkaWN8/K+zJGelBGWZoMZMo7Cw84pqy5lw71EaKChb0gymCBWVYOZVHdYmeIpMEqS5xl/ZoGb/X1HFcmeHikrj0oorXaJMW3PkVdq0tOmrecVVUVpQbHtwWorA5oGrqCDhGpgVawSUaY6xB+ycasos1t3OKfXeBbDduFal4ixPoMUKu7KaImnASsqVu2X1RvO69moON3Nk74Rn3JrBRyxpNXiHyVr2/xkxJ1E7A/tg8mz4ehh+et4djZrkHJJH5AnpkYi8JCPynozJhDDvh/fT++398U/8b37h6631wGvWPCQ7za/+ApgkRZU=</latexit>

(a, 2 <latexit sha1_base64="rQZK3yV/CTuPgitXbny3dwhZv6I=">AAAD0HicbVLLjtMwFHUmPIbymoElm0CF1JGqKhmBgF0RSLBgUR5lKjVlcJybjlXbiWxnmCqqEFt+iu/gA9jCL3CdhhFNx1Hs63OO7WufmxSCGxuGP70d/9LlK1d3r3Wu37h56/be/p2PJi81gzHLRa4nCTUguIKx5VbApNBAZSLgKFm8cPzRKWjDc/XBLguYSTpXPOOMWoSO971JrOALy6WkKq3ibFXFSS5Ss5Q4rDobpGTISmpPGN2ikoZKkhZTCmRKlYJ2KbbI/HTlugu5EegC2QKH+H79ubAlSkQJKKrfoaonLYGG9Jx3cYueawB1LljPUJJChgmtQblsRHqOlwwHUT8IB49dF606PdpPDmKuAnyB4N0nFVsuwdST471uOAjrFmwHURN0SdNG6MXnOM1ZKUFZJqgx0ygs7Kyi2nIm3POUBgrKFnQOUwwVxaNmVZ3lKniISBpkucZf2aBG/19RJXJjh4pK41zGlc430+YceBE3LW32dFZxVZQWFFsfnJUisHngCixIuQZmxRIDyjTH3AN2QjVlFstw45R67wLYZl5npeIsT6GFCntmNUXQgJWUK3fL6rnmdSnWGG7mwN5LPufW9N9ghav+K/RucfBPiJ5EbQe2g/Hh4NkgfPuoOxw25uySe+QB6ZGIPCFD8pqMyJgw74f3y/vt/fHf+0v/q/9tLd3xmjV3yUbzv/8FsxtMAA==</latexit>

(a, b) 6= (0, 0) such that


<latexit sha1_base64="TRwvv1G5eAn0NhV4qNnXI7AkS+s=">AAADxXicbVLbjtMwEHU2XJZy24VHXgIVUitVVbICAW9FILEPIBWJsiu11eI4k6xV2wm2U7aKKn6Kn4FX+BDGaVjRdB3FHp9zZjz2TFwIbmwY/vT2/GvXb9zcv9W5fefuvfsHhw8+m7zUDCYsF7k+jakBwRVMLLcCTgsNVMYCTuLFG8efLEEbnqtPdlXAXNJM8ZQzahE6O/Q+zBR8Y7mUVCXVLF1XszgXiVlJXNadLVIyZCW154zuUHFDxXGLKQUypUpAuxRbZL5cu+lKbgy6QLbAZfa4/pzZEsWiBBTV71DVm5ZAQ3LJO7tFZxpAXQo2O5QkkGJCG1CuGpHO8JLhMBoE4fC5m6J1p0cHcR8jfg164SDsd84OuuEwrEewa0SN0SXNGGMBvsySnJUSlGWCGjONwsLOK6otZ8K9SWmgoGxBM5iiqagEM6/q1NbBU0SSIM01/soGNfq/RxXLrQgVlcaVFj1dsUybc+BV3LS06ct5xVVRWlBsc3BaisDmgeuqIOEamBUrNCjTHHMP2DnVlFnsva1T6tgFsO28LkrFWZ5ACxX2wmqKoAErKVfultVrzev+qzEM5sDeW55xawbvsa3V4B0WbNH/J8SaRO0K7BqTo+GrYfjxWXc0aoqzTx6RJ6RHIvKCjMgxGZMJYd4P75f32/vjH/vKt/5yI93zGp+HZGv43/8Cj6BGYg==</latexit>

a> x + bt  a> x0 + bf(x0 ) 8(x, t) 2 epi(f).


<latexit sha1_base64="gKk5IaY2io031NQOnCNEtGc+uss=">AAAD+HicbVNdb9MwFE0bPkZhsIEED7wYKqRWq6oUgYC3IpDggYciUTapKcVxbjqrthNsZ7SKivgrPCFe+Tf8Av4G12k20WyOYh+fc+xc+95EmeDGBsGfRtO/dPnK1Z1rres3dm/e2tu//dGkuWYwZqlI9VFEDQiuYGy5FXCUaaAyEnAYLV45/fAEtOGp+mBXGUwlnSuecEYtUrP9xvdQwVeWSklVXITJugijVMRmJXFYt7ZEyVCV1B4zek6KKimKakouUMlVDNqFWBPTk7XrLtRGoDNUMxzCB+XjYM0UiRzQVN5DUU5qBg3xme5wTZ5rAHVm2MzQEkOCAW1IuapMeo6HDPqDHgn6T103WLfop9CmGVkeRDYU8IWczmfBQZR0cOh+C5NUUyFIZ9mz3ZArElpY2gIyvu4k3X5rttcO+kHZyHkwqEDbq9oIc/Y5jFOWS1CWCWrMZBBkdlpQbTkT7hpzAxllCzqHCUJFJZhpUZ5mTR4hExOMCV9lScn+v6KI5NYOBZXGVQOudPk1dc2RF2mT3CbPpwVXWW5Bsc2Hk1wQmxJXiCTmGpgVKwSUaY6xE3ZMNWUWy3XrK+XeGbDtuJa54iyNocYKu7SaImnASsqVO2XxUvOyZEsON3Nk5zWfc2t67/BPUL03mONF99SIORnUM3AejB/3X/SD90/aw2GVnB3vvvfQ63gD75k39N56I2/sscbf5m7zbvOeX/g//J/+r4212ajW3PG2mv/7H7CvWJ0=</latexit>

t
As x0 2 int(dom(f)) , the corre-
sponding supporting hyperplane is
<latexit sha1_base64="129EdeA4kvN+m0HhZRhYDwpD8o8=">AAAD1HicbVLLjtMwFE0mPIbymoElm0CF1EpVlSIQsCsCCRYsikSZkZqqOM5Nx6ofke0MraKsEFt+iq/gE9jCD3CdhhFNx1Hs43OO7Wvfm+ScGRtFP/2D4MrVa9cPb3Ru3rp95+7R8b1PRhWawpQqrvRpQgxwJmFqmeVwmmsgIuFwkqxeO/3kHLRhSn60mxzmgiwlyxglFqnFsR/HEr5QJQSRaRlnVRkniqdmI3CoOjuioKgKYs8o2ZOSRkqSllJwVAqZgnYhtkR1XrnuUm0COkc1xyF+WH8OtkwJLwBN9TuU9aRl0JBe6A635KUGkBeG7QwtKWQY0JYUm8akl3jJaDgahNHwmetGVWe9iGImYwtrWzJpq94WpkpUvazfXxx1o2FUt3AfjBrQ9Zo2wXx8jlNFCwHSUk6MmY2i3M5Loi2j3D1RYSAndEWWMEMoiQAzL+tIq/AxMmmYKY2/tGHN/r+iTMTODiURxmUaV7rcmbbmyMu0WWGzF3O8cF5YkHR7cFbw0KrQFVmYMg3U8g0CQjXD2EN6RjShFktx55R67xzoblzrQjKqUmix3K6tJkgasIIw6W5ZvtKsLseaw80c2XvDlsyawXuscjl4i/lb9f8ZMSejdgb2wfTJ8OUw+vC0Ox43yTn0HniPvJ438p57Y++dN/GmHvV/+L/83/6f4CSogq/Bt631wG/W3Pd2WvD9L5Y2TxA=</latexit>

epi(f)
nonvertical, i.e., b < 0 . Then, <latexit sha1_base64="DSkleB5a1F51RPRe2niCfb5ybus=">AAADuHicbVJdb9MwFHUWYKN8bfDIS6BCGlJVJQg0EDwUgQQPPBSJskltNRznJrPqj8h2xqqov4dfwyuIf8N1GiaazlHs43OO7Wvfm5aCWxfHf4Kd8Nr1G7t7N3u3bt+5e2//4P5XqyvDYMK00OYkpRYEVzBx3Ak4KQ1QmQo4ThfvvH58DsZyrb64ZQlzSQvFc86oQ+r0IBjNFHxnWkqqsnqWr+pZqkVmlxKHVW9DlAxVSd0Zo1tS2kpp2lEqgUqlMjA+xI6oz1e+u1IbgylRLXGYPWo+DzumVFSApuYd6mbSMRjILnWPO3JhANSlYT1DSwY5BrQm5bI1mQIvGQ+TQRQPX/guWfXSN/Hpfj8exk2LtkHSgj5p2xjf/Nss06ySoBwT1NppEpduXlPjOBP+GSoLJWULWsAUoaIS7LxuollFT5DJolwb/JWLGvb/FXUqN3aoqbQ+m7jS58d2NU9epU0rl7+c11yVlQPF1gfnlYicjnwhRRk3wJxYIqDMcIw9YmfUUOaw3DZOafYugW3GdVEpznQGHVa4C2cokhacpFz5W9ZvDW9KruFwM08evucFd3bwCStZDT5gjhZP/xkxJ0k3A9tg8mz4ahh/ft4fjdrk7JGH5DE5JAk5IiPykYzJhLDgR/Az+BX8Dl+HNCxCvrbuBO2aB2SjheYvpb5Cxw==</latexit>

f(x0 ) a> (x x0 )
f(x) f(x0 ) b 8x 2 Rn .
(a, b)
<latexit sha1_base64="b6xqG0g7WI8urZSvZyVvQav0CY8=">AAAD/3icbVPLjtMwFE0bHsPwmBlgh0CGCqkjdaoUgYBdEUiwYFEQZUZqOsVxbjpWbSfYztAqyoYNv8IKseVT+Am+geu0jGg6jmKfnHNsX9/rRJngxgbB70bTv3Dx0uWtK9tXr12/sbO7d/OjSXPNYMhSkeqjiBoQXMHQcivgKNNAZSTgMJq9dPrhKWjDU/XBLjIYSzpVPOGMWqQme41voYIvLJWSqrgIk7IIo1TEZiFxKLfXRMlQldSeMLohRSspimpKLlDJVQzahVgT09PSdedqA9AZqhkO4f3qcbBmikQOaKryUFQfNYOG+Ex3uCZPNYA6Myy/0BJDggEtSblYmfQUDxl0ex0SdJ+4rodOC3Nr7EIASdrz/XAKnx2YBPsHYaIpK+hxaNOMtOcHjiyLqAw7YZJqKgSZh1wRTB15f6y6k91W0A2qRjZBbwVa3qoNsHSfwjhluQRlmaDGjHpBZscF1ZYz4bKZG8gom9EpjBAqKsGMi+pQJXmITEwwEHyVJRX7/4wikmsrFFQadylwpiuzqWuOPE8b5TZ5Ni64ynILii03TnJBbErcfSQx18CsWCCgTHOMnbATiqmzeGvXdqnWzoCtxzXPFWdpDDVW2LnVFEkDVlKu3CmLF5pXN7ficDFHtl/xKbem8xZ/CNV5jaWe7f8zYk169QpsguGj7vNu8O5xq99fFWfLu+M98Npez3vq9b033sAbeqzxp3m7ebd5z//qf/d/+D+X1mZjNeeWt9b8X38B4Qtbwg==</latexit>

x =) y = a
is a subgradient at x0
x0 <latexit sha1_base64="3yFV7uEtgyth9kuhjQ5VNim9xUg=">AAAD13icbVLLjtMwFHUmPIbhMTOwZBOokAZpqBIEAhZIRSDBgkWRKDOiqYrj3HSs2k5kO0OrKBIrxJaf4h/4B7aw5joNI5qOo9jX5xzb1z43KQQ3Ngx/elv+hYuXLm9f2bl67fqN3b39mx9MXmoGI5aLXB8n1IDgCkaWWwHHhQYqEwFHyfyl449OQRueq/d2WcBE0pniGWfUIjTd96axgs8sl5KqtIqzuoqTXKRmKXGod9ZIyZCV1J4wukElLZUkHaYUyJQqBe1S7JD5ae26c7kh6ALZAof4TvO5sCNKRAkoat6haiYdgYb0jHdxh55pAHUmWM1QkkKGCa1AuWxFeoaXDPvRYRD2H7suQqWFhTV2KSCIuUTDwATL5w/iTFNW0bpK6uleL+yHTQs2g6gNeqRtQ7TkU5zmrJSgLBPUmHEUFnZSUW05E+6VSgMFZXM6gzGGikowk6pJtg7uIZIGWa7xVzZo0P9XVIlc26Gi0jizcaWzz3Q5B57HjUubPZ1UXBWlBcVWB2elCGweuDoLUq6BWbHEgDLNMfeAnVB8FovVuHZKs3cBbD2vRak4y1PooMIurKYIGrCScuVuWb3QvKnIBsPNHHjwis+4NYdvsdDV4Wu0cH7/nxA9iboObAajh/1n/fDdo95g0JqzTW6Tu+SAROQJGZA3ZEhGhHk/vF/eb++P/9H/4n/1v62kW1675hZZa/73v6MIUJ4=</latexit>
b <latexit sha1_base64="skFoEQKtuh2DA8n3OXWGhUAz5Lw=">AAADuXicbVJdb9MwFHUWPkb52uCRl0CFNKSpSiYQIPHQCSR44KFIlE1qq+I4N62pPyLbGa2i/h9+Da/Av+E6DRNN5yj28TnH9rXvTQvBrYvjP8FeeO36jZv7tzq379y9d//g8MEXq0vDYMi00OY8pRYEVzB03Ak4LwxQmQo4SxdvvX52AcZyrT67VQETSWeK55xRh9T0MDgdK/jOtJRUZdU4X1fjVIvMriQO686WKBmqkro5oztS2khp2lJKgUqpMjA+xJaoL9a+u1IbgClQLXAYP64/D1umVJSApvodqnrSMhjILnWPW/LMAKhLw2aGlgxyDGhDylVjMjO8ZNxLjqO498J3ybqznMad6UE37sV1i3ZB0oAuadoAH/3rONOslKAcE9TaURIXblJR4zgT/h1KCwVlCzqDEUJFJdhJVYezjp4ik0W5NvgrF9Xs/yuqVG7tUFFpfTpxpU+QbWuevEoblS5/Nam4KkoHim0OzksROR35SooyboA5sUJAmeEYe8Tm1FDmsN62Tqn3LoBtx7UsFWc6gxYr3NIZiqQFJylX/pbVqeF1zdUcbubJo3d8xp09/oilrI7fY5IWz/4ZMSdJOwO7YHjSe92LPz3v9vtNcvbJI/KEHJGEvCR98oEMyJCw4EfwM/gV/A7fhGk4D79trHtBs+Yh2Wqh/QvWpUMU</latexit>
152

Main Take-Away Points

• Definitions: open and closed sets; interior, closure and


boundary of a set; closed functions; closure of a function
• Strict separating hyperplanes: a convex closed set C can
be strictly separated from any point not in C; thus, C is the
intersection of all closed halfspaces containing C; a proper
convex closed function f is the envelope of all affine functions
below f
• Weak separating hyperplanes: a convex set C can be
weakly separated from any point not in C; thus, C has a
supporting hyperplane at each boundary point; a proper
convex function has a subgradient at each point of the
interior of its domain

You might also like