22 Linear Programming 3: The Simplex Method The simplex method is presented and applied to the sotut f n of a certain elase of tineaw prox obLema Gaussian elimination Chapter 21: Linear Programming 2 INTRODUCTION At the end of Chapter 21, we described the simplex method as a procedure for moving from one basic feasible solution of a linear programing problem to an adjacent basic feasible solution in such @ way that the value of the objective function never decreases. In this chapter we shall describe the algebraic details of this proce- dure. The general Linear progranning problen in n variables was stated as Problem 21.1 on page 304, However, in order to simplify the presentation of the simplex method, we shall restrict ourselves to Linear programming problens having the following special form 323324 / The Simplex Method PROBLEM 22.1 Find daluee of stys ty5.++4 2, which mazimize eee tog t i tos, in?n £91 ann £2, In Problem 22.1, the condition that each of the m constraints be a < inequality is not restrictive since it can easily be shown that any Linear programaing problem can always be written with all < con- straints, It is the condition b;>0 for j=1,2,...,m that is the real restriction, Nevertheless,’a large class of practical problems are of this form, and the procedures developed in this chapter for Problem 22.1 are used in the application of the simplex method to the general Linear programming problen. To convert Problem 22.1 to ono in standard form (see Problem 21.2 on page 304) we introduco m slack variables 416,491 ++) Som? one for each of the m constraints, to obtain PROBLEM 22.2 Find values of sys tys+0+4 fp4q, which maxim Brom togey tie 9,8, + Oa beet FO subject toLinear Programming III / 325 "1 * Sma * E€ we can find an optimal solution to Problem 22.2, then the values of the variables 21, 23,...,, will provide an optimal solution to probles 22.1. THE SIMPLEX TABLEAU In order to more clearly describe the steps in the simplex nethod, let us examine the following specific problen of the type we are considering in this chapter: PROBLEM 22.3 Find values of ay, ty and ts which maximize tae, subject to To convert Problen 22.5 to one in standard form, we add slack varia~ bles 4, a5. tg to obtain326 / The Simplex Method PROBLEM 22.4 Find values of 2.2m ty ty 2, and ae which masinine a ry dts + On, +O, + Ong subject to 2ey- 254 Sey ey =5 Behe Bey tee Boy +6 tag 4 and 20. Hin pu ge Es yum, For our purposes, it will be convenient to rephrase Problem 22.4 in the following equivalent fore: PROBLEM 22.5 which satisfy values of ay ty ty 22, in By they ty yt dey ~ 2 * = (22.0) treo and ouch that 21, tp. B55 24 By te are nonnegative and 5 te ae Imye as posstbie. In this formulation of the problem, 2 is treated as a varisble on @ par with xy through 2g, and the equation defining = in terms of the 2 is treated as an additional constraint. Thus our problen is to find a solution of the linear system of four equations in seven un- knowns given by (22:1) in which one of the variables, =, is as large as possible and the othor six variables are nonnegative. ‘The usual procedure for solving a linear systen of equations is to construct the augmented matrix of the systea and apply Gaussian elimination or Gauss-Jordan elimination to it to put i in rox-echelon form (or reduced row-echeton form). The row-echelonLinear Programming 111 / 327 form then determines the augnented matrix of an equivaient Linear system which is easily solved. The simplex method proceeds along similar lines. Namely, the augmented natrix of the linear systen is constructed and a variation of Gaussisn elimination, called elimination, is applied to obtain augmented matrices in which basic feasible solutions to the linear programing problem can be deter- mined by inspection. Let us return to Problem 22.5 to see how this is done. ‘The auguented matrix of the linear system (22.1) is [a 203 04 0 09 0 6s L ¢ 2 GC fem 8 gy aay pee ee arte tener eteeeigee rect te aS eee ees oe ee gO From the way the 1's and 0's are distributed in the 4th, Sth, 6th, and 7th colums of this matrix, one particular solution of (22.1) can be seen by inspection; namely, 4 + 2220, 2, 1, eee 4, 30. 22.39) 3 571+ 6 Im terms of the corresponding linear programming probles, this is (22.4) It is easily seen that x' is a basic feasible solution of linear progranming Problem 22.4, according to Definition 21.5 on page 249. ‘The three variables x4, 2g, ag are the basic variables, and their corresponding values are found as the first three entries of the last colum of (22.2), The value of s is the last entry of this column. That x! is a basic feasible solution is of crucial impor- tance, since it is awong the basic feasible solutions that we can hope to find an optinal solution. To see how we may go about find- ing another basic feasible solution, we rewrite: (22.2) with some ‘additional Labeling:328/ The Simplex Method Tableau 22.1 Seger era eee set si s ae : 2 a a : 2 a 1 4 co : a] . 2 oe 4 S os Each column has been Labeled with its corresponding variable from the linear system (22.1). On the right, we have labeled the entries of the last column with the corresponding variables whose values they determine in the solution given by (22.3). We have also dravn a vertical and horizontal line within the matrix in order to high- Light certain entries which will be useful to us later on. In the field of Linear programming, the augmented matrix is re~ ferred to as a tableau. In particular, Tableau 22.1 above is called the initia? tabieu of the problen. Ne shall call the last row of the tableau the objective row, since it arises fron the objective function of the original probien. In Tableau 22.1 we have also shaded four particular colums. It can be seen that these four columns are identical to the coluans of the 4x4 identity matrix. Indeed, it was exactly this fact that pernitted us to find the solution given in (22.3) s0 easily. We had only to set those variables not corresponding to these four colums equal to zero, and then the values of the variables corresponding to the four colums were found in the last column of the augmented matrix, ‘This suggests a way of proceeding to a new solution of the linear system. We apply appropriate elementary row operations to Tableau 22.1 to arrive at a new tableau which again contains the four col- ums of the 4x4 identity matrix, but this time in different posi- tion, To see how to do this, consider the following tableau, which is just Tableau 22.1 with one of its entries shaded. (We postpone for the moment a discussion of why this particular entry was chosen.) Tabbeau 22.2Linear Progranming IIT / 329 our objective will be to use olenentary row operations to replace the shaded entry by a “I” and obtain zeros everywhere else in that coluin, To do this we first divide the third row by six to obtain Tableau 22.3 “4 Next we perform the following three elementary row operations, 1, Add -3 times the 3rd row to the Ist row. 2. Add 2 times the 3rd row to the 2nd row, 5. Add 4 times the Srd row to the 4th row. ‘The result is the following tableau: we a = oe = 85 aa Te can be Seen that Tableau 22.4 contains within it the four columns of the 4x4 ddentity matrix, though not in their usual order. Con- sequently, if we set the vatiables not associated with these colunns equal to Zero, we obtain the following solution to linear system (22.1) aye Spe US, ge 0, 58/3. (22.5) As before, we have labeled the entries in the last column with the variables’ whose values they determine in this solution.330/ The Simplex Method Solution (22.5) specifies the following vector solution of Problem 22.4: xr [705) and 5 = 8/3. 22.6) We see that x" is a basic feasible solution with basic variables 23, 24, 25. We also see that x! as given in (22.4) and x" are adjacent bavte feasible eoluttone since they have m-1=2 basic variables in common; namely, x4 and zs. That is, the elementary row operations we performed on Tableau 32.1 to obtain Tableau 22.4 took us from the basic feasible solution x’ to the adjacent basic feasible solution x". In addition, the value of the objective function increased from 350 to =8/3. ‘The shaded entry in Tableau 22.2 is called the pivot ontry of that particular tableau. The column in which it Lies 4s called the pivot calm, and the row in which it lies is called the pivot ray. After the elenentary row operations were applied to obtain Tableau 22.4, the pivot column contained all zeros, except for a one in the pivot entry position. Thus the pivot colum was converted to one of the colums of the 4x4 identity matrix, and the variable 23 corre- sponding to the pivot column was converted from a nonbasic variable to a basic variable. At the same timo, the variable zg which label- ed the pivot rox, was converted fron a basic variable to a nonbasic variable, That is, 5 replaced zg as a basic variable in going from Tableau 22.1 to Tableau 22.4. For this reason, 23 is called the entering variable and 2g is called the departing Sariable of Tableau dae, Let us see if we ean increase the value of the objective func- tion above the value == 8/5 attained in Tableau 22.4. Below we have rewritten Tableau 22.4 with a pivot entry shaded. Again, we post- pone until the next Section a discussion of how this pivot entry was chosen. Tebleau 22.5 Enter =,Linear Programming III / 331 We apply the elementary row operations to the first column to con vert it to a coluan with a "I!" in the pivot position and zeros everywhere else. Thus, 21 will be the entering varlable and z5 will be the departing variable, as the arrows in the tableau indicate. ‘The reader can casily verify that the elenentary row operations nec- essary will produce the next tableau: Tableau 22.6 Setting those variables not corresponding to the shaded columns equal to zero yields the solution = 7/60 ay 0, zy" U2, 2429/12, mg" 0, y= 0, 3 25/6. 22.7 For the linear programing problew, we then have the following basic feasible solution and objective function value: 16 [%] yi = hen) 4 ° ° 23/6. (22.8) As we shall show in the next section, the value 2= 25/6 is the Ia fest value the objective function can assime over the feasible set. ‘Thus we have reached an optimal solution. For the solution to Prob- lem 22.3, we discard the slack variables 4, 25, a, and write ge 1/12 (22.9) as the optinal solution, with the corresponding maxim 2 23/6 for the objective function. This example illustrates the kinds of calculations required to implenent the simplex method. In the next section we shall discuss how to choose the pivot entry and how to determine if an optimal solution has been reached.332 / The Simplex Method Steps IN THE SimPLEX METHOD In this section we shall outline the steps in the simplex method and give an example. In the next section we shall discuss their mathonatical justification. The simplex method consists of the folowing five Steps: STEP 1 Conetrust the initial tableau. STEP 2 Zest for optimality. If the tableau ytelde an optinal eolution, then eto; othemtee, eontinue to Step 3. STEP 3 Determine the pivot oolwm. STEP 4 Determine the pivot row. STEP 5 Apply the elementary row operations to obtain ail zeros in the pivot colum, exept for a None" in the pinot row. Return to Step 2. ‘The details for Steps 2, 3, and 4 are as follows: Test for Optimality. If all of the entries in the ob- Jective row are nonnegative (ignoring the rightmost entry) The tableau determines an optinal solution. Determination of Pivot Colum. Chodse the pivot colum so that it contains the most negative entry of the ob- jective row (ignoring the rightmost entry). Determination of Pivot Row. Ignoring the objective row, divide each positive entry of the plyot column into the last entzy in its rox, Choose the pivot row to be one which yields the smallest such ratio. ‘The reader should return to the tableaus constructed in connection with Problem 22.5 to verify that the pivot entries in each tableau Were selected according to the above rules, and elso to verify that Tableau 22.6 determines an optinal solution to the problen. Let us apply the simplex method as described above to the fol- Jowing example: EXAMPLE 2261. pandt vatuas of ey) eye ay shtch nance am Se, +40, + 2gLinear Programming 111 / 333 subject to Se, +2eyt dey £15 ayt2y+ 8g <7 at et ays 6 and fe Bp Hy SOLUTION 15 standard form, this problen is Maximize" 2 = 32) +40) + 205+ 004 + 025 + 0, subject to Bey + dey desea, "15 te, 7 tage 6 and 1g > 0. ‘The initial tableau for the problen is then Tableau 22.7 The objective row contains negative entries, so that the initial tableau does not determine an optimal solution. The nost negative entry, -4, 1ies in the second column, so that the second column will be the pivot column. To determine the pivot row, we evaluate the fotlowing ratios:334/ The Simplex Method Ase row: 15/2= 7%. 2nd row: 7/2= 54. Sed row: 6/1=6. ‘The 2nd row yields the smallest ratio so that it will be the pivot row, So far ve have the following: Tabteay 22.8 We now apply the following elementary row operations to Tableau 22.8: Divide the 2nd row by 2. Add -2 tines the 2nd row to the Ist Tow. Add -1 times the 2nd row to the 3rd row. Add .4 times the 2nd row to the 4th row. ‘The resulting tableau is the following: Tapieau 22.9 ‘The objective row still contains a negative entry, so that we have not yet reached an optimal solution. The new pivot colum is the first since it contains the only negative entry of the objective row. To determine the pivot row, we evaluate the following ratios: z Ist rows 6/2=4, 2nd row: (7/2)/(4/2) 7, 3rd rows (5/2)/ (3/2)Linear Programming III / 335 The third row yields the smallest ratio, so that it is the new pivot rox, So far, we have Tableau 22.30 & fe ty my te ty 8 Boe um ot 4 so wl clan, Desart [Mae 2 72 8 2 OO) 72] + a seelo2) 0-2 0-2 1 0 | 5/2] 4 We now apply the following elenentary row operations to Tableau 22.10: Divide the 3rd row by 3/2. Add -2 times the 3rd row to the Ist row. ‘Add ~i; times the 3rd row to the 2nd row. Add I times the 3rd row to the 4th row. The resulting tableau is Tableau 22.11 ‘The objective row of this tableau does not contain any negative en tries and so this tableau deterines an optimal solution. The basic variables in the optinal basic feasible solution are =, 22, and 24 as the righthand Labeling indicates. The optimal solution is 8, 55/3, 2278/3, 25" 0, 2414/3, 5 "0, 260, 2= 47/3.336/ The Simplex Method For the original problem posed in this example, we discard the slack Variables 24, 25, 5 and simply write 5/3, 2,7 8/3, #5=0, 5m 47/3, We conclude this section with sone remarks concerning complica- tions which may arise in the use of the simplex method as we have described it: 1, Im Step 3 it is possibte that there is a tie for the nost nogative entry In the objective row. In that case, any. one of them nay be chosen, and no complications arise. 2, In Step 4 it is possible that there is more than one row with the same smaliost ratio. In that case, any one of then may be used to determine the pivot row and no conpli- cations arise in the calculations. However, if such a tle arises, it can be shown that the basic feasible solution determined by the next tableau will be degenerate (i.e., will have a basic variable whose value is zero). As discussed in the last chapter, it is degeneracy which may bring about cy- cling. But as we also mentioned, it is more a theoretical problem than a practical probien, 3. In Step 4 it is possible that no entry in the pivot column is positive, in which case our technique for evaluating the pivot row is meaningless. Tt can be shown that if this Situation arises, the problem has an wibounded aolution. JUSTIFICATION OF THE STEPS IN THE SIMPLEX METHOD Let us return to the Linear progranning problem in +m varia bles posed in Problem 22.2. Suppose at some point in our calcula tions we have arrived at Tableau 22.12, (In Exercise 22.11, we ask the reader to show that in any tableaa the column labeled "2!" al- ways has the form indicated.) Thus the current basic variables are yy Byprs++1 Zyq Mith corresponding values dy,d,,.--) 4), and the current value of the objective variable is w. Let us see if the entry Jyq WOULG make a suitable pivot entry. The entering variable would be z, and the departing variable would be agus TE Yyg# 0, the elenentary row operations in Step 5 of the simplex method wilt pro- duce @ tableau having the form of Tableau 22.25,Linear Progranming III / 337 Tableau 22.12 i i Y2 7" Ye * 4} sn 422 °** Hae 4, "Fa 4, B- alee, ‘The pivot colwm now contains all zeros except for the "1" in the previous pivot entry position. All of the other entries in the tableau have new values, which we indicate with asterisks, except338 / The Simplex Method in the last column where we have explicitly written the new values in terms of the entries in Tableau 22.12, First, let us examine ‘the value of the entering variable #, Bang (22.10) Since 4,>0, we see that we mist have y,,>0 in order to satisfy the constraint #20, Let us list this fact as OBSERVATION 22.4 The pivot entry must be positive in onder that the new tableau determine a feavible aolution, ‘The remaining basic variables have values given by Sep" Me must have 2,20 for the new solution to be feasible, Now if for some £ we have y,,<0, then (22.11) states that for that 2, 29420 since d,>0, d,>0, and y,,>0. On the other hand, if for Lp-VigdelYpy FOR E=2, 2005 sone ¢ we have yz.>0, then (21-11) requires that 45 - Yeah lng 2 (22.12) in order that 5,20 for that ¢. Equation (22.12) can also be writ- ten as a (22.13) ne ~ Yee In other words, Eq. (22,13) mist be satisfied for 211 those 7 for which ygq? 0 in order that the new tableau determine a feasible solution. We state this as OBSERVATION 22.2 In order that the nev tableau determine @ feasible solution, the following must be true: The ratto of the elenent in the rightmost cotum of the pivot rev to the pivot entry mist be the enatleat of the eorweaponding rabioe in all of the other rove which contain positive en ties tn the pivot colum (ignoring the objective raw).Linear Programming 111 / 339 Next, let us examine the new value u* of the objective function. Fron Tableau 22.13, we see that wa D- Od /Yyy - (22.14) Ideally, we would want the increase in the objective function (22.15) to be as large as possible, But this would require that we compute the quantities -264 Ng for all possible values of » and # to find the largest one. Usually this is not done because of the large munber of calculations this would require. Instead, the entering variable zz 1s chosen so that Oe is as nogative as possible, Since d.>0 and ¥,,>0, Eq. (22.15) then guarantees that ise. that the objective function does not decrease. As discussed in the last chapter, this always eventually leads to an optinal solution, except for the very remote possibility of cycling. Let us sumarize this as follows: OBSERVATION 22.3 Chosse the pivot colum eo that tt con tains the most negative entry of the objective nav (tg noving the entry in the rightmost column). Equation (22.15) also tells us the following: If all of the c's are nonnegative then the value of the objective function cannot increase, regardless of the choice of pivot entry. In this case, we mst have already attained the maximum value of the objective function, ‘Thus, we have OBSERVATION 22.4 If all of the entrice in the objective rou, except for the rightmost entry, ane nonnegative, ‘the tableau determines an optinal eotution.340 / The Simplex Method ‘The above four observations justify the steps in the simplex nothod. Readers interested in pursuing Linear programing in nore ée- tail are referred to the following texts: S. I Gass, Linear Programming, 4th ed. New York: NeGraw-Hiill Book Company, 1975. L. Cooper and D, Steinberg, Methods and Applications of Linear Programing, Philadelphia: W. 8. Saunders Company, 1974. Exercises Jn Beorotaes 22.1 to 22.8 solve the given Linear programming prob- ten by the simplex method. Ped Maximize 2= 52, + 42, subjeet to and 22.2 Maximize a= 2c, +2) subject to Bey ey <4 Beye aye = gS and By B20 22.3 Maximize 5535, - 22, + 65 subject to 2ey- Sey tty <2 yt #08555 and Hyp Bop Fy 200Linear Programming [11 / 344 22.4 Maximize 5= 22) 2)-25 subject to Peps beyt 2s? By ¢82,+ ty < Bey- 4ey- 3 < and 2, 2a, 22.5 Maximize 2= 52) subject to 2ay-Saytes- 2,26 Bt tny- 25+ 2e, 4 and ye pe 22.6 Maximize a= .- 2+ 3ryt2, subject to B= eat yt Sey 8 2H, + 3ey- yt ey <5 yt eyo Sey tary 26 and 22.7 Solve Exanple 20.1 by the sinplex wathod. 22.8 Solve Exercise 20.1 by the simplex method. 22.9 Solve Exercise 20.6 by the simplex method. 22.10 Solve Exercise 20.7 by the sinplox mothod. 22,44 Show that in any tableau the column labeled "2" always has the form indicated in Tableau 22.12.