Contents Foreword Preface Authors’ Profiles and Copyright Convention and Problem Categorization List of Abbreviations List of Tables List of Figures 1 Introduction 1.1 Competitive Programming - 1.2. Tips to be Competitive 1.2.1 Tip 2: Quickly Identify Problem Types 1.2.2 Tip 3: Do Algorithm Analysis . 1.23 Tip 4: Master Programming Languages 1.24 Tip 5: Master the Art of Testing Code 1.2.5 Tip 6: Practice and More Practice 1.2.6 Tip 7: Team Work (ICPC Only) 13. Getting Started: The Ad Hoc Problems - 14 Chapter Notes i i 2 Data Structures and Libraries 2.1 Overview and Motivation 2.2 Data Structures with Built-in Libraries 2.2.1 Linear Data Structures 2.2.2 Non-Linear Data Structures 2.8 Data Structures with Our-Own Libraries . 23.1 Graph 23.2 Union-Find Disjoint Sets 3.3 Segment Tree . A Fenwick Tree 24 Chapter Notes 3. Problem Solving Paradigms 3.1 Overview and Motivation 3.2 Complete Search 3.2.1 Examples 2 Tips Divide and Conquer xiii xiv 30 aL aCONTENTS © Steven & Felix 3.3.1 Intorosting Usages of Binary Search oo... ooo ee eee eee AT 34. Greedy. . Eee ee ra sees BL SS4c1 cerebro cetera a ee eee 35 Dynamic Programming... 0.00 cece BB 3.5.1 DP Illustration eee : : wees BB 35.2 Classical Examples. 000 eee ee eee eee OL 3.5.3 Non Classical Examples o.oo... eee eee cece es 6 3.6 Chapter Notes... eee : : Mo: 4 Graph a 4.1 Overview and Motivation ©... 066. c eee e eee e eee e eee 4.2 Graph ‘Traversal... EE Heese ese eater eoe eee 42.1 Depth First Search (DFS) : fl a 422 Breadth First Search (BFS)... .. . Ree eee Era: 423. Finding Connected Components (in an Undirected Graph)... 0.02... 73 4.24 Flood Fill - Labeling/Coloring the Connected Components fl a 42.5 Topological Sort (of a Directed Acyclic Graph)... 0.00... eee ees 4.2.6 Bipartite Graph Check... . . Seer eee 42.7 Graph Badges Property Check via DES Spanning Tree fl 6 428 Finding Articulation Points and Bridges (in an Undirected Graph) . 7 4.2.9 Finding Strongly Connected Components (in a Directed Graph) 80 43° Minimum Spanning Tree a 43.1 Overview and Motivation 4 43.2 Kruskal’s Algorithm Mt 433° Prim’s Algorithm 85 434 Other Applications . 86 44 Single-Source Shortest Paths RE ees ees peor eet: 44.1 Overview and Motivation 9 9 91 93 6 9% 6 98 44.2 SSSP on Unweighted Graph 44.3 SSSP on Weighted Graph 44.4 SSSP on Graph with Negative Wat Cycle. 4.5 All-Pairs Shortest Paths : : 4.5.1 Overview and Motivation 452 Explanation of Plyd Warsal' DP Solution 5.8. Other Applications . Ce eter eee ett 46 aaa el EEE eet EeSt eee eee 46.1 Overview and Motivation»... 1... ; ; ee LOL 4.6.2 Ford Fulkerson’s Method... 0.000000 eee eee eee eee ees IO 46.3 Edmonds Karp’s .. 2... 0.2200 eee eee eee eee es 102 4.6.4 Other Applications... 6... ees ‘| ; ce 104 4.7 Special Graphs... - reece ete ere ee Pee ore ATA Directed Acyclic Graph... 1. 2s... csc ssserese eres es 107 47.2 Tree Eee eee ; ; Heed: 473 Bolerian Graph o.oo. ccc e cece cece eve eeee ees ell 474 Bipartite Graph 02.00 eee eee ee TM 48 Chapter Notes... REA ‘| ; se 9 5 Mathematics 121 5.1 Overview and Motivation 206. .00eeeeeeeeeeeeeeee ee DI 5.2 Ad Hoc Mathematies Problems 121 53 Java BigInteger Class... ca ce we 15 BBL Basic Features eee ee eee ee 1B 9.3.2 Bonus Features 126CONTENTS © Steven & Felix 54 Combinatorics ree 129 5.4.1 Fibonacci Numbers . 129 5.4.2 Binomial Coefficients 130 543 Catalan Numbers 1 5.44 Other Combinatorics 11 5.5 Number Theory 133 5.5.1 Prime Numbers 133 5.5.2 Greatest Common Divisor (GCD) & Least Common Multiple (L 135 5.5.3 Factorial 5 136 5.5 Finding Prime Factors with Optimized ‘Trial Divisions 136 5.5.5 Working with Prime Factors ee, 137 5.6 Functions Involving Prime Factors 138 5.5.7 Modulo Arithmetic . 40 55.8 Extended Buclid: Solving Linear Diophantine Equation Mi 5.5.9 Other Number Theoretic Problems v2 5.6 Probability Theory 42 5.7 Cycle-Finding M3 5.7.1 Solution using Eficient Data Structure 43 87.2 Floyds Cycling Algoritn 43 5.8 Game Theory ; M5 58.1 Decision Tree 45 5.8.2 Mathematical Insights to Speed-up the Solution 46 5.83 Nim Game ea 46 5.9 Powers of a (Square) Matrix a7 5.9.1 The Idea of Efficient Bxponentiation 7 5.9.2. Square Matrix Exponentiation us 5.10 Chapter Notes 49 6 String Processing 1st 6.1 Overview and Motivation. 151 6.2 Basic String Processing Skills . . 151 63 Ad Hoc String Processing Problems 153 64 String Matching Ae 156 64.1 Library Solution 156 64.2 Knuth-Morris-Pratt (KMP) Algorithm 156 64.3. String Matching in a 2D Grid . 158 6.5 String Processing with Dynamic Programming 160 65.1 String Alignment (Edit Distance) 160 6.5.2 Longest Common Subsequence 161 65.3 Palindrome . . 162 6.6. Suffix Trie/Tree/ Array 163 6.6.1 Suffix Trie and Applications 163 6.6.2 Suffix ‘Tree 163 6.6.3 Applications of Sufix Tree 164 6.64 Suffix Array ; 166 6.6.5 Applications of Sufix Array 170 6.7 Chapter Notes va 7 (Computational) Geometry 175 71 Overview and Motivation 5 72 Basic Geometry Objects with Libraries 176 7.2.1 OD Objects: Points 176 72.2 ID Objects: Lines . WwCONTENTS © Steven & Felix 723° W Objects: Circles... 6... Re eee eee 1 724 2D Objects: Triangles eee eee ee ee 188 725 2D Objects: Quadrilaterals 185 726 3D Objects: Spheres... 6... Pere HEH eee Hea H7.as7]aD, Objects) Otlines ee ee-e st teria stai te apace 1B 7.3. Polygons with Libraries 188 Polygon Representation... 0.2.02... 05 PEPE HEH eee eI: Perimeter of a Polygon... 000 eee eee cere eee eee ee ee + 188 Area of a Polygon... ree : : wee 188 Checking if a Polygon is Convex se ove vce vena eee 189 Checking if a Point is Inside a Polygon... oe eee ee es 189 Cutting Polygon with a Straight Line 190 7.3.7 Finding the Convex Hull of a Set of Points... . pe eee eet 1 7A Divide and Conquer Revisited... 0.00 ee 15 75. Chapter Notes 196 8 More Advanced Topies 197 8.1 Overview and Motivation ©... 00 eevee eee eee eee eee eee ee IT 82. Problem Decomposition 197 82.1 Two Components: Binary Search the Answer and Other... . se. 197 8.22 Two Components: SSSP and DP... 2.2... ERE Hee eet Ie 3 Two Components: Involving Graph... . ss. pa FEE ee: 8.24 Two Components: Involving Mathematics ry ses 199 82.5 Three Components: Prime Factors, DP, Binary Search... 0... +++ + - 199 6 Thro Components: Complete Search, Binary Search, Groody ....... . . 199) 83 More Advanced Search Techniques... . iz oes 208 Siac Hinormed Searet tae tree Het reae ee tse H Ree ate eer eee e200: 83.2 Depth Limited Search oo eee Sea ereABapanease” 83.3 Iterative Deepening Search. . . . ry ces 208 8.34 Iterative Deepening A* (IDA*) .. 0 see ee es ERE Hee eee OO: 84 More Advanced Dynamic Programming Techniques «0.0. svc vce eves 1205 84.1 Emerging Technique: DP + bitmask re ses 205 84.2 Chinese Postman/Route Inspection Problem... 000 eevee ee ee «205 843 Compilation of Common DP States . an CA EEE ee ea: 844 MLB/TLE? Use Better State Representation! ry ses 207 84.5 MLE/TLE? Drop One Parameter, Recover It from Others! .. 0... +.» . 208 84.6 Your Parameter Values Go Nogative? Use Offset Technique... ..... . . 209 85. Chapter Notes an A Hints/Brief Solutions 213, B uHunt 225 © Credits 227 D Plan for the Third Edition 228 Bibliography 229CONTENTS © Steven & Felix Foreword Long time ago (exactly the Tuesday November 11th 2003 at 3:55:57 UTC), I received an email with the following sentence: T should say in a simple word that with the UVa Site, you have given birth to a new CIVILIZATION and with the books you write (he meant “Programming Challenges: ‘The Programming Contest Training Manual” [34 coauthored with Steven Skiena), you inspire the soldiers to carry on marching. May you live long to serve the humanity by producing super-human programmers, Although it’s clear that was an exaggeration, to tell the truth I started thinking a bit about and. Thad a dream: to create a community around the project I had started as a part of my teaching Job at UVa, with persons from everywhere around the world to work together after that ideal. Just by searching in Internet I immediately found a lot of people who was already creating a web-ring of sites with excellent tools to cover the many lacks of the UVa site. ‘The more impressive to me was the ‘Methods to Solve’ from Steven Halim, a very young student from Indonesia and I started to believe that the dream would become real a day, because the contents of the site were the result of a hard work of a genius of algorithms and i Moreover his declared objectives matched the main part of my dream: to serve the humanity. And the best of the best, he has a brother with similar interest and capabilities, Felix Halim, It’s a pity it takes so many time to start a real collaboration, but the life is as itis. Fortunately, all of us have continued working in a parallel way and the book that you have in your hands is the best proof. can’t imagine a better complement for the UVa Online Judge site, as it uses lots of examples from there carefully selected and categorized both by problem type and solving techniques, an incredible useful help for the users of the site. By mastering and practicing most programming exercises in this book, reader can easily go to 500 problems solved in UVa online judge, which will place them in top 400-500 within 100000 UVa OJ users ‘Then it’s clear that the book “Competitive Programming: Increasing the Lower Bound of Programming Contests” is suitable for programmers who wants to improve their ranks in upcoming ICPC regionals and IOIs. ‘The two authors have gone through these contests (ICPC and 101) ‘themselves as contestants and now as coaches. But its also an essential colleague for the newcomers, becanse as Steven and Felix say in the introduction ‘the book is not meant to be read onee, but several times’. Moreover it contains practical C++ source codes to implement the given algorithms. Because understand the problems is a thing, knowing the algorithms is another, and implementing them well in short and efficient code is tricky. After you read this extraordinary book three times you will realize that you aze a much better programmer and, more important, a more happy person. ormaties, ‘Miguel A. Revilla, University of Valladolid UVa Online Judge site ereator; ACM-ICPC International Steering Committee Member and Problem Archivist netp://uva.onlinejudge.org; http: //Livearchive. online judge orgCONTENTS © Steven & Felix Preface ‘This book is a must have for every competitive programmer to master during their middle phase of their programming career if they wish to take a leap forward from being just another ordinary coder to being among one of the top finest programmer in the world. book would be: 1. University students who are competing in the annual ACM International Collegiate Program- ‘ming Contest (ICPC) [38] Regional Contests (including the World Finals), 2. Secondary or High School Students who are competing in the annual International Olympiad in Informatics (IOI) (21) (including the National level), 3. Coaches who are looking for comprehensive training material for their students [13], 4. Anyone who loves solving problems through computer programs. ‘There are numerous pro- gramming contests for those who are no longer eligible for ICPC like TopCoder Open, Google CodeJam, Internet Problem Solving Contest (IPSC), ete. ‘Typical readers of t Prerequisites ‘This book is not written for novice programmers. When we wrote this book, we set it for readers who have basie knowledge in basic programming methodology, familiar with at least one of these programming languages (C/C++ or Java, preferably both), and have passed basic data structures and algorithms course typically taught in year one of Computer Science university curriculum. Specific to the ACM ICPC Contestants We know that one cannot probably win the ACM ICPC regional just by mastering the current version (2nd edition) of this book. While we have included a lot of materials in this book, we are much aware that much more than what this book can offer, are required to achieve that feat, Some reference pointers are listed in the chapter notes for those who are hungry for more. We believe however, that your team would fare much better in future ICPCs after mastering this book. Specific to the IOI Contestants ‘Same preface as above but with this additional Table 1. This table shows a list of topies that are currently not yet included in the [01 syllabus {10]. You can skip these items until you enter university (and join that university’s ACM ICPC teams). However, learning thom in advance may bbe beneficial as some harder tasks in IOT may require some of these knowledge,CONTENTS © Steven & Felix Topic Tn This Book Data Structures: Union-Find Disjoint Sets Section 2.3.2 Graph: Finding SCCs, Max Flow, Bipartite Graph Section 4.2.1, 4.6.3, 4.7.4 Math: Bighnteger, Probability, Nim Games, Matrix Power | Section 5.3, 5.6, 5.8, 5.9 String Processing: Suffix Tree/ Array Section 6.6 More Advanced Topics: A*/IDA* Section 8.3 Table 1: Not in IOI Syllabus [10} Yet ‘We know that one cannot win a medal in IOT just by mastering the current version of this hook While we believe many parts of the IOI syllabus have been included in this book ~ which should give you a respectable score in future 1Ols ~ we are well aware that modern IOI tasks requires more problem solving skills and creativity that we cannot teach via this book. So, keep practicing! Specific to the Teachers/Coaches This book is used in Steven’s C3238 - ‘Competitive Programming’ course in the School of Com- puting, National University of Singapore. It is conducted in 13 teaching weeks using the following lesson plan (see Table 2). The PDF slides (only the public version) are given in the companion web site of this book. Hints/brief solutions of the written exercises in this book are given in Appendix A. Fellow teachers/coaches are free to modify the lesson plan to suit your students’ needs ‘WE | Topic Ta This Book DL | Introduction ‘Chapter 1 02 | Data Structures & Libraries Chapter 2 03 | Complete Search, Divide & Conquer, Greedy | Section 3.2.3.4 04 | Dynamic Programming 1 (Basic Ideas) Section 3.5 05. | Graph 1 (DFS/BES/MST) Chapter 4 up to Section 4.3 06 _| Graph 2 (Shortest Paths; DAG-Tree) Section d.ded.5; 71-472 =__| Mid semester break 2 07 | Mid semester team contest 7 08 | Dynamic Programming 2 (More Techniques) | Section 6.5; 8.4 09. | Graph 3 (Max Flow; Bipartite Graph) Section 4.6.3; 4.7.4 10 | Mathematics (Overview) Chapter 5 11 | String Processing (Basic skills, Suffix Array) | Chapter 6 12 | (Computational) Geometry (Libraries) Chapter 7 13 _| Final team contest All, inchiding Chapter 8 = [No final exam: 7 ‘Table 2: Lesson Plan To All Readers Due to the diversity of its content, this book is not meant to be read once, but several times, There are many written exercises and programming problems ( 1198) scattered throughout the body text of this book which can be skipped at first if the solution is not known at that point of time, but can be revisited Inter after the reader has accumulated new knowledge to solve it. Solving these exercises will strengthen the concepts taught in this book as they usually contain interesting ‘twists or variants of the topic being discussed. Make sure to attempt them once. We believe this book is and will be relevant to many university and high school students as ICPC and 101 will be around for many years ahead. New students will require the ‘basic’ knowledge presented in this book before hunting for more challenges after mastering this book. But before you assume anything, please check this book’s table of contents to see what we mean by ‘basic’CONTENTS © Steven & Felix We will be happy if in the year 2010 (the publication year of the first edition of this book) and beyond, the lower bound level of competitions in ICPC and IOI increases because many of the contestants have mastered the content of this book. That is, we want to see fewer teams solving very few problems (< 2) in future ICPCs and fewer contestants obtaining less than 200 marks in future IOls. We also hope to sce many ICPC and IOL coaches around the world, especially in South East Asia, adopt this book knowing that without mastering the topics in and beyond this book their students have very little chance of doing well in future ICPCs and 10s. If such increase in ‘required lowerbound knowledge” happens, this book would have fulfilled its objective of advancing the level of human knowledge in this era, Changes for the Second Edition ‘There are substantial changes between the first and the second edition of this book. As the authors, ‘we have learned a number of new things and solved hundreds of programming problems during the one year gap between these two editions. We also have received several feedbacks from the readers, especially from Steven’s 053233 class Sem 2 AY2010/2011 students, that we have incorporated in the second edition. Here is a summary of the important changes for the second edition: ‘© The first noticeable change is the layout. We now have more information density per page. ‘The 2nd edition uses single line spacing instead of one half line spacing in the Ist edition, ‘The positioning of small figures is also enhanced so that we have a more compact layout, This is to avoid increasing the number of pages by too much while we add more content. ‘* Some minor bug in our example codes (both the ones shown in the book and the soft copy given in the companion web site) are fixed. All example codes now have much more meaningful comments to help readers understand the code. ‘ Several known language issues (typo, grammatical, stylistic) have been corrected. © On top of enhancing the writing of existing data structures, algorithms, and programming, problems, we also add these new materials in each chapter: 1. Much more Ad Hoc problems to kick start this book (Section 1.3) 2. Lightweight sot of Boolean (bit manipulation techniques) (Section 2.2.1), Implicit Graph. (Section 2.3.1), and Fenwick Tree data structure (Section 2.3.4) 3, More DP: Clearer explanation of the bottom-up DP, O(n log k) solution for LIS problem, Introducing: 0-1 Knapsack/Subset Sum, DP TSP (bitmask technique) (Section 3.5.2) 4. Reorganization of graph material into: Graph ‘Traversal (both DFS and BFS), Mini- mum Spanning Tree, Shortest Paths (Single-Sonrce and All-Pairs), Maximum Flow, and Special Graphs. New topics: Prim’s MST algorithm, Explaining DP as a traversal on implicit DAG (Section 4.7.1), Eulerian Graph (Section 4.7.3), Alternating Path Bipartite Matching algorithm (Section 4.7.4). 5. Reorganization of mathematies topics (Chapter 5) into: Ad Hoe, Java Biglnteger, Com- binatories, Number Theory, Probability Theory, Cycle-Finding, Game Theory (new). ‘and Powers of a (Square) Matrix (new). Each topic is rewritten and made clearer, 6. Basie string processing skills (Section 6.2), more string problems (Section 6.8), string matching (Section 6.4), and an enhanced Suffix Tree/Array explanation (Section 6.6). 7. Much more geometric libraries (Chapter 7), especially on points, lines, and polygons. 8, New Chapter 8: Discussion on problem decomposition; More advanced search techniques (A*, Depth Limited Search, Iterative Deepening, IDA*); More advanced DP: more bit- asks techniques, Chinese Postman Problem, compilation of common DP states, dis- ‘cussion on better DP states, and some other harder DP problems.CONTENTS © Steven & Felix ‘© Many existing figures in this book are redrawn and enhanced. Many new figures are added. to help explain the concepts more clearly. © The first edition is mainly written using [CPC and C++ viewpoint. The second edition is now written in a more balanced viewpoint betwoen ICPC versus 101 and C++ versus Java Java support is strongly enhanced in the second edition, However, we do not support any other programming languages as of now. + Steven's ‘Methods to Solve’ website is now fully integrated in this book in form of ‘one liner hints’ per problem and the useful problem index at the back of this book, Now, reaching 1000 problems solved in UVa online judge is no longer a wild dream (we now consider that this feat is doable by a serious 4-year Computer Science university undergraduate). ‘* Some examples in the first edition use old programming problems. In the second edition, some examples are replaced added with the newer examples. ‘* Addition of around 600 more programming exercises that Steven & Felix have solved in UVa online judge and Live Archive between the first and second edition. We also give much more conceptutal exercises throughout the book with hints/short solutions as appendix. + Addition of short profiles of data structure/algorithm inventors adapted from Wikipedia [42] or other sources. It is nice to know a litte bit, more abont the man behind these algorithms. Web Sites ‘This book has an official companion web site at: http://sites.google.con/site/stevenhalin. In that website, you can download the soft copy of sample source codes and PDF slides (but only the public/simpler version) used in Steven's (83233 classes. All programming exercises in this book are integrated in: nttp://felix-halim.net/uva/hunting. php, and in nttp: //uva, online judge. org/index. pap?option=com_ online judge temid=8ucategory=118 Acknowledgments for the First Edition Steven wants to than! ‘© God, Jesus Christ, Holy Spirit, for giving talent and passion in this competitive programming, ‘¢ My lovely wife, Grace Suryani, for allowing me to spend our precious time for this project ‘¢ My younger brother and co-author, Felix Halim, for sharing many data structures, algorithins, and programming tricks to improve the writing of this book. ‘¢ My father Lin Tjie Fong and mother Tan Hoey Lan for raising us and encouraging us to do well in our study and work. ‘* School of Computing, National University of Singapore, for employing me and allowing me to teach C8323 - ‘Competitive Programming’ module from which this book is born. ‘* NUS/ex-NUS professors/lecturers who have shaped my competitive programming and coach- ing skills: Prof Andrew Lim Leong Chye, Dr Tan Sun Teck, Aaron Tan Tuck Choy, Dr Sung Wing Kin, Ken, Dr Alan Cheng Holun. ‘¢ My friend Mham Winata Kurnia for proof reading the manuscript of the first edition, ‘* Follow Toaching Assistants of CS3233 and ACM ICPC Trainers @ NUS: Su Zhan, Ngo Minh Due, Melvin Zhang Zhiyong, Bramandia Ramadhana.CONTENTS © Steven & Felix ‘¢ My $3233 students in Sem2 AY2008/2009 who inspired me to come up with the lecture notes and students in Sem? AY2009/2010 who verified the content of the first edition of this ook and gave the initial Live Archive contribution Acknowledgments for the Second Edition Additionally, Steven wants to thank: ‘¢ © 550 buyers of the 1st edition as of | August 2011. Your supportive responses encourage us! ‘* Fellow Teaching Assistant of C$3233 @ NUS: Victor Loh Bo Huai, ‘© My 83238 students in Sem2 AY2010/2011 who contributed in both technical and presenta tion aspects of the second edition of this book, in alphabetical order: Aldrian Obaja Mui Bach Ngoc Thanh Cong, Chen Juncheng, Devendra Goyal, Fikril Bahri, Hassan Ali Askar Harta Wijaya, Hong Dai Thanb, Koh ZiChun, Lee Ying Cong, Peter Phandi, Raymond Hendy Susanto, Sin Wenlong Russell, Tan Hiang Tat, Tran Cong Hoang, Yuan Yuan, and one other stuclent who prefers to be anonymous (class photo is shown below). ‘* The proof readers: Seven of CS3235 stuclents above (underlined) plus Tay Wenbin. ‘* Last but not least, Steven wants to re-thank his wife, Grace Suryani, for letting me do another round of tedious book editing process while you are pregnant with our first baby. ‘To a better future of humankind, Sreven and FELIX HALIM Singapore, 1 August 2011