GAMS can be used to efficiently solve multiple scenarios or instances of optimization models. The GAMS Scenario Solver and grid computing facilities allow large numbers of scenarios to be solved independently and in parallel. The Scenario Solver generates a model once and then efficiently updates it for each scenario, keeping the model "hot" within the solver. This achieves solution times close to using a solver's native application programming interface. Grid facilities distribute independent GAMS jobs across a computing cluster. These tools enable GAMS to solve thousands of scenarios rapidly.
GAMS can be used to efficiently solve multiple scenarios or instances of optimization models. The GAMS Scenario Solver and grid computing facilities allow large numbers of scenarios to be solved independently and in parallel. The Scenario Solver generates a model once and then efficiently updates it for each scenario, keeping the model "hot" within the solver. This achieves solution times close to using a solver's native application programming interface. Grid facilities distribute independent GAMS jobs across a computing cluster. These tools enable GAMS to solve thousands of scenarios rapidly.
GAMS can be used to efficiently solve multiple scenarios or instances of optimization models. The GAMS Scenario Solver and grid computing facilities allow large numbers of scenarios to be solved independently and in parallel. The Scenario Solver generates a model once and then efficiently updates it for each scenario, keeping the model "hot" within the solver. This achieves solution times close to using a solver's native application programming interface. Grid facilities distribute independent GAMS jobs across a computing cluster. These tools enable GAMS to solve thousands of scenarios rapidly.
GAMS can be used to efficiently solve multiple scenarios or instances of optimization models. The GAMS Scenario Solver and grid computing facilities allow large numbers of scenarios to be solved independently and in parallel. The Scenario Solver generates a model once and then efficiently updates it for each scenario, keeping the model "hot" within the solver. This achieves solution times close to using a solver's native application programming interface. Grid facilities distribute independent GAMS jobs across a computing cluster. These tools enable GAMS to solve thousands of scenarios rapidly.
GAMS Development Corp. GAMS Software GmbH www.gams.com
GENERAL ALGEBRAIC MODELING SYSTEM
GAMS - History • Roots: World Bank, 1976 • Alex Meeraus founded GAMS Development Corp. (USA) in 1987 and GAMS Software GmbH (Germany) in 1995
• New management team in 2016
2 GENERAL ALGEBRAIC MODELING SYSTEM
GAMS - Product General Algebraic Modeling System • Pioneered Algebraic Modeling Languages: Model is executable algebraic description of optimization problem Made mathematical optimization available to broader audience (domain experts) Increased productivity tremendously 2012 INFORMS Impact Prize • Evolution through more than 25 years of R&D and user feedback • Maturity through experience and rigorous testing 3 GENERAL ALGEBRAIC MODELING SYSTEM
Agenda
GAMS – Basic Concepts
Application: Solving Scenarios in the Cloud
4 GENERAL ALGEBRAIC MODELING SYSTEM
GAMS at a Glance Powerful algebraic modeling language with open architecture and uniform interface to all major commercial and academic solvers (30+ integrated)
Robust, scalable state-of-the-art modeling technology for
complex, large-scale modeling applications
Model Different layers Platform Solver Data Interface
5 GENERAL ALGEBRAIC MODELING SYSTEM
Separation of Model and Platform
Supported Platforms
Model 6 Platform Solver Data Interface GENERAL ALGEBRAIC MODELING SYSTEM
Separation of Model and Solver
Supported Solvers and Model Types
Model 7 Platform Solver Data Interface GENERAL ALGEBRAIC MODELING SYSTEM
Separation of Model and Data
• Declarative Modeling: x(j), j∈{1,…}
• ASCII: Initial model development ASCII GAMS
Application
• GDX: Binary Data layer
(“contract”) between GAMS and applications GAMS – Platform independent – Direct GDX interfaces and GDX Application SOLVER general API
Model 8 Platform Solver Data Interface GENERAL ALGEBRAIC MODELING SYSTEM
Separation of Model and User Interface
• Open architecture and interfaces to other systems
No preference for a particular user interface
• Smart Links to popular environments: Excel, MATLAB, R, ...
• .Net, Java, Python, C++,… • Wrapper classes that encapsulates a GAMS model • No modeling capability: Model is written in GAMS
Model 9 Platform Solver Data Interface GENERAL ALGEBRAIC MODELING SYSTEM
Encapsulation of a GAMS Model
Simple Interface to interact with GAMS • Classes to communicate input data and results • Classes to change options like the solver to use • Classes to create and run model instance(s)
Main Program Data/Results Wrapper Classes (.Net, Java, Python, C++) Options GAMS Model Create / Run
Model 10 Platform Solver Data Interface GENERAL ALGEBRAIC MODELING SYSTEM
Agenda
GAMS – Basic Concepts
Application: Solving Scenarios in the Cloud
11 GENERAL ALGEBRAIC MODELING SYSTEM
Solving “many” independent Scenarios
1. Small Ratio of MP solution time/GAMS overhead
GUSS/Scenario Solver
12 GENERAL ALGEBRAIC MODELING SYSTEM
Simple Serial Solve Loop - Basics
Both declarative and imperative Elements
Set s /s*s10/; Loop
Parameter A_s(s,i,j) “scenario data”, xlo_s(s,i,j) ”scenario lower bound for x.l”, Generation em_s(s,i) “scenario solution for e.m; Loop(s, Solution A(i,j) = A_s(s,i,j); x.lo(j)=xlo_s(s,j); Update solver mymodel min z using lp; xl_s(s,j) = x.l(j); em_s(s,i) = e.m(i); );
13 GENERAL ALGEBRAIC MODELING SYSTEM
Simple Solve Loop - Performance
Different options to call the solver from GAMS
• trnsport.gms (LP) solved 500 times (CPLEX)
Loop Setting Solve time (secs) Generation Solvelink=%Solvelink.ChainScript% 52.221 Solution
Solvelink=%Solvelink.CallModule% 37.366 Update
Solvelink=%Solvelink.LoadLibrary% 03.252
14 GENERAL ALGEBRAIC MODELING SYSTEM
Scenario Solver - Basics
• Generates model once and updates the algebraic model keeping the model “hot” inside the solver Generation • Platform independent, works with all solvers • Performance close to native solver API Solution Set s /s*s10/; Parameter Update A_s(s,i,j) “scenario data”, xlo_s(s,i,j) ”scenario lower bound for x.l”, em_s(s,i) “scenario solution for e.m; Set dict / s. scenario. ´´, A. param. A_s, x. lower. xlo_s, x. level. xl_s, e. marginal. em_s /; solve mymodel min z using lp scenario dict; 15 GENERAL ALGEBRAIC MODELING SYSTEM
GUSS/Scenario Solver 2. Large ratio i.e. only MP time is relevant (pre/post processing not critical) Grid Facility, NEOS, Gurobi/Cplex Server
17 GENERAL ALGEBRAIC MODELING SYSTEM
Grid Computing Facility
GAMS jobs in a distributed environment Scalable: supports large grids, but also works on local machine Platform independent, works with all solvers/model types Only minor changes to model required
1. Submission of jobs GAMS
2. “Grid Middleware” GAMS GAMS GAMS GAMS
– Distribution of jobs Job 1 Job 2 Job .. Job n
– Job execution
3. Collection of solutions GAMS
4. Processing of results 18 GENERAL ALGEBRAIC MODELING SYSTEM
GAMS/Kestrel Remote Solver Execution on NEOS Servers Stay in your GAMS environment Results are being processed as with any local solver Remote Cluster Local Machine (NEOS) --- Executing KESTREL: elapsed 0:00:00.006 Connecting to: http://neos-server.org:3332 NEOS Solver: xpress Model transport /all/; NEOS job#=3631352, pass=iJLdAkhP Option lp=kestrel; transport.optfile=1; Check the following URL for progress report : http://neos-server.org/neos/cgi-bin/nph-neos- $onecho > kestrel.opt solver.cgi?admin=results&jobnumber=3631352&pass=i kestrel_solver xpress JLdAkhP $offecho Job 3631352 dispatched password: iJLdAkhP Solve transport using lp minimizing z; ---------- Begin Solver Output ----------- Job submitted to NEOS HTCondor pool.
FICO-Xpress 24.3.3 r48116 Released Sep 19,
2014 LEG x86 64bit/Linux
19 GENERAL ALGEBRAIC MODELING SYSTEM
Solving “many” independent Scenarios
1. Small Ratio of MP solution time/GAMS overhead
GUSS/Scenario Solver 2. Large ratio i.e. only MP time is relevant (pre/post processing not critical) Grid Facility, NEOS, Gurobi/Cplex Server 3. If entire model run including pre processing/MP solve/post processing is costly Parallel/asynchronous execution of entire model in the cloud
20 GENERAL ALGEBRAIC MODELING SYSTEM
Application – Cloud Computing
xyz Energy Company Challenge • Scenario Analysis: Solve 1,000+ scenarios (MIPs, one hour) every week overnight
Issues: • Automation • Security • Licensing 21 GENERAL ALGEBRAIC MODELING SYSTEM
Application – Cloud Computing
xyz – Energy Company Implementation: • Amazon Cloud: 1,000+ parallel machines (instances), Python, GAMS + OO Python API • Automated setup, including • Starting instances • Prepare / Submit / Run GAMS jobs • Collect results • Stop instances
22 GENERAL ALGEBRAIC MODELING SYSTEM
Application – Cloud Computing
Protecting IP and Sensitive Data Options to hide sensitive information: • Extrinsic function libraries / External Equations • Encrypted source files / secure work files • Obfuscated work files: change all the names and other documentation related to a specific model run
23 GENERAL ALGEBRAIC MODELING SYSTEM
Application – Cloud Computing
Obfuscated Files – An Example
Normal Solver Log
Reduced LP has 5 rows, 6 columns, and 12 nonzeros. Presolve time = 0.00 sec. (0.00 ticks)
• Preparation of obfuscated binary • Control of instances
model file • Submission of model and • Preparation of data for every worker scenario data to workers • Mapping of results into original • Collection and merge of results namespace 26 GENERAL ALGEBRAIC MODELING SYSTEM
Software Licensing: • Gurobi and IBM offer per-usage license • Client with strong preference for annual license fee, not a per-usage license
27 GENERAL ALGEBRAIC MODELING SYSTEM
Application – Cloud Computing
45 Provided Model Instances • Statistics: • 163,608 – 1,959,550 rows • 84,930 – 983,587 var. (32,240-258,796 dis.) • 447,537 – 6,068,729 NZ • CPLEX, SCIP, and CBC • 60 minutes, gap max. 1% • Manual option tuning for SCIP (thanks to Gerald Gamrath & Ambros Gleisner)
28 GENERAL ALGEBRAIC MODELING SYSTEM
Application – Cloud Computing
Results • CPLEX: All instances solved to optimality • SCIP: • Could solve all 45 instances • But: After 60 min. 2 instances with gap > 20% • CBC: • Did also well • But: After 60 min. no solution for some instances (< 10%)
29 GENERAL ALGEBRAIC MODELING SYSTEM
Application – Cloud Computing
Proposed Strategy • Run all instances simultanuesly with SCIP and CBC „hardware“ costs: $0,07 per instance hour
• After 60 minutes take the best solution
• If necessary solve „difficult“ instances with CPLEX
(outside the cloud)
30 Thank You Europe USA GAMS Software GmbH GAMS Development Corp. P.O. Box 40 59 1217 Potomac Street, NW 50216 Frechen, Germany Washington, DC 20007, USA Phone: +49 221 949 9170 Phone: +1 202 342 0180 Fax: +49 221 949 9171 Fax: +1 202 342 0181 info@gams.de sales@gams.com
GAMS Development Corp. GAMS Software GmbH www.gams.com
