Lesson 4: Developing Models

Lesson objectives
 To introduce the student to the practical steps of model development
 At the end of the lesson the student should be in a position to design a simple simulation
model

Introduction
 Model development refers to the process of creating, testing, and refining a mathematical
or computational model that represents a system or process of interest.
 The goal of model development is to create a model that accurately represents the system
or process and can be used to make predictions or test hypotheses.
 The process of model development typically involves the following steps:
i. Define the problem or system of interest and identify the variables that need to
be modeled.
ii. Choose an appropriate modeling approach based on the available data and the
goals of the model. This could involve choosing between different types of models,
such as linear regression, neural networks, or decision trees.
iii. Collect and preprocess data to be used in training and validating the model. This
may involve cleaning and transforming data, handling missing values, and
selecting relevant features.
iv. Train the model using the data and an appropriate algorithm. This involves
adjusting the model parameters to minimize the difference between the predicted
and actual values.
v. Validate the model by testing its performance on a separate set of data that was
not used for training. This helps ensure that the model is not overfitting to the
training data and can generalize to new data.
vi. Refine the model as needed based on the validation results. This could involve
adjusting the model parameters, choosing different features or algorithms, or
collecting additional data.
vii. Deploy the model for use in real-world applications.

 The process of model development can be iterative, with each round of testing and
refinement leading to a better model. It is also important to evaluate the assumptions
and limitations of the model and to communicate the results and uncertainties to
stakeholders.
 The generic steps of model development can be grouped into five blocks of the modeling
cycle
Figure 1: Main steps of a modeling cycle

1.1. Model conceptualization

 Model conceptualization is the initial stage of developing a model, where the general
ideas and concepts underlying the model are formulated.
 This stage involves defining the problem or research question to be addressed, identifying
the key variables that need to be included in the model, and selecting an appropriate
modeling approach.
 The process of model conceptualization typically involves the following steps:
i. Identify the problem or research question: The first step is to clearly define the
problem or research question that the model will address. This involves
understanding the context and purpose of the model and the stakeholders
involved.
ii. Define the scope of the model: Once the problem or research question is
identified, the next step is to define the scope of the model. This involves
specifying the system or process to be modeled, the relevant variables and their
relationships, and the time frame and spatial scale of the model.
iii. Identify the key variables: The next step is to identify the key variables that need
to be included in the model. This involves considering the variables that are most
relevant to the problem or research question, as well as any other factors that
may influence the system or process being modeled.
iv. Select an appropriate modeling approach: Once the key variables are identified,
the next step is to select an appropriate modeling approach. This may involve
 choosing between different types of models, such as statistical models, simulation
models, or machine learning models, based on the available data and the goals of
the model.
v. Develop a conceptual diagram: Finally, the conceptualization stage involves
developing a conceptual diagram that represents the general ideas and concepts
underlying the model. This diagram should include the key variables, their
relationships, and any assumptions or limitations of the model.
 The process of model conceptualization is an important first step in developing a model, as it
helps ensure that the model is well-defined and meets the goals and requirements of the
stakeholders. It also provides a clear roadmap for the subsequent stages of model
development.

1.2. Model formalization

 Model formalization is the process of translating the conceptual model developed during the
model conceptualization stage into a precise and quantitative representation that can be
simulated or analyzed.
 Simply, formalization refers to the process of translating a conceptual model into a computer-
readable ‘language’.
 This stage involves specifying the mathematical or computational equations that describe the
relationships between the variables in the model, and defining the data structures and
algorithms needed to implement the model.
 Critical steps in model formalization include:
i. Selecting a modeling approach: The first step in model formalization is to select a
modeling approach that can represent the relationships between the variables in the
model. This may involve choosing between different types of models, such as linear
regression models, differential equation models, or agent-based models.
ii. Defining the equations and parameters of the model: The next step is to define the
mathematical or computational equations that describe the relationships between
the variables in the model. This involves specifying the parameters and variables in
the model, and defining the functional relationships between them.
iii. Specifying the data structures and algorithms: Once the equations and parameters
are defined, the next step is to specify the data structures and algorithms needed to
implement the model. This may involve defining the input data needed to run the
model, as well as the output data generated by the model.

1.3. Model parametrization

 Model parametrization is the process of assigning values to the parameters in a
mathematical or computational model.
 Parameters are the numerical values that determine the behavior of the model and
govern the relationships between the variables in the model.
 The process of model parametrization involves selecting appropriate values for the
parameters that best represent the real-world system being modeled.

 The process of model parametrization typically involves the following steps:

i. Identify the parameters: The first step in model parametrization is to identify the
parameters in the model. These are the numerical values that appear in the
mathematical or computational equations that describe the relationships
between the variables in the model.
ii. Determine the range of possible values: Once the parameters are identified, the
next step is to determine the range of possible values that each parameter can
take. This may involve reviewing literature or other sources to find estimates for
the parameter values, or conducting experiments or simulations to estimate the
values.
iii. Select initial parameter values: After the possible range of values for each
parameter is determined, the next step is to select initial values for the
parameters. This may involve using educated guesses or estimates based on
previous studies or observations.
iv. Fit the model to the data: The next step is to fit the model to the available data by
adjusting the parameter values to best match the observed data. This may involve
using optimization techniques to find the parameter values that minimize the
difference between the model predictions and the observed data.
v. Evaluate the model performance: Once the parameter values are estimated and
the model is fit to the data, it is important to evaluate the performance of the
model. This may involve comparing the model predictions to additional data, or
conducting sensitivity analyses to evaluate the impact of different parameter
values on the model predictions.
 vi. Refine and adjust the parameters: Based on the evaluation results, the parameters
may need to be refined and adjusted to improve the model performance. This may
involve adjusting the parameter values, or modifying the model structure or
assumptions.
 The process of model parametrization is a critical step in model development, as it
determines the behavior and accuracy of the model predictions.
 It involves selecting appropriate values for the parameters based on available data, and
refining the parameter values to improve the model performance.

1.4. Model calibration

Figure 2: Graphical depiction of model calibration process

 Model calibration is the process of adjusting the model parameters to improve the fit of
the model to observed data. Model calibration involves finding the optimal values of the
model parameters that result in the best possible agreement between the model
predictions and the observed data.
 The process of model calibration involves the following steps:
i. Collect and preprocess data: The first step in model calibration is to collect and
preprocess the data that will be used to calibrate the model. This may involve
cleaning and filtering the data, and transforming it into a suitable format for use
in the model.
ii. Specify the model structure: The next step is to specify the structure of the model,
including the equations, variables, and parameters that will be used to simulate
the system being modeled.
iii. Select initial parameter values: Once the model structure is specified, initial values
are selected for the parameters of the model. These initial values may be based
on previous studies, literature, or expert knowledge.
iv. Run simulations and compare to data: The model is then used to simulate the
system being modeled, and the results of the simulations are compared to the
observed data. This comparison is used to evaluate the fit of the model to the
data.
 v. Adjust parameters and repeat: If the fit of the model to the data is not satisfactory,
the parameters of the model are adjusted and the simulation is run again. This
process is repeated until the model provides an adequate fit to the observed data.
vi. Evaluate model performance: Once the model has been calibrated, it is important
to evaluate its performance. This may involve comparing the model predictions to
additional data, conducting sensitivity analyses to evaluate the impact of different
parameter values on the model predictions, or using statistical tests to evaluate
the goodness-of-fit of the model.
 The process of model calibration is an iterative process that involves adjusting the model
parameters until the model provides an adequate fit to the observed data.
 Calibration is an important step in model development, as it helps to ensure that the
model accurately represents the system being modeled and provides reliable predictions.

1.5. Model analysis

 Model analysis is the process of evaluating and interpreting the results of a mathematical
or computational model. Model analysis involves examining the behavior and predictions
of the model, as well as the assumptions and limitations of the model.
 The process of model analysis involves the following steps:
a. Evaluate model predictions: The first step in model analysis is to evaluate the
predictions of the model. This may involve comparing the model predictions to
observed data, or conducting sensitivity analyses to evaluate the impact of
different parameter values on the model predictions.
b. Identify model limitations: It is important to identify the limitations of the model,
including assumptions, simplifications, and uncertainties. This may involve
conducting uncertainty analyses to evaluate the impact of different sources of
uncertainty on the model predictions.
c. Interpret model behavior: The next step is to interpret the behavior of the model,
including the relationships between the variables and the mechanisms that drive
the behavior of the system being modeled.
d. Conduct sensitivity analyses: Sensitivity analyses can be conducted to evaluate the
impact of different parameter values on the model predictions, and to identify the
parameters that have the greatest impact on the model behavior.
e. Conduct robustness analyses: Robustness analyses can be conducted to evaluate
the impact of different model structures and assumptions on the model
predictions, and to identify the assumptions that have the greatest impact on the
model behavior.
f. Communicate results: The final step is to communicate the results of the model
analysis to stakeholders, including policymakers, scientists, and the general public.
This may involve developing visualizations and summaries of the results, as well
as explaining the implications of the results for decision-making.
  The process of model analysis helps to ensure that the model is reliable and provides
accurate predictions. It involves evaluating the behavior and predictions of the model, as
well as identifying the limitations and assumptions of the model.

1.6. Model validation

Figure 3: Model validation involves comparing model outputs to reality

 Model validation is the process of evaluating the performance of a mathematical or

computational model by comparing its predictions to independent data that were not
used in the model development.
 The purpose of model validation is to ensure that the model provides reliable and
accurate predictions when applied to new situations or scenarios.
 Model validation involves the following steps:
o Collect independent data: The first step in model validation is to collect
independent data that were not used in the model development. These data
should be representative of the system being modeled and should be relevant to
the questions or problems that the model is intended to address.
o Specify validation metrics: Validation metrics are used to evaluate the
performance of the model. These metrics should be relevant to the questions or
problems that the model is intended to address and should be based on the
independent data that were collected.
o Evaluate model performance: The model is then applied to the independent data,
and the performance of the model is evaluated using the validation metrics. If the
model provides accurate and reliable predictions, it is considered to be validated.
o Refine the model: If the model does not provide accurate and reliable predictions,
it may be necessary to refine the model by adjusting the model structure or
 parameters. The process of refinement may involve iterative cycles of model
development and validation until the model provides satisfactory predictions.
o Document the validation process: The final step is to document the validation
process, including the independent data that were used, the validation metrics
that were specified, and the results of the validation analysis. This documentation
is important for ensuring the reproducibility and transparency of the model
validation process.
 The process of model validation provides an objective evaluation of the performance of
the model.
 Validation helps to ensure that the model provides reliable and accurate predictions when
applied to new situations or scenarios, and increases confidence in the use of the model
for decision-making.

1.7. Methods for model validation

1. Cross-validation: In cross-validation, the model is trained on a subset of the data and
tested on the remaining data. This process is repeated several times using different
subsets of the data, and the results are averaged to evaluate the performance of the
model.

2. Holdout validation: In holdout validation, a portion of the data is set aside for testing, and
the model is trained on the remaining data. The model is then evaluated based on its
performance on the testing data.
3. Bootstrap validation: In bootstrap validation, the model is trained on multiple samples of
the data, each of which is randomly drawn with replacement from the original data. The
results from each sample are then combined to evaluate the performance of the model.
 4. Sensitivity analysis: Sensitivity analysis involves varying the inputs to the model to
evaluate the impact on the model's output. This helps to identify the inputs that have the
greatest impact on the model's performance and can help to improve the accuracy and
reliability of the model.
5. Out-of-sample testing: In out-of-sample testing, the model is tested on data that was not
used in the model development process. This helps to assess the generalizability of the
model and to identify any potential biases or limitations in the model.

1.8. Application of models to decision making

 Models are often used in decision-making processes to help individuals or organizations
make informed choices based on available data and information.
 Models can help decision-makers to understand complex systems and evaluate the
potential impacts of different scenarios or policy choices.
 In using models for decision making the following steps are followed:
i. Define the problem: The first step in using a model for decision-making is to clearly
define the problem that needs to be addressed. This may involve identifying the
relevant stakeholders, defining the decision criteria, and specifying the available
data and resources.
ii. Develop the model: The next step is to develop a model that can address the
problem. The model should be based on relevant data and information, and
should be validated to ensure that it provides reliable and accurate predictions.
 iii. Identify potential scenarios: Once the model has been developed and validated,
potential scenarios or policy choices can be evaluated using the model. These
scenarios should be based on the decision criteria and should reflect the range of
options available.
iv. Evaluate the scenarios: The model can be used to evaluate the potential impacts
of each scenario, based on the available data and assumptions. This may involve
conducting sensitivity analyses to evaluate the impact of different assumptions or
inputs on the model predictions.
v. Make the decision: Based on the results of the model analysis, a decision can be
made that best addresses the problem and decision criteria. The decision may
involve selecting a particular scenario or policy choice, or may involve developing
a plan of action based on the model predictions.
vi. Monitor and refine: Finally, the decision-making process should be monitored
over time to evaluate the effectiveness of the decision and to identify
opportunities for refinement or improvement. This may involve updating the
model as new data and information become available, or refining the decision
criteria based on the outcomes of the decision.

1.9. Obstacles to adoption of models in decision making

i. Some of the obstacles to effective use of models in decision making include the
following:
i. Data limitations: Models require data to provide reliable and accurate predictions,
and data limitations can affect the quality and usefulness of the model. Data may
be incomplete, outdated, or inaccurate, and it may be difficult or expensive to
collect the necessary data.
ii. Model complexity: Some models may be highly complex, making it difficult for
decision-makers to understand the assumptions and limitations of the model. This
can lead to confusion or misinterpretation of the model results, and can
undermine the usefulness of the model.
iii. Technical expertise: Using models often requires technical expertise, which may
not be available or accessible to all decision-makers. This can lead to over-reliance
on model results without a thorough understanding of the underlying
assumptions and limitations.
iv. Time and resource constraints: Developing and using models can be time-
consuming and resource-intensive, requiring significant investment in data
collection, model development, and validation. This can be a barrier to effective
use of models, particularly in situations where decisions must be made quickly.
v. Lack of transparency: In some cases, models may be seen as "black boxes" that
are difficult to understand or interpret. This can reduce transparency in the
decision-making process and undermine confidence in the model results.
1.10. Application of models
Example of real world applications of models include
i. Epidemiological models: Epidemiological models are used to study the spread and impact
of infectious diseases, and to evaluate the potential impact of interventions such as
vaccination or social distancing measures. During the COVID-19 pandemic,
epidemiological models were widely used to inform public health policy decisions,
including the timing and duration of lockdowns, and the distribution of vaccines.

Example:
Yang HM, Lombardi Junior LP, Castro FFM, Yang AC (2021) Mathematical modeling of the
transmission of SARS-CoV-2—Evaluating the impact of isolation in São Paulo State (Brazil)
and lockdown in Spain associated with protective measures on the epidemic of CoViD-19.
PLoS ONE 16(6): e0252271. https://doi.org/10.1371/journal.pone.0252271

ii. Climate models: Climate models are used to study the Earth's climate system and to
project future climate change scenarios. These models are used to inform climate policy
decisions, including the design of mitigation and adaptation strategies, and the evaluation
of the potential impacts of climate change on natural and human systems.
iii. Economic models: Economic models are used to study the behavior of markets and the
impact of policy decisions on the economy. These models are used to inform a wide range
of economic policy decisions, including tax policy, trade policy, and monetary policy.
iv. Transportation models: Transportation models are used to study the behavior of
transportation systems, and to evaluate the potential impacts of infrastructure
investments or policy changes. These models are used to inform transportation planning
decisions, including the design of new transportation networks, the pricing of
transportation services, and the evaluation of the potential impacts of transportation
policy changes.

v. Financial models: Financial models are used to study the behavior of financial markets
and to evaluate the potential risks and returns of different investment strategies. These
 models are used to inform a wide range of financial decision-making, including asset
allocation, risk management, and the valuation of financial instruments.

Additional reading
1. Kerr, L. A., Cadrin, S. X., & Secor, D. H. (2010). Simulation modelling as a tool for examining the
consequences of spatial structure and connectivity on local and regional population dynamics.
ICES Journal of Marine Science, 67(8), 1631-1639.
2. Higgins, S. I., Richardson, D. M., & Cowling, R. M. (2001). Validation of a spatial simulation model
of a spreading alien plant population. Journal of Applied Ecology, 38(3), 571-584.