1 Computational Mechanics Meets Artificial Intelligence

Leon Herrmann
Stefan Kollmannsberger
Chair of Data Engineering in Construction
Bauhaus-Universität Weimar

Deep Learning in Computational Mechanics – an introductory course,

Herrmann et al. 2025

website book
Leon Herrmann & Stefan Kollmannsberger || Deep Learning in Computational Mechanics || Bauhaus-Universität Weimar 1
Contents
• 1 Computational Mechanics Meets Artificial Intelligence (& Introduction to PyTorch):
• What is Artificial Intelligence?
• History of Artificial Intelligence
• Recent Achievements of Artificial Intelligence
• Artificial Intelligence in Science
• Challenges
• Computational Mechanics Meets Artificial Intelligence
• 2 Fundamental Concepts of Machine Learning
• 3 Neural Networks
• 4 Introduction to Physics-Informed Neural Networks
• 5 Advanced Physics-Informed Neural Networks
• 6 Machine Learning in Computational Mechanics
• 7 Material Modeling with Neural Networks
• 8 Generative Artificial Intelligence
• 9 Inverse Problems & Deep Learning
• 10 Methodological Overview of Deep Learning in Computational Mechanics
• 11 The Future of Deep Learning in Computational Mechanics
Leon Herrmann & Stefan Kollmannsberger || Deep Learning in Computational Mechanics || Bauhaus-Universität Weimar 2
What is Artificial Intelligence? Artificial Intelligence: A Modern Approach, Norvig et al. 2020
Artificial Intelligence
• “Intelligence exhibited by machines/computers“
• (Total) Turing test requires: natural language processing, knowledge representation, automated reasoning,
machine learning, (computer vision, robotics)
Intelligence
• Human or rational?
• Intelligent thoughts or intelligent behavior?

Or?
Machine Learning
• “Learn from data & generalize to unseen data
(without explicit instructions)”
Deep Learning
• “Training (deep) neural networks”
Inspired by Rebekka Woldseth, author of ”On the use of
artificial neural networks in topology optimisation”

Leon Herrmann & Stefan Kollmannsberger || Deep Learning in Computational Mechanics || Bauhaus-Universität Weimar 3
History of Artificial Intelligence Artificial Intelligence: A Modern Approach, Norvig et al. 2020

Superintelligence: Paths, Dangers,

Strategies, Bostrom 2014

Leon Herrmann & Stefan Kollmannsberger || Deep Learning in Computational Mechanics || Bauhaus-Universität Weimar 4
History of Artificial Intelligence Artificial Intelligence: A Modern Approach, Norvig et al. 2020
• The inception of artificial intelligence (1943-1956)
• Basic physiology of the brain → artificial neurons (on/off); updating rule as Hebbian Learning; SNARC
• Early enthusiasm, great expectations (1952-1969)
• Turing “a machine can never do X”; models were based on logic and symbolic reasoning; (GPS, Lisp, perceptron)
• A dose of reality (1966-1973)
• Overconfidence: models based on “informed introspection” & “intractability of attempted problems”; Lighthill
• Expert systems (1969-1986)
• Instead of general-purpose tools; domain-specific knowledge; (DENDRAL, Mycin, R1); Fifth Generation Project
• The return of neural networks (1986-)
• Reinvention of backpropagation
• Probabilistic reasoning and machine learning (1987-)
• Reaction to failure of expert systems; learn from experience → adaptable & incorporation of uncertainty
• Hidden Markov Models (Reinforcement Learning); Bayesian Networks; TD-Gammon
• Big data (2001-)
• World Wide Web: large datasets (billions-trillions of samples); ImageNet (challenge), IBM’s Watson
• Deep learning (2011-)
• Hardware improvements (GPU: 1014 − 1017 vs CPU: 109 − 1010 Flops); (Deep CNNs in AlexNet); AlphaGo
Leon Herrmann & Stefan Kollmannsberger || Deep Learning in Computational Mechanics || Bauhaus-Universität Weimar 5
 Recent Achievements in Artificial Intelligence

https://media.freemalaysiatoday.com/wp-content/uploads/
2022/05/lifestyle-garry-emel-pic-110522.jpg
Cats versus dogs https://media.freemalaysiatoday.com/wp-
content/uploads/2016/03/AlphaGo.jpg

https://commons.wikimedia.org/wiki/File:C12orf29_AlphaFold.png

Leon Herrmann & Stefan Kollmannsberger || Deep Learning in Computational Mechanics || Bauhaus-Universität Weimar 6
Artificial Intelligence in Science
Check www.aitracker.org for other trends

Publications in all fields Publications in computational mechanics

Leon Herrmann & Stefan Kollmannsberger || Deep Learning in Computational Mechanics || Bauhaus-Universität Weimar 7
Challenges
• “Generate an image of Isaac Newton in front of a blackboard on which his three laws are written in mathematical
notation and chalk.“
• Follow-up: “The laws on the blackboard are incorrect. Please add the correct formulations. If you are unable to do
so, simply focus on the second law, which is F=m*a.”

Generated with
DALL-E-3

Leon Herrmann & Stefan Kollmannsberger || Deep Learning in Computational Mechanics || Bauhaus-Universität Weimar 8
 https://openai.com/ind
ex/attacking-machine-
Challenges learning-with-
adversarial-examples/
Limitations in deep learning in general
• Neural networks break in unpredictable ways → can be consistently fooled
• Deep learning is not robust due to sensitivity to hyperparameters → requires extensive tuning
• Neural networks are uninterpretable, i.e., limited explainability → limits reliability
See chapter 11 for details
Problems in deep learning in computational mechanics
• Reproducibility crisis (bias towards positive results, sensitivity, transparency)
• Fair evaluation metrics are disregarded (breakeven threshold, meaningful metrics, statistical assesements)
• State-of-the-art is not considered 𝑇data + 𝑇train
𝜏=
𝑇simulation − 𝑇surrogate
Good scientific practice for deep learning in computational mechanics
• Honest assessments & explanations (consider the state-of-the-art & proper metrics)
• Proposed methods should be robust towards hyperparameters (no extensive tuning for a novel problem)
• Careful & narrower selection of problem types (not general-purpose solution)
• → domain-specific improvements
Towards a meaningful integration of neural networks in
computational solid mechanics, Herrmann 2025
Leon Herrmann & Stefan Kollmannsberger || Deep Learning in Computational Mechanics || Bauhaus-Universität Weimar 9
Example from topology optimization
The mean squared error
𝑚
1 2
𝑀𝑆𝐸 = ෍ 𝑥left𝑖 − 𝑥right𝑖
𝑚
𝑖=1
between the two structures is very small (2.5 ⋅ 10−3 ), due to one pixel difference.

Structural compliance (inverse of stiffness)

𝑐 = 𝑭𝑇 𝒖
Is different by one order of magnitude
For more details,
see Chapter 9

compliance= 351 compliance= 4729

Leon Herrmann & Stefan Kollmannsberger || Deep Learning in Computational Mechanics || Bauhaus-Universität Weimar 10
Computational Mechanics Meets Artificial Intelligence
Computational Mechanics
Abstraction of physical systems (reality) through simplified mathematical models (often differential
equations), which are discretized and solved numerically for insight into real-world behavior
Exemplary tasks
• Efficient solutions techniques for forward problems, e.g., finite element, difference, and volume
• Identification tasks (inverse problems), e.g., inferring material distribution/properties from measurements
• Optimization, e.g., finding the optimal material distribution that maximizes stiffness

Machine Learning Machine Learning, Mitchell 1997

”a computer program is said to learn from experience E with respect to some class of tasks T and performance
measure P, if its performance at tasks T, as measured by P, improves with experience E”

Where can machine learning be applied in computational mechanics?

• Identification of mathematical models from data (instead of relying on hand-crafted models)
• Acceleration of forward solvers and optimizers
• Streamlining of pipelines to avoid human experts within the processes

Leon Herrmann & Stefan Kollmannsberger || Deep Learning in Computational Mechanics || Bauhaus-Universität Weimar 11
Computational Mechanics Meets Artificial Intelligence
Deep learning in computational mechanics: a review, Herrmann et al. 2024
• Simulation substitution
• Data-driven modelling Simulation with graph neural networks; DMD; Transfer
learning
• Physics-informed learning Hamiltonian/Lagrangian neural networks; SINDy; (PINNs)

• Simulation enhancement Input-convex neural networks for material modeling;

EUCLID; Neural networks as ansatz function of inverse
quantities; Superresolution; Differentiable physics

• Discretizations as neural networks Hardware acceleration with GPUs; (HiDeNN)

• Generative approaches Generative design; Realistic data generation; Anomaly

detection; Transformers for natural language processing

• Deep reinforcement learning Control engineering tasks: autonomous flight; robots;

Alternative gradient-free optimizer

Leon Herrmann & Stefan Kollmannsberger || Deep Learning in Computational Mechanics || Bauhaus-Universität Weimar 12
Contents
• 1 Computational Mechanics Meets Artificial Intelligence (& Introduction to PyTorch):
• What is Artificial Intelligence?
• History of Artificial Intelligence
• Recent Achievements of Artificial Intelligence
• Artificial Intelligence in Science
• Challenges
• Computational Mechanics Meets Artificial Intelligence
• 2 Fundamental Concepts of Machine Learning
• 3 Neural Networks
• 4 Introduction to Physics-Informed Neural Networks
• 5 Advanced Physics-Informed Neural Networks
• 6 Machine Learning in Computational Mechanics
• 7 Material Modeling with Neural Networks
• 8 Generative Artificial Intelligence
• 9 Inverse Problems & Deep Learning
• 10 Methodological Overview of Deep Learning in Computational Mechanics
• 11 The Future of Deep Learning in Computational Mechanics
Leon Herrmann & Stefan Kollmannsberger || Deep Learning in Computational Mechanics || Bauhaus-Universität Weimar 13
1 Computational Mechanics Meets Artificial Intelligence
Leon Herrmann
Stefan Kollmannsberger
Chair of Data Engineering in Construction
Bauhaus-Universität Weimar

Deep Learning in Computational Mechanics – an introductory course,

Herrmann et al. 2025

website book
Leon Herrmann & Stefan Kollmannsberger || Deep Learning in Computational Mechanics || Bauhaus-Universität Weimar 14