Simulation Driven Design

Benchmark Report
Getting It Right the First Time

October 2006
 The Simulation Driven Design Benchmark Report

Executive Summary

G et it done quickly. That’s the message manufacturers are hearing from the mar-
ket. They must develop more products that are more complex for their custom-
ers, but, most of all, they must get to market on time. When trying to figure out
how to get things done more quickly, these manufacturers face a seeming para-
dox: should they take more time to perform simulations in design so they can save time
and money in testing? Some have found the answer is not only” yes,” but that early simu-
lation assists them to hit their product development targets. How? Interestingly enough,
it’s actually quite simple.

Key Business Value Findings

• Best in class manufacturers their hit revenue, cost, launch date, and quality targets for
86% or more of their products.
• Best in class manufacturers average 1.6 fewer prototypes than all others.
• Best in class manufacturers of the most complex products get to market 158 days
earlier with $1,900,000 lower product development costs.
• Best in class manufacturers of the simplest products get to market 21 days earlier
with $21,000 fewer product development costs.

Implications & Simulation

• All best in class manufacturers use simulation in the design phase compared to only
75% of laggards.
• Best in class manufacturers are 63% more likely to provide CAD-embedded simula-
tion to their engineers.
• Best in class manufacturers are 48% more likely to provide technologies to transfer
models from CAD to independent preprocessors to their analysts.
• Best in class performers are 42% more likely than all others to provide specific ex-
amples to users for training.

Recommendations for Action

• Perform more simulation of product performance in the design phase.
• Provide CAD-embedded simulation capabilities to engineers.
• Use training materials and specific examples to get new users up to speed.
• Employ technologies that transfer geometry from CAD to independent pre-
processors for analysts.
• Track requirements and regulatory product compliance prior to design release.

All print and electronic rights are the property of Aberdeen Group © 2006.
Aberdeen Group • i
 The Simulation Driven Design Benchmark Report

Table of Contents

Executive Summary .............................................................................................. i

Chapter One: Issue at Hand.................................................................................1

The Ultimate Goal: Hit Ever-shrinking Time-to-market Windows ................... 1
Early Simulation Cultural Challenges? A Misconception, Not a Reality.......... 2

Chapter Two: Key Business Value Findings .........................................................5

Varying Prototype Costs and Time across Product Complexity ..................... 6
Avoiding Physical Prototypes with Virtual Prototypes .................................... 7

Chapter Three: Implications & Simulation ...........................................................8

All Best in Class Performers Utilize Simulation in the Design Phase............. 8
Familiar Environments: Engineers Access Simulation through CAD ............. 9
Design Reuse: Analysts Transfer from CAD to Preprocessors .................... 10
Tracking Configurations with the Simulation Model or Data Management... 11
Educating the User: Formalized Training Programs .................................... 12
Keeping an Eye on Requirements and Change Orders............................... 13

Chapter Four: Recommendations for Action ...................................................... 15

Laggard Steps to Success........................................................................... 15
Industry Average Steps to Success ............................................................. 15
Best in Class Next Steps ............................................................................. 16

Appendix A: Research Methodology .................................................................. 17

Appendix B: Related Aberdeen Research & Tools ...............................................1

All print and electronic rights are the property of Aberdeen Group © 2006.
Aberdeen Group
The Simulation Driven Design Benchmark Report

Figures

Figure 1: Challenges to Simulation-driven Design................................................2

Figure 2: Responses to Simulation-driven Design Challenges.............................3

Figure 3: Best in Class Hit Targets on an 86% Average or Better ........................5

Figure 4: Best in Class Perform More Simulations Earlier....................................8

Figure 5: Best in Class Provide Engineers CAD-embedded Simulation...............9

Figure 6: Best in Class Provide Experts Independent Simulation Tools ............. 10

Figure 7: Best in Class Track Simulation Configurations.................................... 12

Figure 8: Best in Class Performers Provide Formalized Training Programs....... 13

Figure 9: Manufacturers Track Compliance to Requirements

and Change Orders........................................................................................... 14

Tables
Table 1: Top Five Business Pressures and Strategic Actions ...............................1

Table 2: General Characteristics of Product Complexity Categories ....................6

Table 3: Prototype Costs and Time per Product Complexity ................................7

Table 4: PACE Framework ................................................................................. 18

Table 5: Relationship between PACE and Competitive Framework ................... 19

Table 6: Competitive Framework........................................................................ 19

All print and electronic rights are the property of Aberdeen Group © 2006.
Aberdeen Group
 The Simulation Driven Design Benchmark Report

Chapter One:
Issue at Hand

• Manufacturers are improving product performance (72%) and development efficiency

Key Takeaways

(51%) in order to address shortened time-to-market (70%) constraints.

• Manufacturers identify shorter time to market (72%) as a top business pressure to
adopt early simulation, yet cite lack of time (44%) as the top challenge.
• Manufacturers are responding to challenges such as lack of expertise (39%) and
complicated product behavior (28%) with formal training programs.
• Manufacturers, in fact, did not identify any of the expected cultural challenges in im-
plementing early simulation as actual issues.

W hile the concept of using simulation early in the product development cycle
initially emerged almost a decade ago, it remains a frequently pursued initia-
tive today. Although one would expect an elevated level of use of simulation
in the up-front design phase by now, in fact, the pressure for shorter time to
market and engineering cultural challenges have prevented manufacturers from succeed-
ing with this new paradigm. Yet some of these companies are overcoming these barriers
to realize tangible business benefits.

The Ultimate Goal: Hit Ever-shrinking Time-to-market Windows

In one form or another, manufacturers adopting simulation earlier in the product devel-
opment process are reacting to pressure for shorter time to market by improving quality
and development efficiencies (Table 1).

Table 1: Top Five Business Pressures and Strategic Actions

Business Pressures Strategic Actions

Shortened time to market 70% Improve product performance or quality 72%

Customer demand for new products 51% Improve development efficiency 51%
Increasingly complex customer require- 40% Reduce base development costs 20%
ments
Accelerating product commodization 30% Develop markets via breakthrough innovation 18%
Threatening competitive products 20% Iterate product design more often 16%
Source: AberdeenGroup, October 2006

All print and electronic rights are the property of Aberdeen Group © 2006.
Aberdeen Group • 1
The Simulation Driven Design Benchmark Report

Based on the results of Aberdeen research,

Elgin Sweeper Company
clearly the top pressure driving manufac-
turers to adopt earlier simulation is short- “We are definitely pressured to get to de-
ened time to market (70%). In response to sign release more quickly in order to keep
this pressure, manufacturers are following up with the competition. We need to get
two main strategies: improving product to market first to win market share. We’re
performance or quality (72%) and improv- turning to simulation to minimize our test-
ing development efficiency (51%). At first ing phase of product development.”
glance, it’s not immediately clear how Jay Abrams, Elgin Sweeper Company
these relate to one another or how they re-
late to simulation early in the product development process. However, follow-up inter-
views with survey respondents found that they were looking to early simulation to ac-
complish two goals: to arrive at a good design earlier and to minimize time spent in the
verification and testing phase of product development.
In addition to their time concerns, manufacturers identify customer demand for new
products (51%) and increasingly complex customer requirements (40%) as secondary
business pressures driving earlier simulation. Correspondingly, they are turning to strate-
gies such as improving development efficiency (51%) and iterating the product design
more often (16%).
Overall, manufacturers are turning towards early simulation to save time in product de-
velopment as well as to get to better designs. However, the ultimate goal is to meet the
shorter time-to-market windows that are today’s reality.

Early Simulation Cultural Challenges? A Misconception, Not a Reality

In the simulation-driven design trend, one would expect major cultural barriers to requir-
ing engineers to perform simulation earlier in product development because they are be-
ing asked to do more in the same amount of time. In reality, the expected cultural chal-
lenges were not cited as major obstacles (Figure 2).

Figure 1: Challenges to Simulation-driven Design

Lack of time 44%

Lack of expertise 40%

Complex product behavior 33%

Physical test correlation 26%

Software and hardware costs 19%

0% 10% 20% 30% 40% 50%

Source: AberdeenGroup, October 2006

All print and electronic rights are the property of Aberdeen Group © 2006.
2 • AberdeenGroup
 The Simulation Driven Design Benchmark Report

In fact, the top challenge, lack of time

The Holland Group
(44%), aligns with the top business driver,
shortened time to market (70%). However, “We run analyses so we can test out a de-
some manufacturers are caught in a conflict sign before we actually spend money on a
between reducing time in the testing phase prototype. Overall, while it takes more
and spending more time in the design phase time to run a simulation, we’ll save time
performing early analyses. Coming to the later by avoiding multiple rounds of pro-
realization that a company must allow en- totypes.”
gineers more time to perform analyses up- Tave Hass, The Holland Group
front in order to save time and cost later in
the development cycle in itself is a cultural challenge.
The challenges of lack of expertise (40%) and complex product behavior (33%) for per-
forming simulation within engineering organizations provides ample motivation for
manufacturers to respond with formal training rollout programs (Figure2). In fact, the top
four responses to challenges in this case reflect just that.

Figure 2: Responses to Simulation-driven Design Challenges

FEA concepts education 47%

Software training 45%

Capture and deploy best practices 40%

Acquire easy to use software 29%

Educate executives on benefits 26%

0% 10% 20% 30% 40% 50%

Source: AberdeenGroup, October 2006

Manufacturers are acquiring easy-to-use software (29%) and providing software training
(45%) in order to reduce the technological barriers of performing simulation for non-
expert and infrequent engineering users. Simultaneously, manufacturers are pursuing
FEA concepts education (47%) as well as programs to capture and deploy best practices
(40%) in order increase the quality of analyses, so the results are more dependable and,
consequently, yield better designs. These four tactics directly address the second and
third top challenges: lack of expertise and complicated product behavior.

All print and electronic rights are the property of Aberdeen Group © 2006.
Aberdeen Group • 3
The Simulation Driven Design Benchmark Report

While some cultural issues, in fact, are inherent

CTS Corporation
in the challenges indicated by survey respon-
dents, many of the common perceptions are “Correlation to physical test is a large
actually untrue. Less than 16% of manufactur- challenge for us. For one, the testing
ers identified issues such as only analysts per- lab can’t perfectly match the idealized
form simulation, difficult to get engineers to setup within the finite element simu-
perform analyses, and little confidence in re- lation tool. Also, the molded parts that
sults as challenges to performing simulation are tested are slightly different
upfront. because of manufacturing variation.
And, finally, the material properties
Overall, the message is clear. Manufacturers of our injection-molded parts are
are addressing the secondary challenges of lack orthotropic, meaning they aren’t the
of expertise and complicated product behavior same in all directions. Correctly set-
with formal training and education programs ting up the simulation to accurately
and easy-to-use software. The paramount cul- reflect the reality of the actual mate-
tural challenge for manufacturers is grasping rial properties is difficult.”
the reality that they must spend more time per-
forming simulations upfront in order to save Dave Pfaffenberger, CTS Corporation
time in the testing phase. Outside of that, the
cultural challenges to introducing simulation
during design are minimal.

All print and electronic rights are the property of Aberdeen Group © 2006.
4 • AberdeenGroup
 The Simulation Driven Design Benchmark Report

Chapter Two:
Key Business Value Findings

• Best in class manufacturers their hit revenue, cost, launch date, and quality targets
Key Takeaways

for 86% or more of their products.

• Best in class manufacturers average 1.6 fewer prototypes than other companies.
• Best in class manufacturers of the most complex products get to market 158 days
earlier with $1,900,000 fewer product development costs.
• Best in class manufacturers of the simplest products get to market 21 days earlier
with $21,000 fewer product development costs.

W hile some manufacturers are adopting simulation early in the product devel-
opment cycle, Aberdeen research shows that they face serious challenges.
While some are taking steps in response, their strategies and tactics are only as
good as the results they deliver. To get a clear picture of which strategies and tactics are
successful, Aberdeen categorized survey respondents by measuring five key performance
indicators (KPIs) that provide financial, process, and quality measures (Figure 3). This
classification subsequently enabled differentiation between the “best practices” of the top
performers and the practices of lower-performing companies.

Figure 3: Best in Class Hit Targets on an 86% Average or Better

100% 87% 87% 89% 91%

86%
77%
80% 66% 64%
60% 63%
58%
60% 46% 45% 48%
45%
40%
20%
0%
Product Product cost Development Product Product
revenue targets cost targets launch dates quality
targets expectations

Best in class Average Laggard

Source: AberdeenGroup, October 2006

Based on aggregate scores incorporating all five metrics, those companies in the top 20%
achieved “best in class” status; those in the middle 50% were “average”; and those in the
bottom 30% were “laggard.” As expected, companies in the different performance cate-

All print and electronic rights are the property of Aberdeen Group © 2006.
Aberdeen Group • 5
The Simulation Driven Design Benchmark Report

gories show substantial differences – with best in class hitting all five marks at an 86% or
better average.

Varying Prototype Costs and Time across Product Complexity

One of the primary reasons manufacturers
pursue simulation early in the product Power Tool Manufacturer
development lifecycle is to test product “Most recently, not only were we able to
performance virtually. Products that are reduce prototypes, but we were able to skip
virtually tested have a higher chance of a prototype qualification phase altogether.
passing physical prototype testing the first Typically, we would develop machined
time. Overall, this translates to less time prototypes, but in this case we went right to
and lower development costs in the prod- a prototype casting.”
uct development lifecycle.
Translating reduced prototypes into hard
costs and time depends on the complexity Ricon Corporation
of the product. To get a clear picture of “Our products move through a range of
how prototype costs and time varied ac- motion. We could always visualize the
cording to product complexity, Aberdeen movements in our heads, but we started
categorized survey respondents’ products using simulation as a means to virtually
by measuring three key indicators: num- verify the motion prior to building proto-
ber of parts, length of development lifecy- types.”
cle, and number of engineering disciplines Ray Reynolds, Ricon Corporation
incorporated. This measurement subse-
quently enabled differentiation of levels of product complexity. The following table de-
scribes the general characteristics of each of the product complexity categories from this
study’s research (Table 2).

Table 2: General Characteristics of Product Complexity Categories

Product Complexity Number of Parts Length of Development
Low Less than 50 Between a week and a year
Moderate Between 50 and 1,000 Between a month and 5 years
High Between 50 and 10,000 Between 1 and 5 years
Between 1,000 and 100,000 Between 1 and 20 years
Very High

Source: AberdeenGroup, October 2006

Based on these product complexity categories, one can see a logical progression in the
corresponding increase in time and costs as complexity increases (Table 3).

All print and electronic rights are the property of Aberdeen Group © 2006.
6 • AberdeenGroup
 The Simulation Driven Design Benchmark Report

Table 3: Prototype Costs and Time per Product Complexity

Product Complexity Time to Build Prototype Cost to Build Prototype
Low 13 days $7,600
Moderate 24 days $58,000
High 46 days $130,000
Very High 99 days $1,200,000
Source: AberdeenGroup, October 2006

Avoiding Physical Prototypes with Virtual Prototypes

Is there any truth to the suggestion that using
simulation during design eliminates unneces- Plastics One
sary additional rounds of prototyping? The “Because we make over-molded elec-
answer is “yes.” Aberdeen research finds that tronics products, we’ve explored a few
the best in class average 3.0 prototypes com- different types of simulations for differ-
pared to 4.6 for all other manufacturers. ent purposes. For one, we’ve used
Applying this difference of 1.6 prototypes to mold-filling simulations to determine
the different categories of product complexity the base placements for injection-
yields compelling results. The best in class molding gates. Also, because over-
manufacturers of the products with very high molding can be rough on electronics,
complexity get to market 158 days earlier we explore various scenarios of failed
with $1,900,000 lower product development electronics components so that when we
costs than average performers. At the oppo- do experience problems, we know
site end of product complexity spectrum, the which scenario actually occurred.”
best in class manufacturers get to market 21 Steve Heckman, Plastics One
days earlier and spend $12,000 less on prod-
uct development costs than average performers.
Obviously there are very real benefits in early simulation that translate into a direct im-
pact on time to market and product development costs.

All print and electronic rights are the property of Aberdeen Group © 2006.
Aberdeen Group • 7
The Simulation Driven Design Benchmark Report

Chapter Three:
Implications & Simulation

• All best in class manufacturers use simulation in the design phase compared to three
Key Takeaways

out of four laggards.

• Best in class manufacturers are 63% more likely than all others to provide CAD-
embedded simulation to their engineers.
• Best in class manufacturers are 48% more likely than all others to provide technolo-
gies to transfer models from CAD to independent preprocessors to their analysts.
• Best in class performers are 42% more likely than all others to provide specific prod-
uct simulation examples to users for training.

A
s noted earlier, the aggregated performance of surveyed companies determined
whether they ranked as best in class, industry average, or laggard. In addition to
having common performance levels, each class also shares characteristics and
practices in four key categories – processes, organizational structure, technology
usage, and performance measurement.

All Best in Class Performers Utilize Simulation in the Design Phase

While many manufacturers have focused on adopting simulation earlier in the product
development process, best in class performers employ simulation throughout the product
development lifecycle (Figure 6).

Figure 4: Best in Class Perform More Simulations Earlier

100%
100% 90% 88%
78% 74% 72%
80% 69%
57%
60% 53%
40%
20%
0%
Design phase Test phase Post design release

Best in class Average Laggard

Source: AberdeenGroup, October 2006

In fact, every single one of the best in class performers surveyed for this report uses
simulation in the design phase as opposed to roughly three out of every four laggards.
And not only is this difference in the design phase, but the 20% difference continues in
the test and post-design release phases.

All print and electronic rights are the property of Aberdeen Group © 2006.
8 • AberdeenGroup
 The Simulation Driven Design Benchmark Report

Finally, one can see that the best Smith Aerospace

in class performers place the
highest emphasis on performing “We perform analyses to make sure we are arriving
simulation upfront in the design at the most weight and cost effective design. In
phase as opposed to during the short, we want to get the biggest bang for our buck.
other phases of development. Additionally, an analysis identifies all the potential
Overall, this early use enables failure modes whereas building and breaking a pro-
those leading manufacturers to totype will reveal just one… the one that broke the
reduce the number of prototypes
prototype.”
necessary to pass quality tests as
well as to avoid unnecessary Jochen Hessemann, Smith Aerospace
change orders after design re-
lease.

Familiar Environments: Engineers Access Simulation through CAD

Given that many manufacturers are performing simulations early in the design phase of
product development, the next question is “how is this being accomplished?” Tradition-
ally, the responsibility for completing simulations has commonly fallen on the analysis
group dedicated to the task. However, the current, ongoing trend is to push the simpler
and directional analyses upstream, into the hands of product engineers, as way of aug-
menting the efforts of the dedicated analyst groups.
How can companies get engineers to take on this additional task? Common sense says to
make it simple and easy to do. This starts with how the engineers access simulation capa-
bilities (Figure 8).

Figure 5: Best in Class Provide Engineers CAD-embedded Simulation

40% 36% 36%

32%
35% 30% 27%
30%
25% 22%
20% 16%
15%
10%
5% 0%
0%
Embedded within Transfer from Only in Mixes of all
CAD application CAD to pre- independent pre- options
processor processor
application

Best in class All others

Source: AberdeenGroup, October 2006

While a number of options for accessing simulations are available to engineers, only few
are typically used. These include simulations embedded within the CAD application and

All print and electronic rights are the property of Aberdeen Group © 2006.
Aberdeen Group • 9
The Simulation Driven Design Benchmark Report

transferred from CAD to preprocessor applications. The first option, embedding simula-
tion capabilities within a CAD application, keeps the engineer in a familiar environment
and removes the additional step of transferring geometry over to another application.
Sometimes this second option is necessary for an advanced setup such as finite element
mesh adjustment and additional geometric idealizations. Development of simulation
models only within independent preprocessors requires users to duplicate the design ge-
ometry.
Overall the best in class are ARA Engineering
63% more likely than all other
manufacturers to access simu- “CAD embedded simulation can be very fast and accu-
lation capabilities directly rate for the fundamental assessments of your product.
within CAD applications (35% We use these in the middle of the design cycle to di-
versus 22%). Conversely, the rectionally confirm our design decisions as we pro-
best in class never utilize inde- ceed.”
pendent preprocessors. Some Roxanne Abul-Haj, ARA Engineering
manufacturers do transfer from
the CAD application to an independent preprocessor, but there is no significant differ-
ence between best in class and all others in this practice. All in all, the conclusion is
clear. The best in class provide access to simulation capabilities for their engineers di-
rectly through CAD applications and through independent preprocessors only when nec-
essary.

Design Reuse: Analysts Transfer from CAD to Preprocessors

A similar question – how are simulations accessed by analysts – yielded dramatically
different results (Figure 9).

Figure 6: Best in Class Provide Experts Independent Simulation Tools

60% 54%
46% 46%
50%
40% 31%
30%
20% 13% 10%
10% 0% 0%
0%
Embedded within Transfer from Only in Mixes of all
CAD application CAD to pre- independent pre- options
processor processor
application

Best in class All others

Source: AberdeenGroup, October 2006

All print and electronic rights are the property of Aberdeen Group © 2006.
10 • AberdeenGroup
 The Simulation Driven Design Benchmark Report

As shown, none of the best in class Smith Aerospace

manufacturers have their analysts’
access simulation capabilities in “We export geometry from the CAD tool and
CAD applications. Given that many import it into a standalone package. Besides sav-
simulation vendors purposefully do ing time, when we transfer geometry we know
not expose all simulation capabili- we are analyzing the exact design the engineer
ties, especially the advanced ones, developed. Re-creating it can introduce mistakes
through CAD applications, analysts in the geometry and might not accurately repre-
find that they can’t even setup ad- sent the design you’d want to analyze.”
vanced analyses. However, those Jochen Hessemann, Smith Aerospace
analysts do want to take advantage
of the modeling work that has already taken place. As a result, analysts at many manufac-
turers perform a transfer from CAD to a preprocessor application in an effort to reduce
the overall simulation cycle. Interestingly enough, because analysts want to take advan-
tage of the already completed design work, they never develop the entire model only in
an independent preprocessor.
On the whole, at best in class manu-
facturers compared to other compa- GHSP
nies, analysts are 48% more likely “We use standalone analysis tools because the
to transfer from the CAD applica- capabilities are more encompassing. We do take
tion to an independent pre-processor the geometry from the CAD tool into the stand-
instead of accessing simulation ca- alone tool so we don’t have to create the geome-
pabilities through a CAD applica- try there.”
tion or building the simulation Mike Hoyt, GHSP
model independent of the CAD ap-
plication.

Tracking Configurations with the Simulation Model or Data Manage-

ment
During the setup of a simulation, many simplifications and idealizations are made to de-
signs in an effort to minimize mesh times and computation solve times. While this is a
common occurrence, many users don’t make the effort to accurately capture the derived
configuration of the model that was analyzed. This can cause issues if the product later
fails and a root cause investigation is required as part of a change process because the
details of the simulation configuration were not documented. Overall, there are a number
of options manufacturers can use to track these simulation configurations (Figure 7).

All print and electronic rights are the property of Aberdeen Group © 2006.
Aberdeen Group • 11
The Simulation Driven Design Benchmark Report

Figure 7: Best in Class Track Simulation Configurations

50% 46%
39%
40%

30%
18% 20%18% 20%
20% 14%
12%
8% 6%
10%

0%
Not tracked Spreadsheet CAD model Simulation Data
model management

Best in class All others

Source: AberdeenGroup, October 2006

In fact, most of the best in class performers use the simulation model as a means to track
the configuration of the product that was actually analyzed. In addition, they are more
than twice as likely to use data management to track the simulation configuration through
a centralized data management tool.
Overall, the conclusion is that the best in class track simulation configurations at least in
some way, most frequently by using the simulation model or by using data management
tools (and using them twice as frequently as other manufacturers).

Educating the User: Formalized Training Programs

Based on the fact that manufacturers are
beginning to assign the simpler analyses to Liebherr Mining
their engineers, it’s easy to see why train- “As far as training goes, we use all the
ing, education, and easy-to-use software available means to get users up to speed.
are the highest ranked responses to the We utilize outside training in addition to
challenges of adopting simulation in the inside training. We have many users that
design phase of product development. A are experienced and can pretty much coach
closer look reveals that the best in class them on the job.”
train and educate their users by variety of Vladimir Pokras, Liebherr Mining
means (Figure 8).

All print and electronic rights are the property of Aberdeen Group © 2006.
12 • AberdeenGroup
 The Simulation Driven Design Benchmark Report

Figure 8: Best in Class Performers Provide Formalized Training Programs

100%
74% 78%
80% 72%
66% 65%
60% 47% 52%
40%
40%
20%
0%
Tutorials Generic Specific Training
examples examples materials
Best in class All others

Source: AberdeenGroup, October 2006

Best in class manufacturers are

more likely to provide three types of KONE Elevators and Escalators
training list here more than other “We actually use the Web help fairly extensively.
companies and two of them mark- We email questions to our software vendor sup-
edly more than all other companies. port group, and they post answers on their Web-
Specifically, a large percentage of site.”
best in class manufacturers use tuto- Andy Jahn, KONE Elevator and Escalator
rials (66%), specific examples
(74%), and training materials (78%). Tutorials and training materials help these compa-
nies provide both FEA concepts education and software training. Specific examples help
address concerns about evaluating complicated product behavior by showing users how
to apply the software to the manufacturer’s specific products. Interestingly, best in class
performers are 42% more likely to provide specific examples to their users (74% vs.
52%) than other companies.
Overall the conclusion is clear. The best in class performers are much more likely to pro-
vide training materials to their users to address their lack of expertise and complicated
product behavior – the primary challenges they cited to performing simulation earlier in
the product development cycle.

Keeping an Eye on Requirements and Change Orders

Overall, the ultimate goal is to
manufacture a product that wins in Large Aerospace and Defense Contractor
the market or satisfies the cus- “We get requirements from the customer regard-
tomer’s needs. How do manufactur- ing fatigue life, deflection, weight, natural fre-
ers ensure this is in fact the case? A quencies, and other areas. The goal is to get as
wide variety of optional tactics are close to all of them as possible.”
available (Figure 9).

All print and electronic rights are the property of Aberdeen Group © 2006.
Aberdeen Group • 13
The Simulation Driven Design Benchmark Report

Figure 9: Manufacturers Track Compliance to Requirements and Change Orders

Measures that are tracked prior to Measures that are tracked after product
product shipment or delivery to customer 96% shipment or delivery to customer
100%

80% 71%
68% 62%
60%
43%
36%
40% 28% 28%
20% 14% 14% 11%14%
8%
0%
0%
Requirements Regulatory Part Number of Customer Number of Warranty
compliance compliance qualification change orders satisfaction prototypes costs
levels measures
Best in class All others

Source: AberdeenGroup, October 2006

Prior to design release, the majority of manufacturers primarily track requirements com-
pliance. Tracking compliance to product performance requirements acts as a key enabling
mechanism for early corrective action to work-in-process design. Additionally, a large
number of manufacturers track regulatory compliance. With the number of regulations
increasing worldwide, this comes as no surprise. Interestingly enough, tracking part
qualification levels, a common practice in many industries, is markedly lower for best in
class performers compared to laggards. Overall, these measures are commonly verified
through simulation prior to the development of a product prototype.
After design release, many manufacturers track the number of change orders logged
against a specific product. Companies use this measure to conduct a “post-mortem”
judgment on the practices used during that product’s development. Specifically, best in
class manufacturers are 55% more likely than other companies (i.e., 96% versus 62%) to
track the number of change orders against a product. Outside of this measure, other post-
design-release measures are infrequently used. Because simulation should be able to pre-
dict product failures that are the root cause of many change orders, manufacturers use this
“post-mortem” judgment to identify the simulation approaches used associated with that
product as good or bad.
The conclusion is clear. Best in class performer most commonly measure product per-
formance by evaluating requirements and regulatory compliance prior to design release
and by tracking the number of change orders after design release.

All print and electronic rights are the property of Aberdeen Group © 2006.
14 • AberdeenGroup
 The Simulation Driven Design Benchmark Report

Chapter Four:
Recommendations for Action

• Perform more simulation of product performance in the design phase.

Key Takeaways

• Provide CAD-embedded simulation capabilities to engineers.

• Use training materials and specific examples to get users up to speed.
• Employ technologies that transfer geometry from CAD to independent preprocessors
for analysts.
• Track requirements and regulatory product compliance prior to design release.

R egardless of the fact that performing more analyses upfront in the design phase
takes more time, manufacturers are engaging in this practice to save time and
money in creating physical prototypes, so they can hit shrinking time-to-market
windows. The following actions can help them address the challenges of early simulation
as well as enable them to improve their performance levels from “laggard” to “industry
average,” or from “industry average” to “best in class,” or even from “best in class” to
number one in their market.

Laggard Steps to Success

1. Perform more analyses in the design phase.
Use simulation capabilities to virtually prototype products in the design phase to
save prototype costs and time in the testing phase.
2. Don’t force engineers to use independent preprocessors. Adopt CAD-embedded
simulation capabilities instead.
Requiring engineers to perform new tasks in completely such new and unfamiliar
applications as independent preprocessors increases the barriers to success. In-
stead, provide simulation capabilities through CAD tools, an environment with
which they are more familiar.
3. Employ training materials and specific examples of company products to bring
users up to speed.
Training materials will enable users to understand FEA concepts as well to use
new simulation software applications. Examples that are specific to your business
and products will help users better understand how apply the technology in their
everyday work.

Industry Average Steps to Success

1. Provide analysts with the ability to transfer geometry from CAD to independent
preprocessors instead of CAD embedded simulation.

All print and electronic rights are the property of Aberdeen Group © 2006.
Aberdeen Group • 15
The Simulation Driven Design Benchmark Report

Simulation capabilities embedded within CAD applications are consciously lim-

ited in an effort to make things simpler for engineering users. Analysts need ac-
cess to the advanced capabilities in independent preprocessors. However, trans-
ferring geometry from the CAD application to these independent preprocessors
allow them to take advantage of and reuse designs, shortening the time required
to perform analyses.
2. Measure the number of change orders as lagging indicator of best practices.
Manufacturers should track the number of change orders related to failed product
performance as a post-design-release measure of the successful use of simulation
in the design phase. It will also facilitate efforts to continuously improve the use
of simulation.
3. Track the simulation configuration through simulation models or data manage-
ment.
In order to understand what was simulated after the product has been released
and launched, formally track the product configurations along with the idealiza-
tions and simplifications required by simulations through a simulation model or
data management tool.

Best in Class Next Steps

1. Track requirements and regulatory compliance more heavily prior to design re-
lease.
While tracking change orders is a good lagging indicator of the successful appli-
cation of simulation to a product’s performance, measuring leading indicators
such as requirements and regulatory compliance will result in minimizing proto-
type costs and time.
2. Reinforce use of CAD-embedded simulation capabilities — instead of independ-
ent preprocessors — for engineers.
While using an independent pre-processor provides access to advanced simula-
tion capabilities, manufacturers should reinforce the use of the simulation capa-
bilities embedded in CAD applications. This way, engineers can avoid learning a
new application that they will infrequently use. Overall, this practice will lower
the barrier to performing simulation earlier in the design phase.

All print and electronic rights are the property of Aberdeen Group © 2006.
16 • AberdeenGroup
 The Simulation Driven Design Benchmark Report

Appendix A:
Research Methodology

D uring August 2006, Aberdeen Group and Desktop Engineering and NAFEMS
examined the experiences and intentions of more than 270 enterprises regarding
their mechanical design simulation methodologies. Responding engineering ex-
ecutives completed an online survey that included questions designed to deter-
mine the following:
• The degree to which mechanical design simulation impacts corporate strategies, op-
erations, and financial results
• The structure and effectiveness of existing mechanical design simulation
• The benefits, if any, that have been derived from mechanical design simulation initia-
tives
Aberdeen supplemented this online survey effort with telephone interviews with select
survey respondents, gathering additional information on mechanical design strategies,
experiences, and results.
The study aimed to identify emerging best practices for mechanical engineering and de-
sign and provide a framework by which readers could assess their own mechanical design
capabilities.
Responding enterprises included the following:
• Job title/function: The research sample included respondents with the following job
titles: engineering and design staff (34%), engineering and design managers (25%),
internal consultants (11%), engineering and design directors (7%), and senior man-
agement (CEP, COO, CFO) (6%).
• Industry: The research sample included respondents predominantly from manufac-
turing industries. Automotive manufacturers represented 22% of the sample. Manu-
facturers in aerospace and defense accounted for 21% of respondents. Industrial
equipment manufacturers followed at 16%. Other sectors responding included com-
puter equipment and peripherals, high technology, telecommunication manufacturers,
services, and logistics.
• Geography: North American respondents accounted for 46% of respondents fol-
lowed closely by EMEA respondents at 43%. Remaining respondents from Asia-
Pacific region and South America represented the remaining 11% of the respondent
pool.
• Company size: About 33% of respondents were from small businesses (annual reve-
nues of $50 million or less), 40% were from midsize enterprises (annual revenues be-
tween $50 million and $1 billion), and 28% of respondents were from large enter-
prises (annual revenues above US$1 billion).
Solution providers recognized as sponsors of this report were solicited after the fact and
had no substantive influence on the direction of the Simulation-driven Design Benchmark

All print and electronic rights are the property of Aberdeen Group © 2006.
Aberdeen Group • 17
The Simulation Driven Design Benchmark Report

Report. Their sponsorship has made it possible for Aberdeen Group, Desktop Engineer-
ing, and NAMFEMS to make these findings available to readers at no charge.

Table 4: PACE Framework

PACE Key

Aberdeen applies a methodology to benchmark research that evaluates the business pressures, actions,
capabilities, and enablers (PACE) that indicate corporate behavior in specific business processes. These
terms are defined as follows:
Pressures — external forces that impact an organization’s market position, competitiveness, or business
operations (e.g., economic, political and regulatory, technology, changing customer preferences, competi-
tive)
Actions — the strategic approaches that an organization takes in response to industry pressures (e.g., align
the corporate business model to leverage industry opportunities, such as product/service strategy, target
markets, financial strategy, go-to-market, and sales strategy)
Capabilities — the business process competencies required to execute corporate strategy (e.g., skilled
people, brand, market positioning, viable products/services, ecosystem partners, financing)
Enablers — the key functionality of technology solutions required to support the organization’s enabling
business practices (e.g., development platform, applications, network connectivity, user interface, training
and support, partner interfaces, data cleansing, and management)

Source: Aberdeen Group, Month 2006

All print and electronic rights are the property of Aberdeen Group © 2006.
18 • AberdeenGroup
 The Simulation Driven Design Benchmark Report

Table 5: Relationship between PACE and Competitive Framework

PACE and Competitive Framework How They Interact

Aberdeen research indicates that companies that identify the most impactful pressures and take the most
transformational and effective actions are most likely to achieve superior performance. The level of com-
petitive performance that a company achieves is strongly determined by the PACE choices that they make
and how well they execute.

Source: Aberdeen Group, Month 2006

Table 6: Competitive Framework

Competitive Framework Key

The Aberdeen Competitive Framework defines enterprises as falling into one of the three following levels of
FIELD SERVICES practices and performance:
Laggards (30%) — FIELD SERVICES practices that are significantly behind the average of the industry,
and result in below average performance
Industry norm (50%) — FIELD SERVICES practices that represent the average or norm, and result in aver-
age industry performance.
Best in class (20%) — FIELD SERVICES practices that are the best currently being employed and signifi-
cantly superior to the industry norm, and result in the top industry performance.

Source: Aberdeen Group, Month 2006

All print and electronic rights are the property of Aberdeen Group © 2006.
Aberdeen Group • 19
 The Simulation Driven Design Benchmark Report

Appendix B:
Related Aberdeen Research & Tools

Related Aberdeen research that forms a companion or reference to this report includes:
• The Product Innovation Agenda Benchmark Report (September 2005)
• The Product Lifecycle Management for Small to Medium-Size Manufacturers
Benchmark Report (March 2006)
Information on these and any other Aberdeen publications can be found at
www.Aberdeen.com.

Aberdeen Group, Inc. Founded in 1988, Aberdeen Group is the technology-

260 Franklin Street driven research destination of choice for the global
Boston, Massachusetts business executive. Aberdeen Group has over 100,000
02110-3112 research members in over 36 countries around the world
USA that both participate in and direct the most comprehen-
sive technology-driven value chain research in the
Telephone: 617 723 7890 market. Through its continued fact-based research,
Fax: 617 723 7897 benchmarking, and actionable Simulation, Aberdeen
www.aberdeen.com Group offers global business and technology executives
a unique mix of actionable research, KPIs, tools,
© 2006 Aberdeen Group, Inc. and services.
All rights reserved
Month 2006
The information contained in this publication has been obtained from sources Aberdeen believes to be reliable, but
is not guaranteed by Aberdeen. Aberdeen publications reflect the analyst’s judgment at the time and are subject to
change without notice.
The trademarks and registered trademarks of the corporations mentioned in this publication are the property of their
respective holders.