MODULE 3

LESSON 1. CONCEPT DEVELOPMENT AND TESTING

Learning Objectives: After reading this module, you should be able to:
1. discuss the difference between idea generation and concept testing
2. understand how to undertake Concept testing

1.1 DIFFERENCE BETWEEN CONCEPT TESTING AND IDEA GENERATION

During the idea generation stage a company may have several ideas, however they will
go through all the ideas and arrive at only a few that will be taken up in the next stage.
In this stage the development team tries and get responses of the various product
ideas/ concepts from some potential customers in the target market. This testing
allows the product development team to understand if the concept meets the
customers’ expectations, whether it can be improved and to get an idea of the potential
sales of the product. In this phase product concept can be discussed with the customer
at several points in the development for example in the initial stage the product may
only be described to a customer verbally to get a feedback on whether the product
actually meets the customer‘s expectations as is being thought by the company
because this is the basis on which the company is developing the product. The product
can also be shown to the customer once it has been partially developed in order to get
a clearer response from the customer and also an indication of the potential sales that
the company can get from the product.

Concept testing closely follows the process of generating ideas. Idea generation and
concept testing both are similar in nature because both aim to reduce the number of
concepts under consideration. However, concept testing is different from Idea
generation because idea generation is more judgmental while concept testing is based
on getting a direct feedback from potential customers and so is not so judgmental. Now
we cannot go to the customer will a large number of ideas as it will confuse him and
not allow a clear decision to be taken. Hence at the idea generation stage itself the
company must screen ideas and shortlist only a few that can be taken for concept
testing to the customer.

In concept development and testing we have not only to discuss the concept with the
customer it may also involve showing him the prototype of the product or other
methods of making him understand the product, its utility and thus getting the
relevant feedback from him.

Based on the feedback we receive from the customer the company should be able to
evaluate the customers’ acceptance of the product, its potential sale and hence the fit
into the business objectives of the company.

PRODUCT MANAGEMENT Page 1

This concept testing may or may not be carried out. The decision to carry it out or not
depends on:

 The type of product and

 Time available for development

 Cost of development versus the cost of launch

 Cost of failure vs cost of development

For example, if we launch a website and there is an error it can be corrected at a later
point of time also but if we launch medical equipment on which life depends and it
malfunctions the consequences can be very serious.

1.2 CONCEPT DEVELOPMENT AND TESTING PROCESS

Concept testing can be carried out by following the following steps:

1. DEFINE THE PURPOSE OF THE CONCEPT TEST

The first step in concept testing is to define what and why we want to test about
the concept. The product development team must know what the areas they want
to test are and for what areas they need answers. It is only then that the test can
be structured in a proper manner. The concept test is an experimental process
and like for any experiment its objective must be defined before we undertake
the test. It is the definition of the test that will tell us how to measure the
responses of the customer. For example, if the company is developing a new
washing machine it may need to know how much piped water is available in
houses in that country or region. If we do not define this parameter in the test
we may just discuss the concept without determining the amount of water
available. In other words, we will not measure the water availability in the test.
Generally, a product concept test will need an answer to the following primary
questions:

a. Does the product meet the customers need?

b. How can the product be improved?

c. Can the product be enhanced to meet additional needs?

d. How frequently will the customer need the product?

e. How much sales will the product generate?

f. Should the development be continued or not?

PRODUCT MANAGEMENT Page 2

 All questions in the survey design will have a bearing on the product and along
with it the business strategy of the company. For example if the product meets
the customer‘s needs but is not likely to generate sales then the purpose of
development is lost.

2. CHOOSE THE TARGET CUSTOMER TO SURVEY

Once we have decided what we want to ask the customers we have to choose the
customers from whom we will ask the information. Choosing the target customer
sample who will be surveyed for testing the concept is an extremely important
step. This is because we are going to base the entire product development
concept on the feedback received from this customer sample. If we have chosen,
the customer sample wrongly this customer sample gives the wrong feedback
and it will be disastrous for the product development and the company. For
example, if the customer sample with whom we have discussed the concept likes
the product and approves it but these customers do not represent the target
market segment because of a wrong selection then the product will most likely
fail.

In order to overcome such a problem companies undertaking such a survey like

to include as many potential target segments in a survey as possible. However,
such a survey is very expensive and it is not possible for all companies to
undertake it. So many times companies try and include two or three major
segments in the survey. This allows the company to get a mix of opinions and so
arrive at a more balanced decision.

Another important factor while undertaking the survey is to decide the number
of customers to be surveyed. Naturally the more the customers who are surveyed
the greater is the possibility of a more unbiased or balanced decision. Here again
more customers to survey means more cost and so a trade-off needs to be arrived
at between number of customers to be surveyed and cost to be spent. The factors
which determine the size of the test consumers are:

 Factors for selecting a small sample size

o If the test is done during the early part of the developmental process

o If the test is only to gather some qualitative data about the product

o If it is expensive or time consuming to survey customers

o If the investment needed for developing the product is small

 Factors for selecting a large sample size

o If the test is being carried out at a later stage of development. In later

stages of the development we have moved ahead on the development
and now have certain definite questions to be answered to proceed to
the next stage of development. This needs a larger number to answer
them so as to get the information without bias.

PRODUCT MANAGEMENT Page 3

 o If we are trying to get an estimate of the demand of the product.

o If it is possible to survey customers quickly and without much cost.

o If the investment needed to develop and launch the product is high e.g.
an airplane, medical equipment, car, etc.

So the actual numbers of customers surveyed will vary drastically from a

small number to a very large number depending on the above mentioned
factors. Let us see what would happen if we develop a website and host it
quickly after conducting a brief survey of only a few customers. It is
possible that because all information on the needs of the customer was
not captured due to a small sample and also due to the speed of the
development some aspects of the website do not function properly. In such
a situation the company developing the website can continue to correct it
in stages since the information is not critical. However, if we were to do
this for say a medical equipment on which life depended and this machine
did not give all the needed functions because we did not ask enough
customers or were in a hurry to launch it the consequences would be
disastrous.

3. DECIDE THE METHOD OF GATHERING INFORMATION

Once the customer sample has been decided we now have the task of actually
going out and collecting the information. The product development team has the
responsibility of determining the best way to collect the information since they
are the ones who are setting up this experiment. In the initial stages of the
product development information is taken in a more unstructured manner this
means that the questions are such that can have qualitative answers and are
not so easily quantifiable. In the later stages of development questions can be
more structured. This means that in the initial stages the interaction with the
customer must be done by the development team themselves while a more
structured survey done at later stages can be given to a research agency for
collecting and collating the information.

There are many ways in which information can be collected:

 Face to face interaction – this type of surveys are the most common and
by far the most effective method. In this method the surveyor meets the
customer face to face and ask him the questions which have been decided
in the questionnaire. Just before beginning the detailed questionnaire the
customer is asked a few questions to confirm that he is actually from the
target segment for which the questionnaire is planned. So the customer
may be asked general questions about his age, income, education, whether
he is the user of a certain product category, etc. to validate him. If this is
the customer from our target segment, then the survey continues or else
it stops. The face to face interaction with the customer can be done by
meeting him in

o Retail outlets, malls, or other public areas where the customer may
be.

PRODUCT MANAGEMENT Page 4

 o Trade show booth – these are good places to talk to the customer
since here we have a relevant customer. In the trade show
companies show their prototypes or concept products. Here
customers are focused on the product and are likely to give more
relevant answers.

o Focus groups – These are a set of customers (between 5 to 10 at a

time) from the target segment who are invited by the company to
come to a fixed location where all of them are explained or shown
the product concept. During their session the concept is discussed
with them and their responses are recorded. Since this is a face to
face discussion the product development team is able to get a lot of
qualitative inputs from the customers. Inputs may be on does the
customer find the product useful, what needs it fulfils, what more
should be in the product, what kind of colors are suited for the
product, does the customer find it easy to use, etc.

Generally the price is not discussed because it tends to either

over value or undervalue the product in the consumer‘s eye even
before he begins to answer the questions. If price needs to be
determined, then the customer must be asked to specify what he
would pay for the product.

The focus groups have an advantage that they are better for
presenting several alternatives to the concept and getting a
separate feedback on each, or they can also be sued for getting
ideas in improving the product.

 Telephone – today technology is also being used for trying to reduce the
time needed in getting a feedback on a product concept. Telephones can
be sued where the product concept can easily be explained to consumers
without actually having to see a prototype, or drawing, etc.

In using telephones, the calls can be made to a o Predetermined set of

customers like young men, or young women, or college going students, etc.

o Or we can make cold calls – meaning that we call without pre-

categorization and once we get responses we set them off depending
on the response category.

Today mobile phones have allowed companies to target questions to

specific categories of customers by profiling their mobile numbers
based on their age, sex, profession, income category, etc. However,
the limitation is that because of its overuse customers are not
always willing to respond to such calls.

 Postal mail – this was an option that used to be followed in earlier days
when the prevalence of electronic media was not so pervasive. Earlier a
telephone was only land line based and so who would answer a phone was
not certain and so asking questions on phones was not simple. It was
therefore better to send a questionnaire to a customer and along with that
send him a prepaid return envelope and hope that the customer will fill

PRODUCT MANAGEMENT Page 5

 the questionnaire and send it back. In most cases it was seen that the
response was not more than 1-2% and it was slow. In order to increase the
responses companies sometimes gave incentives to customers to send in
their responses.

 Email – this is option is an improved version of the postal mail. It has

virtually the same issues as the postal mail system. Here people have the
option to put filters in their mail to block unwanted mail. However, this
remains a quick and cost effective option to collect feedback from
customers. It also has the advantage that data collected can be integrated
into the database without needing to actually feed the data into it.

 Internet – the internet can be used very effectively in getting a feedback

from the customer. Here the product development team can put not only
a questionnaire but can also show images of the product‘s concept or
simulated videos showing how the product will work and what benefits the
customer can get from it. Here the customers can be asked to comment on
the product or fill the feedback form that is on the website. Filling this
form automatically updates the information database.

In addition, today the internet blogs provide a virtual space where potential
customers can come and discuss products and provide suggestions and
solutions. In a way we can say that these are similar to the focus groups
that companies use for getting a feedback. 210 In cases like software
companies sometimes release a beta version on the net for people to
download and use before giving their comments on its features and
benefits.

Each method of gathering information has a bias towards the type of customer
it is suited for – internet/ email is more suited towards technology savvy
customers; the focus group discussions are good for products where the product
development team needs feedback on qualitative characteristics of the product.
However even though each process has some limitations, for some products it is
a good idea to use a test method with a limitation because for using the product
this functionality is needed e.g. a software based product needs ability in the
customer to use technology. However, it is not such a good idea if TV based
internet (IPTV) is to be tested.

Open ended interactive formats are better in the concept development stage since
it allows the customer to choose his reply himself.

As tests get more focused a more structured format can be used. For these type
of surveys, a market research company can be hired.

4. COMMUNICATE THE CONCEPT

So we have decided what to ask, to whom to ask and how to ask. The next step
is how to show him the concept. Please note that the concept must be
communicated to the customer in such manner that the ideas of the product
development team do not dominate his thoughts. The customer should have the

PRODUCT MANAGEMENT Page 6

freedom to use his own thoughts to give the feedback. Some of the ways in which
the concept can be shown to the customer are:

 Verbal description – here the customer is given a verbal description of the

product. This is usually done in the initial stages of the products
development and when the product is still in the conceptual stage of
development and no firm description is available.

 Sketch – The product concept can also be shown as a single or a series of

sketches. These can show the concept and how it can be used. This type
is again used in the initial stages of the product development.

 Photo and rendering – Sometimes product concepts can be shown as

images that have been developed on computer simulation. This allows the
customer to get a more realistic impression of the product. This type of
visualization allows the product development team to show the customer
different colors and also the detail of some functional or ergonomic
features.

 Storyboard – a storyboard is a concept that was first developed for making

cartoons by Walt Disney and later for films.

According to Christopher Finch in The Art of Walt Disney (Abrams, 1974),

Disney credited animator Webb Smith with creating the idea of drawing
scenes on separate sheets of paper and pinning them up on a bulletin
board to tell a story in sequence, thus creating the first storyboard.

So the product development team uses the storyboarding method to show

the concept to the customers who can by seeing this understand what the
product can do.

 Video – the advantage of a video is to show how the product works and
how it can be used. This method is used where it is difficult to take the
actual product to the customers. For example, if we want to show how a
machine works it is easier to take a video of the machine and its
functionality and show it to customers at different locations.

 Simulation - It is the imitation of some real thing or process. The act of

simulating something generally means showing certain key characteristics
or behaviors of a selected physical or abstract system.

In simulation a user may actually control the use of a product and get a
real feel of how the product will respond. This can give him a near real
feeling of the product. Simulation is used in safety engineering and tests,
for training, development, etc. For example, we have a flight simulator in
which pilots train how to fly aircrafts before that actually go on a plane.

During product development this method is used where the system can be
used to show the effect of using different use parameters e.g. in a chemical
process where the change in parameters can lead to different
consequences.

PRODUCT MANAGEMENT Page 7

 Simulation is also used when the real system cannot be engaged, because
it may not be accessible, or it may be dangerous or unacceptable to engage.
Interactive media

The product development team can also use a combination of Video plus
simulation to explain the concept better.

Developing simulation is expensive. This type of method is used in highly

technical products where the cost of developing the simulation is
significantly lower than the cost of developing the product.

 Physical appearance models – In this method the concept is shown as a

model of the product. At times this model also has some functions also
incorporated in them so that customers can get an idea of how the product
will work.

Generally, this type of method is used either when the product

development team has a clear idea of what they have to develop or it is in
the later stages of the development of the product by which time a large
extent of the work has been done.

 Working prototypes – This type of method is used when the product

development is almost complete and the product development needs the
final inputs for the customer. However, care must be taken because
customers tend to equate this product with the final product – so the
positives and negatives of the prototype rub off onto their judgement and
feedback. Secondly since the prototype is generally one of the few products
developed by the product development team and all care has been taken
and probably better material has been used to manufacture it this
prototype may perform better than the final product that comes off the
actual production.

The choice of the survey format must match the communication method used
because how we show the product concept to the customer depends on how we
are asking him to respond. So for example we cannot demonstrate a working
model while using a phone survey.

In addition to this we must take care that while showing or communicating the
product concept to the customer:

 Must not oversell the concept to the customer.

 Must give only that much information as much as the customer will get
during selling. This ensures that the responses from the customer will be
similar to the way he will respond when he gets the advertising stimulus.

 Price must not be given but customer must be asked for it. Giving a price
to the customer tends to undervalue or overvalue the product depending
on how the customer perceives the price. SO in order to overcome this
problem it is better to ask the customer what he is willing to pay for the

PRODUCT MANAGEMENT Page 8

 product.

5. CAPTURING CUSTOMER RESPONSE

The product development team must faithfully capture the customer response
in order to ensure that the development progresses in the right direction. Here
in order to capture the information is the right manner the tests are made with
formats that provide for:

 Questions with multiple choice answers.

 In order to ensure that the answers given by the customer are validated as
correct and consistent the questionnaire also has questions that ask asks
why he gave the answer.

 The questionnaire also has similar questions in different parts of the

questionnaire that ask similar information in a slightly different manner.
This again helps the development team to validate and understand the
correctness and consistency of the customers thinking.

 The questionnaire also captures the intention of the customer to buy the
product, the frequency of purchase, the certainty of purchase, etc. Here
we may capture answers like:

o Definitely buy

o Probably buy

o Might or might not

o Probably would not buy

o Definitely not buy

Depending on the method used in interacting with the customer the

product development team in addition to structured responses from the
customer interaction can also capture of a large amount of unstructured
data. This data is also important and needs to be recorded and analyzed.

6. INTERPRET RESULTS

Once the information has been captured it must be collated and structured. This
information must then be structured in a hierarchy of importance. This
hierarchy must be segregated by primary and secondary needs. From the
responses the redundant responses must be removed and further all information
should be categorized by customer need so that we can see what important needs
have to be worked upon first and what can be worked on later.

PRODUCT MANAGEMENT Page 9

Since customers are shown multiple options, results must be interpreted based
on the difference in responses and the concept preferred by the customers. They
must also factor in cost of manufacturing and other considerations (Like possible
sales, customer acceptance, profit, etc.).

If manufacturing costs are very different and price of the product has not been
communicated to the customer during the survey, then the company must use
its own judgement in proceeding ahead with one of the products selected by the
customer.

Some factors like sales of the product have to be calculated by using various
forecasting methods. However even this data needs to be compared with the sales
data that the company may have had for similar products or industry data that
exists in the market to check if this bears out the forecast.

In case the company has no history of the product and neither does the market
then care must be taken

In concept testing models we are not able to factor in:

 Effect of word of mouth – especially when product benefits are not obvious
word of mouth plays an important role in promoting the products

 Fidelity of concept description – if the final product differs substantially

from the one described in the test – forecast may differ

 Pricing – if price deviates significantly from that indicated in survey or from

customer expectations

 Level of premium – spending on advertising and promotion can add to the

cost of the product. This is weakly accounted in forecasting models by
factoring in awareness/ availability term and via materials used to present
the concept(s)

A Team must reflect on results and qualitative and quantitative results obtained.
In order to be sure that th answers received are true they must ask two key
questions

o Was the concept communicated in the way that the customer response is
likely to be true?

o Is the resulting forecast consistent with the observed sales rate of similar
products?

All results whether used or not must be documented so that they are available
for similar products in the future.

Another learning that the product development can get is that their results for
the tests can be compared with future results. This benefits the team because
testing in future can benefit from the experience in analyzing the difference
between forecast and actual.

PRODUCT MANAGEMENT Page 10

 The benefits from this survey is :

o Overall size of market – can be increased by considering alternative

markets

o Availability/ awareness of the product – can be enhanced through

distribution and promotion

o Fraction of consumers likely to purchase – can be enhanced by changes

in product design (and advertising) that improve the attractiveness of the
product

1.3 SUMMARY

Many people ask is there a difference between development of an idea and testing of
the concept. They say that ultimately in both we are taking inputs for the customer
and analyzing his responses and arriving at decisions on how to take the product
development forward. This is true however at the time of the generation of the idea the
concept is in a fuzzy stage and no clear cut decisions are available. In that stage we
are still trying to work through a large number of options to arrive at a few shortlisted
options. The process of concept development begins from here and takes forward the
development. During the concept development stage all interaction with the customer
takes place with a view of understanding on how to develop or improve on the few
options that have been shortlisted.

It is during this phase of development that the company actually begins to commit
financial resources for the development of the product. This phase of the development
is a more experimental phase in which interaction with the customer is planned to
determine the answers to a few predetermined questions. The need for such an
interaction with the customer arises when the product development team needs to
clarify the choices they have to make in the development of the product.

This kind of testing can be done at various stages in the development of the product –
meaning that the product development team can interact with the customer just after
the ideas have been finalized in order to understand the functionality needed in the
product or they can meet him towards the end of the development of the product so as
to get a more clear feedback on the acceptance of the product by the customer and the
amount of sales that can be expected so that the necessary changes can be
incorporated before launch.

There is a six step method that can be used to collect this information from the
customer

You may now proceed to the next lesson. 

PRODUCT MANAGEMENT Page 11

REFERENCES:

1. Evans, Joel R and Berman, Barry (2007) Marketing Management, New Delhi,
India, Cengage Learning, Page 356-363
2. Ulrich Karl T and Eppinger, Steven D, (2000) Product Design and Development,
New York, USA, Irwin McGraw – Hill page 107-159
3. Crawford, Merle and Benedetto, Anthoni Di, (2004) New Products Management,
New York, USA, Page 83-91, 118-130

PRODUCT MANAGEMENT Page 12