Sensory Discrimination Tests and Measurements: Sensometrics in Sensory Evaluation

Preface

The second edition of this book is similar to the first in that it focuses on sensory tests and measurements from a statistical perspective. However, it expands greatly upon the first in the following aspects:

The second edition extends the discussion of sensory measurement from Thurstonian discriminal distance f1-math-0001 (Chapter 2) to the area (R-index and Gini-index) under the ROC curve in Signal Detection Theory (Chapter 3) and to wider sensory measurements, including sensory threshold (Chapter 12), sensory risk (Chapter 13), time-intensity (Chapter 14), sensory shelf life (Chapter 15), the performance of a trained sensory panel and panelists (Chapter 16), consumer emotions and psychographics (Chapter 17), and the relative importance of attributes (Chapter 18).

The second edition extends the discussion of sensory discrimination tests from main difference tests (Chapter 4) to similarity/equivalence tests (Chapter 5) and Bayesian tests (Chapter 6). Chapters 7–11 discuss novel methods for modified discrimination tests, multiple-sample discrimination tests, and replicated discrimination tests.

More R/S-Plus built-in programs, packages, and codes are used in the second edition. Some of the tables for statistical tests used in the first edition are replaced by R/S-Plus codes. The R/S-Plus codes and some of the data files used in the book are listed in Tables A.1 and A.2 in Appendix A and are available from the companion Web site, www.wiley.com/go/bi/SensoryDiscrimination. The R packages (R Development Core Team 2013) used in the book are listed in Table A.3 in Appendix A and can be downloaded from www.r-project.org.

The title of the book has been changed to reflect the expanded and changed contents of the second edition. The title of the first edition was Sensory Discrimination Tests and Measurements: Statistical Principles, Procedures, and Tables, while is the title of the second edition is Sensory Discrimination Tests and Measurements: Sensometrics in Sensory Evaluation.

The book is organized as follows:

Chapter 1 briefly describes sensory discrimination methods.

Chapter 2 and 3 discuss sensory effect measurement using distance index, Thurstonian f1-math-0002 , and the area indices R-index and Gini-index.

Chapter 4–6 discuss sensory discrimination tests, including difference testing, similarity (equivalence) testing, and the Bayesian approach to discrimination testing.

Chapter 7 and 8 discuss modified and multiple-sample discrimination tests.

Chapter 9–11 discuss replicated discrimination tests based on the beta-binomial (BB) model, the corrected beta-binomial (CBB) model, and the Dirichlet–multinomial (DM) model, respectively.

Chapters 12–18 discuss diverse and specific sensory measurements in a broad sense, from measurements of sensory threshold (Chapter 12) to measurements of the relative importance of attributes (Chapter 18).

The assumed readers of the book are researchers and practitioners in the sensory and consumer field, as well as anyone who is interested in sensometrics. The book is intended to be a useful reference for modern sensory analysis and consumer research, especially for sensometrics. It is different in its objective from the textbooks widely used in the sensory field (e.g., Amerine et al. 1965, Stone and Sidel 2004, Meilgaard et al. 2006, Lawless and Heymann 2010) and from common guidebooks (e.g., Chambers and Wolf 1996, Kemp et al. 2009). It is also different in perspective and focus from the books on quantitative sensory analysis and applied statistics in sensory and consumer research (e.g., O'Mahony 1986b, Næs and Risvik 1996, Meullenet et al. 2007, Mazzocchi 2008, Gacula et al. 2009, Næs et al. 2010, Lawless 2013), although it has some topics in common with these.

References

Amerine, M. A., Pangborn, R. M., and Roessler, E. B. 1965. Principles of Sensory Evaluation of Food. Academic Press, New York.

Chambers, E. VI and Wolf, M. B. 1996. Sensory Testing Methods (2nd ed.). ASTM Manual Series MNL. ASTM International, West Conshohocken, PA.

Gacula, M., Singh, J., Bi, J., and Altan, S. 2009. Statistical Methods in Food and Consumer Research (2nd ed.). Elsevier/Academic Press, Amsterdam.

Kemp, S. E., Hollowood, T., and Hort, J. 2009. Sensory Evaluation. A Practical Handbook. John Wiley & Sons, Chichester.

Lawless, H. T. 2013. Quantitative Sensory Analysis: Psychophysics, Models and Intelligent Design. Wiley-Blackwell, Oxford.

Lawless, H. T. and Heymann, H. 2010. Sensory Evaluation of Food: Principles and Practice (2nd ed.). Springer, New York.

Mazzocchi, M. 2008. Sensory for Marketing and Consumer Research. Sage, Los Angeles, CA.

Meilgaard, M. C., Civille, G. V., and Carr B. T. 2006. Sensory Evaluation Technique (4th ed.). CRC Press, Boca Raton, FL.

Meullenet, J. F., Xiong, R., and Findlay, C. J. 2007. Multivariate and probabilistic Analysis of Sensory Science Problems. Blackwell, Ames, IA.

Næs, T. and Risvik, E. 1996. Multivariate Analysis of Data in Sensory Science. Elsevier, Amsterdam.

Næs, T., Brockhoff, P. B., and Tomic, O. 2010. Statistics for Sensory and Consumer Science. John Wiley & Sons, Chichester.

R Development Core Team. 2013. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna. Available from: http://www.R-project.org/ (last accessed April 14, 2015).

Stone, H. and Sidel, J. 2004. Sensory Evaluation Practices (3rd ed.). Academic Press, Amsterdam.

Acknowledgments

I am grateful to Professor Michael O'Mahony, Professor Hye-Seong Lee, Dr. Carla Kuesten, Dr. Herbert Meiselman, Julia Chung, Yaohua Feng, and Pooja Chopra, who are the co-authors of our papers published in recent years.

I would like to thank the Wiley editor David McDade and project editor Audrie Tan for their encouragement in the completion of this project.

I wish to dedicate this book to my wife, Yulin, and my daughter, Cindy.

Jian Bi

Richmond, Virginia

December 2014

About the companion website

This book is accompanied by a companion website:

www.wiley.com/go/bi/SensoryDiscrimination

The website includes:

R/S-Plus codes for downloading

Data files for downloading

The lists of R/S-Plus codes and data files available on the website are provided in Appendix A on page (insert page number) of this book.

Introduction

1.1 Sensometrics

This book is about sensometrics, focusing on sensory discrimination tests and measurements in the domain of sensory analysis. Sensometrics is a subfield of, or an area related to, sensory and consumer science. According to Brockhoff (2011), Sensometrics is the scientific area that applies mathematical and statistical methods to model data from sensory and consumer science. It is similar to psychometrics in psychology, biometrics in biology, chemometrics in chemistry, econometrics in economy, politimetrics in macropolitics, environmetrics in environmental sciences, and so on. Sensometrics has experienced rapid growth in both academia and industry within the last 2 or 3 decades. It plays an important role in modern sensory analysis and consumer research, especially in the coming Big Data era.

1.2 Sensory tests and measurements

The basic functions of sensory analysis are to provide reliable sensory measurements and to conduct valid tests. Statistical hypothesis testing is the theoretical basis of sensory tests. Statistical tests include both difference tests and similarity (equivalence) tests. The Thurstonian model (Thurstone 1927) and Signal Detection Theory (SDT) (Green and Swets 1966, Macmillan and Creelman 2005) are the theoretical basis for sensory effect measurement. Psychometric functions provide invariable indices that are independent of the methods used for measurements. Notably, the Thurstonian discriminal distance c01-math-001 (or c01-math-002 ) (ASTM 2012) and the area (R-index) under the receiver operating characteristic (ROC) curve in SDT have been widely accepted and are used in both food and sensory fields. Daniel M. Ennis (1993, 1998, 2003) and Michael O'Mahony (1979, 1992), among others, should be particularly thanked for their insight and foresight in introducing the methodologies into these fields and for tirelessly promoting their research and application over recent decades.

Sensory measurement takes on a broad range of meanings and contents. Besides sensory effect measurement using Thurstonian discriminal distance and area under ROC curve, the following measurements can also be regarded as different types of sensory measurement: sensory threshold measurement, sensory risk measurement, time intensity measurement, sensory shelf life measurement, trained sensory panel/panelist performance measurement, consumer emotions and psychographics measurement, and attribute relative importance measurement. Specific statistical methodologies are used for different types of sensory measurement.

1.3 A brief review of sensory analysis methodologies

Sensory analysis can be divided into two types: laboratory sensory analysis and consumer sensory analysis. In laboratory sensory analysis, a trained panel is used as an analytical instrument to measure the sensory properties of products. In consumer sensory analysis, a sample of a specified consumer population is used to test and predict consumer responses to products. These have different goals and functions, but share some methodologies.

Discriminative analysis and descriptive analysis are the main classes of methodology for both laboratory and consumer sensory analyses. Discriminative analysis includes discrimination tests and measurements. In this book, discrimination tests are used to determine whether a difference exists between treatments for confusable sensory properties of products (difference test), or whether a difference is smaller than a specified limit (similarity/equivalence test), usually using a two-point scale or a rating or ranking scale. Discrimination measurements are used to measure, on a particular index, the extent of the difference/similarity. There are two sources of sensory differences: intensity and preference. A discrimination test is used when testing difference/similarity of intensity; a preference test is used when testing difference/similarity of preference. Descriptive analysis is used to determine, on a rating scale, how much of a specific characteristic difference exists among products (quantitative descriptive analysis) or to characterize a product's sensory attributes (qualitative descriptive analysis). Quantitative descriptive analysis for preference is also called acceptance testing.

Acceptance or preference testing is of very limited value for a laboratory panel (Amerine et al. 1965) but is valuable in a consumer analysis setting. Laboratory discrimination testing, using a trained panel under controlled conditions, is referred to as Sensory Evaluation I, while consumer discrimination testing, using a sample of untrained consumers under ordinary consumption (eating) conditions, is referred to as Sensory Evaluation II (O'Mahony 1995). Confusion of the two will lead to misleading conclusions. Controversy over whether the consumer can be used for discrimination testing ignores the fact that laboratory and consumer discrimination tests have different goals and functions.

The distinction between discriminative analysis and quantitative descriptive analysis is not absolute from the viewpoint of modern sensory analysis. The Thurstonian model and SDT (see Chapters 2, 3) can be used for both discriminative analysis and quantitative descriptive analysis. The Thurstonian c01-math-003 (or c01-math-004 ), a measure of sensory difference/similarity, can be obtained from any kind of scale used in discriminative and descriptive analysis. A rating scale, typically used in descriptive analysis, is also used in some modified discrimination tests.

The following types of analysis are the important topics and methodologies of sensory analysis: sensory threshold analysis, sensory risk analysis, time intensity analysis, sensory shelf life analysis, trained sensory panel/panelist performance analysis, consumer emotions and psychographics analysis, and sensory attribute relative importance analysis.

This book is primarily concerned with methodology, mainly from a statistical point of view, of sensory discrimination tests and measurements, including laboratory and consumer sensory analyses.

1.4 Method, test, and measurement

In this book, a distinction is made among three terms: sensory discrimination method, sensory discrimination test, and sensory discrimination measurement.

In sensory discriminative analysis, certain procedures are used for experiments. These procedures are called discrimination methods (e.g., the Duo–Trio method, the Triangular method, the ratings method).

When discrimination procedures are used for statistical hypothesis testing, or when statistical testing is conducted for the data from a discrimination procedure, the procedure is called discrimination testing (e.g., the Duo–Trio test, the Triangular test, the ratings test). In this book, discrimination testing is referred to as both difference testing and similarity/equivalence testing for both preference and intensity (Chapters 4, 5). Bayesian statistical tests are also discussed, in Chapter 6. In Chapter 7, some modified discrimination tests are discussed. Multiple-sample discrimination tests are discussed in Chapter 8. Replicated discrimination tests are discussed in Chapters 9–11.

When discrimination procedures are used to measure, or, in other words, when an index (e.g., Thurstonian c01-math-005 (or c01-math-006 ) or R-index) is produced using the data from a discrimination procedure, the procedure is called a discrimination measurement (e.g., Duo–Trio measurement, Triangular measurement, ratings of the A–Not A measurement). Effect measurement includes distance measure c01-math-007 and area measure R-index (or Gini-index). Besides the effect measurement discussed in Chapters 2, 3, other sensory measurements are discussed in Chapters 12–18. Both sensory testing and measurement are of importance and are useful. However, generally speaking, sensory measurement is more important and more useful in practice. Sensory measurements provide indices of the magnitude of sensory effects.

1.5 Commonly used discrimination methods

1.5.1 Forced-choice methods

The Two-Alternative Forced Choice (2-AFC) method (Green and Swets 1966): This method is also called the paired comparison method (Dawson and Harris 1951, Peryam 1958). With this method, the panelist receives a pair of coded samples, A and B, for comparison on the basis of some specified sensory characteristic. The possible pairs are AB and BA. The panelist is asked to select the sample with the strongest (or weakest) sensory characteristic. The panelist is required to select one even if he or she cannot detect the difference.

The Three-Alternative Forced Choice (3-AFC) method (Green and Swets 1966): Three samples of two products, A and B, are presented to each panelist. Two of them are the same. The possible sets of samples are AAB, ABA, BAA or ABB, BAB, BBA. The panelist is asked to select the sample with the strongest or the weakest characteristic. The panelist has to select a sample even if he or she cannot identify the one with the strongest or the weakest sensory characteristic.

The Four-Alternative Forced Choice (4-AFC) method (Swets 1959): Four samples of two products, A and B, are presented to each panelist. Three of them are the same. The possible sets of samples are AAAB, AABA, ABAA, BAAA or BBBA, BBAB, BABB, ABBB. The panelist is asked to select the sample with the strongest or the weakest characteristic. The panelist is required to select a sample even if he or she cannot identify the one with the strongest or weakest sensory characteristic.

The Triangular (Triangle) method (Dawson and Harris 1951, Peryam 1958): Three samples of two products, A and B, are presented to each panelist. Two of them are the same. The possible sets of samples are AAB, ABA, BAA, ABB, BAB, and BBA. The panelist is asked to select the odd sample. The panelist is required to select one sample even if he or she cannot identify the odd one.

The Duo–Trio method (Dawson and Harris 1951, Peryam 1958): Three samples of two products, A and B, are presented to each panelist. Two of them are the same. The possible sets of samples are A: AB, A: BA, B: AB, and B: BA. The first one is labeled as the control. The panelist is asked which of the two test samples is the same as the control sample. The panelist is required to select one sample to match the control sample even if he or she cannot identify which is the same as the control.

The Unspecified Tetrad method (Lockhart 1951): Four stimuli, two of A and two of B, are used, where A and B are confusable and vary in the relative strengths of their sensory attributes. Panelists are told that there are two pairs of putatively identical stimuli and to sort them into their pairs.

The Specified Tetrad method (Wood 1949): Four stimuli, two of A and two of B, are used, where A and B are confusable and vary in the relative strengths of their sensory attributes. Panelists are told that there are two pairs of putatively identical stimuli and to indicate the two stimuli of specified A or B.

The Dual Pair (4IAX) method (Macmillan et al. 1977): Two pairs of samples are presented simultaneously to the panelist. One pair is composed of samples of the same stimuli, AA or BB, while the other is composed of samples of different stimuli, AB or BA. The panelist is told to select the most different pair of the two pairs.

The c01-math-008 method (Lockhart 1951): c01-math-009 samples with M sample A and N sample B are presented. The panelist is told to divide the samples into two groups, of A and B. There are two versions of the method: specified and unspecified. This is a generalization of many forced-choice discrimination methods, including the Multiple-Alternative Forced Choice (m-AFC), Triangle, and Specified and Unspecified Tetrad. The c01-math-010 with larger M and N can be regarded as a specific discrimination method with a new model. Unlike the conventional difference tests using the c01-math-011 with small M and N based on a binomial model, the c01-math-012 with larger M and c01-math-013 can reach a statistical significance in a single trial for only one c01-math-014 sample set based on a hypergeometric model. The methods that use a new model are particularly useful for assessing the discriminability of sensory panels and panelists; these are discussed in Chapter 16.5.

1.5.2 Methods with response bias

The A–Not A method (Peryam 1958): Panelists are familiarized with samples A and Not A. One sample, which is either A or Not A, is presented to each panelist. The panelist is asked if the sample is A or Not A.

The A–Not A with Remind (A–Not AR) method (Macmillan and Creelman 2005): Unlike the A–Not A, which is a single-sample presentation, a reminder (e.g., sample A) is provided before each test sample (sample A or Not A) in order to jog the panelist's memory.

The Same–Different method (see, e.g., Pfaffmann 1954, Amerine et al. 1965, Macmillan et al. 1977, Meilgaard et al. 1991, among others, for the same method under different names): A pair of samples, A and B, is presented to each panelist. The four possible sample pairs are AA, BB, AB, and BA. The panelist is asked if the two samples that he or she received are the same or different.

The ratings methods discussed in the book include ratings of the A–Not A, A–Not AR, and Same–Different methods.

1.6 Classification of sensory discrimination methods

Sensory discrimination methods are typically classified according to the number of samples presented for evaluation, as single-sample (stimulus), two-sample, three-sample, or multiple-sample methods. This classification is natural, but it does not reflect the inherent characteristics of the methods. In this book, the discrimination methods are classified according to the decision rules and cognitive strategies they involve. This kind of classification may be more reasonable and profound. In the following chapters, we will see how methods in the same class correspond to the same types of statistical model and decision rules.

1.6.1 Methods requiring and not requiring the nature of difference

There are two different types of instruction in the discrimination method. One type involves asking the panelists to indicate the nature of difference in the products under evaluation; for example, Which sample is sweeter? (the 2-AFC and the 3-AFC methods); or Is the sample A or Not A? (the A–Not A method). The other type compares the distance of difference; for example, Which of the two test samples is the same as the control sample? (the Duo–Trio method); Which of these three samples is the odd one out? (the Triangular method); or Are these two samples the ‘same’ or ‘different’? (the Same–Different method). The two types involve different cognitive strategies and result in different percentages of correct responses. Hence, the discrimination methods can be divided into these two types: methods using the skimming strategy and methods using the comparison of distance strategy (O'Mahony et al. 1994). The two types of methods can also called specified or unspecified method.

1.6.2 Methods with and without response bias

Response bias is a basic problem with sensory discrimination methods. Many authors hav eaddressed this problem (e.g. Torgerson 1958, Green and Swets 1966, Macmillan and Creelman 2005, O'Mahony 1989, 1992, 1995). Sensory discrimination methods are designed for the detection and measurement of confusable sensory differences. There is no response bias if the difference is large enough, but response bias may occur when the difference between two products is so small that a panelist makes an uncertain judgment. In this situation, how large a difference can be judged as a difference may play a role in the decision process. Criterion variation (strictness or laxness of a criterion) causes response bias. A response bias is a psychological tendency to favor one side of a criterion. Response bias is independent of sensitivity. This is why the methods with response bias (e.g., the A–Not A and the Same–Different methods) can also be used for difference testing. However, response bias affects test effectiveness (power).

Forced-choice procedures can be used to stabilize decision criteria. Hence, most sensory discrimination methods are designed as forced-choice procedures. A forced-choice procedure must have at least three characteristics: (1) Two sides of a criterion must be presented. The two sides may be strong and weak, if the criterion is about the nature of the difference between products. The two sides may be same and different, if the criterion is about the distance of the difference between products. Because a single sample or two samples of the same type cannot contain two sides of a criterion, evaluating a single sample or the same type of sample is not a forced-choice procedure. Because a single pair of samples or a pair of samples of the same type cannot contain two sides of a criterion concerning the distance of a difference, evaluating a single sample pair or a pair of samples of the same type is not a forced-choice procedure, either. (2) A panelist should be instructed that the samples presented for evaluation contain the two sides of a criterion. (3) A response must be given in terms of one clearly defined category. The don't know response is not allowed.

1.6.3 Methods using multiple sets and only one set of samples

In conventional discrimination tests using forced-choice methods, such as the c01-math-015 method with small M and N, we cannot get a statistical conclusion from a response for only one set of samples, because even for the perfect response for a set of the samples, the chance probability (e.g., 1/3 in the 3-AFC) is still larger than any acceptable significance level. Hence, multiple sets of c01-math-016 samples are needed. A binomial model is used for analysis of the proportion of correct responses. However, we can get a conclusion based on responses in a c01-math-017 table for only one set of c01-math-018 samples with larger M and N in a Fisher's exact test.

1.6.4 Methods with binary and ratings data

The responses in forced-choice methods are binary. The responses in the methods with response bias may be binary or ratings. The ratings of the methods represent degrees of sureness of a judgment or different decision criteria. For example, the responses in an A–Not A test are A/Not A (i.e., 1 or 2). The responses in a ratings of the A–Not A test may be a six-point scale with (1, 2, 3, 4, 5, 6) corresponding to (A, A?, A??, N??, N?, N).

Table 1.1 describes the classifications of sensory discrimination methods.

Table 1.1 Classifications of sensory discrimination methods

Chapter 2

Measurements of sensory difference/similarity: Thurstonian discriminal distance

2.1 Measurement of sensory difference/similarity

Discrimination testing can tell us whether there is a significant difference/similarity between products. However, it cannot tell us about the degree or extent of the difference/similarity. Measurement of sensory difference/similarity using a suitable index is highly desirable.

2.1.1 Proportion of correct response in forced-choice methods

The proportion of correct response in a discrimination test using a forced-choice method is an important test statistic. However, it is not a good index by which to measure sensory difference or discriminability, because it is not independent of the methods used. Obviously, for an identical pair of stimuli, the proportions of correct response using the 2-AFC and the 3-AFC methods will be different, because the two methods contain different guessing probabilities. Even for methods with the same guessing probability, such as the 2-AFC and the Duo–Trio methods or the 3-AFC and the Triangular methods, the same probability of correct responses using different methods reflects different sensory differences or discriminabilities, as revealed by the famous so-called paradox of discriminatory nondiscriminators (Gridgeman 1970). In this paradox, judges gave a higher proportion of correct responses to the 3-AFC than the Triangular test, for the same stimuli. Byer and Abrams (1953) first noted this from their experimental data, and many studies have confirmed it (e.g., Hopkins and Gridgeman 1955, Raffensberger and Pilgrim 1956, Frijters 1981a, MacRae and Geelhoed 1992, Stillman 1993, Geelhoed et al. 1994, Tedja et al. 1994, Masuoka et al. 1995, Delwiche and O'Mahony 1996, Rousseau and O'Mahony 1997). Frijters (1979a) was the first to explain and solve the paradox in theory.

2.1.2 Difference between two proportions in the A–Not A method or the Same–Different method

For a given pair of stimuli, A and B, if the A–Not A method is used, we get two proportions, c02-math-001 and c02-math-002 , where c02-math-003 is the proportion of response A for sample A, and c02-math-004 is the proportion of response A for sample Not A. If the Same–Different method is used, we get two proportions, c02-math-005 and c02-math-006 , where c02-math-007 is the proportion of response same for the concordant sample pairs and c02-math-008 is the proportion of response same for the discordant sample pairs. The expected difference between c02-math-009 and c02-math-010 is not the same as the expected difference between c02-math-011 and c02-math-012 for a given sensory difference. Hence, the difference between the two proportions cannot be treated as a measure of sensory difference.

2.1.3 Thurstonian model

Louis Leon Thurstone (1887–1955) was a US pioneer of psychometrics who developed a profound theory by which to measure sensory difference using the 2-AFC method (Thurstone 1927). This theory assumes that for a given stimulus, a resultant sensation in a subject is a variable and follows a probability distribution model. Such distributions have different cases, the most important and widely used of which is Thurstone's (1927) Case V. In this case, it is assumed that two different stimuli generate two sensation distributions with different mean sensations, but with equal standard deviations and zero correlations. The assumption of zero correlation can be relaxed to an assumption of equal correlations between pairs (Mosteller 1951). The standard distance between the two means of the distributions, c02-math-013 , is used as a measure for sensory discriminability or sensory difference. Sensory difference in terms of c02-math-014 can be estimated from the observed proportion of correct responses, or from other proportions in different discrimination methods.

The Thurstonian models discussed in this book cover all psychometric functions in different discrimination methods based on different decision rules. The psychometric functions for the forced-choice methods describe the relationship between c02-math-015 and the probability of correct response, c02-math-016 . The psychometric functions for the methods with response bias describe the relationship between c02-math-017 and probabilities of hit and false alarm. The probability of hit is the probability of response A for sample A in the A–Not A method, or the probability of response same for the concordant sample pair in the Same–Different method. All the psychometric functions for the discrimination methods are based on the principles of Thurstone's theory. The principles of this theory are also the basis for the Signal Detection Theory (SDT) (Green and Swets 1966, Macmillan and Creelman 2005). SDT was established originally in electrical engineering in the early 1950s, in the context of visual and auditory detection, and has since been applied to a wide range of perceptual, cognitive, and other psychological tasks. In SDT, the measure for sensory discriminability or sensory difference is usually denoted as c02-math-018 . In this book, c02-math-019 and c02-math-020 are interchangeable, but we often use c02-math-021 as an expected value and c02-math-022 as an estimate of c02-math-023 .

2.2 Thurstonian discriminal distance, δ or d′

2.2.1 Decision rules and psychometric functions for forced-choice methods

Bradley (1957) first derived psychometric functions based on different decision rules for the 2-AFC (Duo), Duo–Trio, and Triangular methods in a Memorandum presented to the General Foods Corporation in the United States. The results were announced in abstracts in 1958 (Bradley 1958a, 1958b) and were published in detail in 1963 (Bradley 1963). Ura (1960) independently derived the psychometric functions for the three methods. David and Trivedi (1962) gave further details of the results. The psychometric functions for the 3-AFC, 4-AFC, and m-AFC are given by Birdsall and Peterson (1954), Green and Birdsall (1964), and Hacker and Ratcliff (1979); Frijters (1979a) gave the logistic variants of the psychometric function for the 3-AFC and Triangular methods; Ennis et al. (1998) gave the psychometric functions for the Specified and Unspecified Tetrad methods; and Macmillan et al. (1977) and Rousseau and Ennis (2001) gave the psychometric functions for the Dual-Pair (4IAX) method.

The decision rules and psychometric functions for some forced-choice methods are given in this section.

2.2.1.1 The 2-AFC

Assume c02-math-024 are sensations evoked by samples A and B, respectively. Sample B has stronger sensory intense than sample A. A correct response will be given when c02-math-025 . Based on this decision rule, the probability of correct response in this method is:

2.2.1 equation

where c02-math-027 is the cumulative distribution function of the standard normal distribution.

2.2.1.2 The 3-AFC

Assume c02-math-028 are sensations evoked by two samples of A and c02-math-029 is a sensation evoked by sample B. Sample B has stronger sensory intensity than sample A. A correct response will be given when c02-math-030 and c02-math-031 . Based on this decision rule, the probability of correct response in this method is:

2.2.2 equation

where c02-math-033 is the standard normal density function.

2.2.1.3 The 4-AFC

Assume c02-math-034 are sensations evoked by three samples of A and c02-math-035 is a sensation evoked by sample B. Sample B has stronger sensory intensity than sample A. A correct response will be given when c02-math-036 , and c02-math-037 . Based on this decision rule, the probability of correct response in this method is:

2.2.3

where c02-math-039 .

2.2.1.4 The Duo–Trio

Assume c02-math-040 are sensations evoked by two samples of c02-math-041 and c02-math-042 is a sensation evoked by sample c02-math-043 . Sample c02-math-044 is selected as the standard sample. A correct response will be given when c02-math-045 . Based on this decision rule, the probability of correct response in this method is:

2.2.4

2.2.1.5 The Triangular

Assume c02-math-047 are sensations evoked by two samples of c02-math-048 and c02-math-049 is a sensation evoked by sample B. A correct response will be given when c02-math-050 and c02-math-051 . Based on this decision rule, the probability of correct response in this method is:

2.2.5

Bi and O'Mahony (2013) found that the influential psychometric function for the triangle test in (2.2.5), which was derived independently by Ura (1960), David and Trivedi (1962), and Bradley (1963), can be expressed as closed forms:

2.2.6

2.2.7

where c02-math-055 denotes the standardized bivariate normal cumulative distribution function of c02-math-056 with correlation coefficient between the two variables c02-math-057 is a built-in program (pmvnorm) in S-Plus (Insightful 2001).

2.2.1.6 The Unspecified Tetrad

Assume c02-math-058 are sensations evoked by two samples of A and c02-math-059 are sensations evoked by three samples of B. B has stronger sensory intensity than A. A correct response will be given when c02-math-060 or c02-math-061 . Based on this decision rule, the probability of correct response in this method is given in Ennis et al. (1998) as:

2.2.8

Bi and O'Mahony (2013) indicate that the psychometric function for the unspecified tetrad in equation (2.2.8) can be expressed as closed forms:

2.2.9

2.2.10

where c02-math-065 denotes the standardized bivariate normal cumulative distribution function of c02-math-066 , with correlation coefficient between the two variables c02-math-067 is a built-in program (pmvnorm) in S-Plus.

2.2.1.7 The Specified Tetrad

Assume c02-math-068 are sensations evoked by two samples of A and c02-math-069 are sensations evoked by three sample B. B has stronger sensory intensity than A. A correct response will be given when c02-math-070 . Based on this decision rule, the probability of correct response in this method is given in Ennis et al. (1998) as:

2.2.11

2.2.1.8 The Dual Pair (4IAX)

Assume c02-math-072 are sensations evoked by three samples of A and c02-math-073 is a sensation evoked by sample B. B has stronger sensory intensity than A. c02-math-074 and c02-math-075 are the percepts of the identical pair and c02-math-076 and c02-math-077 are the percepts of the different pair. A correct response will be given when c02-math-078 . Macmillan et al. (1977) give a psychometric function for the 4IAX method in equation (2.2.12). Rousseau and Ennis (2001) give a quite different form of the psychometric function, but it is the same as:

2.2.12 equation

2.2.2 Decision rules and psychometric functions for methods with response bias

The decision rules and psychometric functions for methods with response bias are based on a monadic design under the assumption that all of the responses in an experiment are independent of one another.

2.2.2.1 The A–Not A

Assume c02-math-080 are sensations evoked by samples A and Not A, respectively. A hit is made when c02-math-081 and a false alarm is made when c02-math-082 , where c02-math-083 is a criterion. Based on this decision rule, the psychometric function for the A–Not A method is:

2.2.13 equation

where c02-math-085 and c02-math-086 are the quantiles of c02-math-087 and c02-math-088 for the standard normal distribution, c02-math-089 is the probability of response A for Not A, and c02-math-090 is the probability of response A for A. Although c02-math-091 and c02-math-092 are affected by the adopted criterion, c02-math-093 is not affected by response bias. Equation (2.2.13) has been discussed adequately in SDT for the yes/no task (e.g., Green and Swets 1966). Elliott (1964) created tables of c02-math-094 for this method.

2.2.2.2 The A–Not A with Remind

The A–Not A with Remind (A–Not AR) is a variation on the conventional A–Not A method. For the A–Not AR, unlike the A–Not A, which is a single-sample presentation, a reminder (e.g., sample A) is provided before each test sample (sample A or Not A) in order to jog the observer's memory. The A–Not AR is not new in the psychophysics literature but is relatively new in sensory and consumer science. Macmillan and Creelman (2005) discuss the reminder paradigm, while Lee et al. (2007), Hautus et al. (2009), Stocks et al. (2013), and Bi et al. (2013b) introduced the method into sensory and consumer science literature, and provided an SDT model of the method and parameter estimation.

In the A–Not AR, each test contains two intervals (meaning two sample presentations), the first of which always contains the reminder. If the reminder is c02-math-095 then the presentations are c02-math-096 and c02-math-097 ; if the reminder is c02-math-098 then the presentations are c02-math-099 and c02-math-100 . Instructions for the A–Not AR can vary but, in essence, the participant is asked to decide whether the second presented sample is the same as or different from the reminder. Although the same and different responses are used in both the Same–Different and the A–Not AR methods, the two methods relate to different cognitive mechanisms. The psychometric functions for the A–Not AR method is as in equation (2.2.14) in the differencing strategy, which is the most likely in the A-Not AR, especially when an adequate familiarization procedure is not included in the test for panelists.

2.2.14 equation

where c02-math-102 and c02-math-103 are the quatiles of c02-math-104 and c02-math-105 for the standard normal distribution, PD is the probability of response same if the presented sample is different from the reminder, and PS is the probability of response same if the sample is the same as the reminder.

2.2.2.3 The Same–Different

Assume c02-math-106 are sensations evoked by samples A and B, respectively. A hit is made when c02-math-107 or c02-math-108 , where c02-math-109 is a criterion. A false alarm is made when c02-math-110 . Based on this decision rule, Macmillan et al. (1977) derived the psychometric function for the Same–Different method:

2.2.15 equation

2.2.16 equation

where c02-math-113 is the proportion of the same response for the concordant sample pairs, c02-math-114 is the proportion of the same response for the discordant sample pairs, c02-math-115 ; and c02-math-116 is a criterion. For given proportions c02-math-117 and c02-math-118 and c02-math-119 can be estimated numerically. Kaplan et al. (1978) published tables of c02-math-120 for the method.

2.2.3 Psychometric functions for double discrimination tests

Bi (2001) discusses the double discrimination tests. In the double discrimination tests using the forced-choice method, the probability of correct response is the product of two probabilities of correct response in conventional discrimination methods. Hence, the psychometric functions for the double discrimination tests should be:

2.2.17 equation

where c02-math-122 denotes a psychometric function for a double discrimination method and c02-math-123 denotes a psychometric function for a conventional discrimination method. For example, the psychometric function for the double 2-AFC method should be:

2.2.18 equation

2.3 Variance of d′

Thurstonian c02-math-125 provides a measure of sensory difference or discriminability. It is theoretically unaffected by the criterion adopted or the method used. However, the true c02-math-126 cannot be observed: it can only be estimated from data. We denote c02-math-127 as an estimate of c02-math-128 . The precision of the estimate c02-math-129 can be expressed by its variance, c02-math-130 .

Variance of c02-math-131 is of importance in the Thurstonian model. It describes how close the estimated value, c02-math-132 , is to a true value, c02-math-133 . Moreover, it provides a basis of statistical inference for c02-math-134 s. Variance of c02-math-135 depends not only on the sample size but also on the method used. Gourevitch and Galanter (1967) gave estimates of the variance of c02-math-136 for the yes/no task (i.e., the A–Not A method). Bi et al. (1997) provided estimates and tables for the variance estimates of c02-math-137 for the four forced-choice methods: 2-AFC, 3-AFC, Triangular, and Duo–Trio. Bi (2002a) provided variance estimates of c02-math-138 , tables, and a computer program for the Same–Different method. Bi et al. (2010) provided variance estimates of c02-math-139 , tables, and a computer program for the 4-AFC. Rousseau and Ennis (2001) provided variance estimates of c02-math-140 and tables for the Dual Pair method. Ennis (2012) provided variance estimates of c02-math-141 and tables for the Unspecified Tetrad method. Bi and O'Mahony (2013) also provided variance estimates of c02-math-142 , tables, and a computer program for the Unspecified and Specified Tetrad methods.

Different approaches can be taken to estimating the variance of c02-math-143 . One is the delta method, which uses the Taylor-series expansion with one and/or two variables. Another is to use the inverse of the second derivative of the maximum likelihood function with respect to c02-math-144 . The former will be introduced in this section. The advantage of this approach is that the variance of c02-math-145 can be expressed in a precise equation.

2.3.1 Variance of d′ for forced-choice methods

For forced-choice methods, the proportion of correct response c02-math-146 is a function of c02-math-147 ; that is, c02-math-148 , where c02-math-149 denotes a psychometrical function for a forced-choice method. According to the Taylor-series expansion:

2.3.1 equation

where c02-math-151 denotes an observation value of c02-math-152 . Hence:

2.3.2 equation

Variance of c02-math-154 for the forced-choice methods contains two components: sample size c02-math-155 and the c02-math-156 value, which is determined solely by the method used. Equation (2.3.3) is a general form of the variance of c02-math-157 for the forced-choice methods:

2.3.3 equation

2.3.1.1 The 2-AFC

2.3.4 equation

where c02-math-160 is the observed proportion of correct response in the method and c02-math-161 denotes the density function of the standard normal distribution evaluated at c02-math-162 .

2.3.1.2 The 3-AFC

2.3.5 equation

where c02-math-164 .

2.3.1.3 The 4-AFC

2.3.6 equation

where

.

2.3.1.4 The Duo–Trio

2.3.7 equation

where

.

2.3.1.5 The Triangular

2.3.8 equation

where

and c02-math-171 is the cumulative standard normal distribution function.

2.3.1.6 The Unspecified Tetrad

2.3.9 equation

where

.

2.3.1.7 The Specified Tetrad

2.3.10 equation

where

.

2.3.1.8 The Dual Pair

2.3.11 equation

where c02-math-177 .

2.3.2 Variance of d′ for methods with response bias

2.3.2.1 The A–Not A

According to Gourevitch and Galanter (1967), the variance of c02-math-178 from the A–Not A method is:

2.3.12 equation

It can be expressed as:

2.3.13 equation

where c02-math-181 and c02-math-182 are sample sizes for the samples A and Not A, respectively, and c02-math-183 . From equation (2.3.13), we can see that the variance of c02-math-184 for the A–Not A method depends on c02-math-185 , total sample size c02-math-186 , and sample allocation; that is, the ratio c02-math-187 .

2.3.2.2 The A–Not AR

For the A–Not AR, the variance of c02-math-188 is:

2.3.14 equation

where c02-math-190 and c02-math-191 are sample sizes for samples A and Not A, respectively, and c02-math-192 , where c02-math-193 and c02-math-194 are the quantiles of c02-math-195 and c02-math-196 for the standard normal distribution, PD is the probability of response same if the presented sample is different from the reminder, and PS is the probability of response same if the sample is the same as the reminder.

2.3.2.3 The Same–Different

According to Bi (2002a), the variance of c02-math-197 from the Same–Different method can be estimated from:

2.3.15 equation

where c02-math-199 and c02-math-200 are sample sizes for the discordant sample pairs and the concordant sample pairs, respectively, and

, and c02-math-202 denotes the quantile of the standard normal distribution. The variance of c02-math-203 for the Same–Different method depends on c02-math-204 and c02-math-205 , total sample size, c02-math-206 , and sample size allocation; that is, the ratio c02-math-207 . In most situations, the variance of c02-math-208 in the Same–Different method is mainly determined by the performance of the discordant sample pairs. Hence, in order to reduce the variance of c02-math-209 in the test, sample size for the discordant sample pairs should generally be larger than that for the concordant sample pairs.

2.3.3 Variance of d′ for double discrimination methods

Because the relationship between the psychometric functions for the Double Discrimination methods and the corresponding conventional discrimination methods is c02-math-210 , it can be demonstrated (Bi 2001) that the variance of c02-math-211 for the double discrimination methods can be obtained from:

2.3.16

where c02-math-213 is the c02-math-214 value for the double discrimination methods for c02-math-215 are derivatives of c02-math-216 and c02-math-217 , respectively; and c02-math-218 denotes the c02-math-219 value for the corresponding conventional discrimination methods for c02-math-220 and c02-math-221 . If the observed proportion of correct response in a double discrimination method is c02-math-222 then c02-math-223 .

Because c02-math-224 is always larger than 1/3 or 1/2 in the conventional 2-AFC, 3-AFC, Duo–Trio, and Triangular methods, c02-math-225 is larger than 1. This means that the variance of c02-math-226 for a double discrimination method is always smaller than that for a conventional discrimination method.

2.4 Tables and R/S-Plus codes for d′ and variance of d′

The authors who developed the psychometrical functions provided tables in their papers for c02-math-227 and c02-math-228 (or c02-math-229 ) values for forced-choice methods. These tables were later revised, expanded, and reproduced (e.g., by Elliott 1964, Hacker and Ratcliff 1979, Frijters 1982, Craven 1992, Ennis 1993, Versfeld et al. 1996, ASTM 2012).

2.4.1 Tables for forced-choice methods

Tables 2.1–2.8 give c02-math-230 or c02-math-231 (calculated as a function of c02-math-232 ) and c02-math-233 values for the eight forced-choice methods. For a given c02-math-234 or observed c02-math-235 , there are two values in the tables: the first is the c02-math-236 (or c02-math-237 ) value and the second is the c02-math-238 value. The c02-math-239 values range from c02-math-240 . Pc0 = 0.5 for the 2-AFC, the Duo -Trio, and the Dual Pair methods; Pc0 = 0.33 for the 3-AFC and, Triangular, Unspecified Tetrad methods; Pc0 = 0.25 for the 4-AFC; Pc0 = 0.17 (1/6) for the Specified Tetrad.

Table 2.1 d′ and B value for variance of d′ for the 2-AFC method

Note: There are two values in a cell for a given c02-math-242 value. The first is the c02-math-243 value and the second the c02-math-244 value. For example, for c02-math-245 and c02-math-246 . The variance of c02-math-247 at 0.3950 is c02-math-248 , where c02-math-249 is sample size.

For a specified c02-math-250 value, it is easy to find the corresponding c02-math-251 (or c02-math-252 ) values in different forced-choice methods from Tables 2.1–2.8. We can find that, for a specified proportion of correct response, e.g., c02-math-253 in the 2-AFC, c02-math-254 in the 3-AFC, c02-math-255 in the Duo–Trio, and c02-math-256 in the Triangular method. This means that a specified probability of correct response represents different sensory difference in terms of c02-math-257 . In other words, for a specified sensory difference in terms of c02-math-258 , different probabilities of correct response are evoked in different methods. Obviously, the proportion of correct responses cannot be used as a pure index of sensory difference or discriminability, because it is dependent on the methods used.

Table 2.2 d′ and B value for variance of d′ for the 3-AFC method

Note: There are two values in a cell for a given c02-math-260 value. The first is the c02-math-261 value and the second the c02-math-262 value. For example, for c02-math-263 and c02-math-264 . The variance of c02-math-265 at 0.9189 is c02-math-266 , where c02-math-267 is sample size.

Table 2.3 d′ and B value for variance of d′ for the 4-AFC method

Note: There are two values in a cell for a given c02-math-269 value. The first is the c02-math-270 value and the second the c02-math-271 value. For example, for c02-math-272 and c02-math-273 . The variance of c02-math-274 at 1.1858 is c02-math-275 , where c02-math-276 is sample size.

Table 2.4 d′ and B value for variance of d′ for the Duo–Trio method

Note: There are two values in a cell for a given c02-math-278 value. The first is the c02-math-279 value and the second the c02-math-280 value. For example, for c02-math-281 and c02-math-282 . The variance of c02-math-283 at 1.1784 is c02-math-284 , where c02-math-285 is sample size.

Table 2.5 d′ and B value for variance of d′ for the Triangular method

Note: There are two values in a cell for a given c02-math-287 value. The first is the c02-math-288 value and the second the c02-math-289 value. For example, for c02-math-290 , c02-math-291 and c02-math-292 . The variance of c02-math-293 at 2.0265 is c02-math-294 , where c02-math-295 is sample size.

Table 2.6 d′ and B value for variance of d′ for the Unspecified Tetrad method

Note: There are two values in a cell for a given c02-math-297 value. The first is a c02-math-298 estimate and the second is a c02-math-299 value. The estimated variance of the c02-math-300 can be obtained by c02-math-301 , where c02-math-302 is the number of sample sets in an Unspecified Tetrad test. For example, for c02-math-303 and sample size c02-math-304 , the estimated c02-math-305 , and the variance of c02-math-306 is about c02-math-307 .

Table 2.7 d′ and B value for variance of d′ for the Specified Tetrad method