Available online at www.sciencedirect.

com

ScienceDirect
Procedia Manufacturing 3 (2015) 3521 – 3528

6th International Conference on Applied Human Factors and Ergonomics (AHFE 2015) and the
Affiliated Conferences, AHFE 2015

Outfit Browser – An image-data-driven user interface for self-

service systems in fashion stores
Alexander Piazza, Christian Zagel, Sebastian Huber, Matthias Hille, Freimut Bodendorf
Friedrich-Alexander-University of Erlangen-Nuremberg, Lange Gasse 20, 90403 Nuremberg, Germany

Abstract

Consumers are overwhelmed with products in retail and online environments. Research shows that this choice-overload often
leads to consumer dissatisfaction. As a consequence, assistive consumer technologies are needed to support decision making.
Taking the fashion industry as an example, a common approach is to offer online services or self-service terminals that allow
consumers to filter the product range according to their preferences. Nevertheless, this option is only applicable when the
consumer is aware of his/her actual preferences. The inspiration phase is a neglected area of research that offers huge potential
for the development of new services to support the consumers in the buying process.This paper introduces the concept and
prototype of a novel consumer interface for intuitively browsing a fashion product range based on visual preferences. In addition
to an online version of the service a terminal for usage in retail stores is presented. The key idea of the visual interface is to
position a selected fashion product in the middle of the screen and similar products in the surrounding environment comparable
to terms in a tag cloud. The outer area of the virtual presentation shows products with a higher degree of dissimilarity. By
selecting one of them the visualization rearranges. In the solution presented, product similarity is mainly based on visual features
(e.g., brightness, color temperature, color values) which are extracted from product images. Finally, the functional as well as the
hedonic value of the new approach are evaluated by a preliminary user study.

©
© 2015
2015TheTheAuthors. Published bybyElsevier
Authors.Published ElsevierB.V. This is an open access article under the CC BY-NC-ND license
B.V.
(http://creativecommons.org/licenses/by-nc-nd/4.0/).
Peer-review under responsibility of AHFE Conference.
Peer-review under responsibility of AHFE Conference
Keywords: Image mining; Product browsing; Recommendation systems; Self-service systems; Retail

1. Introduction and motivation

The fashion industry belongs to the largest consumer industries worldwide. In 2011 the total global apparel
exports are worth approx. USD 412 billion and global textile exports USD 294 billion [18]. Online stores offer an
enormous and fast changing product range to their consumers. For instance the German fashion retailer Zalando.de

2351-9789 © 2015 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license
(http://creativecommons.org/licenses/by-nc-nd/4.0/).
Peer-review under responsibility of AHFE Conference
doi:10.1016/j.promfg.2015.07.686
3522 Alexander Piazza et al. / Procedia Manufacturing 3 (2015) 3521 – 3528

has approx. 150,000 products in his range. On the one side consumers tend to visit retailers having large product
offerings to increase the probability to find products, which meet their preferences. On the other side large product
offerings may cause a choice overload, potentially resulting in decreased product satisfaction [13].To mitigate this
situation, assistive consumer technology (ACT) like recommender systems are applied [14]. In traditional fashion
online stores consumers navigate through the product range by following a product tree structure and by filtering
products regarding attributes, like color or size.
The assumption of this research is that traditional navigation is suitable for consumers being aware about their
concrete needs and preferences. Consumers who are still in the preference finding phase need inspiration by looking
at various products. Furthermore, the decision criteria within the fashion buying process are not only based on a
single product, but rather on clothing combinations, which should result in an overall fitting outfit.
The overarching objective of this research is to design and implement an innovative user interface which enables
the user to browse through the product range by showing products that have a certain degree of similarity to a given
one. In clothing items, color attributes belong to the dominant product features. Therefore, the similarity is
calculated based on visual metrics extracted from the product images. Within this research in progress the design of
the Outfit Browser is tested using an empirical user study. It is conductedasalaboratory experiment by showing the
visualization on the so-called “product experience wall”, a human-size full HD display equipped with a touch screen.
This scenario simulates a multi-channel set-up by offering the e-commerce product range within a physical store.
This visualization supports the consumer increating outfit combinations by seeing upper and lower parts of apparel
combined on one screen.

2. Design and implementation of the Outfit Browser

2.1. Image data preprocessing

To generate appropriate recommendations, at first the shape of the product needs to be extracted. Within this
research approx. 3000 apparel product images of a German online retailer areused. In the first step the background
color of product images is removed. Therefore a so-called binary mask is created that indicates parts of the image
showing the product. The basic idea for achieving such a mask is to identify the silhouette of the product and then
assign the inner parts with 1s and the outer parts with 0s. The creation process of the binary mask is illustrated in
Figure 1. First the color image is converted in a greyscale representation. Then it is converted into a binary image
having only edges of the product by applying theCanny Edge Detection algorithm [11].This algorithm first applies a
Gaussian filter for smoothing the image and afterwards detects the edges within an image by calculating the derivate
based on the image pixel intensities.The outer edges of the resulting image represent the silhouette of the product.
Next, the so-called dilatation is conducted to fill potential gaps in the edge silhouette before the inner part of the
silhouette is filled. This process works well in cases of strong contrasts between the product’s color and the
background. Cases of products having low contrasts, resulting in incorrect segmentations,are filtered out
manually.Finally the images are converted from JPEG to PNG graphic file format, as the latter one contains a so-
called alpha channel for annotating transparent areas of the image. This alpha channel is set according to the created
binary mask.

Fig. 1.Processing pipeline for deriving binary masks for segmenting the clothing from the background.
 Alexander Piazza et al. / Procedia Manufacturing 3 (2015) 3521 – 3528 3523

2.2. Extracting visual product information

Colors belong to the most relevant properties for consumers of fashion products. While users of the Outfit
Browser are navigating through the product range, items are visualized,that are very similar or dissimilar to a
product selected. The similarity is defined by metrics oriented on the visual perception of humans. According the
trichromacy theory, humans perceive color based on a combination of three primary colors. In the following,the
applied visual measures are described:

2.2.1. Correlated color temperature extraction

The correlated color temperature (CCT) is a visual metric associating a given color to a temperature measured in
the unit Kelvin. In this research the proposed approach from McCamyis implemented [12]. Therefore, the color
values from the images are converted from the RGB into the CIE XYZ color space by using equation1.

§X· § 0.412453 0.357580 0.180423 · § R ·

¨ ¸ ¨ ¸ ¨ ¸
¨Y ¸ ¨ 0.212671 0.715160 0.072169 ¸ ¨ G ¸ (1)
¨Z¸ ¨ 0.019334 0.119193 0.950227 ¸ ¨ B ¸
© ¹ © ¹ © ¹

Then the chromaticity coordinates x, y are calculated by the equation 2.

X Y
x , y (2)
X Y  Z X Y  Z

Finally the CCT is computed by equation 3, with X e being 0.3320 any Y e being 0.1858.

3 2
§ x  xe · § x  xe · § x  xe ·
CCT ( x, y ) 449¨¨ ¸¸  3525¨¨ ¸¸  6823.3¨¨ ¸¸  5520.33 (3)
© y  ye ¹ © y  ye ¹ © y  ye ¹

2.2.2. Color brightness

Colors are perceived in different intensities. For determining the perceived color brightness, the intensities of the
red, green and blue color channel have to be weighted accordingly based on equation 4 [17].

b perc (r , g , b) 0.299r  0.587 g  0.114b (4)

2.2.3. Color hue

To compare the color hue of the products the colors are converted from the RGB color space extracted from the
images into the L*A*B* color space. The L*A*B* color space represents colors in a way similar to the human
perception. This means that colors that are perceived similar to humans have small distances in the L*A*B color
space, and colors appearing more dissimilar have larger distances. The color space consists of three dimensions,
where the dimension A* and B* describe the hue of the color. For extracting the hue the RGB colors have to be
converted into the XYZ color space according to equation 1. Afterwards the A* and B* component are calculated
according to equation 5 resp. 6 (using equation 7).
3524 Alexander Piazza et al. / Procedia Manufacturing 3 (2015) 3521 – 3528

ª §X · § Y ·º
a* 500« f ¨ ¸ f ¨ ¸» (5)
¬ ©X ¹ © Y ¹¼

ª §Y · § Z ·º
b* 200 « f ¨ ¸  f ¨ ¸» (6)
¬ ©Y ¹ © Z ¹¼

­ q1 / 3 ifq ! 0.008856½
f (q) ® ¾ (7)
¯7.787 q  16 / 116 else ¿

Within the current state of the prototype, the dominant color as well as the average color of the product is
considered. The dominant color is determined by selecting the most frequently occurring value of the red, green, and
blue channel of the RGB color image. The average color is determined by summing up all intensities per RGB
channel and dividing the sum by the number of pixels.

2.3. Software application and user interface

The Outfit Browser is designed as a web application to support both e-commerce stores and retail stores. As the
underlying framework the PHP programming language is used, except for the color extraction that is implemented
in Python. All images are preprocessed by these programs and the resulting color metrics are stored within a
MySQL database.
The system of choice to be used within the retail environment isthe so-called “product experience wall“, a self-
service system designed for fashion stores. In its original setup it represents a context-adaptive outfit
recommendation system that builds on various data gathered through sensor technologies [4]. The device uses
cameras, RFID readers, and connections to third party services in order to deliver a personalized experience to the
user. In combination with the SHORE 1 face detection SDK, it automatically detects the number, genders, as well as
the approximated age of the users. RFID additionally helps to identify products taken in front of the device. Based
on the knowledge gathered the system then pre-filters the products presented [3]. In addition, the device allows
sharing the products via social media using a QR code displayed (see Figure 2(a)).

1
For additional information also see http://www.iis.fraunhofer.de/de/bf/bsy/fue/isyst.html.
 Alexander Piazza et al. / Procedia Manufacturing 3 (2015) 3521 – 3528 3525

a b

Fig. 2.Social media connection and digital avatar (a), Product Experience Wall (b).

The physical setup is constructed using a projector with full HD resolution in combination with an acrylic back-
projection screen. User input is handled through a capacitive touch foil mounted to the screen. This construction
allows presenting products as well as digital avatars in real-life size with the goal of creating a sense of realism and
immersion [4]. Figure 2(b) presents the prototype of the system including the interaction process with the virtual
avatar.On Figure 3(a) the key visualization of the Outfit Browser is depicted. The user interaction is as follows: In
the center of the screen the currently selected outfit is shown. It is separated in a top and bottom apparel, but the
recommendation algorithm works for single apparels like dresses as well. Near the currently selected pieces the
recommendations with the highest shape and color similarities are displayed. To the outer area, recommendations of
dissimilar apparel are given and serve as an inspiration for new products. By clicking on a recommended item, the
visualization automatically calculates new recommendations based on the new selection and updates the application
accordingly. With this approach a customer can rapidly explore the whole product range without clicking through
the typically long product list as found in state-of-the-art e-commerce stores.
Figure 3(b) shows that the application is also suited to support the customer experience in a retail store (when
shown on the “product experience wall”). Besides the deeper shopping immersion caused by the interaction with the
system, it enables the retailer to display a broader range without having each item on stock.

a b

Fig. 3.User interface (a); Self-service system in action (b).

3526 Alexander Piazza et al. / Procedia Manufacturing 3 (2015) 3521 – 3528

3. Evaluation

3.1. Study setup and demographics

The system is evaluated in form of a pre-study within the period of one week during March 2015 and as a
laboratory experiment conducted at the University of Erlangen-Nuremberg. All participants invited to take part in
the experiment are introduced into the general use case of product and outfit recommendations. They are asked to
interact with the user interface shown on the product experience wall and browse through the products displayed.
After having interacted with the system, they are asked to fill out a questionnaire that focuses on the system’s ability
to support the purchase decision making process as well as on the perceived experiences when interacting with the
system. As not only conscious but also subconscious effects may occur all participants additionally get observed
during the interaction process. This observation is conducted as a non-participating and hidden observation in order
to avoid potential side effects [10]. The sample consists of students recruited at the university. Out of a total of 55
subjects, 36 participate in the study. 13 participants are male, 23 are female with an average age of 24.6 (SD=3.80).

3.2. Questionnaire and results

Next to demographic criteria, the users are asked for their mindset towards the system as well as about how the
new method for product search would improve their shopping process and make finding new product and getting
new inspirations easier.
A major element of the questionnaire is represented through the Service Fascination item set [1, 2]. In its original
setting the Service Fascination research model is used to evaluate the attractiveness of technology-enabled self-
service systems and is validated in numerous previous studies. The fascination criteria not only includes the user’s
emotions and intention to actively and repeatedly use the system in the future, but also their intention to actively
promote the system through positive word-of-mouth. In addition the acceptance of the recommended clothing items
are measured. These items are specifically developed to evaluate the acceptance of the recommendations in the
context of online stores.
All questions were answered on a 7-point Likert-type scale labeled at the end points and rated from 0 to 6. The
questionnaires as well as the results of the pre-study are shown in Table 1. These preliminary results indicate that the
Outfit Browser generates a high value for the user and provides a positive experience at the same time.

Table 1.Questionnaire and Results.

6HUYLFH)DVFLQDWLRQĮ  Md x SD Adapted from
I would share my good experience about using the system. 5 4.69 1.411 [6]
I would recommend using the system. 5 4.69 1.327 [6]
Using the system is exciting. 4 3.94 1.756 [7]
Given that I have access to the system, I predict that I would use it. 5 4.14 1.869 [8]
I will frequently use the system in the future. 5 4.28 1.614 [9]
3URGXFW6HDUFKĮ  Md x SD Adapted from
The system…
…showed me products I would not have recognized using traditional 5 4.13 1.721 [15] based on [16]
methods.
…inspired me in regards to new product combinations. 4.5 4.17 1.715 [15] based on [16]
…makes garment shopping easier. 5 4.17 1.483 [15] based on [16]
 Alexander Piazza et al. / Procedia Manufacturing 3 (2015) 3521 – 3528 3527

4. Conclusion

Within this research in progress a running prototype is implemented toenablecustomers to browse a given
clothing product range. According to the first results of the conducted pre-test study, the users perceived the
visualization as a helpful system supporting the search process during garment shopping. Furthermore, the results
indicate that the users perceive the interaction with the system as a positive experience. This is especially applies for
situations when multiple people are using the system together.
Nevertheless the presented research possesses several limitations. The color extraction method is based either on
the dominant color or the average one. This works well with rather mono-colored images, but less with multi-
colored ones. In future research this drawback is addressed by extending the color comparison to additional colors
by adding a color histogram based product comparison, e.g., by applyingthe earth-mover-distance. Furthermore,
texture information like the coarseness or direction of patterns will be integrated by using Gabor or LBP features.
The current evaluation includes 36 participants, which provides only a first indication about the acceptance of the
visualization. In further research a broader user group will be included in the study. Also comparisons to traditional
product search methods will be assessed and compared to the evaluation results of the new approach. This will
provide insights in how well the application is suited for being used in an economic context. This will furthermore
be supported by another comparative test that intends to assess user acceptance and satisfaction when being
integrated into a real web shop. Also the potential of using physical, RFID tagged products as the first filter option
on the “product experience wall” will be evaluated.

Acknowledgements

We would like to thank the “Hans-Frisch-Stiftung”, who supported this research and the development of the
product experience wall.

References

[1] C. Zagel, F. Bodendorf: “Success factors and approaches of service fascination – A research framework“, in: L. Freund, W. Cellary (Eds.),
Advances in the Human Side of Service Engineering, AHFE Conference, 2014, pp. 537-548.
[2] C. Zagel, J. Süßmuth, L. Glomann: “Assessing the influence of gender towards the adoption of technology-enabled self-service systems in
retail environments“, in: N. Mardsen, N. Kempf (Eds.), Gender-UseIT – HCI, Web-Usability und User Experience unter
Gendergesichtspunkten, de Gruyter, Oldenbourg, 2014, pp. 177-190.
[3] C. Zagel, J. Gröger, F. Bodendorf: „Echtzeit-Personalisierung kontextadaptiver Systeme auf Basis von Sensordaten: Design und Entwicklung
einer Präsentationsschnittstelle auf Basis von Websockets“, working paper, Lehrstuhl für Wirtschaftsinformatik II, Friedrich-Alexander
Universität Erlangen-Nürnberg, Nürnberg, 2014.
[4] C. Zagel: „Product Experience Wall: A Context-adaptive Outfit Recommender System“, in: M. Koch, A. Butz, J. Schlichter (Eds.), Mensch
und Computer Tagungsband, Oldenbourg Wissenschaftsverlag, München, 2014, pp. 367-370.
[5] C. Kueblbeck, A. Ernst: „Face detection and tracking in video sequences using the modified census transformation.” Image and vision
computing 24(6), pp. 564-572.
[6] J. G. Maxham: “Service recovery’s influence on consumer satisfaction, positive word-of-mouth, and purchase intentions”, Journal of Business
Research 54 (2001) 11-24.
[7] L. T. Childers, L. C. Carr, J. Peck, S. Carson: “Hedonic and utilitarian motivations for online retail shopping behavior.” Journal of Retailing
77 (2001) 511-535.
[8] V. Venkatesh, F. D. Davis: “A Theoretical Extension of the Technology Acceptance Model: Four Longitudinal Field Studies.” Management
Science 46 (2000) 186-204.
[9] J.-C. Gu, L. Fan, Y. H. Suh, S.-C. Lee: “Comparing Utilitarian and Hedonic Usefulness to User Intention in Multipurpose Information
Systems.” Cyberpsychology, Behavior, and Social Networking 13 (2010) 287-297.
[10] C. F. Graumann: “Grundzüge der Verhaltensbeobachtung”, in: E. Meyer (Ed.), Fernsehen in der Lehrerbildung, Manz Verlag, München,
1966, pp. 86-107.
[11] J. Canny: A computational approach to edge detection. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 6, 1986, pp. 679-
698.
[12] C. S., McCamy: Correlated color temperature as an explicit function of chromaticity coordinates. Color Res., Appl., 1992, 17, pp. 142–144.
3528 Alexander Piazza et al. / Procedia Manufacturing 3 (2015) 3521 – 3528

[13] J. T. Gourville, Soman Dilip. "Overchoice and assortment type: When and why variety backfires." Marketing science 24, no. 3 2005: pp.
382-395
[14] K. B. Murray, J. Liang, G. Häubl: "ACT 2.0: the next generation of assistive consumer technology research." Internet Research 20, no. 3
(2010): pp. 232-254.
[15] D. Baier, E. Stüber: “Acceptance of Recommendations to Buy in Online Retailing”, Journal of Retailing and Consumer Services, 17 (3),
2010, pp. 173–180.
[16] E. Stüber: "Kaufempfehlungen als Personalisierungsansatz im Internethandel." In Personalisierung im Internethandel, pp. 11-41. Gabler
Verlag, 2013.
[17] M. Anderson, C. Srinivasan, R. Motta, M. Stokes:“A standard default color space for the internet”: srgb. Technical report, International
Color Consortium, 1996.
[18] FashionUnited: “Global fashion industry statistics - International apparel”, http://www.fashionunited.com/global-fashion-industry-statistics-
international-apparel, retrieved 9th April, 2015.