Are Consumer Design Evaluations Trustworthy?

1. Introduction

Designers believe their own aesthetic judgment to be independent of consumers’ taste and therefore they are reluctant to gain knowledge from ordinary consumers (Margolin, 1997). However, several recent studies suggest that they try hard to listen to the voices of consumers (Oh and Joo, 2012; 2015). For instance, designers use persona to understand consumers deeply (So and Joo, 2017), IDEO cards to generate user-centered ideas (Lee and Joo, 2017), and scenarios to evaluate design concepts on behalf of consumers (Chung and Joo, 2017). Then, do designers benefit when they consider consumer design evaluations or not? Although this question has not been academically discussed to our knowledge, we want readers to pay attention to the following case. It shows consumer design evaluations may not be as trustable as expected, suggesting that designers are better to follow their own design evaluations.

Hyundai Motors introduced Sonata in 1985. For the past 30 years, it launched seven different models under the same name, and was selected as the best-selling car for 12 consecutive years. In 2009, Hyundai Motors successfully launched the 6^th model of the Sonata called YF Sonata. The sales of this Sonata continued over 10,000 for six months. However, the 7^th model of the Sonata called LF Sonata, launched in March 2014, failed to attract consumers. LF Sonata enjoyed its “new product label” only for the first two months.

Figure 1

YF Sonata vs. LF Sonata

Although the failure of LF Sonata was attributed to many factors, one of them was its plain exterior design without taste. A former Hyundai Motors designer spoke at an interview that “When designing LF Sonata, designers considered the voice of customers about YF Sonata. The previous model received extreme responses. Some customers liked it but others hated it. Therefore, designers decided to go with a plain design for the new Sonata to avoid extremity.”

The above mentioned example suggest that consumer design evaluations might not be trustable. Interestingly, an extensive marketing research argues that consumers’ choices are unstable. Consumers’ choices depend on, for instance, whether they have ample (vs. limited) resources to process information (Shiv and Fedorikhin, 1999), whether they decide which option to acquire (vs. to give up) (Dhar and Wertenbroch, 2000), whether they choose in a windfall (vs. in a standard purchase situation) (O’Curry and Strahilevitz, 2001), and whether two options are presented jointly (vs. separately) (Okada, 2005) to name a few. For instance, say a consumer faces a choice between an option with superior form (i.e., a beautiful mobile phone) that fails to meet functional criteria, and a functionally superior option that is not as aesthetically pleasing. Usually, the consumer favors the latter. However, when both options exceed basic requirements of form and function, the consumer prefers the option with superior form (Chitturi, Raghunathan, and Mahajan, 2007).

In the present work, we propose that consumers construct not only choices but also design evaluations. It has been known that design evaluations are determined by a wide variety of variables such as unity and typicality (Veryzer and Hutchinson, 1998), visibility (Han, Nunes, and Dreze, 2010), and complexity and simplicity (Cho and Kim 2006). However, it has not been studied whether design evaluations depend on contexts. As choices depend on contexts, design evaluations can vary depending on contexts. As such, this research aims at proposing and empirically testing how a contextual variable affects consumer design evaluations.

Our guiding premise is that consumers equate the concept of design with the concept of uniqueness. With this premise, we argue that consumers appreciate design highly when it is unique and less appreciate when it is not, suggesting that consumers evaluate design highly when the product is unpopular. To verify this argument, we conducted a pilot study and three experimental studies in which product uniqueness was manipulated.

Our findings will provide a better understanding of consumers’ design evaluation processes, and so contribute to the ongoing discussion on what is good design from the consumers’ perspective. Note that some design researchers have studied evaluation criteria for good design (Krippendorf, 1995) and they generally conclude that both form and function are the most common evaluation criteria for internationally recognized design awards such as Red Dot Design Award, International Forum (IF) Design Award, and Industrial Design Excellence Awards (IDEA) (Demirbilek, and Park, 2001). However, the question of whether consumers evaluate product design in the same way as professional designers do has rarely been discussed. More importantly, whether consumer design evaluations are stable or not has not been discussed at all (Bloch, 1995; Luchs, and Swan, 2011).

2. Literature review

Product design history tells that designers have pursued to create a unique product, and in order to achieve this goal, they have emphasized either form or function. In the late 1910s, when most designers stylized their products with ornaments, a group of European designers declared themselves Minimalists, stripping their work to its fundamental features and necessary elements (Meyer, 2000). For instance, Dieter Rams adopted the motto of “less for more” and commercialized a line of products by using only few basic geometric shapes and clean color combinations (Lovell and Kemp, 2011). Daily objects from chairs to kitchen utensils were subsequently dominated by Minimalism. Later, another group of designers emphasized product form as a distinguishing factor in order to differentiate themselves from previous Bauhaus designers, initiating a new design trend. For instance, Phillipe Stacks introduced a useless but pleasing orange squeezer called Juicy Salif (Morgan, 1999). Two examples clearly demonstrated that designers consistently pursue uniqueness and they use form and function to achieve it.

We expect that consumers approach the concept of design in the same way that designers have done. When consumers are asked to evaluate the quality of a product’s design, they evaluate how distinctive the product is from other products, mostly in the same category. It is well established in Western culture that consumers want unique products (Snyder, 1992). Because non-conformity appeals to most people, consumers try to build their unique public images through the material objects that they buy and display. Further, consumers tend to pursue uniqueness more strongly in some product categories (Simonson and Nowlis, 2000). According to a study conducted by Berger, and Heath (2007), students at Stanford University were more likely to diverge when selecting different options in domains such as their favorite CD or hairstyle but they were less likely to do so in domains such as dish soaps or paper towels. This occurs because consumption behavior is a safe way to express uniqueness without damaging an individual’s sense of social assimilation (Ruvio, 2008).

More specifically, Tian, Bearden, and Hunter (2001) conceptualized consumers’ need for uniqueness (CNFU) as a three-dimensional construct that underlies different ways in which consumers express their need for uniqueness through consumption. The three dimensions are creative choice counter conformity, unpopular choice counter conformity, and avoidance of similarity. Among the three dimensions, we expect consumers rely on the second dimension, unpopular choice counter conformity, when evaluating design. Prior studies show that unpopular choices enhance self-image and social image because people should take social risks to express their uniqueness (Ruvio, 2008). Compared to choice or consumption behavior, design evaluations are less risky to express their own uniqueness. Therefore, we assume that as a means of establishing uniqueness, consumers will evaluate the design of unpopular products highly rather than popular ones.

Following our guiding premise that consumers believe that a unique product is well designed, we expect that consumers will evaluate a design more negatively upon learning that other consumers chose it. Suppose that a female consumer evaluates which bag is better designed between an aesthetically appealing bag and a functionally superior bag. If she is surrounded by her friends who are attracted to aesthetically appealing bags, she will discount aesthetics when evaluating design. On the other hand, if she accompanied by her business clients who carry functional bags, she will devalue function when evaluating design. This suggests that situation indicates which option is popular, which, in turn, determines which option is better designed. Based on these grounds, we postulate that when others choose aesthetic products, consumers will evaluate the design of functional products highly. When others choose functional products, they will evaluate the design of aesthetic products highly.

Proposition: When aesthetically (functionally) superior products are popular, consumers will evaluate its design lower.

3. Pilot study

We conducted a pilot study to test whether consumers use form and function when evaluating design. Ninety undergraduate students recruited by the participation pool at a large North-American University answered one open-ended question: “What do you consider when you evaluate the design of a product (i.e., how well a particular product is designed)?”

Subjects produced a total of 302 responses (3.39 responses per subject). Two judges, one author and a graduate student who was blind to the research project, categorized the collected responses independently (inter-judge agreement = 83%). Discrepancies were resolved by discussion with two professional designers; one worked as a professional product designer in Shanghai after obtaining his undergraduate degree in USA and the other worked as a graphic designer in Toronto after obtaining her graduate degree in Canada.

Our analysis revealed four important findings. First, majority of subjects considered form and/or function when they evaluate design. Out of 302 responses, 113 responses (37%) were functional attributes (e.g., function, features, efficient, effective, practical, useful, and convenient) and 93 responses (31%) were aesthetic attributes (e.g., sleek, eye-catching, attractive, aesthetically pleasing, looking good, stylish, and colorful). Other responses belong to the groups such as weight (9%), technological advancement (4%), differentiation (3%), simplicity (3%), popularity (3%), and modernity (2%). The remaining 21 responses were not categorized (e.g., compatibility, multi-purpose, durability, and brand). Second, many responses were mutually inconsistent. For instance, technological advancement is frequently accomplished by sacrificing simplicity. Differentiation goes against popularity. Modern products are not classic. Observed inconsistency implied that consumers have ambiguous understanding about design. Finally, of special interest to this study, many subjects responded both form and function. Out of 90 subjects, 59% of subjects answered both form and function. Only 16 subjects answered form only and 16 subjects answered function only. This implies that many subjects paid attention to both form and function, as designers do, when evaluating design. Fourth, few subject explicitly mentioned the variables that have been discussed in the prior studies about design evaluations. Unity, typicality, visibility, or complexity has not been mentioned at all. Only nine subjects mentioned simplicity (Cho and Kim 2006; Han et al. 2010; Veryzer and Hutchinson 1998). This suggests that consumers may rely heavily on form and function rather than academically tested variables when evaluating design. In sum, our pilot study provides evidence that consumers use form and function to form design evaluations.

4. Study 1: Innovative portable device

4. 2. Findings

Our findings show that product situation influences subjects’ design evaluations. When the device was used in school, more subjects chose the functional option better designed (Aesthetic = 33% vs. Functional = 67%). On the other hand, their choice was reversed when the device was used in the business context; more subjects chose the aesthetic option better designed (Aesthetic = 60%vs. Functional = 40%, X²(1) = 3.71, p = .05).

Interestingly, subjects’ purchase intention was not influenced by situation. They were more likely to purchase the functional option regardless of whether the device was used in the school context (Aesthetic = 22% vs. Functional = 78%) or in the business context (Aesthetic = 24% vs. Functional = 76%, X²(1) = 0.02, p > .10).

Figure 2

Design evaluation as a function of situation (Study 1)

5. Study 2: Laptop messenger bag

5. 1. Objective, design, and procedure

In study 2, we aimed at replicating the findings obtained in study 1 and, more importantly, addressing two issues discussed previously. In this study, we provided pictorial stimuli to subjects in order to, first, eliminate any potential ambiguity of the verbal description and, second, make an aesthetic option relatively more appealing than a functional option. Our pictorial manipulation, therefore, allowed us to test whether uniqueness works for both aesthetic option and functional option.

We used the same procedure as used in study 1. We employed a 2 (situation: school vs. business) between-subjects design. In this study, we used a different stimulus, a new laptop messenger bag, to extend our prior findings. In total, 44 undergraduate students were recruited by the subject pool in a large North-American University and a course credit was rewarded for their participation. Half of the subjects were asked to imagine that they use a new messenger bag when they go to school and meet their friends and the other half of the subjects were asked to imagine that they use a new messenger bag when they go to office and meet their business clients. Then, they were provided with a picture of two bags. In the picture, one bag has superior aesthetic attributes (e.g., material, exterior, and texture) and the other bag has superior functional attributes (e.g., laptop sleeve, interior pockets, and exterior pockets). Finally, subjects were asked to answer four choice questions: one was for design evaluation, the other was for purchase intention, and the remaining two were for manipulation check (e.g., aesthetic superiority: which bag is more aesthetically appealing in general? which bag is superior in terms of appearance? and functional superiority: which bag is more functional in general? which bag is superior in terms of usefulness?).

Figure 3

Stimuli (Study 2)

5. 2. Findings

First, our manipulation was successful. Subjects answered that the aesthetic bag is aesthetically more appealing than the functional bag (Aesthetic = 91% vs. Functional = 9%) and it is superior in terms of appearance as well (Aesthetic = 98% vs. Functional = 2%). They also answered that the functional bag is more functional (Aesthetic = 2% vs. Functional = 98%) and it is superior over the aesthetic bag in terms of usefulness as well (Aesthetic = 18% vs. Functional = 82%).

Same with study 1, we found that subjects’ design evaluations were determined by situation. When they imagined to use the bags in school, more subjects evaluated the functional superior option better designed (Aesthetic = 28% vs. Functional = 72%). However, when they imagined to use bags while meeting their business clients, more subjects evaluated the aesthetic superior option better designed (Aesthetic = 68% vs. Functional = 32%, X²(1) = 7.11, p < .01).

As expected, purchase intention was not influenced by situation. However, differently from the prior findings, more subjects chose the aesthetic superior option than the functional superior option regardless of whether they used bags in school (Aesthetic = 64% vs. Functional =36%) or for business (Aesthetic = 74% vs. Functional = 26%, X²(1) = 0.47, p > .10).

Figure 4

Design evaluation as a function of situation (study 2)

6. Study 3: Keyboard

6. 1. Objective, design, and procedure

In study 3, we tested our proposition by manipulating uniqueness directly. We used a pair of keyboards for our stimuli to generalize our previous findings and employed a 2 (situation: school vs. business) x 2 (unique product at schools: functional products vs. aesthetic products) between-subjects design. In total, 62 subjects enrolled in a North-American university participated in this study. Different from prior studies, subjects were provided with two sets of descriptions. First, they were asked to imagine to use keyboards either in school (e.g., “You will use a new keyboard at school”) or for business (e.g., “You will use a new keyboard at office”). Secondly, they were informed which keyboard was more likely to be chosen by students. Half of the subjects were informed that aesthetic keyboards were popular at schools and the other half were informed that functional keyboards were popular at schools.

After manipulating situation and unique product, we provided them with the picture and the verbal descriptions about two keyboards; one is aesthetically appealing whereas the other illuminates its characters and turns on its back lights. Finally, subjects were asked to choose between two keyboards in terms of design evaluation, purchase intention, and manipulation check for aesthetic superiority and functional superiority.

Figure 5

Stimuli (Study 3)

6. 2. Findings

First, our manipulation was successful. Subjects answered that the aesthetic keyboard was more aesthetically appealing regardless of whether it was popular at schools (Aesthetic = 77% vs. Functional = 23%) or not (Aesthetic = 68% vs. Functional = 32%). The same findings obtained from the next question; it was superior over the other keyboard in terms of appearance regardless of whether it was popular (Aesthetic = 81% vs. Functional = 19%) or not (Aesthetic = 71% vs. Functional = 29%).We also obtained reverse patterns for the two functionality questions. Subjects answered that the functional keyboard was more functional whether aesthetic keyboard was popular at schools (Aesthetic = 16% vs. Functional = 84%) or not (Aesthetic = 10% vs. Functional = 90%), and it was superior in terms of usefulness whether aesthetic keyboard was popular at schools (Aesthetic = 13% vs. Functional = 87%) or not (Aesthetic = 16% vs. Functional = 84%).

Next and most importantly, we obtained evidence that subjects’ design evaluations were not determined by situation but determined by the unique product at schools. When subjects were informed that aesthetic keyboard was popular at schools, their design evaluations were consistent with our previous findings. Subjects increased design evaluations of aesthetic option when using for business compared to when using in school (in school: Aesthetic = 47% vs. Functional = 53% vs. for business: Aesthetic = 94% vs. Functional = 6%, X²(1) = 8.33, p < .01). However, this pattern was reversed when subjects’ belief about what is popular at schools were incorrect. When subjects were informed that functional keyboard was popular at schools, subjects decreased design evaluations of aesthetic option when using for business compared to when using in school (in school: Aesthetic = 87% vs. Functional = 13% vs. for business: Aesthetic option = 44% vs. Functional option = 56%, X²(1) = 6.23, p < .05).

Finally, we found that purchase intention was neither influenced by situation nor by others’ choices. More subjects were constantly more likely to buy the functional keyboard regardless of whether it was popular among business people (X²(1) = 0.27, p > .10) or students (X₂(1) = 1.69, p > .10).

Figure 6

Design evaluation as a function of situation and unique product at schools (study 3)

7. Discussion, practical implications and conclusion

In the present work, we aim to address a question about design: whether consumers have a determined, stable, design evaluations or if they construct design evaluations on the spot. By relying on the assumption that consumers may consider unique products to be well designed, we propose that consumer design evaluations may not be determined but constructed. We found evidence from the three studies that consumers make design evaluations using aesthetic and functional attributes. Importantly, they place greater weight on functional attributes for their design evaluations when they use products in school because they expect other students pay attention to aesthetics. On the other hand, aesthetic attributes are emphasized for design evaluation when functional attributes are expected to be heavily considered by business people. Interestingly, situation influences consumer design evaluations but not their purchase intention.

The present work contributes to the academic discussion of consumer decision making. Prior studies examined various contextual variables which influence consumers’ preferences or their choices between hedonic options and utilitarian options (Chitturi, Raghunathan, and Mahajan, 2007; Dhar and Wertenbroch, 2000; O’Curry and Strahilevitz, 2001; Okada, 2005; Shiv and Fedorikhin, 1999). Differently from their findings, we found that consumer design evaluations are shaped by usage. In addition, prior studies suggest that design evaluations are determined by a list of variables such as unity, typicality, or visibility (Cho and Kim 2006; Han et al. 2010; Veryzer and Hutchinson 1998). Differently from them, we found that consumers rely on form and function rather than academic variables and, more importantly, their weighting varies depending on context.

More importantly, our findings provide the following valuable practical implications to contemporary designers. Nowadays, consumers often post their opinions on popular social network services such as Instagram, Facebook, and YouTube and either praise or criticize the design of specific products and services. Therefore, there is a chance that consumer design evaluations could influence professional designers like car designers working Hyundai Motors. However, an application of our findings to the real world situation reveals that consumers are not necessarily trustable about their design evaluations. Their expressed design evaluations may not reflect their genuine views. Instead, they may favor what majority does not choose to appear unique and be different. Since consumer design evaluations depend on others’ evaluation at any time and, therefore, designers may not have to react to voice of consumers.

References

• Berger, J., & Heath, C. (2007). Where consumers diverge from others: Identity signaling and product domains. Journal of Consumer Research, 34(2), 121-134. [https://doi.org/10.1086/519142]
• Bloch, P. (1995). Seeking the ideal form: Product design and consumer response. Journal of Marketing, 59(July), 16-29. [https://doi.org/10.1177/002224299505900302]
• Chitturi, R., Raghunathan, R., & Mahajan, V. (2007). Form versus function: How the intensities of specific emotions evoked in functional versus hedonic trade-offs mediate product preferences. Journal of Marketing Research, 44(4), 702-714. [https://doi.org/10.1509/jmkr.44.4.702]
• Cho, K., & Kim, T. (2006). A Study on the simplicity/complexity and balance of product aesthetic factors. Archives of Design Research, 19(2), 15-30.
• Chung, J., & Joo, J. (2017). Effect of empathy on designers and non-designers in concept evaluation. Archives of Design Research, 30(3), 57-71. [https://doi.org/10.15187/adr.2017.08.30.3.57]
• Demirbilek, O., & Park, M. (2001). A survey of criteria for the assessment of good product design. In Proceedings of the Fourth European Academy of Design Conference (pp. 370-377). Portugal: EAD.
• Dhar, R., & Wertenbrock, K. (2000). Consumer choice between hedonic and utilitarian goods. Journal of Marketing Research, 37(1), 60-71. [https://doi.org/10.1509/jmkr.37.1.60.18718]
• Franke, N., & Schreier, M. (2008). Product uniqueness as a driver of customer utility in mass customization. Marketing Letters, 19(2), 93-107. [https://doi.org/10.1007/s11002-007-9029-7]
• Han, Y. J., Nunes, J. C., & Drèze, X. (2010). Signaling status with luxury goods: The role of brand prominence. Journal of Marketing, 74(4), 15-30. [https://doi.org/10.1509/jmkg.74.4.015]
• Krippendorf, K. (2006). The semantic turn: A new foundation for design. FL: CRC Press. [https://doi.org/10.4324/9780203299951]
• Langlois, J. H., Kalakanis, L., Rubenstein, A. J., Larson, A., Halam, M., & Smoot, M. (2000). Maxims or myths of beauty? A meta-analytic and theoretical review. Psychological Bulletin, 126(3), 390-423. [https://doi.org/10.1037/0033-2909.126.3.390]
• Lee, Y., & Joo, J. (2017). Using design methods to improve design Quality: Verbalizers vs Visualizers. The Design Journal, 20(Sup1), 1471-1484. [https://doi.org/10.1080/14606925.2017.1352672]
• Lovell, S., & Kemp, K (2011). Dieter rams: As little design as possible. Phaidon Press.
• Luchs, M., & Swan, S. (2011). Perspective: The emergence of product design as a field of marketing inquiry. Journal of Product Innovation Management, 28(3), 327-345. [https://doi.org/10.1111/j.1540-5885.2011.00801.x]
• Margolin, V. (1997). Getting to know the user. Design Studies, 18(3), 227-236. [https://doi.org/10.1016/S0142-694X(97)00001-X]
• Meyer, J. (2000). Minimalism: Themes and movements. New York: Phaidon Press.
• Morgan, C. (1999). Starck. Universe Publishing.
• O' Curry, S., & Strahilevitz, M. (2001). Probability and mode of acquisition effects on choices between hedonic and utilitarian options. Marketing Letters, 12(1), 37-49. [https://doi.org/10.1023/A:1008115902904]
• Oh, D., & Joo, J. (2012). Perspective-taking method for user experience research. Archives of Design Research, 25(3), 235-245.
• Oh, D., &Joo, J. (2015). Effect of mechanical perspective-taking method on evaluating user experience. Archives of Design Research, 28(1), 219-231. [https://doi.org/10.15187/adr.2015.02.113.1.219]
• Okada, E. (2005). Justification effects on consumer choice of hedonic and utilitarian goods. Journal of Marketing Research, 42(1), 43-53. [https://doi.org/10.1509/jmkr.42.1.43.56889]
• Ruvio, A. (2008). Unique like everybody else? The dual role of consumers' need for uniqueness. Psychology and Marketing, 25(5), 444-464. [https://doi.org/10.1002/mar.20219]
• Shiv, B., & Fedorikhin, A. (1999). Heart and mind in conflict: The interplay of affect and cognition in consumer decision making. Journal of Consumer Research, 26(3), 278-292. [https://doi.org/10.1086/209563]
• Simonson, I., & Nowlis, S. (2000). The role of explanations and need for uniqueness in consumer decision making: Unconventional choices based on reasons.Journal of Consumer Research, 27(1), 49-68. [https://doi.org/10.1086/314308]
• Snyder, C. (1992). Product scarcity by need for uniqueness interaction: A consumer catch-22 carousel?. Basic and Applied Social Psychology, 13(1), 9-24. [https://doi.org/10.1207/s15324834basp1301_3]
• So, C., & Joo, J. (2017). Does a persona improve creativity?. The Design Journal, 20(4), 1-17. [https://doi.org/10.1080/14606925.2017.1319672]
• Tian, K., Bearden, W., & Hunter, G. (2001). Consumers' need for uniqueness: Scale development and validation. Journal of Consumer Research, 28(1), 50-66. [https://doi.org/10.1086/321947]
• Townsend, C., & Sood, S. (2012). Self-affirmation through the choice of highly aesthetic products. Journal of Consumer Research, 39(2), 415-428. [https://doi.org/10.1086/663775]
• Veryzer, R. W., & Hutchinson, J. W. (1998). The influence of unity and prototypicality. Journal of Consumer Research, 24(4), 374-385.