Why Should GUI Designers Create Visual Novelties? : The Mechanisms of First Impression Formation in User Experience

2. 1. Overview of User Experience Research

Early HCI research focused on how quickly and safely users could accomplish tasks since hardware breakdown or software failure occurred frequently during the early development of products and services. After the 1970s, as technology stabilized, researchers such as Gibson (1978), presented the concept of the affordance of objects exposed on the exterior of products or systems, and the importance of the user interface began to be mentioned (Norman, 1988). By the mid-1990s, research on the user interface design had accelerated, along with the development of internet technology. Nielsen (1994) defined usability using five quality components and emphasized the usability of user interface design in web services. Furthermore, in ISO/IEC 9141(1998), usability was defined as ‘the degree to which a software can be used by specified consumers to achieve quantified objectives with effectiveness, efficiency, and satisfaction in a quantified context of use.’ According to these definitions, it can be seen that the sub-dimensions of usability also include practical aspects such as efficiency or effectiveness, together with satisfaction and even the psychological state of the individual, and this means a tendency to evaluate user experience by centering generally on usability.

As system performance improves together with the level of usability, various experiential, affective, and meaningful aspects were found to be main factors that caused qualitative differences in the user experience. In this process, it was proven that there are two independent attributes in the previous notion of usability, which are inherent usability (meaning judgment is easy to use), and apparent usability, (which is related to aesthetics) (Kurosu & Kashimura, 1995; Tractinsky, 1997). Since then, HCI studies have treated usability and aesthetics as independent dimensions, and furthermore, the relationship between the various dimensions of the user experience has been studied. ISO 9241-210 (2009) defines user experience as ‘a person's perceptions and responses resulting from the use or anticipated use of a product, system or service.’ It means that user experience includes all the user’s emotions, beliefs, preferences, perceptions, physical and psychological responses, behaviors and accomplishments that occur before, during and after use. However, considering the huge amount of design projects of websites and apps that are regularly renewed, it is necessary to clarify the effects of first impressions before use.

2. 2. Framework for Formulating User Experience

In UX research, the influential relationships among various factors in user experience were studied. Thüring and Mahlke (2007) suggested the CUE-Model (Component-model of User Experience), combining the factors of the Technology Acceptance Model (Davis, 1989) and Hassenzahl’s UX model (Hassenzahl & Tractinsky, 2006). The CUE-Model shows that human-technology interaction is conducted through physical interaction, with the components of system properties, context or task parameters, and user characteristics. Through this process, users make a cognitive judgment, which includes the perception of instrumental qualities and the perception of non-instrumental qualities (meaning aesthetic). These two perceptions influence the emotional reactions of the user, and finally, these three components comprise the consequences of the user experience (see Figure 1). In research that empirically evaluates user experiences of screen-based images in digital media, like websites, usability, aesthetics and emotion can be considered typical attributes of the perception of instrumental qualities, the perception of non-instrumental qualities and emotional user reactions. Additionally, they demonstrated that first formed usability and aesthetics have an influence on the emotion.

Figure 1

CUE-Model (Thüring & Mahlke, 2007)

The term ‘usability' refers to evaluation by behavioral experience ‘during and also after use (or behavior)’, and influences other experience aspects. The term 'perceived usability', formed by the impression ‘before use (or behavior)’, is influenced by quick-formed subjective decisions such as aesthetic judgments and other user experience aspects (Brady & Phillips, 2003; Lazarus & Lazarus, 1991).

The processes of user experience formation between two situations aren’t explained in the same model. Therefore, we need to investigate the difference between the first impression mechanism and the general user experience mechanism based on usability.

2. 3. Aesthetic Judgment & Emotion in the First Impression

The viewpoint of ‘visual aesthetics’ can be broadly divided into the expressive attributes of the object’s beauty, such as alignment, proportion, symmetry and animation, and the subjective judgment of the agent sensing beauty, such as feelings of pleasure (Ginsborg, 2005; Lavie & Tractinsky, 2004; Tractinsky et al., 2006). In the first impression of the user experience, the aesthetic judgment of the subject should be centered in the discussion more than the expressive attributes of the object, because the aesthetic judgment deals with the judgment of the recognized subject matter rather than perceived subject matter. Lavie and Tractinsky (2004) demonstrated that the visual aesthetic judgment of a website is comprised of two independent dimensions; classical aesthetics and expressive aesthetics. Classical aesthetics are related to ‘apparent usability’ (Kurosu & Kashimura, 1995) or the ‘visual clarity’ dimension (Nasar, 1999) through design attributes of the website that can be expressed as classical, orderly, and clear. Meanwhile, expressive aesthetics are related to creativity and originality through the design attributes that the designer can express.

In general user experience models like the CUE-Model (Thüring & Mahlke, 2007), claim that emotion represented by the feeling of pleasure is known as a psychological response formed by the effect of usability and aesthetics. Then, as Lindgaard et al. (2006) questioned, how can we explain the formation of emotional dimensions at reliable levels upon the first sight (quick looking or quick browse), even before a relatively slow cognitive judgment is formed? There were two major streams of theory related to the relationship between emotion and cognition. One was cognitive theories based on the hypotheses asserted by Lazarus and Lazarus (1991) and one was a cognitive theory by Ortony and Turner (1990). They asserted that emotions are generated according to how individuals perceive stimuli or the object. Against the backdrop of this hypothesis, user experience models such as the already discussed CUE model posit that the usability of the user and cognitive judgment have an influence on the formation of emotion. The other was differential emotion, theories based on the hypothesis claimed by Zajonc and Markus (1982). Against the background of this hypothesis, and as research is gathered about rationality and independent emotion, differential emotion theory posits that unlike the cognitive system, which is detailed but slow in speed, emotional experience proceeds independently from cognitive judgment. It is fast and rough, and occurs before cognitive judgment in time.

These kinds of different views on emotion and cognition cause differences in definitions and categories of emotion. In Cognitive theories, the experience of emotion includes various cognitive components, including activating appraisals, subsequent desires, and intentions (Ortony & Turner, 1990). On the other hand, in Differential Emotion Scale theory, emotional experience is a feeling state or motivational condition that comes as a direct and immediate product of the particular neural processes associated with that emotion (Izard, 1989). These varying perceptions on emotion empirically present the results of research on neurophysiological responses, which evade responses to environmental stimuli (Cacioppo, Gardner, & Berntson, 1999), while Differential Emotion theories became widely accepted and activated the research on independent emotion (J.-h. Kim, Lee, Min-kyu, 2006) (see Figure 2). In the emotional dimension of this neurophysiological point of view, ‘Pleasure,’ ‘Arousal,’ and ‘Dominance’ are divided into different dimensions (Berlyne, 1974; Detenber & Reeves, 1996; Mehrabian & Russell, 1974). In PAD, the two most used dimensions are hedonistic pleasure and arousal. Pleasure is a continuous emotional response ranging from positive pleasure to negative unpleasantness, while arousal is the excitement of the autonomic nervous system and a continuous response to emotions ranging from activeness, excitement, and wariness to a calm, drowsy, or peaceful emotion (Lang, 1980).

Figure 2

S-O-R framework by Mehrabian & Russell

In today’s HCI field, although it is possible to produce the first impression in the user experience in the short period of 0.5 seconds, it is necessary to analyze the basic emotional state in the quick and rough neurophysiological response, which is independent from the comparatively slow cognitive system. In addition, quickly formed neurophysiological emotional effects (emotional states) such as this predict the influence of a user’s cognitive judgment of usability or even aesthetic judgment (response of approach or avoidance).

4. 1. Variables and Measurement

The definition of each variable is as follows:

First, Visual novelty (VN) is a characteristic of perceiving an object and making a judgment on it based on the user’s previous experience. Therefore, the degree of what participants evaluated as new was measured.

Second, the neurophysiological emotional evaluation has been conducted with Pleasure (PL)’ and Arousal (AR)' as the variables, as defined by Mehrabian and Russell (1974), Mehrabian (1996). These emotional responses are mainly measured by Visual self-reporting based on the facial expression pictures of emotional states – ‘pleasure’ and ‘arousal’ – designed by Lang (1980). The result of this evaluation has a high correlation with the measurement of physiological emotion and less prone to influence by cognitive process than a Self-report based on emotional vocabulary expressions, so it is appropriate to evaluate emotions in a short period of time such as when forming first impressions.

Third, Aesthetic judgment is a subjective judgment about certain attributes inherent in a target object. To measure this, two dimensions, the Classical aesthetics (CA) and Expressive aesthetics (EA) dimensions proven by Lavie and Tractinsky (2004) was used. It has been proven that the classification can be equally applied when translated into Korean (Yang, 2011). Originally classical aesthetics used to consist of five sub-categories, but among them ‘pleasure’ is redundant in the emotional dimension, therefore it was excluded. Finally, each question is composed of a total of nine factors: ‘clear,’ ‘clean,’ ‘symmetric,’ ‘aesthetics,’ ‘creative,’ ‘fascinating,’ ‘use of special effects,’ ‘original,’ and ‘sophisticated.’

Fourth, Perceived Usability (PU) is a prediction of website usability. It is not a usability evaluation of behavioral experience during and also after use, but an evaluation of impression, so the question is consolidated. Generally, only one question, ‘easy to use’ is asked (Lindgaard, Dudek, Sen, Sumegi, & Noonan, 2011), but in this study, considering that the purpose of visiting a new website is to search for information, an additional question of ‘easy to find information’ was added resulting in two questions.

The questionnaire composed of fourteen items was translated into Korean and measured on a 7-point scale. (strongly disagree=1, strongly agree=7).

4. 2. Stimulus & Process

Websites can be evaluated by different standards according to their service purpose and function. Therefore, the stimulations to be provided to participants were limited to corporate identity websites in the industries of finance, IT, services and manufacturing, where a strong impression of the first screen is important.

First, stimulants were selected through a similar process to the study of Tuch et al. (2012) which examined user experience with regards to complicated website screen images. The URL of about 2,500 top websites in the categories of finance, IT, and services & manufacturing according to a web log analysis site (www.rankey.com) were collected. Next, excluding the websites of companies with high recognition, the remaining websites were screen captured and saved in JPG format. Websites with the following eye-catching elements in the captured screen image were eliminated: intro pages with only images and no content, shopping malls websites, webpages with a strongly noticeable ad, webpages with intense background colors, and webpages written in a foreign language. This led to a final selection of 250 screen images. Finally, of the 250 images, the visual novelty of the screen images was evaluated, and a final 18 images were chosen to present various degrees of novelty (see Figure 4).

Figure 4

Stimulus

The study was conducted in a manner to avoid website screen images with a similar degree of novelty to be shown consecutively.

In the experiment, a participant was informed of the experiment method and the number of repetitions. Next, one of the 18 website screen images was displayed on the PC screen. As soon as the image was presented, participants immediately answered the questionnaire. The same process was repeated 18 times.

4. 3. Participants

A total of 514 university students participated in this research. But respondents who had seen one of the stimulant websites before, or in the case that they had prior knowledge of a website, were excluded from the analysis. Insincere respondents were excluded as well. Finally, the responses given by 329 respondents were analyzed, of which male students were 179(54.4%), and female students were 150(45.6%).

5. 1. Factor Analysis

The factors analysis using a total of 11 questions defined from CA, EA, and PU variables, were explained by three factors with 75% of the total variance. Fact 1, Factor 2, and Factor 3 could be represented EA, CA, and PU variables, except a component ‘aesthetic’. ‘Aesthetic’ which had been designed as a component of CA was loaded with the components of EA, and the coefficient was smaller than other components. Thus component ‘aesthetic’ was excluded.

In factor 1, the five terms of ‘creative,’ ‘fascinating,’ ‘use of special effects,’ ‘original’, and ‘Sophisticated’ were used as components explaining EA, and the Cronbach’s alpha was 0.905. In factor 2, the three terms of ‘clear,’ ‘clean,’ and ‘symmetric’ were used as components explaining CA, and the Cronbach’s alpha of CA was 0.829. In factor 3, the two terms of ‘perceived ease of use’ and ‘perceived ease in searching for information’ were used as components explaining PU and the Cronbach’s alpha was 0.789.

Table 1

Factor Analysis of Components, Rotated Component Matrixa

5. 2. Effect Analysis of Novelty

Research question 1 addressed whether the degree of novelty in the screen image on the website affected user emotions, aesthetic judgment, and perceived usability. For verification of this, MANOVA was conducted to investigate whether novelty had an effect on each variable (see Table 2).

Table 2

Tests of Between Subjects Effect (Independent Variable: VN)

Analysis results showed that VN had a significant influence (p<.001) on all variables; PL, AR, EA, CA, and PU. Therefore, research question 1 was surpported significantly. The effect size η2 of VN on PL and AR was large at .196 and .195. In the two dimensions of Aesthetic judgment, the effect size of VN on EA was .357 which is very large, and the effect size on CA was at an intermediate level of .095. The effect size of VN on PU was about average at .063.

The results of the examination were able to confirm the following: First, Visual novelty was identified as an important factor which impacts Pleasure, Arousal, Expressive aesthetics, Classical aesthetics, and Perceived usability in first impression formation. Second, going against expectations that the degree of Visual novelty would influence Arousal the most, it was found that it had a more significant effect on Expressive aesthetics.

5. 3. Structural Model Analysis

The result of this research was validated through Structural Equation Modeling (SEM). The result of examining the condition of normal distribution of the structural equation model (skewness <2, kurtosis <4) (Hong, Malik, & Lee, 2003), the skewness and kurtosis of the used variables satisfied the condition of normal distribution necessary for applying the structural equation model (-.238 < skewness <.054, -.489 < kurtosis < .235). Also, to validate the causal relationships established in this study, the correlation amongst all used variables was analyzed, and it showed a significant positive correlation for all variables at the p<0.01 level. The model fit was evaluated through RMSEA, TLI, and CFI which is not sensitive to sample size, considered the parsimony of the model and had an established good of fit index. An RMSEA value of less than .05 suggests a good model fit, .06 to .08 an acceptable fit, and .10 would indicate a poor fit (Browne & Cudeck, 1993), and in the case of TLI and CFI, the index varies according to the continuum from 1 to 0, and any value above .09 is considered a good fit (Bentler, 1990; Tucker & Lewis, 1973; re-quoted from J.-h. Kim, Kim, & Hong, 2009)

The results of the structural model [Baes model] analysis, the fit indexes of this research model is TLI, CFI were .933 and .922, which were considered a good fit (see Table 3: Base model). But the index for RMSEA was 0.84, which was slightly higher and for PL → PU, and AR → PU, the regression weight showed a negative (-) value. Therefore, PL and AR did not have a statistically significant impact on PU (see Figure 5). It means that PL and AR do not directly influence PU.

Table 3

Comparison of the Good of Fit : Base model and Modified model (N=5922)

Figure 5

Base model

Model specification which found an optimized model by opening a path that could result in a different path was conducted for four paths where the relationship had been not clearly identified in previous studies; PL → PU, AR → PU, PL → EA, and AR → CA. As a result of the comparison of the Specification Search (Exploratory SEM), a highly optimized model with an excellent PRATIO was selected, which excluded PL → PU, AR → PU, and AR → CA (see Figure 6). The model fit index excluding the three path coefficients showed the CFI and TLI = above .90 which was a good fit (see Table 3: Modified model). The value for RMSEA (Barrett, 2007), which has recently become the most used index in model fit decisions was .082. It was higher than the recommended threshold value of .05, but according to Chen et al.(Chen, Curran, Bollen, Kirby, & Paxton, 2008), if the RMSEA is near .08, although it may not be a close-fit to the theoretical true model, it is a good-fit.

Figure 6

Modified Model

Research question 2 questioned if the emotional response to the first impression of the screen image might excel the perceived usability through mediating aesthetic judgment of the two dimensions of classical aesthetics and expressive aesthetics, and was validated by the path coefficients estimated through the study model because the model fit had been validated.

According to each parameter estimation value expressed in table 4, first, the regression weights for CA and EA in the prediction of PU were significantly different. The standardized total (direct and indirect) effect of CA on PU was .71. That was, due to both direct (unmediated) and indirect (mediated) effects of CA on PU, when CA went up by 1 standard deviation, perceived usability went up by .71 standard deviations. And the standardized total effect of EA on PU was .18.

Table 4

Parameter Estimation Values of [Modified Model] (N=5922)

Table 4

Direct and Indirect Effect of Key Variables

Second, the regression weight for PL in the prediction of CA and the regression weights for AR in the predictions of both CA and EA were significantly different. The standardized direct effect of PL on CA was .58. The standardized direct effect of PL on EA was .46. And the standardized direct effect of AR on EA was .36.

Third, due to the fact that the regression weights for PL and AR in the prediction of PU hadn't been significantly different in Base Model analysis results, no direct effect could have been found. But, in the Modified Model, the standardized indirect effect of PL on PU was .50. The mediated effect of PL having impact on PU through CA was .41, which was the multiplication of PL → CA path coefficient .58 with CA → PU path coefficient .71.

The standardized indirect effect of AR on PU was .06. The mediated effect of PL having impact on PU through EA was .09, which was the multiplication of PL → EA path coefficient .47 with EA → PU path coefficient .18. In addition, the mediated effect of AR having impact on PU through EA was .06, which was the multiplication of the AR → EA path coefficient .35 with EA → PU path coefficient .18.

The results of the examining were able to confirm the following;

Classical aesthetics and Expressive aesthetics increased with a higher level of Pleasure and Expressive aesthetics increased with a higher level of Arousal. In other words, Pleasure had an impact on Classical aesthetics and Expressive aesthetics, and Arousal only affected Expressive aesthetics. Among the many variables, only Classical aesthetics and Expressive aesthetics influenced Perceived usability, and Perceived usability increased as well with the higher level of Classical aesthetics and Expressive aesthetics. Classical aesthetics especially strongly influenced Perceived Usability.

In total, the following causal structures were identified through model fit validation. First, the User experience model in first impression formation, different from the general user experience models forming emotional responses regarding usability and aesthetics, showed that during first impression formation, very fast-forming emotional responses preceded Aesthetic judgment or Perceived usability. Second, the emotion formed at this time influences aesthetic Judgment. Pleasure influences Classical aesthetics and Expressive aesthetics, and Arousal influences Expressive aesthetics. Meanwhile, emotion did not directly impact Perceived usability. Third, Pleasure is not a variable that directly explains Perceived usability, but it was found that it impacts Perceived usability through mediated variables; Classical aesthetics and Expressive aesthetics. Classical aesthetics had high impact on Perceived usability, whereas Expressive aesthetics had little impact. However, Classical aesthetics did not mediate Arousal having an impact on Perceived usability, but only mediated Pleasure having an impact on Perceived usability. In contrast, Expressive aesthetics mediated both Pleasure and Arousal to have an impact on Perceived usability, but the impact was small.