Archives of Design ResearchArchives of Design Research

Home > Vol. 37, No. 1

Nonverbal Communication of Symptoms: Visualization and Cognition of Symptom Pictograms on User Comprehension
  • Qi Wang : Department of Human Environment & Design, Master’s student, Yonsei University, Seoul, Korea
  • Yesel Jun : Department of Human Environment & Design, PhD candidate, Yonsei University, Seoul, Korea
  • Hyunju Lee : Department of Human Environment & Design, Professor, Yonsei University, Seoul, Korea

Background The issues of language barrier involving medical services and thus directly relating to health concerns remain a challenge. Communication is particularly critical between pharmacists and visitors as judgments of symptoms mostly rely on the subjective descriptions given by the visitors. The nonverbal communication of pictograms, as a visual language, is a reliable form of symptom communication because it does not depend on users’ mother tongue and requires no prior learning experience. To this end, the present paper explores the visual and cognitive factors influencing users’ comprehension of symptom pictograms.

Methods The study was carried out in two stages of an observational study and a user survey. In the observational study, symptom pictograms currently being used were collected and classified by their visualization strategy through a Delphi process involving ten design experts. Then, symptom pictogram stimuli were developed as a reinterpretation of the sample. These stimuli were evaluated by 238 participants (104 American, 134 Chinese) from the general public through a user survey on the cognitive factors of semantic distance, complexity, concreteness, and familiarity. Chi-squared and logistic regression analyses are used to analyze the data.

Results Analyses revealed that user comprehension is generally higher for symptom pictograms with a mixed combination of both depictive and abstract motifs, whose relation with the referent symptom is arbitrary rather than direct or indirect, whose semantic distance is closer to the referent symptom, is less complex, is less concrete, and is more familiar.

Conclusions The perceptive cognition, more so than the visualization strategy, should be given particular attention when developing symptom pictograms. Their cognitive aspects should be not only beneficial to the overall comprehension but also well-balanced such that the synergy resulting from discreetly combining the advantages of being semantically close, less complex, less concrete, or more familiar surpasses the disadvantage that may result from the limitation of visualization strategy feasibility. Specific cultural differences are also discussed.

Keywords:
Cognition, Comprehension, Nonverbal Communication, Symptom Pictogram, Visualization.
pISSN: 1226-8046
eISSN: 2288-2987
Publisher: 한국디자인학회Publisher: Korean Society of Design Science
Received: 07 Feb, 2022
Revised: 04 Apr, 2022
Accepted: 04 Apr, 2022
Printed: 31, May, 2022
Volume: 35 Issue: 2
Page: 7 ~ 25
DOI: https://doi.org/10.15187/adr.2022.05.35.2.7
Corresponding Author: Hyunju Lee (hyunju@yonsei.ac.kr)
PDF Download:

Citation: Wang, Q., Jun, Y., & Lee, H. (2022). Nonverbal Communication of Symptoms: Visualization and Cognition of Symptom Pictograms on User Comprehension. Archives of Design Research, 35(2), 7-25.

Copyright : This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/3.0/), which permits unrestricted educational and non-commercial use, provided the original work is properly cited.

1. Introduction

The number of foreigners visiting Korea continues to rise and thus the issues occurring from language barriers remain as challenges to overcome (Ministry of Culture, 2018). The number, which slightly decreased in 2020 due to the COVID-19 pandemic, persistently increased from 2016 to 2019 with a yearly growth rate of 7.2%. Currently, the figure has exceeded 2 million, taking up 4% of the entire Korean population (Korea Immigration Statistical Yearbook, 2021). Among the obstacles foreigners encounter due to language barrier, those occurring at medical service sites require particular attention as they directly relate to health issues (Cho, 2021; Lee et al., 2011; Tian et al., 2021). Moreover, unlike hospitals, where medical practitioners objectively diagnose to determine the symptoms of patients, pharmacists, more often than not, must rely on the subjective descriptions made by pharmacy visitors. Hence, accurate and effective communication of symptoms is critical especially between pharmacists and visitors.

Cross-culturally consistent communication is necessary in order to minimize language barrier issues. The nonverbal communication of visual language minimally depends on users’ mother tongue. One particular form of such visual language, the pictograms, is an appropriate means to communicate symptoms at pharmacies as it is likely to be understood without prior learning experience regardless of users’ age or education level. However, careful considerations must be given in developing symptom pictograms as misinterpretations can cause confusions that lead to serious medicine misuse (Tijus et al., 2007).

Even medical journals publish visual representations of symptoms with no consideration as to the cognitive rules or criteria (Moriyama et al., 1994). Several attempts have been made to explore the nonverbal communications of medical symptoms (e.g., Bellamy et al., 2020; Moriyama et al., 1994; Richler et al., 2012; Tack et al., 2014; Zhao et al., 2017). However, exhaustive research to define the constituents of effective symptom pictograms are still lacking. Hence, in-depth research into the nonverbal communication of pictograms conveying medical symptoms is needed especially for those that are invisible to the naked eye and commonly experienced by pharmacy visitors. The present paper studies the cognition and understanding of symptom pictograms that are not exposed and solely experienced by symptom sufferers.

The purpose of the paper is to provide empirical evidence of the influence of visual and cognitive factors of symptom pictograms on users’ comprehension, thereby establishing a design guideline in developing universally effective symptom pictograms. To this end, the research questions of the current study are as follows.

RQ1. How are symptom pictograms currently designed and what are the characteristics of their visual representation strategies?

RQ2. What are the visual factors that chiefly influence symptom pictogram comprehension?

RQ3. What are the cognitive factors that chiefly influence symptom pictogram comprehension?

2. Literature review
2. 1. Symptoms

In a broad sense, symptoms include all health abnormalities occurring from illnesses and diseases. More specifically, they can be divided into symptoms and signs (The Great Encyclopedia of Nursing Science, 1996). Strictly speaking, symptoms and signs differ in that symptoms are the subjective experiences that cannot be identified by a third party and signs are the objective phenomena observable by a third party and diagnosed through medical examinations (Unified Medical Language System). For instance, a headache is a symptom whereas red spots is a sign. Because signs are manifested outward and thus observable, there exists a general understanding as to their physical look. However, symptoms cannot be observed, and so special considerations are required to accurately and effectively convey their meaning.

2. 2. Pictogram and semiotics

The word, pictogram, is a compound of “picto,” meaning picture, and “gram,” meaning message. It is a graphic symbol that visually represents objects, forms, and concepts, to convey meaning that the general public can easily understand. Accordingly, meaningful symbols are not born out of an individual’s intuition but require explorative research in order to optimize their effectiveness of meaning delivery. The grammar of symbols includes the means by and the conditions under which something becomes a symbol (Robin, 1968). The mechanism governing pictogram comprehension should thus signify the relation between the sign and users’ personal experience of the referent (Kolers, 1969).

Charles Sanders Peirce (1839-1914) proposed a triadic model of semiotics that includes the representamen, the object, and the interpretant (Figure 1). He argued that the representamen is not bound to be interpreted solely as the target object but that diverse interpretations can result depending on the interpretant. That is, the interpretant in Peirce’s model defines and mediates the relation between the representamen and the object.


Figure 1 Peirce’s triadic model
2. 3. Pictogram functions and evaluation

Fundamentally, pictograms’ main function is to visually deliver a meaning. Hence, the precondition of evaluating the effectiveness of a pictogram is that it is visually perceived. As pictograms are a form of visual language intended to convey meaning of the referent in the most effective, accurate, and quickest way, they must be accompanied by a general sharing of what it symbolizes regardless of the users’ language and prior knowledge. In this sense, pictograms are national and international systems of agreement and regulations (Kwon, 2009). The International Organization for Standardization established the level of sufficient effectiveness as above 66% of correct interpretation for a pictogram to be selected for use (ISO9186). Several research have attempted to define the standards of evaluating the effectiveness of pictograms in various circumstances as given in Table 1.

Table 1
Previous work on evaluation standards of pictograms

Author(s) Year Scope Measures of evaluation
Park et al. 2018 Pictograms on hospital websites Ability to prevent safety issues, legibility, visibility, universality, comprehensibility, consistency, interestingness, suitability
Wang & Park 2018 Pharmaceutical pictograms Accuracy, trendiness, visibility, formativeness, ability to attract, memorableness
Kim 2017 Pictograms in elderly medication education Universality, legibility, consistency, interestingness, suitability
Prada et al. 2016 Symbols Aesthetic appeal, familiarity, visual complexity, concreteness, valence, arousal, meaningfulness
Yu & Yoon 2015 Pictograms at retail spaces Intuitiveness, accuracy, consistency, harmony
Park 2014 Olympic pictograms Simplicity, realism, artistry, creativity, familiarity
Chan & Chan 2013 Pharmaceutical pictograms Familiarity, concreteness, complexity, meaningfulness, semantic distance
Nakamura & Zeng-Treitler 2012 Healthcare pictographs Lexical category, semantic category, representation strategy
Park 2012 Safety pictograms Universality, legibility, consistency, interestingness, suitability
McDougall & Isherwood 2009 Icons Icon concreteness, icon familiarity, visual complexity, semantic distance, name agreement, function concreteness, function familiarity, number of syllables
Ng & Chan 2008 Icons Color, shape, size, familiarity, concreteness, complexity, meaningfulness, semantic distance
Ng & Chan 2007 Traffic signs Familiarity, concreteness, simplicity, meaningfulness, semantic closeness
Min 2000 Pictograms on medicine packages Meaningfulness, consistency, usability
(conspicuousness, ability to attract, legibility)
McDougall et al. 1999 Symbols and icons Concreteness, complexity, meaningfulness, familiarity, semantic distance
Moriyama et al. 1994 Symptom symbols Comprehensibility

2. 4. Classification of symptom pictograms
2. 4. 1. Visualization type

Figurative visualization type

The referents symbolized by pictograms include natural objects, artifacts, phenomena, time, situation, and abstract concepts. There are largely two ways to figuratively visualize these referents. Namely, meanings can be expressed depictively, either implicitly or explicitly; or abstractly (Ota, 1987). The figurative visualization of symptom pictograms in the current study is classified into either depictive, abstract, or mixed, which combines both depictive and abstract motifs (Table 2).

Table 2
Figurative visualization types of pictograms

Type Depictive Abstract Mixed
Definition Represents a lexical (implicit) or a derivative (explicit) meaning A simplified form containing a part or parts of a concept A combination of depictively and abstractly visualized motifs
Example
(Park, 2010)
Exit
Medical office
Emergency call

H1a. User’s comprehension of symptom pictograms significantly differs by the type of figurative visualization.

Relational visualization type

Pictograms are communicated based on preestablished agreement and convention and so their visual representation should be evaluated not only on their figurative form but also on the relation with their targeted referent (Nakamura & Zeng-Treitler, 2012). There are three basic strategies of relational representation: a direct relation, in which the sign and its referent are visually similar; an indirect relation, in which the sign and its referent are semantically associated; and an arbitrary relation, in which the sign and its referent are connected by a social convention (Montagne, 2013). Among referents represented by pictograms, those that can be expressed directly or arbitrarily are relatively limited solely due to the nature of the referents. Hence, indirect visualization is fairly common in pictogram designs (Familant & Detweiler, 1993; Nakamura & Zeng-Treitler, 2012).

Peirce’s early classification of semantic associations include three categories of direct, implied, and arbitrary (Hartshorne & Weiss, 1932; Siné J.P. McDougall et al., 1999). Direct signs almost exactly imitate the physical look of the referent. Implied signs require a reasoning process and are represented using a part or parts of a concept that are necessary for interpretation. Arbitrary signs lack a natural association with its referent and cannot be interpreted intuitively. The resulting sign can be categorized into an icon, an index, or a symbol, depending on whether its semantic association is direct, implied, or arbitrary, respectively, and pictograms range over all three (Hwang, 2008). McDougall et al. (1999) argued that Peirce’s classification categories signify a continuum of semantic proximity such that icons represent the closest and symbols represent the farthest semantic distance.

Similarly, other studies have also classified signs according to their relationship with their referent. Park (2010), based on Peirce’s triadic model, categorized the semantic relation employed in safety signs into descriptive, metaphorical, and arbitrary. Nakamura & Zeng-Treitler (2012) used three evaluative axes of lexical, semantic, and representational categories to identify three basic representation strategies of visual similarity, semantic association, and arbitrary convention. The current study classifies the relational visualization into three categories of direct, indirect, and arbitrary (Table 3).

Table 3
Relational visualization types of pictograms

Type Direct Indirect Arbitrary
Definition The sign and its referent are visually similar The sign and its referent are semantically associated The sign and its referent are connected by a social convention
Example
(Nakamura & Zeng-Treitler, 2012)
Thermometer
Last
Female

H1b. Users’ comprehension of symptom pictograms significantly differs by the type of relational visualization.

2. 4. 2. Cognitive factors

Contrary to the visualization strategy of pictograms, which is deliberately selected and applied by designers, cognitive factors can only be measured by estimating users’ subjective perception. Many studies have attempted to quantify pictograms’ visual complexity, among others. Attneave (1957) devised a formula and Forsythe et al. (2003) proposed an image processing technique to calculate visual complexity. However, there are limits to efficient and effective complexity quantification and, more importantly, such method is not guaranteed to appropriately reflect users’ actual perception. So far, subjective measures remain to be the most common method to estimate the cognitive factors of pictograms. The four cognitive factors the current study covers are semantic distance, complexity, concreteness, and familiarity (Table 4).

Table 4
Cognitive factors of pictograms

Factor Semantic distance Complexity Concreteness Familiarity
Definition The closeness of the relationship between the sign and the referent The amount of detail or intricacy Symbols are concrete if they depict real objects and abstract, otherwise The frequency with which symbols had been encountered
Rating Not closely related Very strongly related Very simple Very complex Definitely abstract Definitely concrete Very unfamiliar Very familiar
Example
(McDougall et al., 1999)
Archive
Break glass to access
First aid
Desk set
Cloth track
steaming
Tape
cassette
Convert
multiple
files
First aid

Semantic distance

Signs that are semantically closer to their referent have been found to show a higher comprehension rate (Goonetilleke et al., 2001; S. McDougall & Isherwood, 2009; Siné J. P. McDougall et al., 2001)

H2a. The semantic distance between pictograms and their referent symptom influences users’ comprehension.

Complexity

The visual complexity of signs is commonly evaluated by the number of points, lines, and planes making up the sign. Although visually complex signs contain more details and thus provide users with more information to be utilized in interpretation (Biederman, 1987; McDougald & Wogalter, 2014), they may also require that much more time to cognitively process (Alario et al., 2004; Byrne, 1993). Such makes visual complexity undesirable for searching tasks with limited time but desirable for identification tasks, where accurate comprehension is of highest priority (Lloyd-Jones & Nettlemill, 2007).

H2b. The complexity of symptom pictograms influences users’ comprehension.

Concreteness

Concrete signs are highly associated with their referent in its visual depiction, whereas abstract signs commonly involve geometric shapes. (Siné J.P. McDougall et al., 1999). In general, it has been found that concrete signs are likely to be interpreted more efficiently, more accurately, and more swiftly (Arend et al., 1987; Sine J.P. McDougall et al., 2000; Passini et al., 2008; Roger & Oborne, 1987; Stotts, 1998). It is important to note, however, that personal differences could exist in what is considered concrete and what is considered abstract because even if a sign symbolizes a real object, users may not perceive it as concrete if its form does not correspond to their own conception (Siné J.P. McDougall et al., 1999). Also, if the visualization of a concrete sign and its referent are related indirectly, the concreteness could rather disturb the interpretation process (S. Min, 2014).

H2c. The concreteness of symptom pictograms influences users’ comprehension.

Familiarity

Perceived familiarity of signs includes the case of accumulated experience resulting from users’ repeated exposure to the sign itself and the case in which users are unfamiliar with the implied meaning of the sign but rather familiar with the physical form of the targeted object. Users’ prior experience in encountering the signified object or the sign itself positively affects interpretation (Siné J. P. McDougall et al., 2001). Higher frequency of prior experience yields greater accessibility to the semantic representation as part of users’ long-term memory (Hancock et al., 2004).

H2d. The familiarity of symptom pictograms influences users’ comprehension.

3. Observational study

The purpose of the observational study is to collect symptom pictograms currently being used at pharmacies in Korea and to deduce their visualization pattern through a Delphi process.

The scope of data collection includes the Korea Tourism Orgnization, the Korean Pharmaceutical Association, pharmaceutical associations and their branches of district subdivisions with high percentages of foreign visitors, large-scale pharmacy franchises, and private pharmacies located at major tourist sites introduced in the Korea’s official travel guidebook. In result, the Korea Tourism Organization, the Seoul Pharmaceutical Association of Gangnam District, and the largest franchised pharmacy in Korea, the Onnuri Health & Communications, are found to provide symptom pictograms for foreigners. Among the symptoms covered by these three organizations, 27 are commonly included in all three, 13 of which are subjective symptoms and 14 are objective signs. The current study covers only the 13 symptoms since observable signs can directly be represented according to its visual form. Hence the original sample of symptom pictograms for which design experts are invited to classify their visualization type are 39.

A total of ten experts participated in the Delphi process (Table 5). Their years of experience in the design field vary from 9 to 20. The process took place from the 18th to the 24th of November 2021 in two stages.

Table 5
Demographic characteristics of the expert group

Participant Sex Age Academic degree Years of experience in design field
Expert 1 Female 40 to 49 PhD 9
Expert 2 Female 30 to 39 Master’s 13
Expert 3 Female 40 to 49 PhD 20
Expert 4 Female 40 to 49 Master’s 18
Expert 5 Female 30 to 39 Master’s 9
Expert 6 Female 40 to 49 PhD 11
Expert 7 Female 40 to 49 PhD 20
Expert 8 Female 40 to 49 PhD 15
Expert 9 Female 30 to 39 Master’s 9
Expert 10 Male 30 to 39 Master’s 12

In the first round of the Delphi process, the experts evaluated and classified all 39 symptom pictograms into their figurative and relational visualization types. A detailed description of the research background and each visualization type were provided ahead. All pictograms were instructed to be evaluated only on the representation of the symptom itself, and not the representation of the body or the body part that signifies the location of the symptom as the human body is always represented depictively and directly. A second round was executed to resolve on those pictograms whose classification was not agreed upon in the first round. The visualization types of all 39 symptom pictograms were classified within two Delphi rounds (Table 6).

Table 6
Visualization classifications deduced by Delphi method

Symptom Visualization type
Figurative Relational
Depictive Abstract Mixed Direct Indirect Arbitrary
Sore throat A ● ●
Sore throat B ● ●
Sore throat C ● ●
Fever A ● ●
Fever B ● ●
Fever C ● ●
Headache A ● ●
Headache B ● ●
Headache C ● ●
Toothache A ● ●
Toothache B ● ●
Toothache C ● ●
Menstrual cramp A ● ●
Menstrual cramp B ● ●
Menstrual cramp C ● ●
Indigestion A ● ●
Indigestion B ● ●
Indigestion C ● ●
Heartburn A ● ●
Heartburn B ● ●
Heartburn C ● ●
Gastritis A ● ●
Gastritis B ● ●
Gastritis C ● ●
Constipation A ● ●
Constipation B ● ●
Constipation C ● ●
Muscular pain A ● ●
Muscular pain B ● ●
Muscular pain C ● ●
Arthritis A ● ●
Arthritis B ● ●
Arthritis C ● ●
Insomnia A ● ●
Insomnia B ● ●
Insomnia C ● ●
Dry eye A ● ●
Dry eye B ● ●
Dry eye C ● ●

4. Main study

Among the 39 symptom pictograms evaluated by Delphi method, the symptoms for which only one type of visualization was observed were eliminated. Hence, the main study includes ten symptoms for which two different visualization types were found. Study stimuli were developed by reinterpreting the visualization strategies employed by the collected sample and conforming to the Standard Guideline of Graphic Symbol Development (Korean Agency for Technology and Standards of the Ministry of Trade, 2016). All pictograms were developed as rectangles with rounded corners using a grey scale to control the color factor. They were uniformly designed with a black background and, referring to the collected sample, non-colored motifs were colored white and colored motifs were colored grey. As the white spaces between motifs within a pictogram may influence users’ perception, the location and size of motifs were designed according to the commonality observed from the collected sample. Two pictograms of differing visualization types were developed for the ten selected symptoms, resulting in a total of 20 stimuli.

The survey was executed by a professional research agency between the 26th of November and the 1st of December 2021. Two groups of subjects were recruited for the survey in order to observe possible cultural differences: the Americans, who were identified as the largest national group of Western foreigners residing in Korea; and the Chinese, who were identified as the largest national group of Eastern foreigners residing in Korea (Korea Immigration Service Statistics, 2020). First, the participants used 7-point Likert scales to evaluate the 20 stimuli on their complexity (“1=Very simple” to “7=Very complex”), concreteness (“1=Definitely abstract” to “7=Definitely concrete”), and familiarity (“1=Very unfamiliar” to “7=Very familiar”). Then the participants guessed the meaning (the target symptom) of the stimuli either by choosing from a list of options that include two to three symptoms that occur in the same body part as the given stimuli or by directly writing down a symptom in case no option is deemed appropriate. Lastly, the participants were told the actual symptom the given stimuli symbolizes and to evaluate the semantic distance between the symptom and the given stimuli, also using a 7-point Likert scale of “1=Not closely related” to “7=Very strongly related”. The scale labels were adopted from the study by McDougall et al. (1999). Data were analyzed using SPSS Statistics 25.0.

A total of 238 participated in the study, among which 104 were of American nationality and 134 were of Chinese nationality (Table 7). The average Cronbach’s alpha of the survey items was found to be above 0.9.

Table 7
Demographic characteristics

American
N(%)		 Chinese
N(%)		 Total
N(%)
Sex Male 58(24.4) 81(34.0) 139(58.4)
Female 46(19.3) 53(22.3) 99(41.6)
Age Under 20 17(7.1) 6(2.5) 23(9.7)
20 to 29 22(9.2) 44(18.5) 66(27.7)
30 to 39 21(8.8) 44(18.5) 65(27.3)
40 to 49 24(10.1) 21(8.8) 45(18.9)
50 and above 20(8.4) 19(8.0) 39(16.4)
Total 104(43.7) 134(56.3) 238(100)

5. Result
5. 1. Main result
5. 1. 1. Visualization type

On symptom pictogram comprehension, chi-squared tests revealed no statistically significant difference among the figurative visualization types but a statistically significant difference among relational visualization types (χ2=20.961, df=2, p<.001). Further analyses revealed that there exist two groups of distinct relational visualization types in symptom pictograms, namely, one group of direct and indirect relations and one group of arbitrary relation. Specifically, users’ comprehension is higher for arbitrarily related symptom pictograms than for directly or indirectly related symptom pictograms (Table 8; Figure 2).

Table 8
Chi-squared analyses of figurative and relational visualization types on symptom pictogram comprehension

Figurative Relational
Depictive
N(%)		 Abstract
N(%)		 Mixed
N(%)		 Direct
N(%)		 Indirect
N(%)		 Arbitrary
N(%)
Comprehended 1,191(71.5) 996(69.7) 1,212(72.7) 1,006(70.4) 2,192(70.8) 201(84.5)
Uncomprehended 475(23.5) 432(30.3) 454(27.3) 422(29.6) 902(29.2) 37(15.5)
Total 1,666(100.0) 1,428(100.0) 1,666(100.0) 1,428(100.0 3,094(100.0) 238(100.0)
χ2=3.400, df=2, p=.183 χ2=20.961, df=2, p=.000


Figure 2 Symptom pictogram comprehension by relational visualization type
5. 1. 2. Cognitive factors

Logistic regression analyses were carried out for cognitive factors on symptom pictogram comprehension. In result, symptom pictogram comprehension was found to increase as the semantic distance becomes closer (B=.492, Wald=487.045, p<.001), as the visual complexity decreases (B=-.069, Wald=16.276, p<.001), as the concreteness decreases (B=-.088, Wald=23.501, p<.001), and as the familiarity increases (B=.217, Wald=118.836, p<.001). That is, users’ comprehension is higher for semantically closer, visually less complex, less concrete, and more familiar symptom pictograms (Table 9).

Table 9
Logistic regression analyses of cognitive factors on symptom pictogram comprehension

B Wald OR p
Semantic distance .492 487.045 1.635 .000
Complexity -.069 16.276 .933 .000
Concreteness -.088 23.501 1.092 .000
Familiarity .217 118.836 1.242 .000

5. 2. Result by nationality

Overall, a logistic regression analysis revealed that American respondents showed a greater symptom pictogram comprehension rate than Chinese respondents (B=.821, Wald=143.836, p<.001). The odds ratio (OR) of this association is found to be 2.272. That is, compared to Chinese, American respondents showed 227.2% greater comprehension of symptom pictograms (Table 10).

Table 10
Logistic regression analysis of nationality on symptom pictogram comprehension

B Wald OR p
Nationality
(Chinese vs. American)		 .821 143.836 2.272 .000

5. 2. 1. Visualization type

For American respondents, the figurative visualization type was found to significantly influence symptom pictogram comprehension (χ2=14.685, df=2, p<.001). Further analyses revealed that there exist two groups of distinct figurative visualization types, namely, one group of depictive and abstract types and one group of mixed type. Specifically, Americans’ comprehension is higher for mixed visualized symptom pictograms than for depictively or abstractly visualized symptom pictograms. No statistically significant difference among figurative visualization types was found for Chinese respondents (Table 11; Figure 3).

Table 11
Chi-squared analyses of figurative visualization type on symptom pictogram comprehension by nationality

Depictive
N(%)		 Abstract
N(%)		 Mixed
N(%)
American Comprehended 560(76.9) 496(79.5) 617(84.8)
Uncomprehended 168(23.1) 128(20.5) 111(15.2)
Total 728(100.0) 624(100.0) 728(100.0)
χ2=14.685, df=2, p=.001
Chinese Comprehended 631(67.3) 500(62.2) 595(63.4)
Uncomprehended 307(32.7) 304(37.8) 343(36.6)
Total 938(100.0) 804(100.0) 938(100.0)
χ2=5.469, df=2, p=.065


Figure 3 Symptom pictogram comprehension by relational visualization type of American respondents

On the influence of relational visualization type on symptom pictogram comprehension, a statistically significant difference was found for American respondents (χ2=22.446, df=2, p<.001). Further analyses revealed that all three relational types represent statistically distinct groups. Specifically, Americans’ comprehension is highest for arbitrarily related, followed by indirectly related, then by directly related symptom pictograms (Table 12; Figure 4).

Table 12
Chi-squared analyses of relational visualization type on symptom pictogram comprehension by nationality

Direct
N(%)		 Indirect
N(%)		 Arbitrary
N(%)
American Comprehended 389(74.8) 1,187(81.5) 97(93.3)
Uncomprehended 131(25.2) 269(18.5) 7(6.7)
Total 520(100.0) 1,456(100.0) 104(100.0)
χ2=22.446, df=2, p=.000
Chinese Comprehended 416(62.1) 1,206(64.3) 104(77.6)
Uncomprehended 254(37.9) 670(35.7) 30(22.4)
Total 670(100.0) 1,876(100.0) 134(100.0)
χ2=11.774, df=2, p=.003


Figure 4 Symptom pictogram comprehension of relational visualization type by nationality

For Chinese respondents, the relational visualization type was also found to be statistically significant on symptom pictogram comprehension (χ2=11.774, df=2, p<.01). Further analyses revealed that there exist two groups of distinct relational visualization types, namely, one group of direct and indirect relations and one group of arbitrary relation. Specifically, Chinese’ symptom pictogram comprehension is higher for arbitrarily related symptom pictograms than for directly or indirectly related symptom pictograms (Table 12; Figure 4).

5. 2. 2. Cognitive factors

For American respondents, logistic regression analyses on the cognitive factors revealed that symptom pictogram comprehension increases as the semantic distance becomes closer (B=.399, Wald=70.434, p<.001), as the complexity increases (B=.121, Wald=13.538, p<.001), as the concreteness increases (B=.121, Wald=12.339, p<.001), and as the familiarity increases (B=.096, Wald=4.210, p<.05). That is, Americans’ comprehension is higher for semantically closer, visually more complex, more concrete, and more familiar symptom pictograms (Table 13).

Table 13
Logistic regression analyses of cognitive factors on symptom pictogram comprehension by nationality

B Wald OR p
American Semantic distance .399 70.434 1.491 .000
Complexity .121 13.538 1.128 .000
Concreteness .121 12.339 1.128 .000
Familiarity .096 4.210 1.101 .040
Chinese Semantic distance .529 290.314 1.697 .000
Complexity -.111 17.575 .895 .000
Concreteness -.195 34.548 .823 .000
Familiarity .048 2.248 1.049 .134

For Chinese respondents, symptom pictogram comprehension was found to increase as the semantic distance becomes closer (B=.529, Wald=290.314, p<.001), as the complexity decreases (B=-.111, Wald=17.575, p<.001), and as the concreteness decreases (B=-.195, Wald=34.548, p<.001). No statistically significant influence was found for familiarity for Chinese respondents. That is, Chinese’ comprehension is higher for semantically closer, visually less complex, and less concrete symptom pictograms (Table 13).

5. 3. Other result

For the entire respondent sample, the cases where comprehension most sharply differed between two pictograms symbolizing the same symptom were the cases of toothache (Toothache A: 81.9%; Toothache B: 62.6%) and gastritis (Gastritis A: 31.9%; Gastritis B: 69.3%). For the case of toothache, Toothache B with a substantially lower comprehensibility has mixed figurative visualization type and its semantic distance with its referent is closer. However, Toothache A is less visually complex and less concrete (Table 14). This implies that having a visualization type that generally increases comprehension does not guarantee higher overall comprehension. Rather, the benefit arising from a combination of desirable cognitive factors may override the benefit of an advantageous visualization strategy. Further, for the case of gastritis, Gastritis B, with over twice the comprehensibility than Gastritis A, is of a relational visualization type that generally shows higher comprehension rate and, cognitively, is semantically closer and less complex (Table 14). This demonstrates that a synergic effect can occur from combining beneficial visual and cognitive factors.

Table 14
Results on cases of toothache and gastritis

Visualization type Cognitive factor(M)
Figurative Relational
Depictive Abstract Mixed Direct Indirect Arbitrary Semantic distance Complexity Concreteness Familiarity
Toothache A 81.9% ● ● 5.82 2.82 5.29 5.52
Toothache B 62.6% ● ● 6.00 3.23 5.46 5.52
Gastritis A 31.9% ● ● 4.89 3.21 5.23 5.32
Gastritis B 69.3% ● ● 5.44 4.12 4.81 5.08

6. Discussion

On the influence of figurative visualization type on symptom pictogram comprehension, no statistical significance was found for the entire respondent sample. However, for American respondents, who generally showed a higher comprehension rate than Chinese respondents, comprehension was found to be higher for mixed figurative type than for depictive or abstract types. This suggests that in developing symptom pictograms, rather than making motifs all depictive or all abstract, it is more beneficial and effective to depictively represent the motifs that are more natural to be represented depictively and abstractly the motifs that are more natural to be represented abstractly. That is, combining motifs that are expressed in its own ideal form benefits the effectiveness of the overall pictogram.

On the relational visualization type, symptom pictogram comprehension is found to be higher for arbitrarily related visualization than for directly or indirectly related visualizations. Because symptoms, as opposed to signs, are subjective experiences unobservable by the naked eye, it is highly doubtful that the general public will share a clear understanding of its physical form. Hence, it is plausible that symptom pictograms can generally be expected to show a higher comprehension rate when they are arbitrarily related to the referent symptom.

On the cognitive factors of symptom pictograms, users generally showed a higher comprehension rate for pictograms whose semantic distance is closer to their referent and that are visually less complex, less concrete, and more familiar. The results on visual complexity and concreteness showed an opposite direction than anticipated from literature review, in which interpretation tend to be higher for more complex and more concrete signs. This may also be due to the fact that there exists no general understanding as to exactly what symptoms look like. That is, simple abstract representations may deliver subjective feelings and sensations more effectively than complex concrete representations.

Lastly, it is worthy to note that the differences found between American and Chinese respondents imply that there may exist modes of visual communication that are specific to or more effective in a certain culture. Specifically, the most conflicting cultural differences were found in the analyses on visual complexity and concreteness. American respondents were found to better comprehend more complex and more concrete pictograms whereas Chinese respondents were found to better comprehend less complex and less concrete pictograms. However, these results should be taken into account with an exceptional caution as they may reflect cultural gaps but may also well reflect the differences in perceptive sensitivity as shown in one study that compared the responses of American and Chinese on pictogram evaluation and found that Americans generally give higher ratings of concreteness than do Chinese (Yong, 2012). At all events, considering the universal validity and value of pictograms as a form of visual language, symptom pictogram development merit particular considerations that embrace the results of both cultures.

7. Conclusion

In developing symptom pictograms, designers are advised to give a particularly careful consideration on the cognitive factors. Depending on the nature of the symptom, the visualization type, whether figurative or relational, can be limited due to its feasibility and the low likelihood that the general public will have an accurate understanding of the physical look of the symptom. Thus, it is advised to develop as many different pictograms as possible within the feasible visualization strategies, and to make final selections based on a comprehension evaluation of their cognitive factors by the targeted audience.

Additionally, as detailed analyses revealed on cases where two different pictograms sharply differed in comprehension for the same symptom, having a semantically closer, less visually complex, less concrete, and more familiar form does not absolutely guarantee higher overall comprehension. Rather, it is more important to balance both visual and cognitive factors appropriately considering the nature of the symptom and the circumstances. A well-balanced combination of advantageous visual or cognitive factors can synergistically benefit the overall comprehension despite a few disadvantageous factors.

The current study is limited in its scope in that it includes only those symptom pictograms currently being used at pharmacies in Korea. Expanding the research to symptom pictograms used in diverse countries and circumstances are needed. Also, more detailed research exploring areas untouched in the current paper such as color and the reversal of positive and negative spaces will further deepen our understanding of what constitutes an effective symptom pictogram.

References
  1. 1 . Alario, F. X., Ferrand, L., Laganaro, M., New, B., Frauenfelder, U. H., & Segui, J. (2004). Predictors of picture naming speed. Behavior Research Methods, Instruments, and Computers, 36(1), 140-155. [https://doi.org/10.3758/BF03195559]
  2. 2 . Arend, U., Muthig, K.-P., & Wandmacher, J. (1987). Evidence for global feature superiority in menu selection by icons. Behaviour & Information Technology, 6(4), 411-426. [https://doi.org/10.1080/01449298708901853]
  3. 3 . Attneave, F. (1957). Physical determinants of the judged complexity of shapes. Journal of Experimental Psychology: General, 53(4), 221-227.
  4. 4 . Bellamy, K., Dowse, R., Ostini, R., Martini, N., & Kairuz, T. (2020). Preferences of resettled refugees on pictograms describing common symptoms of illness. Journal of Immigrant and Minority Health, 22(1), 216-219. [https://doi.org/10.1007/s10903-019-00908-3]
  5. 5 . Biederman, I. (1987). Recognition-by-components: A theory of human image understanding. Psychological Review, 94(2), 115-147. [https://doi.org/10.1037/0033-295X.94.2.115]
  6. 6 . Byrne, M. D. (1993). Using icons to find documents. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems - CHI' 93, June, 446-453. [https://doi.org/10.1145/169059.169369]
  7. 7 . Chan, A. H. S., & Chan, K. W. L. (2013). Effects of prospective-user factors and sign design features on guessability of pharmaceutical pictograms. Patient Education and Counseling, 90(2), 268-275. [https://doi.org/10.1016/j.pec.2012.10.009]
  8. 8 . Cho, H. (2021). Pharmacy utilization and medication use among immigrants in South Korea. Gachon University.
  9. 9 . Familant, E. E., & Detweiler, M. C. (1993). Iconic reference: evolving perspectives and an organizing framework. Internation Journal of Man-Machine Studies, 39(705-728). [https://doi.org/10.1006/imms.1993.1080]
  10. 10 . Forsythe, A., Sheehy, N., & Sawey, M. (2003). Measuring icon complexity: An automated analysis. Behavior Research Methods, Instruments, and Computers, 35(2), 334-342. [https://doi.org/10.3758/BF03202562]
  11. 11 . Goonetilleke, R. S., Shih, H. M., On, H. K., & Fritsch, J. (2001). Effects of training and representational characteristics in icon design. International Journal of Human Computer Studies, 55(5), 741-760. [https://doi.org/10.1006/ijhc.2001.0501]
  12. 12 . Hancock, H. E., Rogers, W. A., Schroeder, D., & Fisk, A. D. (2004). Safety symbol comprehension: Effects of symbol type, familiarity, and age. Human Factors, 46(2), 183-195. [https://doi.org/10.1518/hfes.46.2.183.37344]
  13. 13 . Hartshorne, C., & Weiss, P. (Eds.). (1932). Collected Papers of Charles Sanders Peirce: Volumes I-VI. Harvard University Press.
  14. 14 . Hwang, S. (2008). A study on the form and communication of pictograms as a visual language. Journal of Basic Design & Art, 9(2), 937-947.
  15. 15 . Kim, K. H. (2017). Use of pictogram in elderly medication education. The Treatise on The Plastic Media, 20(4), 20-28.
  16. 16 . Kolers, P. A. (1969). Some formal characteristics of pictograms. American Scientist, 57(3), 348-363.
  17. 17 . Korea Immigration Service Statistics. (2020).
  18. 18 . Korea Immigration Statistical Yearbook. (2021).
  19. 19 . Korean Agency for Technology and Standards of the Ministry of Trade, I. and E. (2016). Standard Guideline of Graphic Symbol Development.
  20. 20 . Kwon, Y. J. (2009). A study about the aspect of the pictogram in various culture. Kyungpook National University.
  21. 21 . Lee, B. S., Oh, I. K., & Lee, E. M. (2011). Foreign residents' experience of Korean health care services. Journal of Korean Academy of Nursing Administration, 17(2), 226-237. [https://doi.org/10.11111/jkana.2011.17.2.226]
  22. 22 . Lloyd-Jones, T. J., & Nettlemill, M. (2007). Sources of error in picture naming under time pressure. Memory and Cognition, 35(4), 816-836. [https://doi.org/10.3758/BF03193317]
  23. 23 . McDougald, B. R., & Wogalter, M. S. (2014). Facilitating pictorial comprehension with color highlighting. Applied Ergonomics, 45(5), 1285-1290. [https://doi.org/10.1016/j.apergo.2013.05.008]
  24. 24 . McDougall, S., & Isherwood, S. (2009). What's in a name? The role of graphics, functions, and their interrelationships in icon identification. Behavior Research Methods, 41(2), 325-336. [https://doi.org/10.3758/BRM.41.2.325]
  25. 25 . McDougall, Siné J. P., Curry, M. B., & de Bruijn, O. (2001). The effects of visual information on users' mental models: An evaluation of pathfinder analysis as a measure of icon usability. International Journal of Cognitive Ergonomics, 5(1), 59-84. [https://doi.org/10.1207/S15327566IJCE0501_4]
  26. 26 . McDougall, Siné J. P., Curry, M. B., & De Bruijn, O. (1999). Measuring symbol and icon characteristics: Norms for concreteness, complexity, meaningfulness, familiarity, and semantic distance for 239 symbols. Behavior Research Methods, Instruments, and Computers, 31(3), 487-519. [https://doi.org/10.3758/BF03200730]
  27. 27 . McDougall, Sine J. P., De Bruijn, O., & Curry, M. B. (2000). Exploring the effects of icon characteristics on user performance: The role of icon concreteness, complexity, and distinctiveness. Journal of Experimental Psychology: Applied, 6(4), 291-306. [https://doi.org/10.1037/1076-898X.6.4.291]
  28. 28 . Min, H. N. (2000). Research on communication function of medicines package design: Focused on pictogram usage. Journal of Communication Design, 5, 140-153.
  29. 29 . Min, S. (2014). The evaluation of cognitive usability of Daegu public pictograms. Journal of Digital Design, 14(4), 545-554.
  30. 30 . Ministry of Culture, S. and T. (2018). International Visitor Survey.
  31. 31 . Montagne, M. (2013). Pharmaceutical pictograms: A model for development and testing for comprehension and utility. Research in Social and Administrative Pharmacy, 9(5), 609-620. [https://doi.org/10.1016/j.sapharm.2013.04.003]
  32. 32 . Moriyama, M., Harnisch, D. L., & Matsubara, S. (1994). The development of graphic symbols for medical symptoms to facilitate communication between health care providers and receivers. Tohoku Journal of Experimental Medicine, 174, 387-398. [https://doi.org/10.1620/tjem.174.387]
  33. 33 . Nakamura, C., & Zeng-Treitler, Q. (2012). A taxonomy of representation strategies in iconic communication. International Journal of Human Computer Studies, 70(8), 535-551. [https://doi.org/10.1016/j.ijhcs.2012.02.009]
  34. 34 . Ng, A. W. Y., & Chan, A. H. S. (2007). The guessability of traffic signs: Effects of prospective-user factors and sign design features. Accident Analysis and Prevention, 39(6), 1245-1257. [https://doi.org/10.1016/j.aap.2007.03.018]
  35. 35 . Ng, A. W. Y., & Chan, A. H. S. (2008). Visual and cognitive features on icon effectiveness. Proceedings of the International MultiConference of Engineers and Computer Scientists, 2, 19-21.
  36. 36 . Ota, Y. (1987). Pictogram Design. Kimoondang.
  37. 37 . Park, J. (2010). An empirical study on the semiotic taxonomy of ISO graphical symbols focused on the comprehensibility of a safety sign in symbolic representation. Archives of Design Research, 23(3), 105-114.
  38. 38 . Park, J., Kim, J., & Lee, H. (2018). Production principles of pictogram used on hospital websites. Journal of Korea Design Forum, 23(1), 123-133. [https://doi.org/10.21326/ksdt.2018.23.1.011]
  39. 39 . Park, S.-S. (2012). A study on the improvement of information designs in a passenger carriage to cope with the negligent accidents of subway: On the target of Busan subway. Kyunsung University.
  40. 40 . Park, S. Y. (2014). Study on the recognition and preference of Olympic pictogram design. Journal of Digital Design, 14(4), 687-696.
  41. 41 . Passini, S., Strazzari, F., & Borghi, A. (2008). Icon-function relationship in toolbar icons. Displays, 29(5), 521-525. [https://doi.org/10.1016/j.displa.2008.07.001]
  42. 42 . Prada, M., Rodrigues, D., Silva, R. R., & Garrido, M. V. (2016). Lisbon Symbol Database (LSD): Subjective norms for 600 symbols. Behavior Research Methods, 48(4), 1370-1382. [https://doi.org/10.3758/s13428-015-0643-7]
  43. 43 . Richler, M., Vaillancourt, R., Celetti, S. J., Besançon, L., Arun, K., & Sebastien, F. (2012). The use of pictograms to convey health information regarding side effects and/or indications of medications. Journal of Communication in Healthcare, 5(4), 220-226. [https://doi.org/10.1179/1753807612Y.0000000012]
  44. 44 . Robin, R. S. (1968). Annotated Catalogue of the Papers of Charles S. Pierce. University of Massachusetts Press.
  45. 45 . Roger, Y., & Oborne, D. J. (1987). Pictorial communication of abstract verbs in relation to human-computer interaction. British Journal of Psychology, 78, 99-112. [https://doi.org/10.1111/j.2044-8295.1987.tb02229.x]
  46. 46 . Stotts, D. B. (1998). The usefulness of icons on the computer interface: Effect of graphical abstraction and functional representation on experienced and novice users. Proceedings of the Human Factors and Ergonomics Society 42nd Annual Meeting, 453-457. [https://doi.org/10.1177/154193129804200502]
  47. 47 . Tack, J., Carbone, F., Holvoet, L., Vanheel, H., Vanuytsel, T., & Vandenberghe, A. (2014). The use of pictograms improves symptom evaluation by patients with functional dyspepsia. Alimentary Pharmacology and Therapeutics, 40(5), 523-530. [https://doi.org/10.1111/apt.12855]
  48. 48 . The Great Encyclopedia of Nursing Science. (1996). Korean Academy of Nursing.
  49. 49 . Tian, Y., Yoon, H., & Choi, M. (2021). Quality of medical service and satisfaction perceived by foreign patients: A systematic review. Health and Social Welfare Review, 41(3), 177-194.
  50. 50 . Tijus, C., Barcenilla, J., de Lavalette, B. C., & Meunier, J. G. (2007). The design, understanding and usage of pictograms. Studies in Writing, 21, 17-31. [https://doi.org/10.1163/9789004253254_003]
  51. 51 . Unified Medical Language System(UMLS). (n.d.). National Library of Medicine. umlsks.nlm.nih.gov.
  52. 52 . Wang, J., & Park, J. S. (2018). A study on the design direction for effective communication of pharmaceutical Pictogram: Focused on Chinese user. Journal of Basic Design & Art, 19(6), 403-416. [https://doi.org/10.47294/KSBDA.19.6.28]
  53. 53 . Yong, G. (2012). Research on graphic symbols design recognition. Zhejiang University.
  54. 54 . Yu, D. W., & Yoon, J. E. (2015). A study of pictograms is retail spaces with a semiotics approach. Bulletin of Korean Society of Basic Design & Art, 16(6), 321-335.
  55. 55 . Zhao, W., Jin, H., Zhang, L., Wang, B., Sun, F., Jiao, H., Sun, C., & Wang, B. (2017). Pictograms to provide a better understanding of gastroesophageal reflux symptoms in Chinese subjects. Gastroenterology Research and Practice, 2017, 1-7. [https://doi.org/10.1155/2017/1214584]
 
 

Korean Society of Design Science

314-8, Korea Design Center 344-1 Yatap 1-dong, Bundang-gu Seongnam City, Gyeongi-do, 463-828, Korea

Tel. +82 (0)31 781 5021, 5022 | Fax. +82(0)31 781 5023

e-mail: ksds@design-science.or.kr