Archives of Design Research

Copyright : This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/3.0/), which permits unrestricted educational and non-commercial use, provided the original work is properly cited.

As artificial intelligence (AI)-based voice agents are developed, user expectations of the agent role are growing from performing voice commands to having conversations (Luger, & Sellen, 2016). Alongside leading companies, such as Google, Apple, Amazon, and Microsoft, Korean companies, SK Telecom, Samsung Electronics, KT, and Kakao, have developed voice agents and entered the market. As the types of voice agents have begun to vary, the scope of functions that can be employed is also expanding. Previously, simple functions, such as playing music, ordering products, and searching for information, have been popular, but more recently, functions have expanded to include reading fairy tales (Xu, Y., & Warschauer, 2019), learning foreign languages (Pastrovich, 2017), or even booking a restaurant reservation (Vincent, 2021). The popularization of voice agents has raised questions about the gender of the agents. Most of the currently commercialized voice agents were initially developed with a female voice as the default. Table 1 indicates that representative global agents, such as Amazon’s Alexa, Apple’s Siri, and Microsoft’s Cortana, and Korean agents, SK Telecom’s Aria, Samsung Electronics’ Bixby, and Naver’s Sally for instance, were all developed with a female voice and even named after women (Lee, 2018). For example, Siri’s name is a Scandinavian female name that means “beautiful woman who leads you to victory” (Abercrombie, Curry, Pandya, & Rieser, 2021).

If a voice agent is designed as female, it can lead to gender stereotyping (Eagly, & Wood, 1999) because the description of the company’s voice agent includes the concept of a digital assistant, which enforces the gender stereotype of an assistant. In other words, the role of an office assistant has historically been undertaken by women and thus the labor is gendered female. Due to the gendering of the labor as female, having a digital assistant who sounds normatively like a woman can be problematic. Thus, voice agent developers adhere to existing gender stereotypes or even produce new stereotypes when designing agents. For instance, a previous study revealed that the use of female-voice agents induces both short- and longterm gender discrimination such as expecting more quick response to female subordinates compared to males (Weisman & Janardhanan, 2020). Also, the study of United Nations reports that female-voice agents can reinforce prevalent harmful gender stereotypes (West, Kraut, & Chew, 2019). The study describes that voice agents induce gender discrimination by perceiving male voices as authoritative while females are helpful. Moreover, the authors discovered that voice agents with female voices are exposed to sexual harassment. Indeed, an AI chatbot designed as a twenty-year-old female character from South Korea called ‘Lee Luda’ was dismissed because of sexual harassment messages from users (Kim, 2021).

Considering the previous study, the biased design of voice agents based on gender stereotypes can cause discrimination. In this regard, Apple announced that users would be able to select the type of voice for their Siri when setting up their phones (Fournier-Tombs, 2021). Similarly, Samsung provides various voice style for voice agent Bixby to allow users set up their own Bixby (Samsung, 2023). On the other hand, Google does not support users to choose the voice of Google Assistant (Google, 2023). The voice and gender of Google Assistant is automatically selected by the language. The differences of voice settings policy between global leading companies reveals that a design standard for voice agents considering gender stereotyping is needed. Nevertheless, existing studies have focused on the agent’s response to the user’s requests rather than on the user’s cognition for voice agents as social interlocutor, especially in gender perspective (Søndergaard, & Hansen, 2018; Hwang, Lee, Oh, & Lee, 2019). Thus, there is a lack of detailed research on how user expectations can vary depending on the agent’s gender.

Therefore, in this study, we investigated how the gender of anthropomorphized voice agents can affect the user’s gender stereotyping of the agent. To this end, we reviewed previous studies on the social roles of gender and voice agents. We then conducted an online survey of 192 people to discover how users perceive the appropriateness of voice commands for each gender of anthropomorphized voice agents. In the following sections, we first introduce social role theory and previous research on voice agents, then describe and discuss the details of the online survey and its results. In discussion, we review the results of the survey and present some design guidelines for voice agents to reduce gender stereotyping induced by using anthropomorphized voice agents. Finally, the future work and conclusions will be described. We expect that this study will reveal the tendency of gender stereotyping of people on voice agents. Simultaneously, we also expect designers of AI-based voice agents would be able to identify some important issues related to the gender stereotypes or discrimination and inspired to design voice agents appropriately with some design implications from the study.

Most voice agents were presented as intelligent virtual personal assistants (Loideain, & Adams, 2020). To explain why voice agents were chosen to be female assistants, the social role theory was reviewed. According to the social role theory, people expect and define certain characteristics based on the activities performed by women and men in society (Eagly, 2013). Social role theory also explains that historically in Western societies, men were more engaged in higher power and status tasks, and women were primarily assigned parenting roles, creating stereotypes that were more suited to men’s agency and women’s communion. For an example, it is natural for male to be perceived as doctors rather than nurses, but they tend to be perceived on the contrary (Wilbourn & Kee, 2010). This forms gender roles, and the typical gender roles arise from the jobs of each gender. In other words, the characteristics needed to perform a task are what trigger gender stereotypes. The division of labor has decided that women should perform domestic or subordinate behavior and men should perform dominant behavior and resource acquisition (Eagly, & Wood, 1999). However, these expectations could lead to confirmation bias which means that people do not consider the other cases such as female doctor and male nurse (Tinsely, & Ely, 2018). In other words, expectations of gender can lead to evaluations of behavior that conform to gender roles, which can be recognized as inappropriate if they are not in the gender role frame (Brescoll, Dawson, & Uhlmann, 2010).

People are subconsciously aware of gender, even if the subject is a computer, under the paradigm of computers being social actors (Nass, Steuer, & Tauber, 1994). According to a previous study, people responded differently to robots based on their perceived gender and tried to use more words when communicating to a robot of the opposite gender than a robot of the same gender (Powers et al., 2005). Not only did people recognize gender in computers, but they also assigned social roles based on the perceived gender of the computers. In other words, once people perceive computer agents as anthropomorphic, gender stereotyping can be applied to them. Nass et al. revealed that computer agents presented as male were rated as knowing more about technology-related topics, whereas computer agents presented as female were rated as knowing more about love and relationships (Nass, Moon, & Green, 1997). In another study, male-voiced computers were subjectively rated as more friendly than female-voiced ones (Nass, & Moon, 2000), which suggests that people have different expectations of the gender roles not only for humans but also for computer agents. These findings indicated that voice agents can be used unintentionally as tools to strengthen fixed social roles based on gender (Robertson, 2010).

As noted above, people ascribe different social roles depending on the subject’s gender, and this can be applied to computer systems. To obtain empirical evidence, we conducted an online survey on how users have formed gender role stereotypes about the voice agents currently in use. In order to investigate how people consider recently commercialized voice agents in gender stereotyping perspective, we chose survey methodology with text-based voice commands instead of voice interactions with agents. It’s because that the voice characteristics of agents can vary the extent of perceived gender stereotyping based on individual differences. The survey was conducted online for a week via Facebook ads in South Korea. A total of 192 people (110 women and 82 men; binary selected) participated in the online survey, and the average age was 23.25 (SD = 6.37). All participants were South Korean whose mother language is Korean. The content of the questionnaire consisted of three main categories. The first category concerned personal information, asking about the age and gender of participants and their experience using voice agents. The second category comprised questions about the gender appropriateness of user commands for voice agents. The last category consisted of questions assessing the extent of the sexism of the individuals. The survey took about seven minutes, and ten participants were given Starbucks vouchers of 5,000 Korean won value through a raffle.

The questionnaires for evaluating the gender appropriateness of user commands for agents were prepared in reference to Table 2, which outlines male and female traits from the previous research on the gender of robots (Eyssel, & Hegel, 2012). Based on the gender stereotypical traits and tasks presented by Eyssel and Hegel, we designed a list of user commands for voice agents from the manual of a commercial voice agent service (SK Telecom, 2019). For an example, a user command “Tell me the information for a recent gaming laptop” was developed based on the one of male stereotypical tasks, servicing equipment. As a result, 65 user commands were written. Based on these user commands, participants were asked to respond to questions on a seven-point Likert scale (1: definitely appropriate for male agents to 7: definitely appropriate for female agents) to judge the gender of the voice agent that would likely offer more appropriate answers for each user command. The exact expression of question was “Considering the gender of voice agents who are likely to give the right answer for each user command, please mark on the closest score among 1 to 7 points (1 point: Male-----7 point: Female).”

Although we listed user commands reflecting gender stereotypical traits and tasks from the previous study, participants were encouraged to select the gender-neutral answers (which is 4 on a seven-point Likert scale) if they had no gender role stereotype of voice agents. On the other hand, there were two groups (male and female) for user commands that could be differentiated significantly compared to the gender-neutral answer if participants had stereotypes of the gender roles of voice agents.

For the data analysis, the scale validity was verified by first conducting a confirmatory factor analysis and reliability test to discover whether user commands for voice agents can be divided into male- and female-trait factors. Next, a one-sample t-test was conducted for male- and female-trait-based user commands to determine if they could be differentiated significantly to gender-neutral commands. Afterward, an independent sample t-test for male and female participants verified differences in participants’ gender role stereotyping regarding the user commands. Finally, to determine whether the individual sexism and age also affected the extent of gender stereotyping of voice agents, two multiple linear regressions were conducted for both male- and female-trait-based user commands.

At first, to establish that user commands can be classified into two groups, a factor analysis was conducted using the principal component analysis and the varimax rotation method with Kaiser normalization. The user commands for which factor loading was less than 0.4 (sixteen user commands) were excluded following the study by Yong and Pearce (2013). In addition, a one-sample t-test was performed to find user commands that are significantly different from 4, the gender-neutral value. As a result, user commands for voice agents could be divided into two factors. Table 3 and Table 4 reveals that user commands from M1 to M22 can be considered male stereotype-inducing commands, whereas F1 to F27 can be classified as inducing female stereotypes, since the mean value lower than 4 indicates male appropriateness and higher than 4 describes female one. Each user command group revealed high reliability (Cronbach’s α = .933 and .911 for male and female groups, respectively). Examples of user commands classified as male stereotyping were “Tell me the schedule for today’s Arsenal [an English Premier League football team] match” or “Tell me the information for a recent gaming laptop.” Examples of female user commands were “Order fabric softener” and “I’d like to make a malatang [a spicy Chinese soup], so let me know the recipe.” Full data for the user commands for M1–M22, F1–F27 and the other sixteen commands are provided in the Appendix.

Second, to verify how the personal characteristics affect the gender role stereotyping of voice agents, several moderating factors, such as gender, age, and individual sexism, were investigated. First, independent sample t-tests were conducted to determine whether any difference exists in gender role stereotyping on voice agents between male and female participants. To this end, we calculated the mean and standard deviation for the survey responses to the user commands in M1–M22 (male-trait-based) and F1–F27 (female-trait-based) commands. Table 5 displays user commands that are expected for the male role, which exhibited no significant difference in gender stereotyping on voice agents between male and female participants (t(190) = 1.96, p > .05). In contrast, gender stereotyping on voice agents induced by female-trait-based user commands exhibited a significant difference between the male and female participants (t(190) = 2.39, p < .05).

Through the online survey and data analysis, we derived two findings. First, participants exhibited gender role stereotypes toward computer agents who do not have an inherent gender about user commands which were developed in the present study based on the prior study of gender role stereotypes (Eyssel, & Hegel, 2012). This finding suggests that people can form gender stereotypes not only from humans, but also from anthropomorphized digital agents with gender. Notably, the results of the survey show that the extent of gender role stereotyping toward a computer agent is powerful, regardless of the recent global trend of feminism in South Korea represented by #MeToo (Gill, & Orgad, 2018). Second, the participants’ gender role stereotyping towards computer agents depended on their gender. As the result of the study, the recognition of gender role stereotypes is more pronounced in female roles and female participants. This finding relates to the modern sexism that Swim et al. noted (Swim, Aikin, Hall, & Hunter, 1995). Modern sexism, characterized by denials or attenuation of gender discrimination (Sarrasin, Gabriel, & Gygax, 2012), hostility toward women, and the absence of policies to help women, has forced women into a more gender-discriminatory environment than men. Considering the characteristics of modern sexism, women could have been forced to be aware much more of their gender roles compared to men which may entail the results of the present study. Nevertheless, this result must be re-verified based on participants from other countries or cultures, since the results may vary according to the level of gender equality of the society.

The gender stereotypes identified in this study could become more widespread and problematic in the future. With the increasing applicability of generative AI, which can autonomously generate content such as text and images based on large language models, various applications have been released. Examples include tutor agents assisting in foreign language education (Marr, 2023) and colleague agents capable of sharing workload in corporate tasks (Schwartz, 2023). Considering these developments, there is a risk that anthropomorphized agents may reinforce gender stereotypes while interacting with users in various situations as AI agents continue to advance. Therefore, we aim to propose design guidelines for AI agents that can minimize gender stereotypes.

Firstly, it involves not revealing the agent’s gender or presenting it in a gender-neutral form. This approach entails excluding design elements that disclose specific traits such as the agent’s name, visual appearance, or voice during interactions with users, minimizing the level of anthropomorphism. This method is particularly suitable for cases when agents primarily interact with users in a task-oriented manner rather than focusing on relationships, especially in text-based interactions. Examples of this include OpenAI’s ‘ChatGPT’ and Microsoft’s ‘Bing.’ It’s important to note that agents interacting with users including features such as voice or visual appearance may be less preferred by users when designed to be gender-neutral or ambiguous compared to instances where gender is clearly presented (Yeon, Park, & Kim, 2023). Hence, it may be more appropriate to employ such approaches in contexts where minimizing anthropomorphism is desirable.

The second strategy is utilizing agents based on diverse characters with different personalities, ensuring that agents performing specific roles are not limited to a single gender, either male or female. This strategy can be applied in situations where agents need to interact using voice or when the agent’s appearance is included in the design. Additionally, it is effective in situations where the level of anthropomorphism is high. An example of this approach is evident in the case of ‘A.’ released by SK Telecom (Sharma, 2023). In ‘A.,’ as depicted in Figure 1, an interface is provided that allows users to interact with agents possessing various characters. By employing a variety of agents within one service, users can choose agents based on their preferences, potentially preventing the reinforcement of stereotypes.

Figure 1 A group of anthropomorphized agents from SK Telecom ‘A.’

Lastly, it could be considered that users personally customizing their own agents based on their preferences through an initial setup process, creating unique agents tailored to individual tastes. This method is akin to the relatively high degree of freedom users have when directly controlling the physical appearance, voice, style, or abilities of their avatars when creating avatars in computer games. If users can directly design the agent they will interact with through initial settings, there is a high likelihood of generating diverse agents for each user. However, in this case, the degree of freedom in the interface for configuring the agent may play a crucial role in determining the effectiveness of this approach.

Despite the interesting findings of the study, there are some limitations. The survey did not present any voice of agents before participants responded to the questions. For this reason, the results might be biased. In other words, participants’ gender stereotyping toward voice agents may come from their pre-existing gender stereotyping by human perspective even if the survey explicitly mentioned voice agents-based evaluation. If the gendered voice delivered to participants, they would perceive the agent’s gender apparently. Interestingly, participants revealed their gender role stereotype regarding voice agents even they have no further information of voice agents especially related to task performance. Also, the set of user commands used in this study has not been validated by any previous studies. Instead, we developed the list of user commands that can be classified into two groups with criteria of gender role appropriateness based on the previous study of gender role stereotypes. The set of user commands that can be perceived as gender-role-biased should be replicated in the following studies. In addition, the preference of female voice agents might be biased since most of participants might have experienced female voice agents first. Therefore, in following studies, it is required to empirically verify how gender stereotypes about voice agents identified through this study affect actual users’ experience with voice agents, especially in user satisfaction or preference. Furthermore, it is suggested to study how to design voice agent services to prevent gender stereotypes of voice agents and the resulting decline in user satisfaction.

In this study, we found gender stereotypes in voice agents among male and female users through online survey with user commands developed from gender stereotyping traits and tasks. Especially, female participants had more strict gender stereotype in female voice agents compared to male participants. Despite some limitations, we anticipate that the types of voice commands identified in this study, associated with inducing gender stereotypes, could be valuable in following studies which include evaluations of AI agents related to gender biases. Simultaneously, the design guidelines for anthropomorphized AI agents we have proposed to prevent gender stereotypes are expected to be useful for design practitioners in creating new agents that are free from gender bias. As people can easily apply their personal expectations of human beings into anthropomorphized agents as interlocutors of social interaction, this human trait induces positive consequences in many cases, such as social support (Torta et al., 2014) and attachment (Banks, Willoughby, & Banks, 2008). However, there are some side effects as well such as the gender role stereotyping, as this study discovered. Considering this, designers, developers and even IT companies should be aware of the unintended consequences so that they can develop agents for gender equality.