Abstract
User experience (UX) has been a hot topic and should be an integral part of the design process – from the early concept to the final product. This article introduces a UX evaluation methodology that helps collect the pragmatic and hedonic attributes of an application in the onboarding process. Moreover, the presented method helps not just to understand the UX (including customers’ unconscious need, desire, and pain points) but also to make the development points clear and straightforward to everyone in the product management team. We focus on a social media platform – Snapchat.
Keywords
The Importance of Onboarding
Social media networks are essential communication channels for the 21st-century person: Generations Y and Z spend an average of 246 minutes per day on their mobile phones (Statista.com, 2018). They use mobile phones for entertainment, to connect with friends and colleagues, to share moments, to stay informed, to shop, and to do just about everything else. Acquiring customers is an essential component for any application development company to generate revenue and grow – and to gain more and more users. It is unavoidable to fulfill the user’s needs and provide positive experiences that keep them loyal. According to Nielsen (2018), 89% of consumer media time spent on mobile apps in all markets, and on average, 3,082 apps are released on the Google Play Store every day and an average of 1,181 apps released on the Apple App Store (42matters.com, 2020). To stay in competition in the application market, the consumers should have a great first experience with the product when signing up. Nowadays, the experience-oriented approach is accepted where the supply is based on the consumer’s experience, specializing in gaining experience (Pine & Gilmore, 1998). The experience-oriented approach is a framework for building consensus over the requirements that extensively consider user experience (UX) with a shared understanding of the fundamental subjects among all users. It means the personal, subjective side of interaction with a product can create emotions and meaning (Law et al., 2009). In today’s industry, design has become increasingly important (Galitz, 2007), and onboarding has become essential in web design in recent years (Morson, 2015).
The first time a novice interacts with a product is called onboarding (Zichermann & Cunningham, 2011). Long-term user retention is strongly related to what happens during users’ first onboarding when the application is installed and first used (Crumlish & Malone, 2009)
So, onboarding is the process in which new users decide to stay engaged or become disengaged. A proper onboarding increases the likelihood that first-time users become retained users, causing increased conversion (Zichermann & Cunningham, 2011). The goal of onboarding is to introduce new users to a website or application and to carefully familiarize the product – to accommodate users and teach them about individual features. As Gladwell (2007) describes in his book, people – in our case users – are trained to make brief judgments based, which means they make rapid inferences about the characteristics and details of a situation or product, with minimal information. When it comes to designing the onboarding process, the preliminary knowledge of a user needs to be considered. For example, how much they know of the application, whether they have ever used it, what advertisements and marketing messages they have encountered. Through consecutive Hook models, successful products reach their ultimate goal of unprompted user engagement – bringing users back repeatedly without depending on costly advertising or aggressive messaging (Eyal, 2014). Successful onboarding means that engagement and emotional connections intensify so that the user will use the application regularly (Zomerdijk & Voss, 2010). However, most users of applications related to health and fitness, productivity, photography, news and magazines, communication, media, music, and social networking become disengaged by the seventh day. Figure 1 shows a retention curve for Android applications and how active users were since the initial installation. Seven days after the initial installation, 40% of users were active on that specific day. The reasons for retentions could be several, but proper onboarding can create an engaging first-time, can give positive UX, and fosters ongoing user engagement (Zomerdijk & Voss, 2010). Users can lose interest when an application is confusing and challenging to navigate. Utilizing application onboarding helps make the first-time user transition as seamless as possible, and it minimizes potential pain points.
App usage retention: Postinstallation (Similarweb.com, 2018).
In our research, we were looking for a better, more transparent methodology to not just understand the first-time users’ UX but also to make the development points clear and straightforward to everyone in the product management team. Development points refer to all the design of the entire process of acquiring and integrating the product, including aspects of design, usability, and function that provide meaningful and relevant experiences to users. In the product development team, the product manager is responsible for effective and quick management, and one of his tasks is to work with multiple teams and fill the gaps between them. This methodology could help product managers effectively delegate work because it identifies two critical factors: the right person and the precise task. We focused on this decisive process in one of the most popular social media applications – Snapchat.
Facts about Snapchat
Snapchat is a mobile photo-sharing network that enables users to share their funniest, ugliest, and most original photos, videos, or messages with other people, and edit them with a selection of digital filters. One of its principal features is that once a message is received and viewed, it is automatically deleted, and becomes inaccessible. It makes the service ideal for sharing quick updates with friends without accumulating media or messages on devices. It is available for iOS and Android devices, and primary usage is free to all. Figure 2 shows a screenshot from the front page of the Snapchat application and a screenshot when one of the filters is used.
Onboarding screenshot and a filter used in Snapchat (own compilation, 2018).
Snapchat is globally known since it was launched in 2011. By 2017 it had 300 million users. The application is popular among teenagers and young adults, as could be seen in Figure 3.
Age and type of users (Statista.com, 2017).
We chose Snapchat because it is ephemeral social media, and ephemerality has been a central component of the UX for many social platforms (Bayer et al., 2016). Moreover, its onboarding is characterized as an unintuitive UX due to the “learn by doing” approach. There are no tutorials, instead of text it focuses on immersing users in an interactive experience, and it allows users to complete gestures rather than a button. One of the most significant advantages of “learn by doing” onboarding is that there is a single workflow, and if it is successful, the user feels satisfaction. However, its disadvantages are that it can become frustrating or too complicated, and there are chances that users make mistakes.
Method
To understand first-time users’ UX and to establish a systematic developmental process, three methods were applied, built on one another (see Figure 4). First, we chose a usability test because it is best used to diagnose problems and to be part of a process that focuses on usability throughout design and development (Dumas & Redish, 1999). The statements from the usability test findings were elaborated with the affinity diagram, as this method helps to cluster and categorize the statements. Affinity diagram (also called KJ Method, after the developer, Jiro Kawakita) is a great technique to externalize and meaningfully organize a large number of ideas and opinions into their natural relationships (Hanington & Martin, 2012). Moreover, it shows the scope of the user problem and helps reveal all issues and worries (Chiou et al., 2014). It also helps overcome the fact that the preserved words of the customers are usually too general or detailed for translation and interpretation (Karsak et al., 2003). Affinity diagram prioritizes customer requirements and initially structures into different hierarchical levels (Kwong & Bai, 2003).
Method of testing user experience in the onboarding process. UX, user experience.
In our case, we used the affinity diagram to highlight the most significant attributes out of the statements from usability test findings.
Our idea was to collate these attributes with a model from Hassenzahl, in order to clarify for everyone in the product management team, what kind of attributes should be developed for better UX and who is responsible for them. It also gives a better, deeper understanding of the UX for the product development team, and it makes to find the right solutions easier.
We chose Hassenzahl’s model (2003), which defines the critical elements of UX and their functional relations. The model assumes that two distinct attribute groups can describe product characters: pragmatic and hedonic attributes. Pragmatic attributes are connected to users’ needs to fulfill their behavioral goals. Manipulation of the environment requires utility and usability in functionality and ways to access this functionality.
In contrast, hedonic attributes emphasize individuals’ psychological well-being, so they are primarily related to the users’ self. In line with previous research on life experiences (Sheldon et al., 2001) and experiences with technology, it is implied that people can have positive experiences with a product. This positive affectivity is related to need fulfillment and may be reflected in hedonic quality perceptions. The negative part of the hedonic quality perceptions refers to users missing positive feedback, and they either become neutral or, in the worst case, they get frustrated and unsuccessful. Hedonic attributes can be further subdivided into stimulation identification and evocation. A product can provide stimulation by its challenging and novel character, which connects to the basic human need for personal development (Schwartz & Bilsky, 1990). Identification by communicating important personal values to others addresses the human need for self-expression through physical objects – their possessions. To be socially recognized is also a powerful inner motivation (Hassenzahl, 2004). Products can also provoke memories of past events, relationships, or feelings (Prentice, 1987).
Usability Test
In the first phase of the examination, we carried out individual, conventional usability studies in the onboarding process to collect UX. Our goal was to discover how users can accomplish their tasks, what their desires, needs are through the onboarding process. Six individuals participated in the usability evaluation, none of whom have previously tried the application. One of the participation segmentation features was that three of them were iOS, and the other half were Android system users. The age segment was represented between 20 and 25 years because the primary users of Snapchat are younger than 35 years, and they were a mix of genders.
Test subjects had to install the application on their own devices so that testing was done in a familiar hardware environment, only focusing on the usability of the application. In the usability test, they went through the registration process and performed some tasks to familiarize themselves with the application (see Table 1). During the usability test, we asked them to think aloud in order to get their inferences, intuitions, mental models, reasons, and decisions while doing the task, to detect cognitive activity that may not be visible (Nielsen et al., 2002). Screenshots were taken to get a clearer picture of their experience.
The Main Tasks of the Usability Test
After the test, we also interviewed the users to get to know their experiences and opinions on the application. The questions related to their engagement, how much they understood the functions and icons, and their ideas for further development.
Results of Usability Test
In this phase, we distilled and summarized all critical and relevant written or spoken statements or actions. After having double-checked the coding, we printed each sentence in a piece of paper, and every note was given a serial number. These 412 statements became the labels for the affinity diagram, which helped to organize them and highlighted the most significant attributes of their UX.
Affinity Diagram
In our case, the affinity group consisted of four people, aged between 18 and 24 years with backgrounds in designer, marketing, and ergonomics studies.
As Scupin (1997) also indicates, our application started with individual work and ended up as teamwork, including the following steps: label making and grouping, chart making, and written explanation. In the beginning, the makers started to cluster the notes in silence, and they carried out the categorizing without talking. It is a novel experience for most people as it was it our case too, it has two positive outcomes: it facilitates outside box thinking and prevents fighting over semantics. After they had made groups from the labels, they started to work in teams – this complies with Diehl and Stroebe’s (1987) findings that group-work generates a significantly higher and broader number of ideas than individuals. The makers started to react instinctively to each other’s opinions: If somebody did not like where one label was placed, they could discuss or move it to where they preferred. Once all labels were entirely in the right group, they named the groups, and they agreed on the labels’ significations. Finally, makers could define four main attributes and 19 specific cases such as icons, design out of 412 statements, as Table 2 shows.
The Main Attributes Received in the Affinity Diagram
Results of Affinity Diagram
The first attribute includes everything connected to the application’s usability. “Information” was another attribute of UXs, gathering all places where users gain knowledge. In the case of Snapchat, users were uncertain of many actions, meanings, and functions presumably due to the “learning by doing” onboarding system. Finally, the last attribute gathered all kinds of opinions and feelings that emerged or were generated in users. With the help of affinity diagram, 412 statements were prioritized and initially structured into different hierarchical levels. In the following, we only focused on the four main attributes, and the 19 specific cases because they gathered all precise feedbacks from the usability tests. These findings were applied to a model to clarify important development points for everyone in the product management team and to provide a deeper understanding of users’ desires and needs.
Model for UX Attributes
In the following phase, we collated the defined 19 cases from the affinity diagram with a UX attributes model by Hassenzahl. The four attributes became the main aspects, and the 19 specific cases have been applied to the UX attributes model. Inserting these aspects and cases in the UX attributes model was a quality phase of the research.
Results of Model for UX Attributes
Table 3 categorizes and visualizes the attributes of further developments. The pragmatic sides refer to its functionality and usability. Accordingly, findings in the weak pragmatic side consist of technology problems, which can be developed by those who work on the application technology in the product management team. For example, the first time users could not be interpreted the “took a screenshot” message because it was available only for a short time. Another pain point was that they did not find the back option and expressed their desire for a better user interface. The lack of tutorial was too challenging for them. However, if they overcame the hurdles of the onboarding system, they expressed considerable relief.
Model of the Essential User Experience Elements of the Snapchat Onboarding Process (Pragmatic Side)
Note. UI = user interface; SMS = short message service.
The strong pragmatic side includes, for example, that the subjects liked the Snapchat logo and its color, and “learning by doing” led in many ways to a feeling of success. They are great feedbacks for the user interface designers, who usually receive the negative ones.
Table 4 shows the hedonic side, and it is separated into three attributes: stimulation, identification, and evocation. The hedonic side includes more subjective experiences of the test subjects. This part of the model could help understand the underlying desires and needs. The strong side of the table collects attributes that fulfill test subjects’ needs and desires, and the weak side gathers attributes that need to be developed or redesigned.
Model of the Essential User Experience Elements of the Snapchat Onboarding Process (Hedonic Side)
Test subjects found Snapchat divisive – some people found it trendy and exciting, and users felt greatly stimulated (strong stimulation). In contrast, others found the application meaningless and lost their enthusiasm with the filter’s uncertainty (weak stimulation). As a result, the product management team could better understand the user’s pain point and could work on how they can increase the filter’s uncertainty. Only one function positively affected users’ identification, if they received a “took a screenshot” sign from their friends (strong identification). They felt that they are not socially recognized; they missed functions like group photo mode, notifications from friends, and getting a recommendation for tagging acquaintances (weak identification). The first-time users had a general feeling of familiarity only about the chat window icon (strong evocation). The default of the application refers to the camera function, and video taking and picture taking were found not to be quick or smooth enough compared to other applications (weak evocation).
The findings in the weak hedonic side demand more complex solutions, so UX designers and product managers need to be involved in problem solving. Meanwhile, the positive contains affirmative feedback and personalized opinions for the product management team members.
According to Hassenzahl’s model, the holistic UX can be delivered if the application can connect and react to users’ needs and motivations. In a usability test, the problems are usually diagnosed. However, this UX evaluation methodology (highlighting main attributes from the usability test with the affinity diagram and then inserting into Hassenzahl’s model) provides a deeper understanding of UX components and users’ pain points.
Conclusion
UX has rendered a central focus of product design and evaluation, and Vermeeren et al. (2010) gave a clear overview of the available UX evaluation methods. They identified several needs for methodological developments and further research questions on UX evaluation methods. Our methodology meets the following findings:
There is a need for attention to the practicability of methods. Our method can be easily employed in product development. It has ease of use, ease of data analysis, and applicability of results for development.
Practical multimethod approaches: Our method effectively collects and analyses data.
A deeper understanding of UX: This method and measure development can be substantially supported (Vermeeren et al., 2016).
It is an increased emphasis on methods that helps us to understand what can be done to improve UX through the whole product lifecycle. In our case study, a usability test, affinity diagram, and the model were built on one another to untangle first-time UX, appreciate the multiple parts of the process, and visualize the positive and negative attributes of the application. Customers’ opinions, requirements, and experience could be analyzed in a way where the positive feedback and further development points are visibly declared. This methodology’s advantage that it does not require any exaggerated input and identifies clear directions for development. Moreover, it helps members (e.g., product managers, UX designers, informatics) in the product management see their responsibilities and the distinct development points.
In our article, we presented a UX evaluation methodology, which could be used in any kind of mobile application. However, there is still a lack of a clear understanding of what UX evaluation method means, and of what characterizes it compared to a usability method. In our case, we believe that the UX perspective adds something to the traditional usability perspective, including emotional, experiential, and hedonic components. In our article, we tried to extend the current set of UX evaluation methods in order to help to understand customers’/users’ needs, desires, and requirements.
Footnotes
.