Measuring User Experience

Details

Published: 05 November 2022

In 1959 Donald Kirkpatrick introduced four levels of evaluation to the learning profession: reaction, learning, behavior, and results. These same levels can be massaged to describe the impact of XD. Thirty years later, Gloria Gery courageously informed us all that we might as well just weigh our students before and after learning, rather than use the metrics we were still using to determine the effectiveness of what we do for organizations.

Current tools and models for measuring design

Many organizations begin and end with user satisfaction measurement.  Measuring the user satisfaction is not enough. Although it is a good product metric, it doesn’t provide the information needed to the design team itself to improve upon.

Design maturity is hard to act on

Design maturity describes how well an organization listens to customer and user input, and how customer and user-centric its design processes are.  Design maturity results in stronger development processes that deliver more consistent products, services, and experiences that delight users, win customers, and enable true differentiation for your brand.  

The next question is, how do we evaluate design maturity?  There are many models out there to describe the continuum on which organizations exist.

• Deloitte has 4 levels

• Invision has 5 levels

• Nielsen/Norman originally had 8 levels, but now has 6 levels

For the purpose of this article, let's use Nielsen Group’s current scale as an example:

While design maturity scales like this are a great way to better understand where the design is at your organization and what it should aspire to, its interpretation can be subjective:

• Executives might argue that the team is on stage 4 because it has a dedicated budget within the company.

• A design team member might claim that it's on stage 2 (developer-centered) because design decisions are made without enough user research.

• Different designers can also disagree about their impact and influence on Product and Engineering, and that can vary by team or individual.

This is a great discussion to have both with your team and with the big bosses. However, a design maturity scale often lacks something tangible to be measured against and to indicate if any progress has been made over time. Increasing the maturity of the design team is not something that can be rushed. It can take years.

Measurable attributes of design maturity

To measure the design maturity, we first need to define metrics that can be used by any design team, regardless of the context of work and team composition.

To do so, we can look at the basic elements of any design project. 

Design system

A style guide is an artifact of design process. A design system is a living, funded product with a roadmap & backlog, serving an ecosystem. It consists of a guideline and repository of patterns, components and assets an organization has, but it goes beyond: it means designing, planning and maintaining a consistent experience, unique to the brand and its tone and voice.

Design process

The design process is, simply, the design process used by the team. A library of methodologies and good practices to ensure a high quality standard and consistency in the work done. Not all problems can be solved with the same approach, but having a main framework can facilitate collaboration and makes other tools and methodologies more accessible.

Design vision

Design vision is the direction where the team sees the design heading. It is formed by principles, values, and even their "blue sky" concept for the business, aligned with the company's mission and goals. 

Whether or not we are actively thinking about these three elements during a project, they are the main ingredients of any design team. Every design project has an interface system, follows a process, and has a vision behind it.

Defining the maturity metrics

Within these three elements in mind, we can define points to be used as metrics for each. These points can vary a lot depending on what is important for the team and for the business, from something specific and tactical to a broader satisfaction score with that element. Here are some examples of what could be used as metrics:

Design system

• The presence, stability, use and effectiveness of interface elements and composition, such as color, typography and iconography, fully mobile-responsive and meeting accessibility standards.

• A clear voice and tone guide, keeping the experience consistent and on-brand.

• The designers' satisfaction working with the current pattern library.

Design process

• A consistent high-level process, from discovery and research to QA and launch tasks.

• Clarity for the team of all design steps and knowledge of the tools and methods available.

• Opportunity (space and time) to study and experiment different methods and cross-team collaboration.

Design vision

• Clear business goals and a clear user market. Every designer should understand the company's mission and high-level roadmap.

• Design principles and vision that the team shares and is confident to use in their projects.

• Design initiatives and projects, aligned to its vision, and not necessarily directly linked to a company's project or product.

• Design literacy in the organization, and how deep and wide is the impact and influence of user-centered design.

 

Setting up a process to measure it

Regardless of the metric points decided for each element, the team will need a rating scale (o to 10? Five stars? Emoji faces?) and specific weights for each point selected.

Defining what and how to measure should be a process owned by the team. Every design team has a different and unique context that needs to be taken in consideration. Any formula has the risk of being too generic. Discussing the metrics and the approach is a great starting point for any evaluation discussion and also makes the team feel like they are owning their self evaluation mechanism.

If the design team concludes that the current state of any of these elements is not satisfactory, they have a clear direction on what to work on and a good argument to ask for more resources for the team if needed.

One can argue that it’s impossible to do well in any of these three areas without doing well the others. It is true, but their level and quality can vary a lot. And that’s exactly why we need to measure it.

Even if the all the scores are initially low, doing this exercise with the team is the best way to check that everyone is on the same page and to measure the progress of the team over time.

On the other hand, if the team seems to have everything under control, there is always new challenges in each of these three elements. Whether because patterns need to be updated, the team has to onboard new members, or a process needs to be reviewed for a new customer channel that is coming up, for example.

The reality is that the design industry is extremely dynamic and the context can switch quickly and it's the same for the design team. Nothing will be perfect — everything can be perfected.

Whether the team is centralized or distributed, with these three elements as talking points in the agenda, it is easier for the designers to assess how they, individually, perceive where they are to each one of them, starting the conversation about what needs to be done and where to invest more.

 

Where to Start in UX

Details

Published: 03 November 2022

If you work in the field of User Experience, you have to know about the following resources. I will keep adding to this list.

Krug, S. (2013), Don’t Make Me Think: A Common Sense Approach to Web Usability, Berkeley, CA: New Riders

This book includes Krug’s Laws of Usability:

1. “Don’t make me think.”

2. “It doesn’t matter how many times I have to click, as long as each click is a mindless, unambiguous choice.”

3. “Get rid of half the words on each page, then get rid of half of what is left.”

Norman, D. (2013), Design of Everyday Things: Revised and Expanded Edition, New York, NY: Basic Books

Formerly known as “Psychology of Everyday Things”, this book introduces the ideas of affordance and mental models. If you see anything by Donald Norman, get it and read it.

Beyer, H and Holtzblatt, K. (1997), Contextual Design: Defining Customer-Centered Systems, Burlington, MA: Morgan Kauffman

While this hasn’t been updated in far too long, this book supplies the techniques and tips to conduct thorough and effective user studies to identify “real users, doing real work, in their real environment.”

Ertmer, P., Quinn J. and Glazewski, K. (2013) The ID CaseBook: Case Studies in Instructional Design, Old Tappan, NJ: Pearson

This is a series of open-ended instructional design case studies that strengthen and encourage successful problem solving, and conceptual, procedural, and analytical skills to be used with a variety of real-world clients and the execution of creative solutions. Oh, and by the way, I co-wrote one of the case studies.

Also, Performance Consulting, by Robinson and Robinson, and Metaphors We Live By, by George Laikoff.

Living in the cloud

Details

Published: 25 October 2022

Once upon a time, boys and girls, we had a computer.  In my case, it was an Apple IIc with a floppy drive and 128k of internal memory.  It was a large and bulky thing with a lot of wires and a heavy green-screen CRT monitor and stuff, and even though it had carry handles and was billed as “portable,” we left it on a desk at home.  If we wanted to do something on the computer, we had to go home to that desk.  When we wanted to show what we were working on to someone else, we either printed it out or copied it to a floppy disk and prayed that the person we were sharing it with had the same kind of computer with the same version of software.  If they didn’t, we were out of luck.

Fast forward 30 years. Today, I am no longer tied to a single device. The internet has become my “PC” – Personal Cloud.  I have two laptops, a chromebook, two tablets and a mobile phone.  They all access my data real-time through the internet, so synchronization is not an issue.  My email, contacts, calendar, documents and music are all online.  Regardless of what device I’m on, or even if I own the device, I can securely access my data to get the job done.  The software is a service in my cloud, so I don’t need to install a program on a local device.  As long as I can get a browser and onto the internet, I have everything I need.  Jumping between devices isn’t an issue when the contact I saved on my phone is automatically available on any device I use to access the internet.  My family is the same way, so we can share a device and the customization is done online for us.

One efficiency tip I’ve heard over and over through the years is you shouldn’t maintain multiple calendars.  Everything should be visible at once.  With Google Calendar, I can see a combined view of multiple private and public calendars, and give people access to mine.  Iif I schedule something on my calendar, it is automatically available to my family.  Group calendars, such as for Boy Scouts or the Marching Band, are included in my view so I know how many conflicting events I have.  When someone asks if I’m free on a given day, I no longer have to tell them I’ll check with the paper calendar at home and get back to them.  My calendar rides in the cloud.

I review and edit papers with my college daughter on Google Docs.  She writes the paper and adds comments and questions, then sends me the link.  I can see her paper and we can even interact real-time within the document.  After I make suggestions and comments, I can watch her make the edits.  I do the same thing with organization meeting agendas.  People on the list can add to the collaborative agenda throughout the month so we don’t have to remember to bring something up.  This was nearly impossible before the cloud.

I’ve nearly gone paperless at work.  I take notes on my tablet using Evernote and can access them on any device anywhere.  The tablet is smaller, lighter and more portable; the battery lasts all day.  The tablet is large enough that typing isn’t an issue, and the autoprompt and autocorrect can actually increase your speed once you get used to it.  One of my favorite advantages the tablet has over paper is the ability to take a photo of the white board we’ve been brainstorming on directly into my notes.  When I’m back at my desk, I can review the notes online and sort by tags or do a full-text search through hundreds of notes to find the information I’m looking for. When I’m away from my desk, all of my notes are always with me, either through my laptop, tablet or phone, or even someone else’s computer.  Try doing that with paper.

The only reason I switch between devices now is for convenience (laptops are easier to type large blocks of text, tablets for watching movies, etc.) or for a tool that hasn’t been ported to the Internet.  I’m starting to resent applications like Axure that require a dedicated installation at one device. In the cloud, I don’t have to worry about whether my software is up to date, or if my version is compatible with yours.  The cloud updates for all of us.  I also have not run into a cloud-based application that costs more than $10.  $600 for Office? When Google Docs is free?  Sure, it does things I can’t do in Google Docs, but the price difference has caused me to rethink a lot of application loyalties.

To be fair, the risk online is greater.  Servers could be hacked, the system can go down, the cloud is outside my control.  If I don’t have access to a wireless network, I have to work tethered to my phone.  If I don’t have a 3G connection on my phone, I’m out of luck.  But really, if I am that remote, I don’t really need to be on my devices.  The risks have to be weighed against the convenience and capabilities, and for me, that decision is easy.

The cloud is where I live, and where everyone will live in the future.  The cloud democratizes computing, in that everyone has access to the power and reach of the Internet. The cloud allows for faster and greater collaboration and sharing and connects us as never before.

The Future of User Experience

Details

Published: 25 October 2022

The field of user experience design has roots in human factors and ergonomics, a field that, since the late 1940s, has focused on the interaction between human users, machines, and the contextual environments to design systems that address the user's experience. With the proliferation of workplace computers in the early 1990s, user experience became an important concern for designers. It was Donald Norman, a user experience architect, who coined and brought the term user experience to wider knowledge.

I invented the term because I thought human interface and usability were too narrow. I wanted to cover all aspects of the person's experience with the system including industrial design graphics, the interface, the physical interaction and the manual. Since then the term has spread widely, so much so that it is starting to lose its meaning.

—Donald Norman

The term also has a more recent connection to user-centered design, human–computer interaction, and also incorporates elements from similar user-centered design fields.

Elements of user experience design

User experience design includes elements of interaction design, information architecture, user research, and other disciplines, and is concerned with all facets of the overall experience delivered to users. Following is a short analysis of its constituent parts.

Visual design

Visual design, also commonly known as graphic design, communication design, or visual communication, represents the aesthetics or look-and-feel of the front end of any user interface. Graphic treatment of interface elements is often perceived as the visual design. The purpose of visual design is to use visual elements like colors, images, and symbols to convey a message to its audience. Fundamentals of Gestalt psychology and visual perception give a cognitive perspective on how to create effective visual communication.

Information architecture

Information architecture is the art and science of structuring and organizing the information in products and services, supporting usability and findability. More basic concepts that are attached with information architecture are described below.

Information

In the context of information architecture, information is separate from both knowledge and data, and lies nebulously between them. It is information about objects. The objects can range from websites, to software applications, to images et al. It is also concerned with metadata: terms used to describe and represent content objects such as documents, people, process, and organizations.

Structuring, organization, and labeling

Structuring is reducing information to its basic building units and then relating them to each other. Organization involves grouping these units in a distinctive and meaningful manner. Labeling means using appropriate wording to support easy navigation and findability.

Finding and managing

Findability is the most critical success factor for information architecture. If users are not able to find required information without browsing, searching or asking, then the findability of the information architecture fails. Navigation needs to be clearly conveyed to ease finding of the contents.

Interaction design

There are many key factors to understanding interaction design and how it can enable a pleasurable end user experience. It is well recognized that building great user experience requires interaction design to play a pivotal role in helping define what works best for the users. High demand for improved user experiences and strong focus on the end-users have made Interaction Designers critical in conceptualizing design that matches user expectations and standards of latest UI patterns and components. While working, Interaction Designers take several things in consideration. A few of them are:

• Create the layout of the interface

• Define Interaction patterns best suited in the context

• Incorporate user needs collected during User Research, into the designs

• Features and Information that are important to the user

• Interface behavior like drag-drop, selections, mouse over actions, and so on

• Effectively communicate strengths of the system

• Make the interface intuitive by building affordances

• Maintain consistency throughout the system

In the last few years, the role of interaction designer has shifted from being just focused on specifying UI components and communicating them to the engineers to a situation now where designers have more freedom to design contextual interfaces which are based on helping meet the user needs. Therefore, User Experience Design evolved into a multidisciplinary design branch that involves multiple technical aspects from motion graphics design and animation to programming.

Usability

Usability is the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use.

Usability is attached with all tools used by humans and is extended to both digital and non-digital devices. Thus it is a subset of user experience but not wholly contained. The section of usability that intersects with user experience design is related to human’s ability to use a system or application. Good usability is essential to a positive user experience but does not alone guarantee it.

Accessibility

Accessibility of a system describes its ease of reach, use and understanding. In terms of user experience design it can also be related to the overall comprehensibility of the information and features. It contributes to shorten the learning curve attached with the system. Accessibility in many contexts can be related to the ease of use for people with disabilities and comes under Usability.

Human–computer interaction

Human–computer interaction is concerned with the design, evaluation and implementation of interactive computing systems for human use and with the study of major phenomena surrounding them.

Human–computer interaction is the main contributor to user experience design because of its emphasis on human performance rather than mere usability. It provides key research findings which inform the improvement of systems for the people. HCI extends its study towards more integrated interactions, such as tangible interactions, which is generally not covered in the practice of user experience. User experience cannot be manufactured or designed; it has to be incorporated in the design. Understanding the user's emotional quotient plays a key role while designing User Experience. The first step while designing the user experience is determining the reason a visitor will be visiting the website or use the application in question. Then the user experience can be designed accordingly.