Insights Podcasts The Testing Show: UX and Usability

The Testing Show: UX and Usability

July 25, 2019

Bias in Software Testing

How often do we look at an application and wonder what the people who were designing it were thinking? When jumping through hoops that seem odd or out of place, it’s easy to ask “how can this have been designed this way?!” If it makes you feel better, there are people who research the areas of Usability and User Experience (UX) specifically.

Michael Larsen talks with Charlotte Dijksman of Test Birds and Nick Leal of Qualitest about the good, the bad and the just plain out there of Usability and UX.













Michael Larsen: Hello everybody and welcome to The Testing Show. It is July, 2019 glad you could join us today. My name is Michael Larsen and I’m your producer and today I am also going to be your M.C. Matt is out for this month, but he will certainly be with us soon again. So I want to say thanks everybody for joining us and today we are going to be discussing what I think is an interesting topic and that is UX and Usability. So to that effect we would like to welcome Charlotte Dikjsman from Test Birds.

Charlotte Dijksman: Hi.

Michael Larsen: And joining us from Qualitest, we’d like to welcome Nick Leal.

Nick Leal: Hi, thanks for having me on the show.

Michael Larsen: Thank you. All right, well let’s just get right to this. So number of years ago I was introduced to usability through two books and they’re from Steve Krug. You might’ve heard of them. “Don’t Make Me Think” and “Rocket Surgery Made Easy”. Is that still a relevant intro to what usability is? I guess better. What is a good broad definition of what usability and UX is?

Charlotte Dijksman: I think it’s good to emphasize the broad definition of usability and UX just because of the sheer range of all the devices that require it in this present day, but I would say I kind of agree with what you said and when we talk about usability and about UX, it’s about a user’s experience in whatever they are looking at or using. It still goes to that saying that people just want a smooth experience from start to finish.

Nick Leal: So to keep it as broad as possible. I would say that usability, user experience is, can I use this and then can I also enjoy what I’m using? USABILITY is… UI is does it work, and then user experience is kind of the next level where it’s “Does something work and does it also give me a good time when I’m using this thing?”.

Michael Larsen: I think that makes sense. I guess in my case, let me give you an example of an application that I think does a pretty good job on this front. I use a fair number of apps right now focused on fitness and losing weight. That’s probably a little bit more personal information that people need to know about me right now. But Hey, there it is, uh, for the purpose of this conversation… because of the fact that I like to walk, hike, bike ride, etc. My device is with me, it’s there to give me feedback. I want to check in with it, but I don’t necessarily want to sit down and drill down into a lot of information. If I’m on a bike ride and I want to see how I’m doing, I want to do that with the minimal number of steps. And so usability is also relative.

Michael Larsen: What is usable in one moment may not be so usable in another. If I’m sitting down at my desk and I have, say, my phone, I’m able to do a lot more intricate maneuvering then I am when I’m literally peddling my bike down the road and I want to do a quick check in, “Hey, how am I doing? How much time do I have?” Usability at that moment changes dramatically. And I think that we need to address that as we’re looking at how our applications are used. So context, I think, is important when it comes to usability. Agree? Disagree?

Charlotte Dijksman: Oh yeah, definitely agree. Yeah. You’re talking about real world situations and how these things are interlinked and translated into functional design and you need to take in to consideration much more not only when are people using these applications or websites or interfaces, but also why and which devices, what age groups do these people fall and what are their capabilities and these are all things that need to be taken into consideration when designing and these are all things that can in fact be tested throughout the design process.

Nick Leal: Yeah, I would agree as well. I think a good thing that you brought up is age group is a perfect example. An example of an application would be an Instagram application or any social media where if you have someone who’s older who’s first using it, there’s different things that you look for as far as relating to usability. Their experience is not going to be the same as someone who’s been using social media their whole entire life. So I would say that’s definitely relative.

Charlotte Dijksman: Yeah. I actually stumbled upon something like this just recently in a test that I was executing. I was working for this client who created this app with a menu bar at the bottom of the screen for smartphone, which is actually a trend and there’s something that we see more and more possibly because of the phone sizes are getting bigger and bigger and a hamburger menu is not always reachable with a thumb. So the menu button goes down, but we were testing with an older target audience between 40 and 65 and just response was mainly, “I don’t understand why this menu is at the bottom of the screen. I would have expected it at the top.” So it’s an interesting conflict there because actually when it comes to trends and the reasoning behind putting the menu bar down lower in to the screen or at the bottom of the screen, it makes sense because it, in theory it’s actually better, but when it comes to user experience it can sometimes differ a little bit from what the trends are and what should be done.

Nick Leal: Yeah, so personally a client that I’ve worked with, their target demographic is the elderly and they make cell phones that are geared for them. So looking at those as opposed to smartphone, it’s clear the differences in design, how they want to make it based on their target demographic.

Michael Larsen: So that’s a good transition point. You just identified that you are working with phones or a service for a particular group and that’s… again it comes back to context. As I said, I use an application differently based on what I am doing. It also goes without saying – actually I shouldn’t say it goes without saying. What’s the point in mentioning it – it should also be mentioned that context isn’t just in the moment. It’s also who is using a product. I realized this a number of years ago… for anybody who’s listened to this podcast for any length of time, you know that I cover topics like accessibility and inclusive design and frankly the interesting thing is is that before I turned 45 I didn’t really think about it very much because I was, for all practical purposes, what would be defined as a normative and I put that in big air quotes, normative user.

Michael Larsen: I didn’t have any vision issues. I didn’t have any exceptional hearing issues, mobility issues. Cognitive issues, I suppose, are a roll of the dice and a matter of opinion, but, but generally speaking I considered myself fairly normal and I think that’s one of the things that we tend to focus on is usability or UX for the normal audience. Once I had to put on a pair of reading glasses or more to the point should I say when I’m in situations where I need to take the reading glasses off, that’s where my interaction with certain applications changes considerably. In the sense that before nice tight, oh lots of information and pretty colors and chunk together stuff. That was really appealing when I was younger and I didn’t have any issues with close vision. Now broader range, brighter, more contrasting colors, the ability of being able to make sense of what I’m looking at without having to do a fine read… those matter to me now. Whereas 10 years ago they didn’t matter to me. To broaden this question, how do we effectively target testing for those specific groups? How do we make sure that we’re not leaving people out?

Charlotte Dijksman: Well, it’s an interesting topic for sure and I could definitely relate to the changing needs over time, which is something that I, I have quite some experience with working with design teams who are designing for specifically all the target groups. We had a retirement target group specifically and the feedback is like you said, needs to be readable. It needs to be a little bit muted and just super clear where everything can be found. Yeah, I can definitely relate to that and when it comes to reaching certain target audiences, I have the pleasure of working for a company that has a massive amount of people connected to their platform. So when it comes to finding a target audience, we have a massive pool of people to look through and to reach out to. I consider it to be a major advantage is the personal connection that we had with all of them.

Charlotte Dijksman: Of course, I think it’s over a quarter of a million people throughout testing and throughout executing all these tests. You build communication with them and when it comes to finding different people or finding very specific people, we can go to the people that we have. We can a lso ask them, “Hey, do you know someone?” is an advantage of being so close to actual people. Also, interestingly, is the need for assistance throughout the process. Whereas maybe a younger person wouldn’t want any assistance and would love to just explore and would maybe feel a bit belittled if they were guided throughout a process. Another person, might feel actually really helped. It might boost his confidence if there’s little tips and tricks everywhere around the page. So those are things you definitely need to take into consideration. I would also like to mention something about reaching specific target groups. I have the privilege of working for a company that has a massive pool of people connected to their platform.

Charlotte Dijksman: There’s just so many people we can reach out to currently working with a tester who’s hearing impaired, so everything we do is of course written. What I personally see as an advantage of having such a large pool of testers to work with is that whenever I’m in a test, I’m constantly talking to the testers and either doing interviews or having written tests and going through the results and building a rapport. So I feel like that personal connection really does allow me to reach people, but also allows them to reach out to people in their environment and say, hey, this is something that is valuable and worthwhile to do and this is something that you can do as well, which I think increases our possibilities of reaching any kind of target group.

Michael Larsen: Let’s switch gears a little bit. Talk about the need for usability. So let’s move this into the testing of usability. Can we identify some trends that are having an effect on usability and the way that we’re looking at usability? I would suggest especially with say the app store model on the phone, whether it’s iPhone or android or something else. We’re at a point now to where the ease of adoption or lack of ease of adoption and the number of choices. If somebody picks up an app and it’s not working for them, they’re not necessarily going to tell you, “Hey, your app doesn’t work for me.” They’re just going to drop it and go to something else. So it’s entirely possible we may never even know that somebody is having a problem with something. How are we able to address that? Are we at a point to where we’re really able to address that and what are some things that are helping drive usability as a practice?

Charlotte Dijksman: Yeah, I can definitely jump in on this one. We are definitely at a point that we can do something about this and that it should be addressed because it really is, as you said, there’s just so much on the market right now. If you look at these app stores or you get a Google play in the app stores, there’s just so much and people might not say anything and just drop the app or they might leave a scathing review and make other people not try out your app. This is something that is definitely a reality and people have opinions, so why not utilize these opinions and incorporate it into kind of a testing cycle to make sure that whatever project or whatever product we’re building it’s actually up to a standard that people like to use. When you talk about usability and I dabble sometimes a little bit, not too much in these usability forums, there are so many things to be considered and there are so many choices you can make just by placing a button somewhere in this app or somewhere on their website. There’s just so many choices you can think of. So many ways you can guide your customers or your target audience through your product that it really does seem like a good idea to have them look at it throughout the process and make changes as you go just to make sure that when you actually do have a finalized product, it meets people’s needs and people will love it .

Nick Leal: As far as trends, I can give you personal trends that I see in my testing tools to make it more usable. Something that I see in my automation is a shift towards AI, artificial intelligence and machine learning and then as well as kind of getting away from needing to be a programmer in order to develop automation tests, moving more towards GUI based testing. In general, all the products that are used to automate applications are becoming a lot more usable and then they’re also becoming smarter utilizing AI techniques. That’s definitely the one thing that I’ve seen.

Michael Larsen:  So one of the areas that I personally struggle with, and I’ll confess this, is there are times when an application will require you to do something. We’ll take testing tools. I think that’s a good model here. In some cases, “Oh, you want to be able to use the testing tool? Here’s how we need to get started with it.” Or “Here’s how you would do your first test.” And unfortunately you find yourself just struggling with trying to figure out what the proper way is and you talk with somebody else. So you have to go to great lengths to either read through tutorials or figure out how something works to be able to say, “Oh yeah, no, what you need to do is…”, and then there’s these 10 steps to be able to make something happen. If you’re using a personal app, if you had to go through 10 steps just to get an application to work for you, chances are you would just drop it and find something else that works more easily. In our everyday testing life with the tools that we use, we may not have that option. We may just have to struggle through it. Have you found industries out there who are more interested in usability than others or who are driving this conversation towards “we want to make our product more usable, especially perhaps to those who are a captive audience”.

Charlotte Dijksman:  So when it comes to industries that are quite interested in improving their usability, personally you get asked quite a lot. They said people see low conversion rates when it comes to their websites or their web shops. There’s a big driver of ecommerce web shops, consumer websites who make a living basically out of their website. And if it’s working well or not, improving the usability on a website like that for them is actually improving their business. So in that way it is consumer usability, but it is to improve this company’s revenue. So that’s something that I consider a big driver and then there’s the automotive industry which we see a lot of testing for, which is actually quite interesting in some cases because their focus of course was not in the digital sphere at all and we actually see a lot of innovation in that area when it comes to creating great UX and creating great digital experiences next to your physical experience when you’re actually driving your cars. Personally, I find that super interesting what’s happening over there.

Nick Leal: Yeah. In an industry that I’ve worked with with previous clients is retail. Even though it doesn’t seem like a software based industry, there’s things that are kind of in the background that you don’t really see it. Also making things more user friendly, not for their customers but for their employees like POS (Point of Sale) systems and things within the store, which this one actually is for the customer, which is the self checkout. I’ve done a lot of testing on that and I seen things like robotics being implemented in with that and a lot of automation. So these systems are really old and they’re trying to get into a modern world and being focused with user experience, it’s hard to do that. So I see a lot of attempts to make it more user friendly in that field as well.

Michael Larsen: So let’s transition here a little bit. We’ve been talking a lot about the need for usability and we’ve been talking about why to do it and a little bit of the how to do it. So what are some trends that you are seeing that are having an effect on usability testing? I remember very well working for a company that was… this is back in the days when the touch pad was a new device on a laptop. One of the things that they were working with, they were looking at how to minimize the needed electronics to thin down a laptop. And one of the things that they happened upon was the idea of saying, hey, why don’t we mix the speaker and your touch pad? And I remember going through, we had a specialist who focused on human factors testing and one of the things that they did was, “Hey, we’d like you to interact with this laptop for a little while and tell us what you think”. And I’m working with it and I’m using the touch pad and some message would come back. And I found it so unnerving that the speaker was actually underneath the touch pad. And so as I’m moving my finger, the touch pad’s vibrating… auuughhhh!!! And I just said, “That weirds me out. I… eww… No, I’m sorry I can’t, I just, I’m, I’m literally working and all of a sudden it’s talking to me and my finger is picking it up”. It just gave me this visceral reaction.

Charlotte Dijksman: Hmm.

Michael Larsen: I ultimately think we didn’t end up going forward with that, but it was one of those early human factors testing to being able to have that experience and getting a crowd of people working on something. Is that still the main Go To for doing usability testing or are there new things that are coming up that are helping us inform that?

Charlotte Dijksman: Well, it is still a massive influence on how usability testing is done. It’s still something that happens on a day to day basis. Actually, my job to do it on a day to day basis. At Testbirds, the company that I work for. We have two ways of testing so we can either test via crowd or test via cloud and I am personally a big fan of crowd testing, which is basically, like you said, it’s getting a group of people together and having them take a look at whatever product needs to be tested. I was interested in this part of your story with how you described your reaction? It immediately made me think as a researcher, how would I interpret that? In your case, obviously, more people had the same reaction and the decision was made not to go forward, but it’s always also good to think about how do we interpret these testing responses from people.

Charlotte Dijksman: With that also comes, how do you ask these questions? Of course you can just get a crowd of people to take a look at something and have them give feedback, but what we’ve really tried to do is think about how are we actually going to ask this question? Are we too leading? We’ll just be objective enough. Will they be influenced by the way we set up a testing script and all these kind of things need to be taken into consideration when it comes to testing with a crowd. It is maybe not as straightforward as it seems. There’s a lot of floating factors that you need to take into consideration to really get some meaningful results. But yes, definitely I would say this is a great way to test for usability and there’s lots of way to do it. We can do written tests, interviews. It’s a little bit more of an old fashioned kind of testing way, but it’s still very popular. Personally, I think that is because, well, in this case we are an agency, so one of her clients wants a product tested and they request interviews. I think there’s always something really magical when you can actually see someone using your product. Although I am super confident in the value of your written test actually, sometimes, it’s preferred because people can do a test in the comfort of their own house if they do it remotely and if they can just do a written test. But being able to see a tester actually go through a process is also something that, like I said, can be a little bit magical and an insight can just kind of appear into your mind. So yeah, testing with real people, I do think it’s the main driver for inspiration and eventually developing something that is valuable.

Nick Leal: And something that I’ve seen, I mean everything you said was really interesting to me, because mainly what I’ve seen as a lot of monitoring tools, which monitor users and how they interact with their program, but it doesn’t really do anything to the point of what you described. Gauging their emotions when they’re doing that. And I think that that is really important. I think that’s where what you guys do with the crowd testing type of stuff really comes into play.

Charlotte Dijksman: Yeah, definitely. That doesn’t mean that any functional testing should not also be part of the testing cycle. Ideally you test both.
Michael Larsen: 22:08 Okay. So to kind of round this out now, because a lot of people who listen to this show are testers themselves, what advice would you give somebody who wants to beef up their skills and be able to focus on usability testing as maybe a broader specialty that they can look at – a broader specialty? Did I just say that? Ummm… Specializing generalist? Generalizing specialist? uh, copyright Alan Page. Sorry. Had to. Uh… But let me rephrase that. What advice would you give to somebody who wants to explore usability testing as a tester to be able to get more involved with usability? There. How’s that?

Charlotte Dijksman: I think that’s a good starting point. When a comes to usability testing as a tester, I think the most important thing is to have an open mind. And if you have looked at the product for long periods of time, you have to realize that your opinion might not be the only opinion. That is something that I feel is really important to take into consideration when it comes to being a tester in an interview. My main advice would definitely be talk from your own perspective. Don’t talk about what do you think others might feel because it’s actually very interesting to look at an individual experience and then as a researcher we can take all those individual experiences and look at them as a whole and have it be well rounded. I think that would also suffice. There’s many ways you can have a go at usability testing. My personal favorite is, like I said, is to test with other people, have other people give feedback. When you look at the certain product for too long you won’t see the nuances anymore like you would if you had just an open mind. So I think that is always something that needs to be taken into consideration.

Michael Larsen: Neil, what do you think about this?

Nick Leal: I don’t think I can give advice because I am a tester who wants to get into that so I can take the advice that Charlotte gives… I mean all that stuff. That makes a lot of sense. I think just being intuitive. That makes total sense and not thinking that your opinion is the only opinion that matters because you have, it’s kind of, you have to have that gauge of it’s not just you that’s using that. The goal of that type of testing is how is this going to affect everybody that uses it. Taking in all that stuff that Charlotte said in it, it makes sense.

Charlotte Dijksman: It is definitely a balance I would say because yes, of course you’re not the only one using it, but if you were the tester or if you’re actually testing the product, then don’t hold back. Give all the feedback that you want because that’s ultimately going to make it better and as a researcher, yes, you need many individual opinions to get to a well rounded result.

Michael Larsen: All right. I think that sounds like a good place for us to wrap this up. I think we’ve covered some great ground here and you’ve definitely given me some new things to think about. I greatly appreciate that. For both Charlotte and Nick, how can people get in touch with you if they want to learn more about you? What are you up to? Are you speaking, are you presenting, are you involved in a broader community where you are?

Charlotte Dijksman: Well that’s a lot of questions. Um, basically yeah, I am always available on Linkedin and the company that I work for (Testbirds) can be found on the Internet. You can go to That is a Europe based company, but we do testing globally. I won’t be speaking at any conferences, but I have recently recorded a webinar which is also available on my linkedin. And I think also, um, the Qualitest website.

Michael Larsen:  and Nick, you’re with Qualitest, but if people want to get in touch with you and get to know a little bit more about what you’re involved with, how can they get ahold of you?

Nick Leal: Same. Linkedin and then as far as inside a quality test where you have all of our internal events and I’m usually at most of those. So same type of stuff as Charlotte.

Michael Larsen: All right. Well, I want to say thanks to both Charlotte and Nick for joining us today. Thank you very much for being part of the show to all of our listeners. We look forward to talking to you again in August of 2019. Have a great day, everybody.

Charlotte Dijksman: Thank you Michael