The Testing Show: Stress Cases
We are all familiar with stress testing when we put an application under load or negative conditions but what about when we put ourselves under those same conditions?
Rachel Kibler and Elle Gee join Matthew Heusser and Michael Larsen to talk about this more human element of software testing that often gets overlooked and how applying it to our everyday testing activities might help us do better testing overall.
- Eric Meyer, My Year Was Tragic. Facebook Ambushed Me with a Painful Reminder.
- Anonyome Labs
- Technically Wrong: Sexist Apps, Biased Algorithms, and Other Threats of Toxic Tech:
- Real Life is Not an Edge Case
Michael Larsen: Hello, and welcome to The Testing Show for April of 2019. Glad you could join us. I’m Michael Larsen, and of course you all know Matt, say hi Matt.
Matthew Heusser: Hello.
Michael Larsen: And today we are really excited to have Rachel Kibler, a test lead at Anonyome Labs.
Rachel Kibler: Hi, so happy to be here.
Michael Larsen: And we’d also like to welcome back Ms. Elle Gee, who is vice president of delivery for Qualitest on the west coast.
Elle Gee: Hi, thanks for having me back on the show.
Michael Larsen: Alright. We’re gonna go ahead and we’re gonna get rolling here. So the topic for this month is stress cases. As I was looking at this and I was thinking about this, I just had to ask, because I originally had thought, stress cases, it sounds like stress testing, but that’s not what we’re talking about, as I’ve come to understand. So I’m gonna lead this to Rachel first, so Rachel what is a stress case?
Rachel Kibler: Stress cases are acknowledging the humanity of the people using our applications. Oftentimes we talk about edge cases being marginal use. Like someone trying to order dinner while scuba diving. That’s never gonna happen, it’s a marginal use. A stress case acknowledges that the people who are using our applications are not marginal, and when they are under stress or in distress they still need to use our application. So one example is Uber. If you are in an unfamiliar part of town, and it’s late at night, and you have 3% battery left. It’s not an uncommon occurrence, and you still need to get an Uber. You need a ride home. Are you going to open the app and be confronted with, “Here’s what’s new in our app,” or are you going to be able to directly get to share your location and get a ride? Does Uber consider low battery, or timeframe, or does it actually consider the humanity in people using their application?
Rachel Kibler: Kind of the canonical example is when Facebook introduced its year in review, the thing that would pop up in your feed was your most popular posts that you could add to, surrounded by confetti, and streamers, and talking about this was your most popular post, don’t you want to celebrate? Well, they assumed that everyone would want to review their year and everyone would want to celebrate it. But Eric Meyer, who’s well-known in software, his six-year-old daughter had died earlier that year, and he was confronted with the face of his daughter surrounded by confetti. Other people had experiences of seeing a sonogram of a baby that had been miscarried. Or the house that burned down and the remnants of this house. So Facebook didn’t consider people when designing the year in review. They considered the happy path and that everyone would wanna celebrate.
Elle Gee: There are other examples that come to mind, and I think maybe they fit in with what you’re trying to describe there Rachel. An example that we might have is we test some life alert and alert-device-type applications and hardware. We recently had a scenario where we were testing whether or not a button was going to alert a network of a possible kidnapping, or another sad situation or tragic situation, would work in certain circumstances. That included going into bathrooms where there are thick cement walls, or walking in between buildings and so on. I think they would be examples of what you’re referring to as stress cases. Will it work in those situations where you need it the most, that people don’t normally put themselves? Would that be an example of what you’re trying to cover with stress cases?
Rachel Kibler: Absolutely. It’s using the application in real life. Walking between concrete buildings is definitely a real-life example.
Matthew Heusser: Just to jump in, if I’m hearing Rachel correctly, a lot of that would be … it’s the 911 app, and I’ve got coverage issues. Or there’s some reason that I’m under stress, and I’ve got this marginal use that is actually very important.
Rachel Kibler: Definitely.
Matthew Heusser: If I’m hearing that right.
Rachel Kibler: Definitely.
Matthew Heusser: I think that is key, when we talk about oh what if you’re on a cellphone, you go between towers and you lost coverage? Well what does the app do? What are the critical pieces of the app? If that were to fail, that could be really bad for me. Maybe it’s nothing. Maybe it’s Pokemon, but I think most of us don’t work on Pokemon.
Elle Gee: But you know what even that Pokemon player, if they’re really dedicated to what they’re doing, and hey I work in an office environment and some of my guys at lunch time, they’re out doing a raid in the parking lot. If they don’t get to complete their raid because of an outage, that affects their day. No it’s not the same situation as the life alert, but their quality of life is affected in a different way. Isn’t it just as important to not only look at the stress cases for the critical 911 situations, but to bring quality of life to every application?
Michael Larsen: So here’s a simple example that I think might fit into this stress case. Sometimes I’ll be doing something, or I’ll need to get a message out, or somebody sent me a picture on my phone or whatever, and I’m on our commuter train. I’m trying to send a picture to my wife, for example, and say, “Oh hey someone sent me this,” or, “Hey here’s a piece of information,” and because of whatever has happened, I’m now between optimal cell towers or something. I’m sending that picture, and what happens is it gets to a certain point and it just stands there. Now until that thing posts, I’m basically just … it’s cranking away and it’s trying to do its level best to send that picture, I don’t have a method to stop it, really. I have to either wait it out or wait for it to go failed to send, and either I can try it again or I can just go eh, it’s not that important. But for that moment, I’m kind of blocked from doing other things. Yes I can stop the phone, but if I stop the phone then there’s a possibility that I’m not gonna be able to continue sending the message either. I’d say that kinda fits into that consideration too. Yes, no, maybe?
Rachel Kibler: Definitely. The stress case part of it is the human element of it. If it is urgent that you get a message to your wife, it needs to work. Or it needs to throw an error, throw a friendly error, or it needs to pick right back up immediately when regaining connection. Having that moment where you are unsure whether your message got through, that’s a good stress case.
Rachel Kibler: One issue that I’ve discovered in a bunch of messaging apps, I call it the stalker blocker issue. If I’m in a group conversation, or if there’s someone who is harassing me, and I block them, because they’re a stalker, but I’m still in a group conversation, what should the behavior be? With some applications if you have a private conversation between two, and one person blocks the other, the stalker can get around it by adding a third person to the conversation, and then all of the messages go through. Others, the messages from the blocked user won’t come through at all, so if you’re in a group conversation and the person is making threats, you don’t see those, and your friends might think that you see them. How do you put the agency on the blocker, and recognize their humanity, that they may need to see critical information but they don’t always want to see it? No messaging app is doing it very well.
Rachel Kibler: It’s one thing that we’re trying to deal with at our company, with our app. Because there is a messaging component to our app. Can we blur out a conversation, or blur out the messages from the stalker so that they can be viewed? It’s an issue of bodily safety, and it’s an issue that comes up, especially in groups of mixed genders where there’s a lot of loyalty to the group, so you don’t want to leave the group, or you don’t want to alienate yourself from the group, but you still want to protect your safety and not see the harassment from one person.
Elle Gee: Rachel, based on your experiences with the stalker issue and the other stress case scenarios you’re providing, I could certainly see a lot of value in how you’re approaching this and what it’s bringing to the test community. How is using stress cases different than what we’re currently, or what people mostly are currently doing, on the engineering flow? What changes, and in your opinion, how are we adding to schedules and timelines or cost, by introducing stress cases to the testing scenario?
Rachel Kibler: That’s a really great question. If you aren’t finding these stress cases before they get to a certain point, there are likely a lot of other issues with your software, and it’s going to add cost and time later down the road. As we all know, it’s so much more expensive and time costly to fix bugs that are found in production rather than the ones that are found before they’re even implemented. Stress cases goes along with accessibility testing a lot of the time. When we make software better for people who need accessibility accommodations, we make software better for everyone. We make the flows easier. When we think about stress cases, we’re making better software. We’re caring more about our users, and we’re increasing the value to shareholders, if we’re a public company. We’re just making the world a better place and our software better. The world a better place is probably hyperbole, but we end up making better software. That’s what we’re all trying to do.
Elle Gee: Would you agree that, like with accessibility, stress cases need to be thought of upfront? Coming in at the end of the requirements phase or even at the end of testing and thinking about this is too late? My reasoning behind this question is I’ve often said of accessibility, you can’t build the building and then go back and work out the foundations are wrong. Accessibility needs to be thought of before you pour those foundations. It’s a part of every step going forward. From this conversation I get the feel that, really, stress cases should have that same level of concern as early on. Do you agree?
Rachel Kibler: Absolutely. One of the ways that I’m hoping to introduce is the concept of a pre-mortem. So if at the beginning of a project we say six months down the road, three months down the road, at the end of two sprints, this has failed spectacularly. What will have gone wrong? People are a lot more creative when thinking about how things could’ve failed. We often create this narrative at the end of a project, saying these are all the reasons why it failed, but at the beginning of a project we’re thinking a lot more creatively of, well, this could’ve gone wrong. Our App Store rating could have dropped by a whole star because users are really angry about this, and then we think more creatively and we can solve problems upfront.
Elle Gee: Awesome, thanks. I agree. I think that this would be something that we should be looking at for all of our projects, and here at Qualitest it’s something that we try to do as well is get people to think upfront, and the possibility to think about stress cases and how they might change the flow of testing is something that we value. This has definitely opened some eyes on that area.
Michael Larsen: Rachel, I’ve got a quick question. The company you work for is called Anonyome Labs. And you’ve got an app that you are currently working with. How might an everyday person be able to use what you’re using? Is this something that everyday people would be able to use? Or is this specific, what you are offering, and it’s not something that, say, an everyday tester would be throwing into their quiver, so to speak?
Rachel Kibler: Stress case testing is something that everyone can do. Our specific app is called MySudo, and it is an app that helps preserve data privacy. You create your sudos, you create these profiles, and each one has an email address, and can have a phone number. Soon we’ll have its own private browser, and you can compartmentalize your life. You can have a Sudo that’s specifically for selling stuff on Craigslist, that is not connected to your name. If you’re like me, I’ve had the same phone number, the same mobile phone number, since 2003. So you can look up my phone number and find out a lot of information about me. I don’t necessarily want to give that information out. So you can have a Craigslist Sudo, and a Tinder Sudo, and an Amazon Sudo. You can compartmentalize your life, preserve your data privacy, and not give all these companies all of the information about you.
Rachel Kibler: The weakest point of any software, that we also deal with, is the human element of it. So there’s the mud puddle test, which if someone dropped their phone in a mud puddle, and slipped and hit their head, and forgot all of their passwords, what information would they lose? We want our software to pass the mud puddle test. That if they forget everything and lose their phone, that information is gone, because we really emphasize data privacy. We emphasize protecting the people. Different companies have various approaches to this, with backups and restores and all of that stuff. The human element is just the weakest element, and we have to always contend with that.
Matthew Heusser: So the whole company, in some ways, is based on addressing a stress case not handled well by existing apps. Most applications don’t handle your desire to compartmentalize your life, your desire to throw away accounts that have become poisoned, or involved with stalkers or whatever, and then re-spool up and start over again but maintain your history. It’s just not something that you’re going to do well on Facebook or anywhere. So your company basically took a stress case and turned it into a business model, is that close to accurate?
Rachel Kibler: Absolutely. We take our data privacy very, very seriously. Every month, every few weeks, there’s another news story about a data breach or a privacy breach. I mean Facebook sold all of its records to various companies. It’s using phone numbers to let people look you up, your two-factor authentication phone number, to let people look you up. It’s scary out there, and we are trying to give the user, to give people, control over their data and control over their privacy.
Elle Gee: So based on your experience and what you’re trying to achieve by your application in giving people privacy, what recommendations would you have for other businesses, or companies like Qualitest who are specializing in testing, in how we would identify stress cases? And how we would be able to make that a higher profile in the testing asset?
Rachel Kibler: I have four main things that you can do. The pre-mortem is a very effective way to discover problems that could be implemented, before they are implemented. So asking, or presenting the scenario that we’ve failed, how did we fail? So that you can come up with solutions to prevent that failure. Another option is to put the tester under stress. A lot of times when we are under stress our cognitive abilities are impaired. It’s the same thing that happens with a cognitive impairment for accessibility. If you put the tester in a stressful situation, and I know this sounds terrible … if the tester really doesn’t like math problems, have him look at calculus stuff, have him do a bunch of math problems for 20 minutes, and then go through your flow. See if your text is read clearly enough, if your buttons are big enough, if there are confusing aspects of your flow, if anything about your app is confusing. If, Elle you said you’re not a morning person earlier, so if you are not at your best in your morning, wake up at 4:00 and do an hour of testing, taking notes that you can come back to. If you put a tester in a stressful situation you’ll get closer to how a person would behave under stress.
Rachel Kibler: Two more options. Having a designated dissenter on your team. On one of my teams here I am the blocker advocate. So every time that we talk about privacy, I advocate for a person who wants the least amount of information put out there. If an email fails, do we want to give that information to someone? It becomes having a designated dissenter who just points out the flaws and advocates for certain positions is a great thing. If you rotate that by sprint or by project or by story, that can really be effective to find problems early on.
Rachel Kibler: Another option is using personas. A lot of times we get blinded by the smiling face, or thinking about, you know, a mom wanting to do her online shopping, but if we put our personas in the worst day of their life, or just in an ordinary crisis … if we think about the persona and what could go wrong for the worst day of their life. Their dog needs to be put down, or they’ve just been in a car accident, or they have a sick child, or any number of things of what could happen on a bad day, and understand how the person would still need to use our app on that day, that can also be effective.
Elle Gee: Oh golly you have me feeling a little bit depressed right now, thinking about my dog-
Rachel Kibler: I know, I’m sorry.
Elle Gee: … or all of those things. But you’re right, because that would totally get my blood pressure up, and I would be fumble-fingered, and all sorts of things would go wrong. Here in our office recently we did do that, and I felt incredibly guilty, but you’re kind of giving me permission to say it’s okay to put people in duress so we can get the best out of our applications.
Elle Gee: An example of what we did here in the office is one of our applications that we test is an emergency alert. Its total goal is to let your network know if you are in distress or in trouble. Which fits very nicely into what you’re talking about. We talked about testing it between buildings and cement and stuff, but we took it to the practical level of saying hey, what if you grab person A and just cart them off? Can she get her fingers to the right buttons? Can she do the right thing? So my team got really involved, and that’s exactly what they did. They grabbed person A, we all heard the scream, and dragged her off. And they were so proud of themselves, but then it was like well that was a small, petite person A, what if we had somebody a little bit more hefty, a little bit more substantial? So we actually turned it into a fun event in our office, but after talking to you I see how, indirectly, we were really getting into stress case testing and using your step four.
Rachel Kibler: Absolutely, that’s a great example.
Matthew Heusser: You know it makes me think about, I don’t know about you, but there have been a few times in my life when I get some very distressing texts, and the blood rushes to your head and your heart starts beating, and not only do you make a very foolish reply on your phone, but then your iPhone autocorrects some important thing in the subject, right? You’re talking about some very complex technology, using some, I don’t know, Linux derivant that is a word that is not commonly used, and your iPhone helps you by autocorrecting you. So your reply makes you look like a fool, and then you have to go and try to fix it later. It’s almost like your iPhone should have a slow down, I’ve gotta get this right mode. Like, “Hey, please reread your message.” I don’t know if that’s ever happened to you. It seems to me that in my most stressful moments of texting, that’s when iPhone autocorrect does me the most favors.
Elle Gee: It gets me every time, and I dearly wish at times I could just do a delete. The applications that allow me to delete that text and just redo it would be far preferable.
Matthew Heusser: Yeah, like an edit button.
Rachel Kibler: Well if I can make a shameless plug, at this very moment, our in-network communication are end-to-end encrypted, and editable and deletable. Stress case right there.
Matthew Heusser: Nice.
Michael Larsen: That sounds supremely helpful. I want to acknowledge the fact that everybody here … we’re doing this in the middle of the day, and I know that we’re taking everybody’s time, this is literally lunchtime for myself and Elle. I can’t speak for Matt or for Rachel. This sounds like a good place where we could say final thoughts about this. Rachel, let’s give you the word since you are the expert here.
Rachel Kibler: I think stress cases are another tool for considering testing. We haven’t been talking about them very much. There are some people who are starting to talk about it. There’s a book out there called Technically Wrong, written by Sara Wachter-Boettcher, and it is amazing, and if you look at her other stuff, her interviews and her other books, they all talk about stress cases in particular, and her work is just so inspiring, and it’s getting the conversation started. I think it’s another tool for testing, and thinking about stress cases just makes our software better for everyone.
Michael Larsen: Excellent. Elle, thoughts?
Elle Gee: I definitely know that we’ve always had a strong focus on personas. Hearing Rachel talk about personas touched us very closely, and aligned with what we’re doing. I want to say that following this conversation and the focus on stress cases, I want to see our Sequoia test take the personas, and the four steps that Rachel described, to another level and bring stress cases into a stronger focus. It’s quite common that usability and stress cases, as described here, or even the use of personas, it represents maybe five to 10% of the testing capability. I think we all agree that it needs to have a stronger focus on that. I’m very glad that in my role I’ll be able to help influence our testing to make sure that that happens. I’m really grateful to Rachel for bringing this to my attention in a new light, that gives me an opportunity to be passionate about testing in a new way.
Michael Larsen: Excellent. Matt, what do you think about all this?
Matthew Heusser: Well the biggest thing, I don’t think we’ve mentioned, is that I think if you did it right early enough, you would smoke out a ton of functional bugs too. So then you’d find important functional bugs earlier, and you could actually reduce and compress timeframes. Anytime we say oh you should add a risk management step here, oh you should add a review step here, oh we should have code review here, I get a little ehhhhhh, because the more steps you add the slower it takes, because that’s how math works. But I think this might be one where we can add a step and go faster. That’s the kind of thing that excites me.
Michael Larsen: I’m with you there, and frankly, when we were starting the lead-up to this conversation, I had a set idea of what we were gonna be talking about, and this has turned out to be a completely different conversation. I’m really excited when that happens, so I’ve got a whole new avenue of ways to thinking about stress cases that I wasn’t even thinking about before, so yay. So alright, we want to give everybody a chance to say what they’re up to, where they’re going. Rachel, I know that you’re going to be speaking someplace pretty soon.
Rachel Kibler: Yes. I am doing the SpeakEasy keynote at Agile Testing Days USA, at the end of June. SpeakEasy is amazing, an amazing program for new speakers, so this is going to be only my second conference talk, and my first keynote. I’ll be talking about this topic, stress cases, how to test for them and how to think about them. How to get that mind shift. I have a couple of other engagements this year that I’ll announce on my blog, which I think are gonna be in the show notes, and yeah I’m just really excited for all the things that are happening right now.
Michael Larsen: Elle, what are you up to?
Elle Gee: What’s coming up for Elle? I’m gonna continue what I’ve always been doing, which is providing testing services for our clients and our stakeholders. This includes, hopefully, going into the future, a stronger focus on stress cases and how they might influence our testing efforts, and help to provide a better quality of life for our users when they hit the market. No big surprises or speaking as it’s coming up for me.
Michael Larsen: Excellent. It feels strange for me to be asking this, Matt what are you up to?
Matthew Heusser: Well, I’m giving a tutorial at Agile Testing Days on implementing continuous delivery with DevOps. Trying to cut my … well testers leading continuous delivery efforts, that’s really what that’s about, testers leading DevOps efforts. I think we have some risk management skills that are critically important for companies that are trying to compress cycle time. Cutting my travel down, so I’m not gonna be a whole lot of places otherwise. Check out my website, I might get up to Kitchener-Waterloo this year. On the research side, we’ve been doing a lot at Excelon on Kubernetes, and cluster management, containers, and build deploy pipelines. That’s what I’m excited about.
Michael Larsen: Excellent. On my end, well by the time that this show airs, the speaking engagement that’s coming up for me will already have happened, it will be STPCon. In addition to that, I’ve submitted for a couple of conferences, haven’t heard back, so I’m not at liberty to say if or when I will be speaking in the near future, so I guess this is just one of those watch this space kind of situations. But with that, I want to say thank you to everybody for your time, and here’s to future days. Thanks so much for joining us.
Elle Gee: Thank you.
Matthew Heusser: It’s been a pleasure.
Rachel Kibler: Thanks so much.