The Testing Show: Accessibility

Panelists:

References:

• Mirai (malware)
• Samsung Galaxy Note 7 users’ phones are blowing up, literally
• The Galaxy J5 is the next Samsung phone to explode, woman says
• Adventures in Tautology – What Makes a Tester a Tester?
• Samsung finally explained why Galaxy Note 7 batteries are exploding
• #a11y
• The War Amps
• Section 508
• Americans With Disabilities Act
• Web Content Accessibility Guidelines (WCAG) Overview
• Making a Web For Everyone – Agile Testing Days 2015
• Inclusive Design Toolkit
• pa11y
• aXe: the Accessibility Engine
• Tenon

Transcript:

MICHAEL LARSEN:  Hello and welcome to The Testing Show.  Today we are joined by a wonderful cavalcade of guests. First off we’d like to welcome our regulars. Justin Rohrman.

JUSTIN ROHRMAN: Good Morning.

MICHAEL LARSEN: Perze Ababa.

PERZE ABABA: Hello everyone.

MICHAEL LARSEN: Garry Heon.

GARRY HEON: Good morning.

MICHAEL LARSEN: Matt Heusser.

MATTHEW HEUSSER: Hey, welcome.

MICHAEL LARSEN: I’m your producer, Michael Larsen, and today we have a special guest. We would like to welcome Alicia Jarvis to the show.

ALICIA JARVIS: Good morning. I’m happy to be here. I’m a testing lead at RBC and I just came back from the Accessibility camp Toronto.

MICHAEL LARSEN: Excellent. So, normally, we let Matt run the show, as he is the moderator, so I am now going to turn the time over to him and let him do his thing.

MATTHEW HEUSSER: Thanks, Michael. Welcome everyone, thanks for coming. Let’s start with the news, specifically this Samsung Galaxy… now, the Note 7 has been recalled and now, apparently, a Note 5 has blown up. That seems like a quality issue to me. What’s that about? Any thoughts?

PERZE ABABA: The funny thing was, in the past couple of weeks, I’ve been traveling back and forth to Asia, and it’s been very apparent and very clear that right after when the US Department of Transportation issued that statement saying “this device is not safe”, a lot of the other government agencies and airlines have really picked up on that. I was in Japan, and then eventually got to the Philippines, and everybody, right before the plane takes off, everybody’s reminded, “if you have a Galaxy Note 7, you have to hand it over to us, and we’ll put it in a safe place, in a box somewhere. You can’t turn it on. You can’t charge it.” It’s very apparent that everyone has seen the effects with this particular problem. It’s no longer airlines now, but I’ve actually seen notes for universities that are banning it, companies that are banning it. There’s even a cell phone provider in New Zealand that’s actually detecting if you’re using a Samsung Galaxy Note 7, and it will block your cellular signal. This is definitely bad for Samsung, and I’m actually very interested in how they would get out of this rut at the moment.

MICHAEL LARSEN: Lithium Ion batteries having strange problems, or getting extremely hot, or short circuiting or causing fires… this is not new. This is not something that is only existing in the Samsung Galaxy world. I’ve actually got discolorations on my thighs because of a Toshiba Sattelite laptop that, in the middle of a flight, just got extremely hot on me. I had to make the decision then and I just unclipped the battery and pulled it out. Pretty much killed what I was working on, but it cooled down and there was a short in the battery and it was causing a problem. Got a replacement battery and never had the problem again. What makes this unnerving, of course, is the fact that these cell phones are with us 24/7, and again, because of the very thin and compact nature of these devices, there just isn’t a lot of margin for error. It’s a whole different world of how you test these because those are outsourced. It’s not like Samsung makes the battery.

MATTHEW HEUSSER: There’s a couple of interesting points in there, and I think there’s a real tie in to software development, and this may be a bit of a stretch, but about ten years ago I was doing a lot of work on technical debt as a metaphor. Basically, people want to have their cake and eat it, too. They want to have “get it to us earlier, maybe even with lower defects, and all the functionality”. When you do that, you take shortcuts that are not one of those three things, and usually it’s the code quality. There will either be defects that don’t matter, we can defer, or the code is just ugly and bad and legacy, and your progress slows down because you have to pay interest on this debt. When I started doing research in this area, one of the common responses was to say, “No, we’re not going to take on tech debt, we’re going to have our cake and eat it, too. We’re going to create metrics and measures. We’re going to measure the cyclomatic complexity and we’re going to measure the code coverage, and we’re going to come up with these percentages, and we’re going to insist on this certain high percentage. It just seemed like a never ending cycle. If you insist on that, too, and you measure it automatically in a system, people will find other ways to take on debt that are more hidden, and you will get crappier code that has good cyclomatic complexity. My point there with the phones is that the apple phones kept getting more powerful but the battery life kept going down, to the point that Apple actually makes a little pack –it’s a carrying case for the iPhone 6- that actually, really, is just a bigger battery, because the battery life is so short. Some manufacturer, kept pushing for lithium batteries to be smaller and thinner and tighter and every possible dimension better, and then BAM! There’s this huge surprise of, “Yeah, it does all those things, but sometimes it blows up!” It’s a result of lazy thinking, a result of, “I want to have my cake and eat it, too” without innovation. Iterating over these devices make them a little bit smaller, a little bit lighter, battery lasts a little bit longer. Sooner or later you’re going to run into a physics problem. Am I wrong?

PERZE ABABA: I would go against the grain on that, Matt, when you say that the question is not really on innovation. I think the problem is too much innovation and less thinking about scalability and maintenance. Mike, you blogged about this, I think, a couple of weeks ago. That seems to be one of the challenges now, where there’s always that expectation to innovate, but then as we push all of these MVPs, or Minimum “Lovable” Products, there’s hardly any conversation of, “What’s going to happen when we start supporting these things?” I think this is the challenge with what you said; you’re looking at a very thin battery that is now very susceptible to fires.

JUSTIN ROHRMAN: Is this really an innovation problem? Is there something that Samsung is doing differently with these note devices that was just radically different from what every other phone manufacturer was doing? To me it just seems like “another phone that’s like the other phones”, but this one, its marked differentiator was combustion… and that’s coming from “I’m not a gadget person.” I have a four year old Samsung Galaxy phone, and it doesn’t blow up on me. That’s one of the perks of never buying new phones; you don’t get the cool all new combustion features.

MICHAEL LARSEN: Is there something unique, and is there something different… I don’t necessarily think that there is, especially in the Android world. I think a lot of these phones are put together by the parts that are available. It wouldn’t surprise me that numerous manufacturers use the same battery pack. Is the problem with Samsung specifically, or is the problem with the source for the battery pack itself? And if the battery pack is made by Samsung, then OK, Samsung’s got something to answer. Somebody’s not doing what they should be doing. If, however, they’re outsourcing it from another company, what’s going on? What else is waiting in the wings that we don’t even know about?

MATTHEW HEUSSER: It is interesting that we don’t have a lot of specifics, it’s, “Yeah, sometimes it blows up!” [laughter].

PERZE ABABA: I actually saw an article… I think it was yesterday, it was by Ricoh, and they were talking about the problem might have been because of the distance between the anode and the cathode of these batteries that they have on these Galaxy Notes. They know that you’re supposed to separate those two, but that its really up to the manufacturers to create that separation, and whatever that distance of separation is. That might really be the case, that it’s close enough, they could cause a spark that could cause all of this. You’re right, Matt, I wish they would get down to specifics. Make this a really good case study on how to not design your battery or components around it, because we can definitely learn from this.

MATTHEW HEUSSER: So the exploding phone is the worst accessibility issue ever. Most of the time, most of us do not have to deal with an exploding phone, but we do have to deal with “entire segments of our customer base can’t use the software” and I think that’s what Michael wanted to… it was Michael’s idea to do this episode on that, and that’s why he invited Alicia.

MICHAEL LARSEN: So one of the things I think is important… first off, Alicia, I want to say, “Thanks very much for coming on the show and being a part of this”. I follow a considerable amount of your dialog, and the conversations that you have, and it’s led me to a lot of interesting insights on testing, especially around accessibility and something else that I am actively involved in as well, which is Inclusive Design. I’m going to step back and ask you to share a little bit about your own exploration and world of accessibility and #a11y advocacy and tell us a bit about why you are the passionate voice that you are.

ALICIA JARVIS: Sure. So, professionally, I started in accessibility, kinda’ much like anyone else, that started in accessibility. Eight years ago, I got a job at Bell Canada. I got it thrown at me, saying, “The CRTC has mandated accessibility, we have this amount of money, and we need to come up with some products and services that make sense”. How can we do this, essentially. That was my professional start, but it actually started way before that. I grew up in the War Amps Child Amputee Program, in which I got introduced to technology at a very early age. Just by being an amputee that’s missing both arms –I do have hands, but basically short arms- so they would throw tech at me. Whether that was artificial limbs, computer at a very young age, all of this stuff, to see what worked for me. That whole concept is what really drives my work today; instead of looking at it from a compliance or a legal aspect, let look at it as what technology is really there for, and it’s to help every one of us live easier, more productive lives. If you design something, we can design it in a way that it can appeal to the majority but also take into consideration people that have different needs. IF you can’t see, how are you going to operate your iPhone, or that sort of thing. So that’s really where I started. Section 508, and what’s happening in the US with the Americans With Disabilities Act, it actually does influence my work a lot, because dealing with different vendors, we’re often dealing with vendors that are from the United States, so having a broad view of the legal aspects and what the different legal components are, really help to talk to your various vendors and start the conversation.

MICHAEL LARSEN: I’ve had a chance to do a number of talks around accessibility, and the way that I often start those conversations is, unfortunately, we start from a point that is not always pleasant, in the sense that  we’re finding ourselves behind the gun and having to take a product that’s  already been designed and shoe-horn accessibility into it, or it’s because we’ve been threatened with legal action. So there’s always something kind of negative, that starts that conversation. Not always, and thankfully, once people kind of get on the ball with it and they realize that, if you design with this in mind up front, it doesn’t have to be painful.

ALICIA JARVIS: Well, I think that’s where the concept of Inclusive or Universal Design comes into play. For designers and business people to realize that their users are not just people that use a mouse. People who use databases a lot won’t necessarily use a mouse. They’re power keyboard users… and that’s one thing to keep in mind, is that a lot of people, especially now with things like Siri and Cortana, less and less people will be using a mouse. They can use other means, including their voice, touch, that sort of thing. So, I think we need to move away from looking at our users as mouse only users right from the design concept. When we start doing that, it becomes easier, because your designers will actually design in a way that’s like “oh, at least I know by the design I know that they’ve thought about it, even though all the tech aspects from a development point of view, and code aspects maybe aren’t quite worked out yet, at least from a design perspective they’ve thought about, “Oh, how are keyboard only users going to navigate through this or how a screen reader user is going to navigate through this, or what does this look like when we magnify it to beyond 200%?”

MATTHEW HEUSSER: Yeah, I think that spells out two very different accessibility, testing-y worlds. One is where you do a gap analysis and do the minimal so that you can pass your audit, and the other one is that you actually design for that customer in mind. I’m a big advocate of the Three Amigos Meeting, where before the code is written, you have the developer, tester and product owner actually talk about what the goal is for the customer. It’s really not new, remarkably not used much, but some companies have actually run with that and they’ve added the graphic designer and the user advocate. Clearly, if we designed for accessibility, it should get easier to test for it. We’ll have less problems when we get on the back end. What’s your check list? When you go in and you’re trying to design for accessibility, what are the kinds of questions that you ask, to make people go, “hmmm, I don’t know how you would do that with a screen reader.”

ALICIA JARVIS: The first thing is to debunk the myth that everything is about a screen reader. Screen reader is usually one of the last things that I test with, only because if you go by WGAG and you test against WCAG, you’ll solve a lot of the screen reader issues along the way. With that, my first question is “how is a keyboard only user going to navigate through this?” The other question, especially from a design point of view, is color contrast. What a lot of people may not know is that a high percentage of particularly men have color blindness. From a design perspective, if your color scheme doesn’t have good color contrast, you’re leaving out a good chunk of your population, including your developers. That’s the first thing on my mind; let’s get the color scheme right. Let’s take out all the color contrast issues right from the design, so that when it gets to QA, and we run it over with a color contrast analyzer tool, from the QA perspective, I already know that it passes. It’s just a matter of testing it against the wireframes or prototype, to make sure they are using the right hex codes. From a screen reader perspective, basically the first question that I sit down and ask the team is, “What is a screen reader?”, because I think it’s important for the tam, including the product owner, the developer, the designer to understand “what are we talking about?” What we’re really talking about is another piece of software that people use to access your software. One thing to keep in mind is that piece of software is exactly that. It has its own bugs, its own challenges, its own things. To really trust one screen reader to test, it won’t produce the best results, so that’s where WCAG comes in, making sure that you’re coding to standard code.

MICHAEL LARSEN: My partner in crime, or the guy I like to refer to as my partner in crime -who’s also up in Toronto, his name is Albert Gareev- Albert and I do a lot of stuff for Weekend Testing Americas, and we do sessions for it and some of the sessions we’ve actually had really good success with are when we discuss Accessibility tools, because for a lot of testers it’s fairly exotic and it’s like, “Ooh, hey, this is interesting. What is this all about?” There are a number of tools that we can get into, where you’re just doing a checking of HTML to verify that things are tagged correctly, and that you’ve included ALT tags where necessary, and that you’ve given enough context on the screen so you understand what’s there, but there was an interesting comment that came up. We were testing, for example, color contrast, and it’s one thing to run through and use a tool, and have it tell you, “You have 76 errors on this page.” It’s another thing entirely to be able to understand what the errors are telling you, and what effective changes you might be able to make. The real question is, “Am I taking time away from the person that is using this product, who needs special accommodation so that they can use it?” And I also point out that accessibility is not limited to people on the far end of the equation. We oftentimes look at it from the perspective of “Oh, you are sightless, so you need a screen reader. Oh, you can’t hear, so you need closed captioning.” All of that, though, comes down to a deeper question, which is, “What is it that you are actually trying to accomplish?” Sometimes I go into method acting mode. If I really want to get into the feeling and understanding of this, I will put myself in a situation where I can’t hear or I can’t see, or I will limit the use of my arms or my fingers to where, you know, I’m greatly hampered. But I always have to remind myself, at the end of the day, I can take those impediments off, and I can go home… well, with the exception of the fact that I’m getting up there in years and now a pair of glasses, readers, are sort of essential for me. I can step away from them, and I can go about my day. Many others cannot. They don’t have that luxury.

PERZE ABABA: Mike, to put specific numbers to that, in the last US Census, which was in 2012, there’s supposedly one in close to six people that is within the accessibility spectrum, which is roughly translated to 56 million people in the United States. So, there’s definitely a need. At this point in time, there should be no more qualms about putting this front and center. And I really like how Alicia points it out in the beginning where she said, “Inclusive Design really needs to be front and center. It needs to be part of the conversation.” For me, it needs to be part of the definition of done.

ALICIA JARVIS: And also, the disability market… companies are starting to realize the market potential in this. The fact is, if you look at the numbers of people with disability worldwide, there’s a market the size of China, and it’s growing. The thing is, we are the single largest minority because every single person is going to be a person with a disability at some point in their lifetime. What I mean by that is that we also all experience “situational disabilities”. There are going to be times when you can’t hear. There’s going to be times where you can’t see. The most common example of that is in a bar, when they’re showing the latest sports game on the television. You have closed captioning usually going on there, because it’s a noisy bar. People can’t see the game, but they can see what the commentary is at the bottom of the screen.

MICHAEL LARSEN: Also, a couple of years ago when I was first doing the talk, on the intersection of Accessibility and Inclusive Design, somebody in the room just happened to raise their hand and say, “You know, we’ve already done a really big push towards developing for accessibility, and it’s a by-product effect, as a matter of fact”, and he held up his cell phone and he said, “the fact that I can go to a number of sites, and the fact that they have optimized it to appear on this screen, they’ve already done 90% of the work necessary to make this product accessible and inclusively designed.

ALICIA JARVIS: Exactly.

MICHAEL LARSEN: The remaining ten percent is taking the rest of the distance.

MATTHEW HEUSSER: As a tester, I’ve actually recently run into a couple of scenarios. So we had a client at a multi-billion dollar company, developing internal software –we had an analyst on the project- and one of our early questions in the requirements gathering was “What about Accessibility?”, and the response was active dismissal. It was “we don’t have to do any of that crap because this is internal software for the team. But what if someone on the team had poor vision? I struggle between “We’re only doing this because we have to make a government requirement”, which is a common attitude, and then there’s “it doesn’t make economic sense for us to do this. We wouldn’t invest that much money for the one team member that needs glasses” with “Well, we’re kinda’ screwing that one team member that needs glasses”. We’ve been talking about the technical pieces, but those sound like business decisions. Gary, do you have any thoughts on that?

GARRY HEON: Well, you know, listening to what you just said, I think the decisions are really short sighted because of the market that is not being addressed if a site is not accessible.

ALICIA JARVIS: To jump in there, I think that, again, goes with a mindset shift on the side of the business, and a business mindset shift in that it’s been proven that people with disabilities, because of how we live our lives -just getting up in the morning, just getting dressed in the morning, doing my morning routine- there are different things that I have to think about that an able bodied person may not have to think about. We end up to be natural born problem solvers, because we have to be, because the world is not designed for us. We have to find ways that work for us. When companies say “oh, it’s just employee facing, it’s just for our team, it’s a product that’s going to be built in house”, I really like to challenge that, because your employees are your first customers. If an employee is not happy in their job or where they are working, and isn’t proud of the company that they’re working for, and can’t do their job because the systems don’t allow them to be the best employee they can be, I think we’re doing a disservice. To all of our employees. I would like to think that my colleagues could live up to their full potential just like anyone else. We need to really shift that mindset of “oh, it’s internal it’s employee facing”, and really think about “what is that really saying to all of our employees when we say that?” Not just to the person with a disability… and really think about it from that perspective.

MATTHEW HEUSSER: Yeah, thank you. One thing I would add before we get going and we’ll do final thoughts and what we’re up to… when we say “oh, it’s internal, it doesn’t count, we don’t have to worry about that, pfft… you’re stupid, you’re silly”, there’s a couple of things going on there. I wonder, if you do good, usable design from the beginning, could you go faster? Instead of looking at it as, “Oh, we’re going to have this extra testing phase where we have to do all this extra work”, could we go faster? Even if you can’t go faster, would it help your developers build skills they’re going to need later in their career? And the answer to that is, “Definitely yes”. The next question would be, “What are we afraid of?” We’re not doing usability, real usability, in any meaningful way. It’s because we don’t have the skills. We don’t know how, and it’s scary, and we think it will slow us down. Those are pretty terrible reasons. To do something, I’ve gotta’ say. That’s circling the drain, and when there’s a change in the marketplace and we need to find a new job, we will have less skills. I’m not buying it, my short answer, and I think that will be my closing thought, and we can go around and do closing thoughts. Justin?

JUSTIN ROHRMAN: I’m mostly interested in the “how” of Accessibility testing. I was working on a project for an agency company earlier this year, and we were making a website for a long distance carrier. One of the things we had to do I accessibility testing, and I was on the project really briefly, and I was presented with what seemed like just a checklist. Like, just a simple list of things I need to go over.  And once the checklist is approved, then the website is officially accessible.  So I think that’s what a lot of people encounter with accessibility testing, and the stories I have heard today lead me to believe that there is much, much more to it than a checklist. I’d like to see actual, real accessibility testing, like live in action.

MICHAEL LARSEN: Come to weekend testing this weekend.

JUSTIN ROHRMAN: [Laughter] I will if I can.

MICHAEL LARSEN: I’m going to give a shout-out to strongly suggest looking at the work that the University of Cambridge in the UK has done, and look at their Inclusive Design Toolkit. I’ve been talking about their information for awhile, and they actually make physical products, because they also test things that are outside of software. The two kits they have, one of which is a tactile tool,, which is a pair of gloves that you can wear. They are these broad bands, and if you adjust them, they make your bending of your fingers harder. The only downside is, they’re not cheap. They cost 200 pounds… I mean 150 pounds, excuse me, 200 dollars to get a pair. But what is affordable, and I would strongly encourage, if anybody wants to do this and wants to get a feel for it, they also have what’s called a progressive glasses pack. So you put on these glasses, and they start to fuzz up your vision. So if you want to see what it feels like to go from imagining you have perfectly good eyesight to wondering what it would feel like to have to use level three reading glasses, just to be able to read something, you can stack these glasses on top. I actually did a blog post about this, and I did an image where I took pictures through the progressive lenses being stacked on top of each other and you go from absolutely crystal clear to “oh my gosh, what am I looking at?” Even if you don’t go and get the physical tools, look at the stuff that the University of Cambridge has done around Inclusive Design.

MATTHEW HEUSSER: OK, Perze.

PERZE ABABA: One of the projects I’ve been working with the past two years, with the company that I’m with right now, is a ton of consumer sites. So you’re looking at close to 800 sites and of the 800, I’m directly responsible for 100 of those, and then you’re looking at, maybe, a little over 100, so if we were to round that number, you’re looking at 10,000 pages that we’re validating. Accessibility is definitely something that we care about, but the challenge now is how do you get to validate close to 10,000 pages on a given pull request? One of the things that we really greatly relied on, starting this year, is a tool called “pa11y” or “Pally” for short. We do understand that, look, lust like in testing there’s the checking side of things and then there’s investigation. We do rely on what we deem as experts to help us in digging deeper beyond the actual automated reports that we get, but having the ability to be able to track what are these things that have been plaguing us over time? Whether it’s the contrast ratio for some of our images and backgrounds, as compared to which particular guideline are we actually having problems against? For those of you who haven’t seen this tool yet, it’s an open source tool that’s based out of Node.JS. You can play around and the reports have been something that I’ve used right now to dig deeper and try to understand what the actual challenges are, so if you’re looking for a tool to help you scale and existing project that you have that needs validation on a regular basis, I would highly recommend using pa11y. Thank you so much for having me in this conversation right now; it has definitely opened my eyes on what else I need to look into, and Alicia, you are definitely an encouragement and an inspiration to me.

ALICIA JARVIS: Oh, thank you. Just to add to that, pa11y is a pretty good tool. There are several tools out there. A few that I use and I’ve started using are aXeCore by Deque, Tenon… Tenon is a little more advanced, so if you’re looking at it from a development point of view, Tenon would probably be very beneficial. There’s a few others. BlueMix, I know. IBM has an experimental tool and IBM also has a mobile accessibility testing tool as well. Those are my kind of go to tools. I know there’s a lot out there, but those are the ones that are really trying to get the false positives down to a minimum. A lot of the tools out there may have a lot of false positives and a lot of problems and issues, o those are my list of go to tools. One thing to keep in mind with tools is, I like to say, it catches the low hanging fruit, don’t use it as “we ran a tool, now we’re done.” It should be “we ran a tool first, now we do our manual checks, and then finally you check with your assistive technology. Whether that’s a screen reader, whether that’s zoom text or Dragon or Siri or any one of those. Using an automated testing tool I absolutely imperative, especially when you want to scale.  You can use it to wake up and get to the office at 9:00 a.m., and look at your reports and say “hey, OK, this is what it spit out, here’s my defects I have to look into further. That’s pretty much my ending point from a testing perspective is I get a lot of questions on “How do we do this?” I think one thing that was said was “we have to get a checklist”, and that’s very important to understand that it’s not just a checklist. You have to start from the beginning with your user stories. It has to be included from a business perspective and all the way through the project lifecycle. So testing is only one component. When you’re testing it’s really imperative that you start with your automated, then your manual, and then any assistive technology that you require. And that’s pretty much closing remarks from me.

MATTHEW HEUSSER: Where can people go to learn more about you and what are you working on? Long time listeners to this show know all about the regulars, so Alicia, do you have a website?

ALICIA JARVIS: You can follow me on Twitter at @ajarvis728. I’m also on GitHub. I just,actually, got on GitHub not that long ago. I don’t have much up there yet, but you can definitely find me on Twitter and on LinkedIn, and definitely check out the hashtag #a11y, and you can get all the updates and se what people are up to in the accessibility space.

MATTHEW HEUSSER: Great. Thank you. And with that, I think it’s time for us to say goodbye. Michael will tell you on the show bumpers how to send us email. We’d love to hear from you and thanks for listening everyone.

PERZE ABABA: Thank you guys.

ALICIA JARVIS: Thanks.

MICHAEL LARSEN: Thanks very much.

JUSTIN ROHRMAN: See ya.