Insights Podcasts The Testing Show: New Year’s TesterLutions

The Testing Show: New Year’s TesterLutions

January 5, 2017

It’s a new year, and it’s that classic time for people to make New Year’s Resolutions, as well as quickly run out of steam trying to actually succeed at them. We discuss ways in which we have set goals, or not set them, how we have been stymied in the past, or how we have pushed on regardless of failing, and the fact that failing is often the key element that helps us progress and ultimately succeed.

Also, in the news, we look back on the fifth anniversary of the death of Christopher Hitchens, how a typo may well have been the root cause of “The Russian Hack” regarding the U.S. Presidential election, and will all exploratory testers be replaced in five years by AI, neural networks and machine learning? We have opinions on all of those.

















MICHAEL LARSEN: Hello and welcome to The Testing Show. Thank you for joining us today. We have a full panel of guests with us today, so let’s start it off. Let’s welcome Jess Ingrassellino.



PERZE ABABA: Hey everyone.

MICHAEL LARSEN: Justin Rohrman.



MATTHEW HEUSSER:  Hey. Welcome to the show, everyone. So I wanted to get started today. We are recording in December, and this will go live in January, but today, the day that we are recording, is the fifth anniversary of the death of Christopher Hitchens. I don’t know if anybody… if you’re familiar with Christopher Hitchens, but he was a journalist and skeptic  in the 20th century. Really well known for challenging the status quo. I think is most challenging, most sacrilegious piece, was “Mother Teresa: Was she really that great?” He looked at it from an angle:  What did she actually do? Who did she actually help?  What have you heard, and what actually happened?  He was a sort of investigative journalist and critical thinker.  He talked about knowledge and… I guess I’d say that as a theist, as a believer, I like Hitchens because he challenges in a way that’s relatively respectful.  He’s a sceptic, so he questions everything. Hear him talking about the cup of certainty, that when the cup of certainty is pressed to your lips, and all of the answers,  I love the easy answers, are right there, that what you should do is you should reject it,  and keep asking questions.  That’s testing in a nutshell. How many times have we added the most value when someone said “We are going to push to production today at 3 o’clock” and we’ve said “Oh, really? Let me see.  Did you know that it does this?”  I think there is a lot of value in that. Not to be negative or cynical or critical or whatever, but just continuing to ask questions and rejecting easy answers and false certainties.  It’s just such a big part of our job; it’s how we move the industry forward.

JESSICA INGRASSELLINO:  I was just thinking, in the way that it relates to The Renaissance Tester show, that questioning can draw on many different disciplines. There is almost always that person who says “Oh wait, but what about this?” As testers, many of us have other interests and other backgrounds. I see people who forget that they are very knowledgeable if not experts, in another realm, and they can bring the questioning that they would use in that realm into they’re testing practice. I also see what worries me a little with newer testers is that they read a book or an article or a website and they’re like “No, this is how I have to do it.” Well, why? Why is that the way that you have to do it?  There are, sometimes, legitimately good answers to those whys, but it’s important to keep the question alive.

MATTHEW HEUSSER:  I’m always skeptical when people say “You’re doing it wrong” or “this is the right way to do it.”  It really should be “If you do it like this, these are the results you will get. Is that what you want? If you do get this other way, these are the results you will get. Which of these two do you prefer, according to your value system?”  it’s always about trade-offs. Right and wrong, yes and no, good and bad, it’s kind of childish, the thought that there is “the right way to do it”.

JESSICA INGRASSELLINO:  It’s also not very applicable to certain situations, which really relates to a lot of dialectical thinking and philosophy, which talks about opposing forces; the yes, the no, the black, the white, the right, the wrong,  to look for the space in between  by saying “Well, on the one hand… on the other hand…  what are the polar pieces of this, and where does it come together, fit in between. How could somebody see this as right? How could somebody see this as wrong? Where’s the middle path? Where does that happen?

MICHAEL LARSEN:  one of the sad things about the drift away from Perl. One of the great Larry Wall’isms, that I still like to consider, is the term “Tim Toady” (TMTOWTDI), as in “there is more than one way to do it”.

JESSICA INGRASSELLINO:  Oh, now I have to look that up. You’ve just given me a research project and you didn’t even know it.


MATTHEW HEUSSER:   oh yeah, yeah, “there is more than one way to do it”. Perl as a programming language has a bunch of different ways to do things, and you can do whatever you want. The number of spaces? Whatever!


MATTHEW HEUSSER:  we don’t care how many spaces you have when you indent. One line…


MATTHEW HEUSSER:  Where you put your curly braces? We don’t care.

JESSICA INGRASSELLINO: I would be such a failure as a Perl programmer [laughter].

MATTHEW HEUSSER:  And in Perl, that’s the argument against it. It’s easier to shoot yourself in the foot. It’s easier to write a bunch of gobbled spaghetti code, but my curled always looked good, because I kind of cared about being consistent.  Next piece of news, There’s so much going on… the Russian hack, and I’m really uncomfortable with calling it a hack, because it was… someone used a phishing attack, which was very low-tech, to get the username and password from Podesta, and then they did that, and then they got a whole bunch of emails and gave them to WikiLeaks. So the American people actually knew what their elected representatives were emailing each other…  and apparently that was done by Russia, and now it’s “hacking the election”.  When I hear that, I think they broke into the ballot boxes or the computers, or they somehow changed the votes, all they somehow change the counting system, but it’s more like “they let the American people see the emails of their elected representatives, talking about things that they were doing that were not good.”  Is that really Russia hacking the election? Is that really a Russian hack?

PERZE ABABA:  is it really a hack, or was that just influence to push public opinion into a different direction?

MATTHEW HEUSSER:   So we talk about “growth hacking”, which is a strategy to get people to use your stuff by giving away free things that you manage to stick in their email signatures or whatever, or providing people incentives. In the old sense of, like, a clever technique to change an outcome, I guess it was a hack.

JESSICA INGRASSELLINO: I feel like it was social engineering.

PERZE ABABA:  I agree, too. Because I mean if you look at that New York Times article that shows how it was framed, and how the mistake was made, was because someone within the DNC looked at that email and said “Look, this is an illegitimate email. Don’t click on it”, and for some reason, that person typed “legitimate” instead of “illegitimate”, and that’s how that link was clicked. Everything else was accessed after that.

MATTHEW HEUSSER:   it was AutoCorrected, right? That was the theory?

PERZE ABABA: I’ve played with that after I read that. If you type “illegitimate” and “legitimate”, I didn’t see any AutoCorrect triggers.

MATTHEW HEUSSER:   if he had spelt it wrong, I don’t know, it’s really weird. I can’t imagine. Maybe he just type-o’d, and forgot the “i”. That’s just ridiculous. So this huge thing, which potentially changed the election, all comes down to “someone made a typo”. So we are going to be in business for a long time, I think…  which brings me to my next piece of news. At Agile Testing Days in Germany, Gojko Adzik was giving a keynote[x], and he said, “all exploratory testing work, those jobs will go away in five years because of the rise of artificial intelligence, neural networks, and machine learning.

JUSTIN ROHRMAN: Well, is he presenting a keynote from 10 years ago?

ALL: [Laughter]

JUSTIN ROHRMAN: That’s not exactly a new sentiment.

MATTHEW HEUSSER:  The first time I heard that job was going to go away was 2004 at STAREast, when I presented on test driven development. Of course, it’s been five years since “Test is Dead”. Test is dead was 2011. Can you believe that? Been five years; not dead yet!

JUSTIN ROHRMAN: I feel like people who make that assertion just don’t really understand what testing is. Like if you think it is going to go away, you clearly don’t understand what testing actually is.

MATTHEW HEUSSER:   From what I could tell, he was very specifically talking to quick attacks, cheat sheet level, “I’ve got Elisabeth Hendrickson’s paper printed out on my desk, and all I do is send in bad input to the thing and look for it to fail” kind of stuff.

JUSTIN ROHRMAN:  So he’s saying “more of the stuff that can be automated will be automated in the future”. That sounds reasonable.

MICHAEL LARSEN:  That’s my take on it, too. Yeah, if it is something that can be codified, then sure, I do think it’s safe to say that it probably will be automated at some point, but you still have to actually understand what a program is doing and what the steps are that matter before you can get to a point of actually automating it. I think we talked about that in the last couple of shows, didn’t we? [Laughter]

MATTHEW HEUSSER:   I think it would be interesting to create a tool that used those things to simulate and got some of the benefits from quick attacks. I’m talking to people about that tool, so I don’t think now is the time to elaborate on it, but I think to say that it’s all going to be done in five years is premature. Newspapers aren’t gone. The radio isn’t gone. I have a hard time seeing human thinking testers gone. There might be less of them. It might be hard to find the “cool jobs” at the next Google. They might be working at a bank, or an insurance company or not the coolest…

JUSTIN ROHRMAN:  Even the assertion that there’s going to be fewer feels a little suspect to me,  just because of the growing software industry. The industry is not slowing down at this point, and seems like it is only increasing.

MATTHEW HEUSSER:   yeah we had Yaron on a few episodes back, and he said “The number of testers is actually increasing, it’s just not increasing as fast as the rest of the industry.” Which I thought was pretty insightful. So you can go from 2-to-1 ratio to a 3-to1 ratio, and it’s like “Man, it feels like there’s less of us” but there is more of everybody.

PERZE ABABA:  Well, you do have to remember that not everybody has that mindset or even culture in the company that can handle types of approaches that are like that. How long has it been since unit testing was introduced? Do we actually have even 80% of the companies all over the world actually using these good practices? We try, but there is definitely another challenge on the other side where the companies that expect for this type of work to be done just doesn’t know how to maintain it or know what to expect from the people that know how to do these things. So there’s also Constant fast revolution as to “where can we use these new techniques?” Honestly, I’m not worried about [laughter].  There is a lot of declarations that have happened before. We’ve never seen them come true. It’s not because these people are charlatans. It’s because they have a very astute view of where they are and where are they are basing their declarations from. We welcome the challenge, and I think the one way that I can spin that is “that’s great. That’s a new thing that we should be testing as well because you employ neural networks and AI to be able to develop new tests. That system needs to be tested, too.

MATTHEW HEUSSER:  That’s really insightful, Perze.  It’s like “Hey, at my company, this problem will be solved in a short period of time. Therefore, this problem will go away for everyone, because everyone is like me, right?” And the answer is “No! [laughter] They’re not!”, but that’s the kind of thinking that gets you those kind of blanket statements. They’re not bad people. They’re not mean people. They’re not evil people. They don’t have the whole picture. That’s really a good way to put it. Thank you. So we should go to the main topic. It’s the beginning of the New Year, which traditionally is the time for people to make resolutions. What do you want to learn next year? What do you want to change in yourself? What do you want to change professionally? How can people think about setting their goals in the new year?

JUSTIN ROHRMAN:  You know, I’ve never been a really big goals guy. I always kind of just do what interests me at the moment. So I may not be a good sample for this.

MICHAEL LARSEN: Well, to be fair, Justin, I think that you are more common than you might think. Most of us, I honestly believe, when we try to set big sweeping goals, more times than not we fail. Not so much because we don’t have resolve, or because were terrible people, but because we haven’t thought through what we are trying to accomplish. I went from being a “well, new year’s resolution and make lists, and let’s see how many of them I actually hold to” to making a conscious decision that I don’t even mention new year’s resolutions; they are not part of my vocabulary. If I do mention a new year’s resolution, I start it three months early [laughter]. Like, I do it in September. So that by the time that January rolls around, this is actually a habit that I have decided that I want to hit. The new year, it’s an arbitrary period of time, but it’s something we’ve culturally gravitated towards. We always pick the period of year that’s the most difficult time to do it. It’s not at all unusual that everybody seems to make this plan about “I’m going to lose 10 or 15 pounds” right after they’ve gone through the feast days of winter. And within two or three weeks they’ve totally fallen by the wayside. I very much believe in the idea of doing something small, doing something every day, being very iterative up; I guess the Agile resolutions if you will [laughter], so that it does have a fighting chance to become a real initiative that I can look back on and go “Oh, hey, that did turn out good.”

MATTHEW HEUSSER:  I guess there’s a couple of things. There is large goals, and there is micro experiments.  Scott Adams, the Dilbert guy, said that he doesn’t like large goals because 99.9% of the time you are living in the shadow of “you haven’t succeeded yet.” Then you get to enjoy that one second of happiness, and then you set a new goal, like, is that really a way to live? Instead, he recommends systems, where you do a lot of something, and you think it will attract what you want, and things will get better.

JUSTIN ROHRMAN:  That might be a good summary for what I do. Basically, every day I have a plan of something small I’m going to do, whether it’s work, the gym, reading, or whatever, and I just make sure to do that thing, but I don’t really have any sort of long-term forecast for what it’s going to turn into. I just do whatever I’m passionate about for that day.

MATTHEW HEUSSER:   Yeah, there’s two ways to look at it. One way to do is to just look backward and say “wow, I got a new job this year” and ” I got to use Python on this cool project, and that landed me a raise, and now my commute is shorter, and it was all really good stuff and I’m very happy with it”. Almost like you would manage an investment account. It’s really hard to predict how you stocks are going to do, but you can look back and say “Twenty percent gain! That’s good enough!” The other way is to sort of have aggressive numerical targets. Personally, in the last six months or so, I’ve shifted a lot from financial benefits to the business, to what I’m going to learn, and physical health, wellness, diet, and travel. Actually travelling less, spending more time with family, but quantifying those is hard. I’m really interested in “the tester as whole person” lately, instead of “let’s talk about soft skills and hard skills and programming languages and failure modes and effects analysis.” That every other podcast we’ve always done. How do you step back and look at yourself as a whole person? How do you balance those things?

JESSICA INGRASSELLINO: I feel like I can speak to that.  I guess I feel that I resist labels, and the reason that I resist labels is because I feel that all of my experiences inform who I am. When I’m with a piece of software… and I’m very deliberately speaking about it from kind of a more humanistic approach, with the software rather than in front of my computer. This year I was really lucky to be able to watch a program at the local community college grow that teaches kids not only coding, but also critical thinking skills through different kinds of projects. I have a business that I have been working on for years. A goal that I have is to start working in the broader community, because the business works with underserved individuals to help them get started doing some coding, some critical thinking, and bringing these workshops where people might not ordinarily have access, which is of huge importance to me. I still teach music. I love it. I will continue to do that. Of course, I have my job, and there are many things that I’d like to learn, but I think one of my goals this year is to do some more writing around the ways that the disparate fields that I have interest in, or that I study, actually seep into my work, without me deliberately calling upon them. Really applying some mindfulness and attention to how poetry or music or philosophy or sociology or other kinds of engineering filter into the work that I do in teaching and testing and in trying to be a more active political participant in my community. It’s so hard for me to separate myself out, and say, “Oh this is who I am as a tester.” It’s who I am as a person.

MATTHEW HEUSSER:   One of the things that impresses me about you, Jess, is that you “get busy”. It’s a phrase that I’ve learned in the past six months that has really impacted me. Not that I wasn’t already busy professionally, but every other aspect of my life, I was a little lazy, and you are a busy. You are not sitting around watching Netflix. You’ve got so many projects going on, and the also to build on each other, to make you a stronger thinker. We all know of the music-math connections, what you are teaching music students. You were going to school at night and you got your doctorate while you were doing testing. The lure of the TV and the Netflix and sitting around… it’s so pervasive. It’s so dangerous. I just wonder if we can help people to just set one goal for them as a person in the new year that helps them to get busy, and what would that be?

JESSICA INGRASSELLINO:  I was just thinking of something said earlier, and it’s “deciding what you value.” When I decide what I value, that’s when it’s much easier for me to push myself when I’m tired to do something, because I might want an extra hour of sleep, or I might want to just chill out and do nothing, but what I value is helping my students become better programmers or to become better critical thinkers or to become better violinists or better trumpet players. Those values guide my actions. It’s a broader concept, so it’s not like “deny myself TV forever. Deny myself this forever.”  It’s “what do I value?  How will my actions be supportive of my values?”

MATTHEW HEUSSER:  I think to build up any new habit, you typically want to give yourself I cheat day once per week. You sleep in one day. You allow yourself to eat junk one-day. You can skip exercising one day. That occasional break. Indefinite, unlimited denying of yourself is going to break down. I think people sign up for the gym, go every day for a week, and then don’t. Same thing with professional development. If you are trying to learn a new skill, I would say by a book, say you are going to do a chapter a night, but one day a week, just skip it and do something else.

MICHAEL LARSEN:  Another aspect that I think most of us tend to fall by the wayside with anything that we plan, we as people are terrible at estimating what it’s going to take for something to stick. We all have good intentions. We get that book, just like you said, “I’ll do a chapter a night”, and the first three or four chapters, that might be doable.  But then many of the other concepts you may find are not relevant, they’re not applicable to you in that given point in time, or they’re using an example that you can’t necessarily get your head around. Because you can get your head around it, you don’t do it, or you figure “oh, that’s not for me.” I think that’s also part of our passivity.  We’re willing to change, but we don’t want to necessarily do the really hard work of making that change. We want to sit down, we want to just let things enter into our brain, and then we get up and go about our day, and we’ve suddenly magically become a different person. No goal that’s of any meaning ever really works that way. You don’t develop a skill like that. You develop this skill by putting in the hard work, by fumbling around, by realizing that you make a mistake after mistake after mistake, and it’s slow for you actually genuinely get good, and many of us get frustrated with that. We get frustrated at how long it takes. For many of us we just stop, or we say “I’ll get back to that tomorrow, it’s too difficult, I’m going to go do something else.” And then we wake up and realize that goal that we wanted to accomplish is now months in the past and we’ve forgotten about it.

MATTHEW HEUSSER:  well yeah, I’ve changed my diet and my exercise in the past six months, so I’m really focused on that, and I think it makes huge difference, because I have more energy, I feel better, I look better, I’m happier, but for the previous 18 years, I would look at that candy bar and that giant-sized Mountain Dew, and say, “Do I value this immediate positive incentive over the delayed, uncertain, long-term, negative consequences? Heck yeah! Tastes great!” So reminding me of my values didn’t help; I think the biggest thing that helped me was habit. Once I got used to going to the gym three times a week. I had a banana for breakfast, protein shake for lunch, a reasonable dinner which would consist of one protein piece (chicken, beef, pork), a vegetable, and a carb, and that’s it. I didn’t count nothing. I lost about 30 pounds and 6 inches off my waist in about four months… five months. The big challenge for me is how to take that and make it sustainable. Stop worrying about weight and start thinking about body fat. Once I got to a certain level of improvement, I had to get much more serious about numbers and nutrition, but I got pretty far with “I’m just doing these three things.” I didn’t do a whole lot of counting. I wonder how many software teams are you trying to “do all the things”, and what they need to do is make one small tweak.

MICHAEL LARSEN:  I think it’s very common. I don’t want to hard on the self-improvement side, but again, this is January, it’s New Year’s resolution time for many. That’s where most of it comes into is the self improvement side of things, and you’re absolutely right. Just by making a couple of changes in your reality, whatever they may be, you’re not necessarily counting anything, but you are quantifying. You are making conscious decisions to make small changes, but those small changes can add up to a lot. At a certain point, after you’ve made those small changes, and you’ve exhausted those “beginner gains”, if you will, it’s when you get into that new state of “OK, now I want to get leaner than I’ve ever been.  Stronger than I’ve ever been. Have greater in endurance than I’ve ever had.” That’s where the real hard work comes in. Your initial plan for what you envision for yourself may not be attainable, at this point in time, or it may not be attainable with the amount of work that you are prepared to do. Does that mean “Don’t do anything?” No, of course not, but it means you have to be reasonable and realistic with what your goals actually are. That’s why believe very strongly in starting small, focusing on little components.

PERZE ABABA: Evaluations as well as self-improvement is definitely true, especially around this time of year, especially when you are in the corporate arena, were you have to deal with your goals and objectives for the next year and it’s a little tricky, because there are some things that are reliant on that. If you deserve a bonus, for example, or if you deserve a raise, or if you deserve a promotion. One thing that I have noticed from what everybody has been talking about earlier, was it all does start with a desire, the desire to be able to do something. From what Matt is saying, the ability to be able to live a healthier life, and also being able to spend time with the family, but there’s definitely that desire that kind of starts these things off. The tricky thing with being in a corporate environment is that it’s just not your desire that’s in play, because it’s somebody else’s desire, too.  So if my boss says, “As part of your goals, you need to expand on these certain things”, you may or may not desire those things. That might be his desire, but you still have to work on it. You know, there’s a lot of theory around it, or at least patterns that we can hinge on, so that we can actually eventually get to that goal. Deal with something that doable, because if you don’t… if it’s “immediately become an expert”, that’s definitely impossible. You’re just gonna’ want to quit. For you to be able to keep going, there’s always these engagements around you. Whether it’s from the community, whether it’s from your co-workers, your team members…  for me it’s from you guys. The way I see are you guys talk about certain things, and I see the passion of what you guys do, I do tend to have an introspection and ask  the question “How do I get there?” There are times when I would reach out to people that I greatly respect and have them help me with the things I know I needed help with, but again, that really does go back to the initial state where I’m honest enough to myself that I’ve identified there is a need for me to be better somewhere. As people in the testing community, who are mostly being looked at as leaders, we could lead through the example that we set. I do look forward to a lot of the changes that are going to happen next year, and I’m pretty excited about that.

MATTHEW HEUSSER:  What I’d really be interested in the comments, and if people want to email the show, it’s thetestingshow (at) qualitestgroup (dot) com. What are your goals for next year? If they’re interesting and exciting and better than what we’ve talked about, let’s get you on the show to talk about it. I guess I should let everybody close with those thoughts. Michael?

MICHAEL LARSEN: I have decided that I want to become a bash expert. The reason is, I do a lot of environment manipulation, just to be able to get machines and systems in a condition where I can do any testing at all. The neat thing is that I’ve realized, with just some of the scripts that I’ve been working on, I’ve had senior developers look at me and go “Wow, that’s really brilliant. You should put that in the source code. We could use that for our setups.” It just struck me, like, I’m actually providing something that nobody else thought about doing. So that’s my personal goal for the next year.


JESSICA INGRASSELLINO:  I plan to keep circling around with no plan, as I kind of highlighted earlier, and just let all of the things kind of absorb, and hopefully keep building on one another, because they have far too many interests, and far too little time, but what a great problem.


JUSTIN ROHRMAN: My main goal for the year is to build a really good conference. I’m working on CAST 2017, which will be in Nashville, Tennessee, in August 2017, and the CFP just opened, so we are taking proposals for that, and trying to build the best conference that we can… and I’ll include a link to that CFP in the show notes.


PERZE ABABA:  I have a primary goal and a secondary goal. The primary goal for me is to be a bit more consistent with my writing. I think I started a bit too aggressive last year with my blog posts, and some articles that I wrote, but that kind of fizzled out. I want to be more consistent with that. The secondary part is really looking more into how to teach developer-specific testing; jumping into TDD a bit more and looking at integration testing and stuff like that from developers mindset. I want to be able to help my group and my company in that manner.

MATTHEW HEUSSER:  Sounds good! I’m going to continue to run my system. I want to lift 220, squat 220, for five sets of five, by the end of the year.

JUSTIN ROHRMAN: Two-twenty-five! Go for two-twenty five.

MATTHEW HEUSSER:  Right, that’s two plates on each side. Yeah yeah!



MATTHEW HEUSSER:  That’s right, bro!


MATTHEW HEUSSER:  My bodyfat… it would be nice to get my bodyfat around 12 by the end of next year but that’s pretty… I dunno, we’ll see. The thing about those goals is that you could improve substantially, and then “Oh, I only got to 220. I didn’t get to 225. I failed. I don’t think of it that way at all. We’re gonna’ keep the business running and keep testing and keep learning. I want to either get somewhere with this neural network stuff, Will be able to discard it as “Nope, it’s not going to work.” The other possibility we might come up with is “Yes, it might work if we can raise 20 million, and hire 20 coders, and work on it for three years.” I don’t know if I’m willing to put that kind of work into it, but, that’s it for me. Thinking about being at QA or the Highway in February. That’s not decided for sure yet. Does anyone else have anything new coming up? I know Justin is looking for writers.

JUSTIN ROHRMAN: I am. I’m the technical editor over at stickyminds (dot) com,  so if you are a writing software tester or a software tester that has never been published, find me on Twitter or send me an email and we can talk.

MATTHEW HEUSSER:  OK. Thanks, everybody. I think I’m going to call that a show. Thanks for coming.

MICHAEL LARSEN: Thanks for having us.


PERZE ABABA: Thank you.


[End transcript]