The Testing Show: Open Source Testing

For many software testers, it can be challenging to build up a portfolio of work that can demonstrate skill, knowledge, and expertise. Having GitHub account for coding skills is one approach but many testers work on proprietary systems where people outside their company cannot see what you are capable of. Open source projects are a way to get involved in an externally visible way but where does a tester start and what can they actually do?

Panelists:

References:

• Jessica Ingrassellino (LinkedIn)
• InfluxData
• Telegraf
• GitHub: InfluxData
• GUI Clients for GitHub
• #heavymetalvocaltiktok
• GUI Clients for GitHub

Transcript:

Michael Larsen (00:00):

Hello, everyone. And welcome to The Testing Show. Glad to have you with us. It’s January, 2021. We are recording this on January 14th, so we are glad to have you with us. And in addition to having “glad to be with us”, we’d like to welcome back a former frequent contributor and co-host of the show, Ms. Jess Ingrassellino. Thanks for being with us today.

Jessica Ingrassellino (00:29):
Hey, thanks. It’s great to be here.

Michael Larsen (00:30):
And of course you are all familiar with our master of ceremonies, Mr. Matthew Heusser. So Matt, do your thing.

Matthew Heusser (00:37):
Hey, thanks, Michael. So last time we had Jess on, she was Director of Test for salesforce.org, which was kind of the non-profit vision of Salesforce. And you’ve moved on since then in particular, the company you’re at now as a manager is doing open source, and you’ve been looking at opportunities for testers to contribute to open source. And I think it’s harder for the tester to make visible contributions that can be understood than it is for a programmer. So tell us about what you’re up to lately.

Jessica Ingrassellino (01:17):
So I am an engineering manager at InfluxData, and my team is the data acquisition team, and we work on a product called Telegraf and Telegraf is massive. And so I sort of encountered this issue of sort of my test background and my test thinking colliding with the needs of this product that I was hired to work with. One of the things that I noticed was that when I came in, we had two developers working full time, but we have thousands of contributors. And there are literally millions of installs of the Telegraf server agent. Everywhere from hobbyists people at home who write and use Telegraf plug-ins, to people who are using it at scale in enterprises and have 10,000 to upwards of 50,000 installations of this. Obviously there’s a really big delta in what two developers can do, who are supporting that product and the contributions. So, you know, one of the things that occurred to me is there’s such a rich opportunity for developers who know a language to go read the code, find a project, make a contribution. There is substantially less opportunity for testers to do the same thing in a way that’s understood. And when I’ve gone to conferences and had people talk about it or gone to sort of open sessions about, “Hey, how can we contribute to open source?” The conversation sort of starts at, “Oh, find a project and look for bugs and then reproduce the bugs, or if you’re not comfortable with code contribute to the documentation”, but that’s also where the conversation stops. Since I’ve been working for about four months now at Influx, I’ve noticed that there really are some gaps that testers could help to fill and sort of the ways in which testers specifically provide value to open source projects.

Michael Larsen (03:35):
So I would like to do a quick interjection here, if I may. Most people who listened to me on this podcast will know that I’m kind of a broken record on this subject, but I honestly think that open source projects are a great place to apply accessibility skills or practice accessibility skills and be able to implement them and help those projects understand where they might be failing in that category. Certain tools are going to be harder to test than others. If it’s a command line tool versus say a UI, okay, that might not be as helpful. But I do want to state that if you are curious about a real, tangible skill, you can bring to an open source project, accessibility is one of them.

Matthew Heusser (04:31):
Jess, tell us a little bit about Telegraf and then, do you have a particular way people can contribute to Telegraf in a visible way?

Jessica Ingrassellino (04:40):
Yeah, and I think that fortunately for everybody who’s listening, that the things that have occurred to me is ways that testers can really contribute are things that are applicable across almost every open source project. And it really depends on sort of the expertise and the interests of the tester. First, I’d say with regard to Telegraf, we support all operating systems and we have a server agent and then we have 270 plugins for that server agent. Right now, we have a lot of monitoring, plugins input, output, processor, and aggregator plugin. So doing lots of things with data. And then we have IOT plugins, plugins that run on an Arduino or Paspberry PI or other small IOT devices in order to get data from those devices at the edge, and then send that data to either the InfluxData instance in open-source or the InfluxData cloud instance, or we even have plugins that send to our competitors. Telegraf is truly open source in that sense. It really is open to whoever to use it, build it, make a plugin that works for them. It’s a massive ecosystem. There are I think, opportunities for testers to say, “First of all, I want to do open source or I want to have GitHub contributions and I either don’t have any, or I work in a product that’s proprietary. So none of my contributions will show up in, GitHub outside of my organizations GitHub account.” These are sort of the two problems that I think we’re solving here. I’m trying to help test yourself is building up that portfolio. So the first thing is find something that’s really interesting to you as a test, or if you’re not interested in financial sector, don’t go to an open source project about financial stuff. It’s not going to be motivating. If you are interested in security or accessibility or front end stuff, then go look for the companies that are doing that work. If you are interested in companies that are doing back-end, data science, command line microservices via the API, database work, go look for a company like InfluxData, a product like Telegraf that sort of has this expansive ecosystem. So first of all, find that space that interests you where you feel compelled. The next thing is, figure out if you can use the product. Something, I think, that a lot of open source products do, or a lot of people recommend, is go find bugs in the product. So I, myself, as a tester, I’ve gone in before, unsuccessfully, gone to GitHub, looked at lots of things. It’s like, okay, I know Python, so I’ll go in. But actually before any of that, of course, download the product, keep track, keep notes of your experience using the product, because whether it’s a visual UI/UX experience, or it’s a back-end experience where you’re using the command line, you’re using the documents, you are interacting with databases, it’s still an experience. As a user and as a tester, keeping track of what’s happening in your user journey is some of the most valuable information that any company can have. And it’s certainly valuable to hear from our users about what that’s like. It certainly helps us make decisions about what we’re doing, helps us make decisions about what we need to document, what we need to write blog posts about ,what we can make videos and tutorials about and what we can invite our users to help us with since it is open source. So all of these things are made helpful just from a tester interacting with and taking good notes. The question is, how do we translate that first experience into a GitHub issue, GitHub interaction? You know, once a tester is familiar with whatever that product does, I definitely recommend looking at the company’s documentation, whether it’s the GitHub docs, the internal docs, because documentation is a great way for exploratory testers who really thrive in the user experience, whatever that is to make a contribution. And some of the things that an exploratory tester can do in the documentation -it’s something that I’ve noticed in Telegraf, especially- is taking the opportunity to say, okay, here are the setup steps that anybody else could use to test this. When I started, we had two developers on my team, we have 270 plug-ins and three operating systems. You can’t test all of that, every time. You can’t even come close. And then the additional work of documenting sort of the “how-to”, it’s not even on the table when you’re looking at that broad product in that broad a community. So something that testers are uniquely qualified and skilled at doing is saying, “Here’s some how to test documentation that I’d like to contribute to your project.” In that way they can have a submitted issue. It can show the step-by-step instructions. And so through that, they’re sort of displaying not only here’s how to test it, but if it’s a product that requires setup of sort of a back-end like Heroku or Linux operating system, or other back-end tools, a certain database, or a certain set of databases, they can show their understanding that they know how to work with those technologies, not just from a resume side, but here’s the actual practical application. And here’s how I helped contribute to a product and helped other people. That’s one really concrete way that testers can contribute to the documentation. And it’s something I know that we are working on, we’d love contributions on, and I haven’t seen a lot of that in other open source products.

Matthew Heusser (10:59):
I’m going to take a step or two back and see if you agree with this. There still are a lot of people maybe who listen to this podcast who aren’t on GitHub. And if you can figure out how to log in to Facebook, you know what the command line is in Windows or Linux or Mac, you can figure out how to create a GitHub account and how to add and remove files from Git. And now are you suggesting for all of these different components you have -you mentioned hundreds of them- You could find one that makes sense for you, or you could use, or you could at least understand, figure out how to test it. And then would they write a read me file like an MD file in GitHub, which they would commit? What does the actual documentation look like?

Jessica Ingrassellino (11:48):
So for our product, that would be a welcome way to make that contribution. I think that depending on the product one chooses, each company and each group of maintainers sort of has their preference as for how contributions are made. So some companies, for example, like everything to be a GitHub issue. And they’ll tell you, “If you’d like to submit an issue, please do these steps. If you’d like to submit a change to documentation, please, to do these steps.” So when in doubt, I think that people can always submit an issue if they’re not sure what they should do or how to contribute, we would be happy to see a ReadMe, but yeah, setting up a GitHub account so that way that they have access to be able to make the issue requests is probably the way that most open source projects interact. Once you have that GitHub account, you can go and you can read each of those processes. And there are a number of ways people can interact with GitHub. There are GitHub UI tools, and there are GitHub command line tools, and there are different apps for GitHub UI interactions. If you’re trying to make a pull request or make code changes, you can also use the prompts on the website. I am a GitHub command line user. So even though I’ve heard that the UI tools, a lot of them are great, like a lot of the desktop tools, it’s just not my preference. So I tend to work in the command line, but really there’s ways every type of person and tester to interact with GitHub and to get those tools on their operating system, whether it’s Linux, Windows, Mac, or command line. So the setup required is pretty flexible to how a person likes to work.

Michael Larsen (13:44):
So to follow on, on this, just a little bit, if I can, oftentimes I think one of the things that can be daunting to people, especially like, “Hey, I’d like to get involved in testing this.” We can totally understand this with, “Oh yeah. You can just create a GitHub account and get to use to how you’re doing all that.” Okay. That’s great. I want to get started. All right. Now you need to download this virtual machine or you need to have this Heroku instance, or you need to have experience with Docker containers or blah-blah-blah. And suddenly you realize that the prerequisite list can become very large. It’s like, “Oh boy!” whether it’s your product or to approach it more broadly, how would you suggest to somebody says, Hey, I want to get involved in more open source projects, or I want to get involved in your open source project. What is the bare minimum they need to be effective?

Jessica Ingrassellino (14:35):
Fortunately, we have multiple entry points. And I think any project, if a company is really smart, they’re listening to people who are using it the first time. I think that that first time through the project, whatever it is, is a useful journey. So why I am saying that, when you’re starting out the first time with our project, for example, one of the projects that we have on our blog for example, is to set up a temperature sensor and have that temperature sensor in an Arduino read into influx DB. Like if that seems like a project you want to do, you can follow along with the process and your bare minimum needs would be to have some kind of a temperature sensor and download Telegraf and download InfluxDB, or set up an InfluxDB cloud account. Let’s say, you’re like, “Hey, I want to learn about it, but I want to buy temperature sensor. I don’t really care about that aspect of things.” You can work with the systems plugin, which allows you to take the system data from your computer and view that data in a time series, either via a local InfluxDB instance, or again, via the cloud instance. I think the starting point varies what I will say to counteract, I guess, the idea that you need to be an expert in all the things in order to add value, or you need to know about Heroku in order to to add value. The reason I would say to pick a project that you’re interested in is because as testers, we need to be aware of emerging technologies. And there’s so many. How do you pick? Whether you decide to do Heroku or what Microsoft services or Kubernetes or AWS or Azure or Google cloud, how do you know where to start? If you pick a project that you are really interested in, something that just fascinate to something that you want to try, a blog post you read, or a product that you’re using at your company, and you think it’s cool, but you don’t know much about it. One of the best ways to learn is by going through the exploratory process. Let’s say that they’re one to locally host a processor plugin. So let’s say they just have a lot of system data and they want to only grab a subset of that data. They can look at the documentation and just follow the steps to try it, to make that work. And they’re going to find that they a) are learning, b) what they need to do for future testing, and if those steps don’t already exist in whatever repository they’re working in writing that information up and putting it in a ReadMe doc. So that way that other people know, “Hey, these are the steps to set this up” is massively valuable. And they’re able to show that understanding. So a lot of people will do that through blog posts, they’ll say, “Oh, I did this thing. I tested this out. So here’s my blog post about it.” Instead of doing a blog post or an addition to a blog post, right? You can still do a blog. Post blog posts are hugely valuable in a different way. Take that and say, “Okay, in order to test this, here’s all the stuff I actually needed to know. Here’s all the things I found out by going through the setup steps. Here’s the six things that seemed obvious to the developers who wrote these instructions or to the technical writers but actually aren’t obvious if you’re testing it, here’s your step-by-step for how to interact with this product in order to test it.” That has massive value because now every developer who wants to test that product, who may also not know the product like they are reading the code, but they might not know how to fully set it up to test it. It provides them value. It provides all future testers value and it provides insight to the maintainers about, “Oh, wow. Maybe we really need to add more information, do more user education. We didn’t highlight this part of our process.” So I don’t think that you have to have the prerequisite knowledge. I think that what you do need is really good exploratory testing skills and excellent documentation skills. So that way that as you’re gaining that knowledge, you are sharing it in the same way that you would share a test report and added value of putting that in a GitHub issue in an open source project is that when you are on a job interview and somebody says to you, “What’s the value? What have you contributed? What do you do with exploratory testing?” If you’re able to point to your test reports or your experience reports in GitHub on multiple projects, suddenly it’s a lot easier to see that value in real time.

Matthew Heusser (19:36):
Well, I think this is really great. And I think you pointed out a real problem in the self-help business world. There are a lot of gurus who go around and say, “I’m going to tell you everything you need to know about owning real estate in this free seminar course.” Go to the seminar. They want you to buy a $200 course to take the $200 online course. They want you to pay it for $5,000 in-person course. And it’s really just about making the money. Getting involved in open source can be very similar in that it’s a great five minute lightning talk, but then how do I do it? So I think you’re laying out a real path to contributing. At this point,. I would say, you go to a job interview to say, look, this is what my tests, this is what the documentation I produce looks like. It’s such an important thing. And it’s something that we so rarely have an opportunity to actually see. You can do crazy stuff when you practice, too, and see if it’s understood, maybe make some jokes with personas, and alliteration, like, who cares? And then you could see how people receive it And if it works and practice it more. So does your company have a website that’ll help people walk through step by step, what they have to do to figure out how to contribute on a particular module or piece of the app?

Jessica Ingrassellino (20:59):
Not yet, but we are actively working on that because we actually want to create an open source testers community. That is my goal is to have a community of testers and users that is as robust as our developer contributor community. So if you’re listening to this show after February 5th, mid February, we are going to have some basic instructions in a ReadMe file on the InfluxData GitHub repository. And I will send the link to Michael for the show notes directly for the Telegraf repository and the ReadMe files. So that way that everybody can see what we have to get started with our open source testing community and contribute suggestions, give us ideas. So that is something that the team and I are working on right now.

Matthew Heusser (22:03):
So I know anything about this podcast, my guess is that there’ll be 2000 people that listen to this podcast. Maybe a little bit more, there might be 200 that are moved to action on open source projects. There might be 20. I think we got 20 people that were long-term committers in your project to start building the community you want to build. I think that’d be great.

Jessica Ingrassellino (22:32):
That would be amazing!

Matthew Heusser (22:32):
Now I will say there are probably like… testers love to critique. It’s what we do. And then, “Oh, she’s just pumping up her business, like trying to get free testing.” Well, maybe. Okay. But I would just say to the audience, if you really feel that way, what have you done to contribute? What are you doing to build the test community in your neck of the woods? And I think Michael and I have both contributed some things. Some of it commercial, some of it non-commercial Jess has done a lot of stuff that’s non-commercial and if you have something that you think is better or more pure, because you’re not motivated, contact us and beyond the show, we are looking for people.

Michael Larsen (23:13):
I’d like to piggyback on this topic just a little bit and say, sure, there is always the risk that if you work on an open source project, yeah, somebody else is benefiting from your labor. If you will. Well, that’s going to be the case in just about anything that you do. My rationale is if that were the logic, I would never have offered to produce a podcast, which I’ve been doing over the past 10 years. The interesting thing is this podcast alone has actually opened up doors and given me opportunities in ways that I would have never anticipated. I actually do voiceover work now for my company because they heard the podcast and said, “Hey, Michael’s got a really good voice. Wouldn’t it be cool if we could leverage that?” or “Hey, Michael has got audio editing capabilities. We might be able to use that.” Technically speaking, I get paid for that. It’s part of my salaried work, but my salary is not too bad. So I’m not complaining. You never know where your next opportunity might show up or might make itself manifest in the work that you do. Sometimes a little bit of free work can book you a lot of well paid work down the road.

Jessica Ingrassellino (24:32):
And just to add to that, Michael, I really want to point out that I’ve come on today, not only to talk to people about contributing to open source in our project, but more importantly, to talk to people about contributing in general. And specifically I’ve noticed that big projects are really lacking in documentation around testing and in test or contributions. And I’ve also noticed during the past six to eight months of the pandemic, as people are experiencing layoffs and looking for work, that many of the job descriptions I’m seeing in many of my colleagues and friends who are reaching out to me are noticing that these employers are asking for GitHub portfolios. A lot of the testers who have been working are either working in proprietary environments where their GitHub work isn’t available to the public, or they’re working in exploratory testing, which doesn’t often give us the opportunity to share the work we do.

Jessica Ingrassellino (25:43):
Because again, it may contain proprietary information, very detailed information about the product and those test artifacts are not typically put into a repository. Of course I’d love it if anybody was interested in influx DB and Telegraf. And if you’ve heard about the work that I’m doing with my company, and you’re like, that sounds miserable, then absolutely don’t contribute because you will be miserable. And that’s not what I’m hoping for. What I am hoping for is to give testers, particularly exploratory testers, a feel actionable path forward to creating and sharing their work on a platform where employers are looking to be able to see the experience and the contributions. So people can find a platform or a product that they love and they can use some of these techniques to help build out their, then I will really feel excited about having accomplished that through this podcast. And if anybody has questions or wants to ask broader questions about what they might consider doing, or any kinds of portfolio building or exploratory testing job searching outside of the direct content of this podcast, please let me know. Please reach out.

Michael Larsen (27:09):
All right. I think this is a good place probably for us to put a pin in it. Last comments. Matt? Jess? Things we’re up to? It’s a new year, what are y’all working on?

Matthew Heusser (27:19):
Well, I would just say across all of our customers right now and all of our projects, everything is behind because of the disruption to America, United States of America, that’s happening in our political situation. So if you are behind at work, forgive yourself and maybe turn off the TV and the Facebook, because I don’t know that that’s long-term healthy for all of us. And that’s just me. That’s just my opinion. I’m just some person that doesn’t represent Qualitest, but we’re all trying to get through this. We want to be here for end with you. Thank you.

Michael Larsen (27:57):
And Jess, I guess if anybody wants to, outside of this project, if anybody wants to get in touch with you, what’s the best way to do that.

Speaker 2 (28:04):
So I’ve got my LinkedIn, JessIngrassellino, Twitter, jess_ingrass, also starting up a new project in February at “Dr. Jess Teaches”. That’s on Twitter and working on the website now. So going to be talking about the intersection of arts technology and education and working on some really exciting large projects that will be coming out in the middle of the year, can’t say too much more about them at the moment, but I am very excited to be able to talk about it a little bit later on.

Speaker 1 (28:43):
Thanks very much. So on my end, most of what I’m doing is in the writing and, well, producing this podcast and working a day job, but I am doing something a little bit different, and this might interest a few of you out there. Many of you know that I am a singer in a hard rock, heavy metal band called Ensign Red. And I’m doing a little project on TikTok, of all things, where I am introducing minute long segments about heavy metal vocal technique. So if you are at all interested in the more extreme side of singing, come check out what I’m doing there. My username on TikTok is “@mkltesthead”, just like it is every place else. And with that, let’s call this a show and I will say thank you very much for joining us. We look forward to talking to you all soon on another episode of the testing show. Take care, everybody.

Matthew Heusser (29:36):
Bye.

Jessica Ingrassellino (29:37):
bye.