Panelists
The world of Utilities is undergoing a massive shift and has been doing so for the better part of a decade. With the advent of new renewable technologies and solar at home, individuals are not only power consumers but are often producers. To this end, Matthew Heusser and Michael Larsen welcome Wayne Tofteroo, Ronald Teteroo, and Scott Swanigan to discuss the changing landscape of utilities and how testing in and around them is full of unique and interesting challenges.
Michael Larsen (INTRO):
Hello, and welcome to The Testing Show.
Episode 141.
Testing For Utilities
This show was recorded on Wednesday, August 9th, 2023.
The world of Utilities is undergoing a massive shift and has been doing so for some time. Individuals are now not only power consumers but often producers. To this end, Matthew Heusser and Michael Larsen welcome Wayne Tofteroo, Ronald Teteroo, and Scott Swanigan to discuss utilities and how testing in and around them is full of unique and interesting challenges.
And with that, on with the show.
Matthew Heusser (00:00):
Welcome back to The Testing Show. We’ve got an exciting talk… this episode, we’re going to be talking about testing for utilities, specifically electrical, natural gas, water, all of these companies that bring things into your house and then send you a bill every month. And we’re in the middle of an exciting time because we’re in the middle of digital transformation for that. We’ve got solar, where people are actually selling their power to the grid instead of renting it, but they might not. If it’s a really cloudy winter, they might need some. So a lot of demand there. Qualitest has some experience there and we’ve got some partners with that. So this week we’ve got Wayne Tofteroo, who is a test director at Qualitest. Welcome to the show, Wayne.
Wayne Tofteroo (00:49):
Hi.
Matthew Heusser (00:50):
Yeah, so Wayne, you’ve been a test director at Qualitest for close to two years. You have a PhD in Chemistry from the University of London. Tell us a little bit about how you added the testing and particular utilities.
Wayne Tofteroo (01:04):
Thanks, Matt. Basically, my career started out as a research chemist strangely, but then I got involved in IT and then I decided to make a career in IT and I rose through the ranks of business analyst, project manager, and program manager, and then became a test leader for one of our largest digital transformation programs for the company I was working for at the time, involved in testing and that changed my career. So I’ve been involved in testing, been involved in utilities now for a number of years. I’ve been involved in several digital transformations, been involved in introducing automation into utilities to support those transformations. So yeah, that brings me here now Matt, in terms of working with utilities and helping utilities drive transformations.
Matthew Heusser (01:49):
Thanks Wayne. We’ve also got [Ronald Tetteroo]… one of Qualitest partners is Tricentis. They make a product called Tosca, which is probably best known for its ability to do all pairs, all try to generate test scenarios in a model based on a GUI. That’s the ten-second version, right? It’s not terrible. No, but you’ve got a strong background with SAP and G SI. So enterprises tie everything together, software that companies use, but then they have to customize the individual bits. You have master’s in economics from Erasmus University, which I used to teach at Calvin College, so if you’ve never heard of Erasmus, he was famous for translating the Greek receive text and he’s from Rotterdam and you’re a licensed European football coach coaching licensed level three.
Ronald Tetteroo (02:39):
Yeah, that’s correct.
Matthew Heusser (02:42):
Tie all that back to utilities. I’m just blown away.
Ronald Tetteroo (02:45):
Alright, well thank you for that. In my background, I started to work with SAP in 2005 and very much I was involved in a lot of complex projects for utilities. I got to be an expert in these complex projects with business process redesign and my eye fell on the fact that when I met the founder of Tricentis, that they had developed something which we call model-based testing like you mentioned, and it allows actually to do these end-to-end process tests much faster and without the scripting necessary, no code. And it allows you to be much more agile. Utilities now are in this transformation to move their old billing systems to transformation on the cloud to make sure that they can cope with what you mentioned on the solar panels that people can also sell their energy, which means that the billing is changing and their billing infrastructure needs to change. And what I learned over all these years is that billing is a key topic for any key executive in the utility because if it breaks down they can’t recognize the revenue. So that’s why I got really intrigued with Tricentis and joined them because there’s a lot of transformation projects going on right now and we see spectacular results if you start early with this estimation. So I’d love to chat further about this during our podcast.
Matthew Heusser (04:07):
Thank you, Ronald. I wish I had more time to go into more detail about these bios because they’re just amazing. Our third guest is Scott Swanigan., I think you’ve been on the show before, right? Scott? I can’t remember.
Scott Swanigan (04:19):
Yeah, I was with you some months back. We were talking about storm readiness testing.
Matthew Heusser (04:23):
Yeah, yeah, that’s what it was. Storm readiness testing. It was a while ago. Welcome back to the show. You’ve got a degree from the Virginia Military Institute?
Scott Swanigan (04:33):
I do, yeah. Yeah, it takes me in the way back machine. Back in the nineties
Matthew Heusser (04:37):
You didn’t pursue a commission, you didn’t become an army officer. Instead you got a degree in mechanical engineering and then rose up through the ranks and now you’re vice president at Qualitest for utilities. Is that right?
Scott Swanigan (04:50):
Yeah, I’ve been in the utility industry since I graduated college. I started out as a system engineer in nuclear power and then I got involved in IT in the late nineties. I served in a capacity of a project manager, quality assurance manager for the implementation of operational systems inside of electric utilities. So I’ve seen a lot change over the last 20, 25 years and as the utilities go through this digital transformation that we’re talking about, the amount of change they have to adopt and change they have to navigate simultaneously is just increased exponentially. Finding efficiencies with automation, automated tools, use of AI, scriptless tool delivery and test automation delivery. It’s very important. I’ve seen it boots on the ground.
Matthew Heusser (05:37):
Well thank you Scott. And for those who don’t know about me, you might say utilities is in my blood a little bit. My father was at the Department of Energy where he was in the senior executive service of the United States and retired as an SES4 in 2001, but he was on the declassification side, so he just missed you when you started. You might’ve had a chance to have run into him, but he was moving over to classification by then and we’ve got Michael Larsen on the show as always. Welcome back Michael. Good to hear from you.
Michael Larsen (06:10):
Thanks. I feel like it’s been a while since I’ve been on the mic, but good to be here.
Matthew Heusser (06:14):
We’ve had a bit of an interesting summer with recording at conferences and I’m gone, but you’re here and I’m here, but you’re gone. So it’s great to have the band back together. And let’s talk about utilities. We were talking off-mic before we hit the record button. So for the benefit of our audience, first question I would ask is what’s different and unique about utilities testing? Why are we talking about an industry vertical? Isn’t all testing the same?
Scott Swanigan (06:40):
I think the utilities have certainly a history, particularly true in the electric utility space where there’s been a slow adoption of technologies. They’re very intentional, their job is to keep the lights on. So operationally they’re very conservative in the types of changes that they adopt. But now with the advent of some of the green energy initiatives and the fact that electric power flow is now bi-directional, whereas before primarily unidirectional, it’s not just power delivery from the generating stations down to the customer, the customer is also selling the power back to the grid. This complicates the whole business of managing an electric grid. It requires new applications, smarter applications to make all that happen. As the world’s changing very quickly, the utilities have to come along and come up to speed. They’re starting to see or have seen some pressure here in recent years to adopt more change. That means more projects, applications being implemented inside their operational space and with more projects come a demand for more resources and demand to do things more efficiently. So here we are.
Matthew Heusser (07:45):
I can see that with multiple inputs to the grid, whether that’s coal or nuclear or wind or solar, the whole dynamic has changed. What used to be a customer might now be a vendor I guess, and then that actually can be spiky. So planning for that’s got to be difficult. There’s a lot of change happening. I’d throw this one to Ronald if it’s all right. So okay, utilities are going through fundamental changes they didn’t have to deal with 20 years ago where you could just send a bill every month. How are modern test approaches going to help with that? I mean realistically what’re going to do?
Ronald Tetteroo (08:24):
Yeah, I say it’s a great topic because ultimately the theme I like is the constant thing right now is the change. And once things change, you need to do quality assurance because if you change code, you want to be assured that you only test what has changed. We’ve seen customers where we use a artificial project to live compare, which looks at changes and detects the change that you made. And we’ve seen utilities using that and finding out that they thought they were covered with all the testing scenarios, but by switching this on and to the pre-production system, they found out that they were only testing 10% of the changes, so they were having a 90% risk. So to be intelligent about what to test, you need to be sure that you test that what’s changed. So that’s the first step. And the second one then is I think the end-to-end process testing where business users come in and where this example I would like to give of a customer of ours who went on the stage in Munich last year in front of the Who’s Who of utilities for SAP, and they were the first one to go into this complex journey to transform their customer experience, starting with streamlining the internal billing systems.
(09:33):
So they had four old ISU billing systems because they acquired companies and they had to put that on one instance on the cloud. And what I liked about this gentleman on the states, the customer experience manager was a business user. He said because we started early with the automation, not during the realized phase of our projects, but starting from day one, testing the changes, the way we worked with SAP and Tricentis and our partners. Basically they were seven months early in delivering this complex project and then he said something very interesting. He said, now I’m agile and now I can be on this agile platform. I can change applications and I can go live in one to two days without production mistakes because I’ve set up my automation and I know what has changed and I test what has changed and I can go live.
(10:24):
That’s not only SAP that can be simple things like a PDF getting information from different source systems than SAP. Everything in the end-to-end chain needs to be tested and that’s why I think this is so exciting to hear that they have made that journey. They’re running 21 billion of revenue now through the new cloud system and they’re able to adapt to their customer’s needs much quicker and when they go live they are much more agile. And I think that’s what we discussed where this automation of testing can be of extreme value to companies to start early, which I think in the world of technology they call shift left.
Scott Swanigan (10:59):
Another aspect of this is the tools are becoming much easier to learn. It’s the scriptless aspect of this is another important dynamic and traditionally user acceptance testing is a little bit more difficult for the business team to come alongside the IT teams, learn the tools that are required to be used in that space and to execute these tests end to end. The technology has really lowered the barrier to entry if you will from a skillset perspective and it’s helped out tremendously to bring the business along and enable them to add the tools that are making the end-to-end user acceptance testing activity a lot easier.
Matthew Heusser (11:34):
So I just heard seven months early, even if that is a big project, that’s a three-year project because utility companies do large projects. That’s still 20% reduction or something like that. What does that mean? Does that mean they had a project and they said we’re going to be done December of ’23 and then whoopsie, we’re going to be done in July and how is that hands-on possible?
Ronald Tetteroo (11:59):
It kind of started to evolve when they started rolling out these different step-by-step projects and they started to see that they were ahead of time and that accumulated so they were also able to plan ahead and to allocate budgets to do other things, which I found fascinating. That’s kind of not a technology thing. It’s like you’re using the advantage of this automation to be early, but you’re going to allocate these budgets which are normally I expected to use on the whole project to other strategic initiatives. So I think that was a key outcome which I really liked. The other one is if you think about the fact that you start early with automation, you can come to these situations. We’ve also seen projects where SAP has this kind of bibles, how to do these steps correctly with the partners and they have a discover phase where you start to discover everything and then they have what they call a realized phase halfway the project. All the projects where we have been involved where they start to test halfway are over time, are over budget because they didn’t start early. Again, that shift left. And I think it’s even customers where we feel uncomfortable to engage because there’s no budget left. Fascinating is here that if you are ahead of the game, you get more confidence, you get also more confidence to do other strategic initiatives and you can get this budget allocated to other areas and I think for key executives and companies, that’s a great message.
Matthew Heusser (13:22):
So I can believe if the company is used to extremely traditional boring after the fact, we write a bunch of test cases, we run a bunch of test cases by hand, we write bugs. By the time we file the bugs, the developers are two weeks or three weeks ahead of us, then we have to triage the bugs, then we have to assign them to a developer who’s different than the one who wrote it in the first place because so much time has passed and then we have to wait another two to three weeks to get a new build and then we do it all over again and then we test the defects out while we’re building new software, which introduces new defects and there’s no trail of breadcrumbs of test automation behind us to check things and find the problem within 24 hours of it being introduced. You really could tighten things up assuming the company really had the maturity to predict in the first place under those circumstances with these new tools, it might be possible to tighten that up. That is an extraordinary claim, but I can start to see how that could be possible. Maybe Michael has some more questions to help us fill that in.
Michael Larsen (14:31):
I do, actually. Yeah. Understand, of course, I have no exposure to this industry other than as the fact that I am a homeowner who gets an electric bill, so for my purposes, I don’t even understand the basics of what’s going into something like this. We’re talking about we’re doing this big change and I guess for my purposes, both from a tester and somebody else who might just be wondering what are utilities even involved with? What are some of the things you’re actually testing and what changed in this process? What got you to where you are today and why was that necessary? I realize that’s a big umbrella of things, but I’m hoping maybe just what were the issues or where were you starting from and what necessitated the change? How’s that?
Wayne Tofteroo (15:21):
You asked the tough questions, huh?
Michael Larsen (15:23):
I guess that’s my job. Yes. (laugh)
Wayne Tofteroo (15:26):
Yeah, no, that is a tough venture change. Go with that. I was at a conference a couple of years ago and I was with a Google engineer and we were looking at what are the new electricity grid diagrams, how they’ll operate. Smart grids, and he recognized it immediately. It was an IT network to him. So we’ve essentially moved in the electricity industry from pylons, generators, suppliers, and distributors to a far more complex, far more IT-rich environment. The only way we can control that or the only way a utility can control that now integrate aspects such as billing and things such as control and maintenance of various components is with smart technology, digital technology, unlike a lot of other companies, a lot of silos. These companies now across the board, water networks, gas networks, so just the networks they’re having to transform very quickly, far more quickly than the companies in other areas have had to transform.
(16:22):
Reasons for that is obviously the introduction of new technology, new software, things such as internet of things such as data-driven decision-making. Someone mentioned billing basically now instead of just billing a bill, a monthly bill, you can bill half hourly for electricity, you can bill half hourly for water. Customers want to be able to see the impact, and do scenarios of what if I use this much water compared to that much water? They want to see that and they want to see that in a mobile application. They want to see that on the phone. They want to see their bill on the phone, they want to be able to interact with the company on the phone. They don’t want to call up a help desk anymore and wait two hours. They want to raise it as a ticket on the phone and then see the response.
(17:04):
And the commercial pressures on these companies now are immense because they’re regulated. Yes, and they have income. Yes, but take the UK for example, or the amount of leaks that water companies have. I think traditionally they used to live with a certain amount, maybe 30% of water losing from the net loss from the network, but now water’s become an expensive commodity. Maintaining water networks is now imperative. So they’re introducing satellite technology to look for leaks, their introducing the internet of things, flow monitoring. They’re collecting data from a vast number of sources. One utility I was looking at had something like 36,000 Internet of things flow monitors in the water network to manage that much data is something brand new for a utility company. And as Scott mentioned earlier on, traditionally these companies have evolved. It’s very much a silo, very static resource that stayed there for maybe 20, 30 years.
(18:00):
Experts in their particular field but not experts in these new technologies. Looking at tools, skill sets, they haven’t got skill sets. So Scott said codeless tools are a must for them, but then the range of technologies, they don’t want it to produce a hundred new tools to manage this diverse range of new software. The fewest number of tools would best, so if we’re looking at say Tosca, they can do the mobile applications, it can do the integrations, the API testing that’s necessary, the data testing, they can do the user acceptance test and SUT testing. You basically are choosing a toolset that cuts down the amount of internal transformation and skill training that you need to do. It’s a huge area with utilities and that’s what makes them different. Becoming a data-driven decision-making organization, managing tens of thousands of new components and old components with the customer pressures, price pressures that they’re now under. That’s why testing so important Without QA they would drown in terms of doing this level of change, being able to do change quickly, accurately as Ronald said is essential now.
Ronald Tetteroo (19:12):
And I think AI is also important here. That what you said, the weekly billing, the pressure that it can come from a marketing campaign which is done maybe in a day and different information you need to then know what to test. So you need to know first be confident that what you’re testing is only what has changed, what you’re interested in, and that’s why you need to be very agile and it needs to be an end-to-end process because it’s interacting, like you said with all these other different systems. It’s not just the billing system but the outcome at the end of the day by being agile that way. What I like again about the utility companies, they mentioned to me, although the world has changed for them and they had to do all these things you just mentioned, the weekly billing, all that complexity, the accuracy of their billing has gone up 4% up the 21 billion. That’s quite an amazing achievement in this new environment. So if you do it right, I think it becomes a competitive advantage for these kind of us to get happy customers and the customers might not go away as quickly as if your customer service will be creating the wrong bill because you’re not as agile as your colleague.
Wayne Tofteroo (20:19):
The other aspect is say field maintenance when you’re maintaining maybe 26,000 components out in the field, take a gas network for example, the price of failure is unacceptable. So you need to be able to test your systems to a hundred percent accuracy, not 99%. They have to be proven. So when you are chasing down, you send a team out to chase down an outage for example, you have to be confident they will get to the right place at the right time. You need that confidence in your software. But at the same time, if you’re also managing changes to your billing system to deal with new components and if you’re managing change to enabling mobile applications and closer integration of customers into your business processes via those mobile outlets, the level of accuracy with the utility company, it has to be a hundred percent. If it gives you the wrong bill, if you’re maintaining your network inaccurately, if you’re not going to the leaks on time, there’s a huge amount of change and the accuracy has to be a hundred percent. So totally get it. Ronald,
Ronald Tetteroo (21:20):
I’m talking about billing, but we’re talking about also security or how you say that safety. I have a great example and that has nothing to do with the cloud. It’s been 10 years ago, but I had a colleague who lived next to me and he was super upset with SAP, really super upset and he used to work in a chemical factory and they have been testing the wrong processes. They had not got the business outcome in their mind, so they were just testing one SAP process to deliver the goods in the warehouse and another process, what they didn’t test is that the goods which came together in the warehouse were highly explosive. So SAP was working, not a problem. They tested the right stuff, but the outcome was that not thought about the fact that the two products came together in the warehouse, which could make an explosion. And you can imagine with utilities, if you think about the speed which people now have to interact, you can’t afford yourself to make mistakes that way. So you need to be also very aware that you trust the automation procedures that you test the business outcomes or the expected outcomes. And that’s so beautiful about model-based automation. You don’t have to script it so you can allow business guys to also think and plan together, which avoids possible catastrophe. So it’s not only the billing, but it could also be a matter of safety.
Wayne Tofteroo (22:37):
Absolutely. I would say utility. Now if it was out thinking about digital transformation, the key enabler is to be able to do testing quickly, efficiently, consistently, and that means bringing automation in. It means bringing in test processes and governance. It’s basically getting that part of the game right before you actually start doing any other changes. If you don’t, then you’re going to build up errors, defects issues, you build up technical debt that will take money, that’ll take time and they’ll destroy the transformation. I’ve probably seen this one transformation where the buildup of technical debt because of badly tested a software or applications basically just destroys your capability of moving forward. So getting it right, getting the right tools, getting the right processes, understanding what needs to happen and making the right tool selections a tool like Tosca for example, which deals with multiple technologies and has a low skill uptake level in terms of training commitment. Those are the types of tools and that’s the types of approaches that companies need to deal with that pace of change.
Scott Swanigan (23:42):
Yeah, Michael, coming back to your question on what are the types of applications that we’re up against in this environment of change? An example might help. Electric utilities. A couple of decades ago it used to be the case, the utility didn’t know that your power was out until you picked up the phone and called them, but with the advent of digital technology, everyone being connected to the internet, the utilities deployed automated meters. I think most everyone has one on the side of their home. These devices, they provide usage data, demand data, tremendous amount of big data back into the utilities operations. So this changes the nature in which let’s say their outage management system needs to operate rather than the outage management system receiving calls in from a customer service representative as an example, it’s now flooded with a tremendous amount of traffic. It needs to be very agile and rapidly digest all that in order to pinpoint where there are power outages and where restoration initiatives need to be taken.
(24:48):
Those changes are being driven into the applications. The application engineers that are delivering these tools hit with all this change, have to be very smart about where they apply their testing resources in order to ensure that the solution’s going to work once it hits production. It’s complicated by the factor that with big data comes innovation and with these innovations, the utilities are finding new ways to use the data and they’re deploying new applications. In recent years, there’s been a move inside the electric utility space to move toward voltage optimization, voltage optimization systems. They use data from the Internet of things, these devices that are in the fields, the AMI meters and other to regulate and keep the voltage on the lines within a certain range in order to save not just the utility but the customer’s money. These applications are starting to sprout up, if you will, by virtue of demand in the market, but also just from the innovation that folks are starting to realize based upon the data that’s available to them.
Wayne Tofteroo (25:57):
It’s interesting, Scott, one of the initiatives in the UK is an open data source. The utilities have realized that they are one of the largest sources of data. The amount of data they have in unstructured format across the utility is massive. Meter readings, maintenance data, flow data. In terms of water networks or electricity networks and gas networks, they have their own metrics, but that data is everywhere in utilities. They’re only now slowly pulling that together and starting to use that as a way of driving those decisions because let’s say one of the business drivers, when you’re a customer of utility and your utility goes up 30%, like it’s happened in the UK, you are going to start asking questions of your utility company, how are you going to do this? How are you going to make this cheaper for me? We pay for inefficiencies.
(26:44):
So there’s a huge amount of pressure, government and business and regulatory for them to become efficient. So the integration of that data and the use of that data would say AI decision-making tools to say, okay, this is now the cheapest way we can maintain that, that work, and this is how we’re bringing that value back to the customer, the end customer. That’s going to be big. It’s big now and it’s getting bigger and as I say, the utilities are sharing that data, which is really interesting and certainly in the electricity industry here with that open data initiative.
Scott Swanigan (27:14):
Yeah. Wayne, you mentioned some of the trends over in the UK. In the United States, electric utilities have been given some regulatory guidance and even government subsidies in order to implement the style of operational technologies, they need to advance in this digital transformation and those subsidies come along with deadlines, so utilities with staff of X that could deliver one or two projects a year of these large projects are now straddled with the responsibility of delivering three, four, or five. So the strain on resources is just tremendous. It’s important to have tools at the delivery level that can lend in this sort of efficiency that we’re alluding to.
Wayne Tofteroo (27:56):
Absolutely, and process. If there’s one thing that a utility needs to have control of is its quality. You can integrate partners in terms of delivery, but at the end of the day, quality belongs to the utility and the performance of those systems belongs to the utility, and that’s why it’s so important I think, in terms of utility owning and initiating the transformation to get the quality right, to get the tooling right from the outset and make sure those are embedded into all the initiatives because failure on any one of those initiatives creates a strain on the rest of the transformation and the timescale is short.
Scott Swanigan (28:33):
Yeah, absolutely. Another thing that I find exciting about the tools that can enable teams today would be the resiliency that’s being built into some of this, and that’s an important aspect tools like Tricentis has to offer. The maintenance that’s required. It’s been a draw on teams. Test automation is created by teams, it’s used by project teams, and then oftentimes these teams, they’re staffed by vendors and those vendors, the personnel walk away from the projects. There’s a lot of turnover in staff and the tools are turned over to operational support teams. When they’re not as resilient to changes in the underlying application, they fall out of practice or fall out of use. That’s problematic. when tools come to the table and they can deliver a sort of resiliency through the use of AI or other methods to where when the application changes, they don’t necessarily have to be modified the test scripts as much. That’s a big value add.
Wayne Tofteroo (29:30):
That’s huge actually, especially as was talking about cloud applications, SaaS applications, SS P applications where maybe a quarterly update on something like or half-yearly update on other products, which may well become monthly updates as we go forward, as companies become more savvy with automation to be able to take a quarterly update, whereas previously you would have business resource on it to do manual tests. That’s how they did it previously, and generations doing that of taking business resource out of the business operation and testing an upgrade and then doing that. Do you think the level of change now being able to enable customers integration to mobile, the backend changes, for example, to make data accessible and changeable, how much testing has to happen based on regression testing for existing processes and things such as billing as well as the new applications that are coming in from different angles? It becomes undoable without a good automation tool and that resilience and that robustness that you are talking about, Scott, it is the thing that changes the game, I think, and enables that pace of change.
Matthew Heusser (30:38):
So just so our audience understands it, with traditional test automation and air quotes, you would write tests against your UI and then you get a new UI with your upgrade in three months and everything would break, and you’d have to go in and you’d have a bunch of different combinations, three times, three times, three times three, whatever the number is, 91, 81 different combinations, 81 different little scripts. You’d have to fix them all and then rerun them. There are ways that you could do better than that, but it was a big maintenance nightmare and you might as well just test it all manually yourself if you had a faster way to do that or a risk-based way to do that. What we’re talking about here is we mathematically analyze those scenarios and we only have one, and if you can rerecord that one and then algorithmically generate the rest, oh, we have a new UI, rerecord, press a button rerun. That process might now take minutes instead of days or weeks. That’s the potential here. That’s what we’re talking about, which is significantly different than I got to write a bunch of selenium stuff in Java. We’re talking about something fundamentally different. So maybe as a sort of parting thoughts, we’ve heard some hints about it, about low-code testing or other techniques. Maybe on the way out we could talk about either where to go for more or parting thoughts about that different style of development. I guess we’ll just go around. Wayne, do you have any thoughts?
Wayne Tofteroo (32:07):
Yeah, I think Scott said earlier on this is this hugely exciting time. I don’t think any other industry has been challenged by the need for change, the amount of change and the need for the precision in that change as the utilities are now being challenged to be able to do this accurately, quickly at low cost and reproduce those results so you can actually maintain that level of agility going forward is a huge challenge. I don’t think any other industry has had it, so that’s what I think makes utilities different. That’s why I think quality is so important, and that’s why I think a good tool selection and good use of technique and process is so important going forward for utilities.
Scott Swanigan (32:50):
Yeah, I would add to that more integrated testing alongside the delivery team is becoming not just a desired trend, but a necessity. So this idea of shifting left or moving to an agile sort of delivery is critical because the pace at which the project teams need to deliver solutions into production, the old waterfall model is starting to break down. It just can’t deliver the goods. Teams are being pressed into a much tighter integration where the various functions, whether that be the code delivery, the environmental integration, the testing, all these functions and team members are working much more closely together. That lends itself to a much better understanding of what’s being delivered. A more rapid need to deliver on testing and testing that is resilient. One sprint builds on another and functionality starts to stack up. Having automated regression tests is critical. Automated regression tests that are resilient and hit the mark, let’s run the tests that need to be run versus the ones that we know are going to pass every time. This is another aspect of the uses of technology that can help teams become more efficient and tremendously so we’ve seen in some cases a sixfold decrease in the amount of time that’s required to deliver on testing through the use of smart technology, which enables teams to look at risk-based assessment of what their test suite should look like.
Ronald Tetteroo (34:16):
But I would like to add, because all the topics have been evolving on our conversations, but the business outcome, I think the key executives in these big utility companies, smaller companies, they’re under pressure where Matt was saying changes might’ve occurred 10 years ago, twice a year, but now they’re on a constant change, could be weekly, could be quarterly. It’s constant and the world is changing. If you can start early detecting these changes and you know exactly how to automate those changes to test the complex environments you’re in, you can discover the defects earlier, go live earlier, and without production mistakes. That gives the confidence to key executives in these huge utilities in all aspects to invest in these kinds of projects because the business outcome can be a tremendous positive way forward to have the confidence to make changes, to get better, to get customer experience in a much better fashion without having the pain to be tracked back that these projects will be on time or will be not successful. I think that the combination of key executives getting confidence and these products, getting more user-friendly can make a big change for moving forward for utilities to be more agile and have business outcomes where they can prove that their investments actually were correct.
Scott Swanigan (35:36):
I agree with all that. Another interesting thing, we kind of take it for granted these days, but we’re in a global economy, and that means global project delivery. Working around the clock is typical for project teams, and the ability to do unattended execution, it can become important. A lot of project teams, they leverage offshore capabilities in the United States out of India and other countries, but even without that, there’s a potential here that technology can come to the table with, again, unintended execution. These tests could be run overnight and then the team wakes up in the morning to digest the results and move forward from there. So it becomes a 24/7 operation that can enhance things significantly.
Matthew Heusser (36:21):
Well, thank you, Scott, Ronald, and Wayne, we’ve kept you long enough. Appreciate you being on the show and to our audience, we’d love to hear from you and we’ll see you next time. Thanks everybody.
Scott Swanigan (36:33):
Thank you.
Ronald Tetteroo (36:33):
Thanks very much.
Wayne Tofteroo (36:34):
Cheers.
Scott Swanigan (36:35):
Enjoyed the conversation. Thank you.
Michael Larsen (OUTRO):
That concludes this episode of the Testing Show. We also want to encourage you, our listeners to give us a rating and a review on Apple Podcasts. Those ratings and reviews help raise the visibility of the show and let more people find us. Also, we want to invite you to come join us on The Testing Show Slack channel as a way to communicate about the show. Talk to us about what you like and what you’d like to hear. And also to help us shape future shows, please email us at [email protected] and we will send you an invite to join the group. The Testing Show is produced and edited by Michael Larsen, moderated by Matt Heusser with frequent contributions from our many featured guests who bring the topics and expertise to make the show happen. Additionally, if you have questions you’d like to see addressed on the testing show or if you would like to be a guest on the podcast, please email us at [email protected].
