Brick and Mortar stores have a lot of software in their operations. From supply chain to ordering, point of sale, inventory management, customer procurement and customer satisfaction, that’s a lot of moving pieces to keep track of. It’s a challenging endeavor to keep it all together, and QualiTest’s Mike Hershkovitz joins Matt and Jess to talk about the ins and outs or testing for the retail space.
Also, in a separately recorded segment, the Amazon purchase of Whole Foods and what it might mean is discussed by Matt, Perze, Justin and Michael. Is it really an additional 400-plus Amazon distribution centers, or is there something else going on here?
Panelists:
References:
- What Amazon’s Purchase of Whole Foods Really Means
- Amazon’s Jeff Bezos Explains Why He Bought The Washington Post
- Now Anyone Can Tap The AI Behind Amazon’s Recommendations
- ALDI US
- The Glass Cage: How Our Computers Are Changing Us
- The Shallows: What the Internet Is Doing to Our Brains
- The Model Office
- Retail and eCommerce Testing
Transcript:
MICHAEL LARSEN: Hi everyone. This is Michael Larsen, the producer of The Testing Show. Due to scheduling challenges, this show was recorded in two parts. The Interview with Mike Hershkovitz of QualiTest was recorded first with Matt and Jess, and then we got together on another day to record the news segment with the rest of the panel. We apologize for the discontinuity, and with that, on with the show…
MATTHEW HEUSSER: Testing for retail, software testing for retail, yesterday. It was incredibly busy. So, we’ve got a very special two-parter for you this week. We have—yes—the interview with Mike, but also some of the news around “retail software” and specifically the Internet that has come out over the past two weeks. It just a nice little intersection. So, we’ve brought together the, kind of, classic talking heads of the show, Michael Larsen?
MICHAEL LARSEN: Howdy.
MATTHEW HEUSSER: Perze Ababa?
PERZE ABABA: Hello.
MATTHEW HEUSSER: And, Justin Rohrman?
JUSTIN ROHRMAN: Hey there.
MATTHEW HEUSSER: To talk about, “What’s going on with Amazon? What’s going on with Whole Foods?” Right? “Is this going to revolutionize the way that we do commerce, or is it just like they bought a grocery store?” I mean, the last time I checked, Amazon bought a newspaper, and it was just, “Eh, they bought a newspaper.” Am I wrong about that?
MICHAEL LARSEN: I don’t know if you’re wrong, but y’all have to forgive me a little bit. Maybe it’s my lack of imagination here. I think there are a few different angles that we can look at with this acquisition and what it might mean. There is a whole, big kerfuffle at the moment—I just love that word—where everybody’s concerned that, “Oh, no. Amazon has bought Whole Foods. Does this mean that stores are going to close down and people are going to disappear and etcetera, etcetera, and robots are going to start doing everything?” That’s been a concern, regardless of whether or not Whole Foods was owned by Amazon or not. But, I guess, my bigger question is, “What does a large online retailer purchasing a classically brick‑and‑mortar retail—?” Let’s face it, grocery stores are as classical brick-and-mortar as you can possibly get, for very obvious reasons, “I need to go grocery shopping for dinner tonight or for the next couple of days.” Yes, there are services that you can get. I’m a subscriber to Imperfect Produce, which is an online grocery shopping service, if you will, but I have limited ability to pick what I’m going to get, oftentimes. Sometimes it’s a grab bag and surprise. I kind of like that, but I don’t get that opportunity to go, “Hmm. I want exactly that thing,” or I want to be able to weigh them and see what I want to work into what I’m cooking a given night. I go down to a store for that; and, very often, when I’m piecing things together, I get a choice of things and I decide, “Oh, you know what? I want to do this.” You know, “If I don’t have saffron, what do I want to use instead? Oh, I guess, I can get some turmeric, or I can get some other deal.” A lot of those are in-the-moment decisions, based on what’s right around you. Is my getting open an app and saying, “Okay. I’m going to scroll through here now, and I’m going to click on certain things,” and that’s going to somehow magically make my shopping experience. I don’t see your every‑day shopper going in and suddenly thinking, “Okay. I have my tablet with me or my pad, and I’m going to start punching in a whole bunch of buttons,” and you’ve revolutioned the store experience where there’s next to nobody in the store and you’re still not having those conversations. I don’t even want to think about how you’re going to manage a meat counter online.
MATTHEW HEUSSER: Ooh, you’ve just… mind blown. There are a couple of things there. So, first of all, I don’t think it’s high tech, “Enable your tablet to connect to your smartphone to tweet when you need milk.” I don’t think it’s about that. I think it’s about, “Amazon just got 400 prime retail locations in suburban areas with affluent hipster, consumerist customers with a complete supply chain.” So, what that means is, these places are good at getting groceries delivered—like fresh lettuce, well before it’s going to go bad, fresh salmon. They have figured out all of those things. So, yeah. You can get your Amazon things shipped to your door, for Prime, for $75.00 a year. Or you could walk—I don’t know—three blocks to the place that you were going to go anyway to get your groceries and pick it up from there and Amazon will pay you. They just call you one of their distribution centers. That’s number one.
JUSTIN ROHRMAN: So, there is like a bowl of interesting points that have been brought up. First of all, I’d like to point that, in most places and most cities, people aren’t walking to the grocery store. In America, it’s just not a pedestrian-friendly culture. But Amazon is going to do what Amazon does, which is process automation. So, here in Nashville, if I order from Amazon, most of the deliveries don’t come from UPS anymore. They come from a guy driving an Amazon van. So, what’s going to happen is, eventually online, people are going to order groceries online and they’re going to impulse buy a bunch of stuff because of the Amazon suggestions machine and then the groceries are going to be delivered to their house an hour later. So, what you’re going to see, over time, is monopolistic behavior in grocery stores because Amazon is going to buy up other chains and probably job loss because of this process automation. You won’t really need check-out people anymore.
PERZE ABABA: Piggybacking on what Justin is saying, there’s definitely two areas that are greatly going to be affected by this. So, you have the consumer side where prices might be driven lower because Amazon is “okay” with having low margins, and they catch up with that with the volume that they sell. But, this is really going to be bad for our existing mom-and-pop grocery stores. On the tech side of things, if you look at the software that the retail industry uses, point-of-sales systems for example or CRM or inventory management, and what Matt mentioned, “supply chain,” and then there’s a little bit of retail accounting that needs to be dealt with. With regards to supply chain and inventory management, the one thing that really excites me from this venture, Amazon has their dash buttons, for example. They have their beacons. They’re jumping into the Internet of Things as well. That can really solidify the existing practices around inventory management, and I’ve seen some experiments with Target as well as CVS where if your app is open while you’re in their store, they can give you push notifications where you’re near something that has a sale. So, they can say, “Hey. There’s a coupon for this right on this aisle—in this area, in this aisle—maybe you should check it out.” Then, for me, that’s actually pretty interesting. It’s pretty exciting, because as they standardize these technologies, it does help me with managing the inventory that I have in my own home. The people that are probably building smart-fridges are pretty excited with this as well, because you have all of this connectivity now. From the short-term perspective, all of these point‑of‑sales systems, CRM, there’s going to be a ton of migration work that needs to be done between Whole Foods, for example, as they bring that into Amazon. So, that’s going to be a ton of work from the tech side of things as well as the testing side of things. But then, as these things get standardized and as with other grocery stores—like Costco, for example—is one of the areas that’s going to be affected by this. It’s what Justin said, we might not know if this will affect us in the near-term; but, in the short-term and even long-term, there are definitely going to be changes in the retail ecosystem because of this.
MATTHEW HEUSSER: I think you’re absolutely right, that they’re going to ALDI’ify the process. One thing Amazon is particularly good at is warehouse distribution and logistics. They’re going to do the same thing with the grocery store, and they’re going to streamline it. They’re going to squeeze that extra 1 percent, 2 percent, or 3 percent profit. It’s possible they get to do Apple-Pay level stuff where there’s self-checkout stuff but even more streamlined, and they get to squeeze another couple percent of profit, which then they can do whatever they want with. They can spin it off and put it on Wall Street or Amazon Groceries are going to get like 10 times better, if you live within 10 miles of a Whole Foods Market. They’re going to deliver it to your door. They’re going to have figured out all of the “how-to’s.” I think that’s a big deal.
JUSTIN ROHRMAN: So just for a point of reference, I just looked this up. Matt mentioned that it would get better for people that live near Whole Foods. There are 431 Whole Foods right now in the US, and I guess you can compare that with Kroger which is another grocery chain. There are 2,778 Kroger. So, if Kroger did something like this, I think it would be a significantly-bigger deal.
MATTHEW HEUSSER: Well, what Whole Foods has is the customers that are willing to spend more for stuff they like.
JUSTIN ROHRMAN: Yeah. Yeah. That’s absolutely right.
MATTHEW HEUSSER: They’re already on eComm.
JUSTIN ROHRMAN: The Whole Foods customers are probably also people that buy Apple products.
MICHAEL LARSEN: This is a good point. Yes.
JUSTIN ROHRMAN: Or Tesla Cars or whatever. Luxury market people.
MICHAEL LARSEN: You know, as is always the case, whenever something like this happens, my guess is that there’s all the things that they expect to have and then the cascading group of effects that “nobody ever anticipated.” It’s the “nobody ever anticipated” that actually has me, [LAUGHTER], raising my eyebrows and looking with great interest.
JUSTIN ROHRMAN: I think that Nicholas Carr would have a lot to say about this. He wrote The Glass Cage. The Shallows. He talks a lot about how automation changes the landscape in terms of jobs, in terms of skill loss, things like that.
[End of News Segment Recording]
MATTHEW HEUSSER: So, we wanted to bring you a real expert in retail commerce, specifically with this Amazon-Walmart land grab. Walmart is in both worlds, right? Walmart is both retail and they have a weaker e-commerce engine, and Amazon is going the other way. To do that, we wanted to talk to Mike Hershkovitz of QualiTest who is a business manager in their West Coast Office. He is a really busy guy, so we wanted to jump right into it. Welcome to the show, Mike.
MIKE HERSHKOVITZ: Thank you very much.
MATTHEW HEUSSER: We also have Jessica Ingrassellino on the line. Welcome back, Jess.
JESSICA INGRASSELLINO: Thank you.
MATTHEW HEUSSER: So, Mike and I have been taking offline a little bit about all of the things that go into “testing for retail.” I’d imagine that’d be somebody like the Gap or Target, any brick-and-mortar traditional presence. There’s actually a whole lot of software in there. So—
MIKE HERSHKOVITZ: That is correct.
MATTHEW HEUSSER: You’ve got the point-of-sale system, you’ve got the order management system. You’ve got the website. You’ve got load testing. You’ve got performance testing. You’ve got PCI.
MIKE HERSHKOVITZ: Mobile devices.
MATTHEW HEUSSER: Justin and I were at a company in Arkansas a while back in retail, and they had the handheld scanner-y things which they were using—
MIKE HERSHKOVITZ: Yeah.
MATTHEW HEUSSER: —for inventory management and such.
MIKE HERSHKOVITZ: They are IBD. It’s a scanner that, nowadays, you can do also some of the scanning with mobile devices. As you may know, most of the mobile devices have the option. Because they have cameras, they can also scan the barcodes and just load it into the point-of-sale. So it’s becoming more and more popular. I don’t know if you’re familiar. I think, with Amazon Store, they opened their own store, which you can actually—they don’t have any salesperson over there—just go in, scan it with your phone (the product that you want to buy), and just pay with your phone. Actually, eliminate human contact in some way but make it more efficient because you don’t need to wait in line. You don’t need to bring cash with you or whatever. Everything is done through the phone.
MATTHEW HEUSSER: So, are you seeing that in retail? Is it personal devices? Are people loading up the company software on their personal device? The employees, I’m asking about? Are they just issuing them a phone?
MIKE HERSHKOVITZ: At the moment, they’re still issuing them a phone; but, as I see it, the future is going to be that everyone, every customer, will have the option to just scan the barcode with his own mobile device. Pay can be with Google Pay, with Apple Pay, or whatever. You know, but they can do it just over the phone. No need for credit card. No need for cash. It will be much faster and efficient.
MATTHEW HEUSSER: So I just mentioned about a half-dozen different ways, and I know I forgot some, “security and penetration testing, auditing.”
MIKE HERSHKOVITZ: Um-hum. Yes.
MATTHEW HEUSSER: When you talk to a company about all of the testing needs that they have that are probably being done by different groups, that are being done inconsistently, where do we even start with how to figure out what we’re doing and how to improve it?
MIKE HERSHKOVITZ: The first thing that we’re doing, when we reach out to a new client, is kind of investigating. Learning, “What’s their current situation?” Learning, “What’s the pinpoint?” Trying to build with them the roadmap of what they want to achieve. For example, one of our clients, they want to replace/upgrade three systems at the same time, which is the point‑of‑sale, order management system, as well as the Omni–channel, the e‑commerce. So, this is a very difficult task, replacing three things at the same time, which means data migration, testing needs to be done, which means also making sure that the systems can work well with each other, and at the same time, choosing the right timing to go live. Because, at the beginning of the plan, for example, (their plan) was to do it in December, which is the worst timing because, you know, Christmas and all those holidays.
MATTHEW HEUSSER: Right.
MIKE HERSHKOVITZ: You don’t want to lose clients.
MATTHEW HEUSSER: Right. Right.
MIKE HERSHKOVITZ: You don’t want to be offline. If you have any glitch, you will lose millions of dollars. So, we sat with them. We worked together with them as partners in order to build the right timing, the right schedule, broken into the bits and bytes of the testing effort, and also working in collaboration with developers, which is important.
JESSICA INGRASSELLINO: Um-hum.
MIKE HERSHKOVITZ: Because they cannot live without each other.
MATTHEW HEUSSER: When did you end up flipping the switch?
MIKE HERSHKOVITZ: In February. It’s a month without holidays, which makes it more comfortable and low risk. You’re trying to make sure that you’re not in the high holidays or any other time of the year where you have a lot of customers working, if it’s online, also in the stores. So, February was a good month to run with that.
MATTHEW HEUSSER: How is the risk picture different for retail? How is testing different compared to—I don’t know—a website?
MIKE HERSHKOVITZ: It’s different; because, in retail, most of them, you have physical stores as well as online. The idea is to, first of all, in some cases, we are testing both of them. We are testing the POS system (which is in stores), the OM system (which is also in the stores), as well as the online purchasing system. It’s different; because, again, it’s something that, if you are doing half work—for example, if you’re just testing the website, the website can work perfectly; but, then, when somebody makes an order, it doesn’t go straight to the POS system. It will say that it does, but the OM system doesn’t get the update. So, there is no new inventory—that can put the company at risk, because they won’t be efficient. The customers won’t like the fact that the orders will take more days than expected. In order to be efficient, in order to be able to move things fast and collaborate with each other, they need to make sure that everything is covered—not only the website, but also the other systems that are actually in every store.
MATTHEW HEUSSER: Okay. So, let me see if I got that right. So, you have your point-of-sale system and they just bought a laptop computer and you need to refresh the inventory, you need to get a new laptop computer in so you don’t ever run out of inventory?
MIKE HERSHKOVITZ: Yes. Correct.
MATTHEW HEUSSER: That flows into the OMS (the order management system)?
MIKE HERSHKOVITZ: Um-hum.
MATTHEW HEUSSER: Now, there’s a couple of different ways to do this, but you’ve either got to go to a warehouse to get a computer or you’ve got to order it from Apple or something?
MIKE HERSHKOVITZ: Yes. Correct.
MATTHEW HEUSSER: Does the OMS do that, or is there a greater ERP system that does that?
MIKE HERSHKOVITZ: Usually it’s ERP, but OMS can work out that as well. But, it depends on the server. There is an ERP system that usually the OMS is connected with, so they make the new inventory. For example, let’s take—everybody knows—Walmart. Now Walmart, they’re working very efficiently. They don’t have much inventory in the store. They have what they have on the shelf. They don’t have big storage places like all the other stores—Target, etcetera. In their case, whatever they have on the shelf, if it’s gone, most cases, it’s gone. So, they need to be very efficient. So the moment they see that one product is not going to be on the shelf anymore, they need to get it shipped immediately. In their case, there is an issue with ERP, with POS, or OMS system, something’s not working there. They won’t have their products on the shelf. Customers will come over there and will be disappointed—that they’re looking for something and it’s not there. Efficiency and making sure that everything works well together, it’s high priority for them.
MATTHEW HEUSSER: Yeah. I’m in Allegan, Michigan. We have a company here called “Perrigo.” It’s Fortune 500, but it’s small. They make knock-off pharmaceuticals, imitation Sudafed, and they had an ERP upgrade. An ERP system conversion. This has been about 10 years ago. It didn’t work. It was so buggy, they put the wrong things on the shelf. They sent more of what the companies already had, and the companies ran out of the things that they needed. The CEO stepped down, and the previous CEO stepped back in to right the ship for a few years. He had been retired. He came out of retirement to get that project done. It was messed up. So, it matters.
MIKE HERSHKOVITZ: Oh, I can give you example. Another thing that we’re doing—for example, for one of our clients—we are doing a load and performance testing every year, especially before Black Friday. Imagine to yourself—I can’t disclose the name—a big retail company that (Black Friday, as you know, is one of the highest, [LAUGHTER], and most profitable times of the year)—that hundreds-of-thousands of people are going into the website, trying to get the best deal ever, and the website crashes. Somebody will actually die. No. [LAUGHTER]. You know, people will be disappointed. The managers of this retail can get fired because they messed up on the most important day of the year; and, if you don’t do the testing incorrectly, making sure that the website can hold the amount of people that we expect them to, to happen on a certain day, initially you kind of multiply it by two at least, just to be on the safe side. This is something that is critical. People can lose their job, just because of this small thing, that is very important to make sure that it is working well.
MATTHEW HEUSSER: Yeah. Absolutely. I’ve been amazed when I talk to companies that they have entire gaps in their risk management strategy, “We do this kind of testing really well, and we do this kind of testing. But, what about from this system to that system?” “Oh.”
MIKE HERSHKOVITZ: Um-hum.
MATTHEW HEUSSER: “We didn’t really think about that.” Especially when that crosses team boundaries. When you have the POS team and the OMS team, then your tests, the integration—
MIKE HERSHKOVITZ: They’re not working together.
MATTHEW HEUSSER: —testing might or might not happen. It might or might not be good. “Who is responsible for it?”
MIKE HERSHKOVITZ: Yeah.
MATTHEW HEUSSER: Right? “It works on our side.” “It works on our side.” “But, does it work for the customer?”
JESSICA INGRASSELLINO: Something else that I have noticed, I have worked in some retail spaces prior to Salesforce, and some of the retail spaces that I’ve worked in, we worked with third-party POS systems. So, we weren’t responsible for the point-of-sale system; and, yet, at the same time, we had to “try to do some testing” with that point-of-sale system, but we didn’t always have the full access. That was something that was really difficult. I wonder, “Does that happen frequently or was that just because I was in a start-up space?” What’s the kind of deal with that, and what would you recommend?
MIKE HERSHKOVITZ: Yeah. So, most of them are POS systems. You know, I think that it’s rare that a company develops their own POS system. Usually, they are taking an off‑the‑shelf product and just work with that. So, they the ability to get to a certain level of testing. It’s not completely open for testing. You know, they won’t be able to go through the bits and bytes. So, the recommendation (and usually what we are doing and also telling our customers), we are putting in some risk mitigation. So, we are raising flags in the beginning of the project to say, “Okay. Those areas are areas that we cannot test, so this is a risk. We put in the flags ahead of time, letting you know that this is a risk, and we can try to work around it.” Now, if we see that in our testing that there are issues that are deeper than what we can handle—because now that we don’t have access to those areas—then we are reaching out to the company that develops this POS system and working with them in order to resolve the issues. We send them the debug report, trying to work it out with them in order to make sure that our client gets the best product ever. I think it’s rare that we’ve gotten to the point that we needed to reach to the developers of the POS system. Usually, we just work on doing the integration because those systems that we are testing usually are something that have been working for a long time. So, the issues in the systems themselves are low. Saying that, we have two clients that they are the developers of POS systems and do the testing for them. So, in that case, I feel very comfortable, [LAUGHTER], because we’re doing the testing also for the retail as well as for the POS system. I mean, it’s extra. So, it makes it a lot easier to have end-to-end testing with that.
MATTHEW HEUSSER: There’s a sort of platonic ideal that, “We get all these systems together and we would automate everything with good-clean data, and we could run through end-to-end and we could simulate the entire order, and we’re going to run the simulator in the cloud or we’re going to run the simulator on our desktop under these very‑controlled conditions. Just going to press a button and we just ran an order all the way through.” I’m not so sure. That’s definitely not necessary. I’m not even so sure it’s ideal. Right?
MIKE HERSHKOVITZ: Um-hum.
MATTHEW HEUSSER: So, if we can, as human beings, actually test the integration with a POS system and say, “This is the kind of data that’s going to come out of it,” and we save that data somewhere and then we automate all of our stuff, which is much smaller, we only have to retest it when we upgrade the POS system. I don’t think that’s huge problem, but it doesn’t conform to the sort of ideal that I commonly hear of, “Press a button, get a green bar.” Or every time we commit, it’s going to run through everything.
MIKE HERSHKOVITZ: No. No. Doing the simulator is nice, but it’s not reality. The reality can change.
MATTHEW HEUSSER: Yeah.
MIKE HERSHKOVITZ: That’s the reason, for example, we recommend (when we can, of course) going to talk with end-users. For example, if it’s a POS system, so it’s the cashier. They use it every day. They are familiar with that. They know what the issues are. Because the engineers that are working on that, can think about the issues that may appears; but, in the reality, the person that’s sitting in front of the POS system and pressing the buttons, he’s the one that knows all the issues and problems with this device. Talking with those individuals is the most important thing, because that way you can build some kind of a testing plan based on that. Whenever we can, we also like to build personas—like choosing five different personas and do the testing based on them. That way you know you cover a very-high range. Taking five different types of people and just, you know, building the test cases based on that, and that gives you some kind of usability of different type of users. Think again. We’ll go back. Let’s take something different. Let’s say, end-users that are customers. I’m taking myself, for example. Now, I have a new baby and I’m starting to buy things that I never bought before. Like, cradles and bottles and diapers and whatever. I’m one type of user. There is another type of user, which is young kids, 20-years-old, or something like that, he doesn’t care about quality. He just wants it fast. He wants to make sure that you get it tomorrow. He wants to make sure that the system works quickly, because he wants, you know, to go out and hang out with his friends. So, those are a different type of people that retail needs to make sure that they address the testing accordingly to their behavior.
MATTHEW HEUSSER: Yeah. So have you tried, or are you familiar with, the idea of a model office? It sounds you might be using a different name. In model office, we pretty much have all of the roles that are actually going to be doing the jobs, and we simulate—
MIKE HERSHKOVITZ: Oh, yes. Yes. Yes.
MATTHEW HEUSSER: —kind of a shop floor, end-to-end, with actual people doing all the work.
MIKE HERSHKOVITZ: Yes. We have something similar. We have something in our interface that we have the actual POS system. We have the other employees that are coming, for example, and running their credit card, and making sure that it’s running smoothly end‑to‑end, including sending back to the inventory system and making sure that we have new inventory. Whenever we can, we’re trying to imitate their reality.
MATTHEW HEUSSER: Great. I know you’re a busy man, and we’re glad to have you for the little time that we had. Where can people go to learn more about this kind of stuff?
MIKE HERSHKOVITZ: People can go to our website: https://www.qualitestgroup.com. They can find a lot of information about retail, about how to test in retail. We have case studies and white papers that they can read and learn more, and of course, reach out to us if they need, as far as doing testing or if they want us to collaborate with them and give them advice as to how to do testing. We are available almost 24/7. So, it’s very easy to reach out to us.
MATTHEW HEUSSER: Great. All right. Thank you for being on the show, Mike. Appreciate your insights, and we’ll have you back again real soon.
MIKE HERSHKOVITZ: Thank you very much, guys.
JESSICA INGRASSELLINO: Thank you.
MATTHEW HEUSSER: All right. Bye.
MIKE HERSHKOVITZ: Thanks. Bye-bye.
[END OF TRANSCRIPT]
