Eran Kinsburner who is Product Management and Quality in a number of very large corporations such as Texas Instruments, GE Healthcare and Sun Microsystems. For the last 10 years he has been the Chief Evangelist at, first at Perfecto and later at Perforce Software. Not less important, Eran is a Best Selling Author of a number of books focusing on software testing and quality of which the last one called Accelerating Software Quality was published last year. With this background you are likely to be the absolute best suited person for this blog, which looks at various aspects of mobile app quality.
Please tell us a few words about Perfecto and Perforce and your role
Thank you, so yes as you mentioned I am Chief Evangelist and a senior director at Perforce Software. I actually oversee a few areas of their portfolio. Perforce is a portfolio company, so obviously, I look at what Perfecto does, content writing, speaking at Perfecto events, and this also includes mobile and desktop web testing, and I am also involved in the sast business, the static code analysis under the clockwork and QAC products under this portfolio. At the end of the day, everything contributes to quality. Whether it’s mobile, web, or automotive. Perforce touches upon a lot of devops practices in the development stage, but also throughout the different phases including quality. I would summarize that I am looking from an Evangelist perspective into the portfolio that relates to the software testing and the quality of different application types, whether that is software test automation, static code analysis, security testing etc.
You are a world renowned expert on Software Quality. Are there any specific issues when it comes to mobile app quality or do you see mobile apps the same way as all other kinds of software?
That’s a great question, and I believe that if we were talking about mobile five or six years ago, maybe my answer would be different. But especially after Covid, digital is king. If there was ever a question before Covid, then certainly after Covid it has become so clear how everything is dependent on digital, and by digital specifically meaning mobile. I believe that today the cause of failure of a software company starts with mobile devices. This is what we all use the most, on any journey we take. We might end our journey on the web, or on a desktop platform, but the majority of time we spend these days is on mobile. This is where mobile quality, mobile security, safety, compliance with accessibility, and other things, are the most important today. That is king. This is what will be the difference between a mobile winner, and a mobile loser. If you don’t provide a good mobile experience, or quality, you will even be disqualified within a market segment. I see that day in and day out. I serve a lot of enterprises across different verticals, telcos, retail, financial insurance, automotive, and they are all investing in mobile growth. You can look at it 80/20. 80% of their funds are spent on mobile growth, 5G, user experience, accessibility, chatbots, voice assistant services, and many more. So as to your point, mobile was the growing king years ago. Today there’s no doubt, it is king.
In a recent blog that you wrote in relation to Tesla you argue about the importance of testing in production but only to a point. Can you elaborate?
Yes, actually the blog that I wrote really instigated a lot of discussion, and I didn’t even mean for that to happen necessarily. I was simply commenting on a recent statement written by Elon Musk about testing in production. Now when you test in production and you look at a specific software, it is not just the software itself that is important, but it’s the type of software that runs in production and who will be impacted by that software, these are things that have a lot of consequences. When Apple releases an iOS15 beta to the public, it is not really dangerous, it actually benefits the developers. The developers can then take the public betas, understand what is coming in the GA, in the final release and they can plan, and actually make their applications better, higher quality, and more compliant with the different regulations. When you release a public beta in a self driving car, and this self driving car moves within occupied cities with different people who may not be consumers of this self driving car, then it begs the question: why should they be “paying the price” for a public beta which is kind of being tested in production, by drivers, who could be new drivers, not really qualified to be driving this kind of car. This was my point. Public betas serve a purpose. They should be used to drive more knowledge and bring more insights to the developers. This being said, however, if it brings risks and has implications on public health that could cause death unfortunately, as we have seen in the past with self driving cars, it needs to be done with a more gradual and managed approach.
How do you see the relationship between QA and R&D? Is there a thumbnail rule for the number of QA people in relation to R&D?
That is a very complex question, and I think it varies between: A. Verticals and B. Maturity of the organization. As mature as you are, today there are concepts such as squads and agile teams, and within these teams you have developers who are doing testing and testers who are doing early TDD, test driven development. I don’t think there is a number or a formula for best practice. It really depends on how the organization is built, how complex the software is, some softwares are harder to automate than others. Harder to automate means more testers. Look at Elon Musk, he needs the entire public to be able to test his software. There is no one right formula for a development manager, as to how many developers vs. testers. You need to assess the software scope. Each iteration has a given scope. You might just implement five user stories, or story points, and that would be a small release, or a small fix. There is some balance that occurs, or some scaling up and down as to the necessary testing and it’s not a given number throughout the year. You need to adapt.
Some companies have argued to eliminate QA departments as part of what they call continuous delivery of code. What is your opinion on this?
That would be a risky statement to make in today’s reality, because each individual has their skill set. You go to the dentist, but you also have the assistant. Remove the assistant, and the dentist might be troubled to doing things that they do not need to be doing. I think there is room in today’s reality, for both testers and developers. I would focus on the objective and less on their title. Developers are getting paid to have x amount of story points implemented in a given sprint or software iteration, and it needs to work top notch. In order to work top notch, they need their own unit testing, or whatever testing can fit in their sprint, supported by a higher scale, and even more professional skill sets and creativity that comes from within the testing departments. So, whether that is covered by COE, or individuals sitting in a given squad or developer team, that is the necessity. I don’t think you can eliminate the need for that. And by the way, it is not a blame game. Developers don’t sign off on a release nor do testers. That is not the way it works. They are a team, together. At the end of the day there is management. When do you know when the product is ready to ship to production. Developers are working on certain aspects, testers are working on certain aspects. Management gets all the data from all of the different personas and needs to decide if it’s good enough to release to production or not.
Do you find that there is a difference in user sensitivity towards poor app quality between different age groups or across different countries? Has it developed over time?
Even the elderly generation is becoming more aware of the power of mobile apps and devices, their expectations as far as app quality and performance are continuing to grow, they are looking at mobile as an assistant technology, rather than just calling their grandchildren. I mean, you see how today we have many apps that can be used as medical devices, your watch can be a medical device. Even the elderly today use mobile technology as an assistant in their daily journeys. Smart watches can test blood, glucose, blood pressure, and even send alerts if something happens. So different age groups today all rely on mobile technology. Take 5G for example, everything today runs on 5G, and everything is connected. Today your cars are connected to traffic lights which can then send information to police, to different cities etc. In my opinion, dependency on mobile technology is only continuing to grow, especially with the new 5G web. You have watches, smartphones, apple home, google hub, amazon, the screen is just a reflection, just a channel of communication. While it is all very exciting, this is also the single biggest point of failure. You spoke about quality, performance, but security is equally as important. Because it takes just one device that is exposed in order to break the chain. We see in mobile how new compliances and policies are being constructed for better cyber security to protect the mobile data and all the systems that correlate.
It isn’t just age groups that should be the question. It is bigger than that. Look at professions. A pilot, regardless of his age group uses different applications, he moves from wifi to airplane mode to 5G. He gets profession specific notifications. The age segment and the professions greatly influence the user journey. The user journey is what makes the difference. What the individual does from start to finish on their mobile device, including not only what apps they are on, but also what apps are running in the background. I recently spoke to Lego, which is an interesting case because although they started out for children, they have a very strong client base that is adult dominated. There is actually an entire show in the US called Lego masters, there are no children there. Just adults. Lego is focusing on making their brand purchasable through their mobile app so adult lego enthusiasts can buy the latest sets, and they have to take into account what is running in the background, in what location, as well as the app itself. Again, the entire user journey is what counts, and these days, there is little to no tolerance for poor app quality.
I’d like to get into some technical questions, concerning apps. Shipbook, as you know, helps promote and maintain app quality. What parameters would you recommend that app owners look at when they need to choose whether to allocate resources for app quality or new features?
It’s kind of like the chicken and the egg. If you don’t innovate, you don’t win in the marketplace and if your quality is subpar you lose the market. It is really a balance between innovation and quality; one cannot be compromised for the other. If you read the Agile Manifesto, or the Devops Manifesto (that is also emerging), this is the exact definition of “agile”: the key is small increments that work, give value to the public, across different types of journeys and platforms, but with high quality. So as an answer to your question, app owners must continue to invest and innovate to remain competitive in the marketplace while investing in quality, performance, accessibility and security. It however is not solely on the shoulders of the testers, but it also is the responsibility of developers, a full cross team collaboration, where everyone is invested. It takes a village as we say to create a winning mobile application. Just building on this, byt the way, you can look at a recent market trend, Flutter. Google is developing Flutter to allow developers a single code base based on Dart that can be deployed to multiple platforms, web, Linux, windows, mobile iOS and Android, etc. This is one of the answers of one of the main technology providers, Google, giving the public one code to manage and maintain, with an excellent UI etc, but with the overarching goal to continue the innovation. So innovation never stops. We can take advantage of the technology out there to continue to create and advance and innovate but never compromise on quality.
Where do you see app quality go in the future?
Let’s make the distinction between app quality and testing. Ensuring app quality requires more intelligent testing, based on data, that can work on a higher scale. Today everyone is talking about doing everything continuously: continuous testing, continuous integration, continuous delivery. When you are doing everything continuously, you are producing a lot of data. To ensure app quality you need something to consolidate all of that data. You have data coming in from production, data from pre-production. How do you assess all of the data and provide actionable feedback to the decision makers? Whether they are development, product, or test managers, at the end of the day, someone needs to control the data coming in from different channels. The controlled intelligence at the end of the software delivery pipeline is what can contribute most to app quality. There are technologies and vendors that are growing as we speak to assess this data and make it available in the future to better app quality.
Do you have a dream app that hasn’t been developed yet?
As someone who is over 20 years in the software industry, and starting out on microsystems early on in the 2000’s, what I have constantly seen as an unsolved problem, is app quality. In mobile especially, to have app quality you must have automation and scalability, maintenance and scalability. This is where a lot of the industry is still facing challenges, to somehow create test automation that is scalable and works consistently. At Perfecto, we try to automate every possible mobile and web scenario. When we finish addressing these main issues, suddenly technology grows again. The pace that technology is growing and our ability to test and ensure quality remains a gap that we have yet to bridge. Think about 5G, infotainment systems, audio and video quality, and accessibility with voiceover. All of these represent innovations in technology that come into contrast with all of the previous technologies that you are attempting to maintain and incorporate. My dream would be to bridge the gap between where we are today as far as technology and our ability to test, at scale, these types of technologies in a signal tool. Because today we have a plethora of open source libraries, and tools, and visual solutions etc etc, and you just get lost. There is no single solution that will cover, end to end everything. I wish there was one test plan for mobile applications in a single framework that can cover everything. Today we have yet to see something like this.
Thank you very much for your time, and for sharing your expertise with us, it was very eye opening.
Shipbook gives you the power to remotely gather, search and analyze your user logs and exceptions in the cloud, on a per-user & session basis.