Interview with Yair Bar-On | Serial entrepreneur whose company, TestFairy, was acquired earlier this year by Sauce Labs
Yair Bar-On is a serial entrepreneur whose company, TestFairy, was acquired earlier this year by Sauce Labs. TestFairy is a mobile beta testing platform, handling app distribution, session recordings, and user feedback, in enterprise and secure environments.
Beta Testing is an important part of overall app quality and therefore we are very pleased that you agreed to join us for this interview for our blog, which focuses primarily on different aspects of app quality.
Please tell us about the background and history of TestFairy.
TestFairy was founded by myself and my co-founder, Gil Megidish back in 2013. We started the company after developing mobile apps in our previous company where we had some quality problems that we needed to solve. We had customers in the field with problems and we tried to solve those problems remotely, we had no idea what was happening over there, and that is how TestFairy began, by solving a real problem. Today TestFairy helps large companies manage their mobile development lifecycle. We do quite a few things across the development lifecycle. It starts with app distribution, which is a process of helping developers put their apps in the hands of users and help them with their beta testing. We can then record videos of exactly what your users are doing with your app. We have in-app feedback capabilities, allowing users to report bugs from the field, and in production we help customers with Remote Support and Remote Logging. So, in other words, with TestFairy you can have testers or real customers use your app, and you’ll be able to solve problems and have bugs reported through a very user-friendly easy interface. All this is just the beginning. We have recently joined Sauce Labs, which is the leading provider of quality software and digital confidence. Sauce has a very powerful service providing developers with real mobile devices in the cloud, that you can use for your testing, as well as emulators and simulators that you can use to test your apps automatically. TestFairy joined Sauce Labs to enrich the offer for mobile developers.
After getting acquired by Sauce Labs, you went from being a small independent company to being part of a much larger organization. What influenced your decision?
Right at the beginning of our talks Sauce Labs has inspired us with their vision. Sauce has acquired five companies over the last year, and you can only guess that more awesomeness is coming. The idea is to expand and become a one-stop-shop for all your digital quality needs. The world relies on code, and you need to make sure your apps, websites, and digital services are all properly functioning. Businesses need digital confidence. Your product needs to work 100 percent of the time. You can’t have your app at a B rating. Anything less than excellent is just not acceptable. What Sauce does is help you test your digital assets, end to end, in a very efficient way, allowing you to test your mobile apps on real devices, on emulators, and on simulators, as I previously mentioned. We have recently acquired API Fortress, which is now a very important part of the Sauce platform. It allows you to test the server APIsthe ones that are used to communicate with your apps, so that your testing is no longer limited to the front-end alone. Another company that joined Sauce is Backtrace, who is the leading crash reporter and error handling solution for mobile, web and gaming consoles. Many of the biggest gaming companies in the world use Backtrace. Crashes are not just about getting a stack trace, and fixing a bug. You need to see trends, and real-time information that can help you understand your users and where they are encountering difficulties. Taking that information and connecting it to your pre-release testing is super powerful and this is what companies are looking for. Imagine you are able to look at a crash and a minute later fix your tests to make sure that this crash will not happen again. All that can be done in real-time, not to mention AI and machine learning capabilities. Another service recently acquired by Sauce is AutonomIQ which is a low code environment that allows people who are not developers to build test automation for services that are used by their company, like Salesforce. The people who write those tests and build them don’t have to be engineers, they can be analysts, they can be product, they don’t need to learn how to code, they just need to know their systems. The last one I’ll mention, which was the first acquisition chronologically, much earlier on, is Sauce Visual. The Sauce visual testing platform allows you to test how your product looks across versions, across environments, platforms, and devices, so that you can look for visual problems during your regular testing.
Looking at Sauce’s impressive portfolio, you can understand why I mentioned that this was inspiring.
Are there any specific issues when it comes to mobile app quality or do you see mobile apps the same way as all other kinds of software?
Absolutely. I’ll start by saying that by definition, mobile is remote. If there's an issue with your app it is always happening somewhere else, not at your desk, or on your screen so you can’t open up a new window and inspect an element and see what is happening on your mobile device on Chrome. Issues happen on a mobile device somewhere in the hand of a person that in many cases you don’t know and cannot communicate with, and under these conditions you need to start solving a problem. That is the beginning. Second, the mobile environment is extremely fragmented. You have so many Android devices, and so many iOS devices, across hardware and software configurations, different levels of OS versions and locals, and this is before we get to the mobile device itself that needs reception, wifi, battery, and more. Mobile is just so much more complicated than web or desktop development. Whenever something goes wrong, the person reporting the bug has a lot of work. They need to take screenshots, they need to pull out logs from their devices, to explain what they did. These are some of the things we are able to solve at TestFairy that help mobile testing be done properly. Together with the things that Sauce Labs has built a powerful platform we manage to make developers' lives easier.
How do you see the relationship between Beta Testing and QA? Can good Beta Testing make companies save on their QA?
There are lots of stages in quality. First, we have to split the subject of QA into two parts, there is automation, and there is testing done by real people. The world of automation is very important for any company we speak to. Companies test their apps with many technologies including Espresso, XCUITest or Appium or on the Web with Selenium and other tools. I don’t think you’ll find companies that don’t do any automation. On the other hand, you have manual testing done by real people, which is not going anywhere, and this incorporates a number of stages. Some companies look at alpha, beta, delta testing and so on. Others will look at QA and dogfood. These are all different stages that are done by different stakeholders in the organization. The typical QA will be done by QA professionals, who are people that work inside the company whose profession is quality. They will test your app or your product manually and automatically. Their job is to find problems. Beta testing, or as we call it “dogfood”, some companies call it “catfood”, or even “monkeyfood”, there are lots of names for “dogfooding”. “Drink your own champagne”, is another term that is very popular. These are terms relating to companies with a process by which company employees use the company app in order to look for problems before the app is released to production. We work with many of the largest companies in the world and help them with their dogfood process. One example that I always like to mention is Groupon. Groupon has thousands of employees who use TestFairy to manage their worldwide internal beta testing. They actually call it “catfood”. Their internal process allows any Groupon employee in the world to report bugs directly to the mobile team in Chicago, in real-time. All they need to do is shake their phone, sketch on a screenshot, and hit the send button. That bug report goes directly to Jira, that is used by the R&D team and they get bugs in real-time. This allowed people to report bugs in a much more convenient way than before and it allows developers to see and fix bugs faster. It is very important to make bug reporting easy. We spoke to lots of companies and asked users if they had a case where they had a bug that they found during internal testing, but that was not reported, and then the same bug showed up in production. The answer was yes in many cases. When we asked them why they didn’t report the bug, the answer was either, it was too complicated to report, or they didn’t have time and they couldn’t be bothered with conference calls or follow ups, or they didn’t want to report the bug at the time since they were sure they knew about it. Then when we showed them TestFairy and for them it was just a life saver, because reporting a bug takes seconds. All you need to do is shake your phone and hit send, and the bug will appear on Jira.
What is special about beta testing is you can do it with people who are not technically savvy. You can do it with literally anyone, the people in finance, or the warehouse manager, operations department, whomever. They are not technically trained, you wouldn’t necessarily talk to them about your mobile app development, but they think like your customers, and they work in the company and want your company to succeed. If you will let them help you test your app, they will very likely be helpful in finding problems and be willing to report them in real-time.
What major trends do you see in app quality over the last 5 or 10 years?
There is a lot of shifting left. Developers do more. And we see a lot of quality going to production. Some of the trends I think we’ll see more and more of is that quality signals from Prod will go back to the development process and the automatic testing process that is done pre-release and will connect, automatically. We hear that from many of the Sauce customers. We see this in the questions we get from our largest clients who built their development process for the upcoming years ahead. I think that this will be a significant trend that we will see evolve.
What is the importance of logging in Beta Testing?
TestFairy can collect logs from remote devices. So if your app writes logs and you want to get logs and put them in your central logging server, TestFairy can help with that. Some of the biggest players in logging are Sumo Logic, Splunk, Coralogix, Log4J, Logz.io, and many others. If you want your logs from all your remote mobile apps to be sent to your logging server, you can do that with TestFairy.
So TestFairy is an intermediary between the mobile apps and the cross-platform logging service providers?
Correct.
Do you have a dream tool that hasn’t been developed yet?
Yes… but I can’t tell you about it. Because if I tell you, it won’t be a secret any more :)
True… true.. Thank you very much for your time, and for sharing your expertise with us, it was very eye opening. You gave us a different perspective on the ways in which companies, and especially larger companies, ensure their apps maintain high quality.
Shipbook gives you the power to remotely gather, search and analyze your user logs and exceptions in the cloud, on a per-user & session basis.