How and why we migrate from Detox e2e to Appium
Two years ago we started using detox tests for our app. Since then we have covered every screen in application at least by smoke test. We have fighting with reliability, when detox tests fails in some mystery cases. E.g. we change some style in one part of app and detox will fail in other part of app when try scroll.
So we started searching for alternative and decide to try Appium in combination of Browserstack.
As CI we use dedicated MAC machine in cloud (Mac mini 2018) on which we are running detox tests and common builds. We are using only android devices for testing for now, because we have about 90% of users on it. We are running tests as pre-merge check. Whole process from open pull-request to approve for merge takes approx. 40 minutes. (git checkout, yarn install, detox build, detox test).
We choose Appium, because can be easily run on cloud, which helps us speed up build process on our MAC. Then we simply added accessibilityLabel props next to testId props. A little challenge was to avoid position:absolute styling, where we didnt have to solve in detox, because we have workaround. (Problem in Appium is described here: http://www.wswebcreation.nl/clicking-on-an-element-with-appium-that-cant-be-found-in-the-ui-tree/). Next difference we found in scrolling, detox needs know element on which you want to scroll, appium not.
What attracted my attention is Appium Inspector (https://github.com/appium/appium-inspector). It’s great tool, which helps check elements tree and showing accessibilityLabels in views. Can help a lot during tunning of tests.
Currently we are running tests in our jenkins cloud, which create a build of apliccation, uploud it on Browserstack and starts tests. It takes about 11 minutes. Appium tests covers about 30% of application. I believe that we reach 20 minutes as maximum when we will have all tests migrated.
Update 01/2022: After migrating all tests whole process takes 29 minutes, which is satisfying from my point of view. I'd hoped in better time, but this is still improvement againt detox.
So when i summarize all in pros/cons, will get following:
Appium pros:
- easy of use with browserstack
- bigger comnunity around then in detox
Appium cons:
- browserstack price (200USD/month for yearly payments)
- looks more stable
Detox pros:
- focused on react native
Detox cons:
- unstability
- mystery failing
When you get here thank you for reading! :)
