Android instrumentation tests
Test the Android app on a real Android runtime, on emulators for every supported API level.
Unit tests run on your JVM. Users run the app on Android, which has its own runtime and supports only part of the Java language features and APIs on older versions. Instrumentation tests close that gap: they run on an emulator, on the same runtime users have.
Write unit tests first. Reach for an instrumentation test when the behavior depends on the real Android system.
Some bugs only show at runtime on a specific Android version. The Jackson library is a real example: from version 2.14 it needs API 26, and nothing fails until the app runs on an older device. Only an instrumentation test, or a crash report from a user, catches that. See this pull request for how it played out.
- Screens, with Espresso and Compose UI tests, checking how a screen behaves when the user interacts with it.
- Code without UI, like foreground services, where you check the data and the interaction with the system.
Most of these tests should not depend on a Home Assistant server. Replace the server with fakes, or with mocks from MockK. Fakes are the better choice: they give you more control over the behavior. Testing against a real server is what the end-to-end tests are for.
On every pull request, the instrumentation tests of :app run on Emulator.wtf, against every API level from androidSdk-min to androidSdk-target. The whole run takes seconds. See continuous integration for how that works with forks.
The tests of :wear, :automotive, :common, and :microwakeword run on the regular Android emulator on GitHub Actions, because Emulator.wtf doesn't offer those targets. That is slower, so they cover fewer API levels.
Android emulators sometimes fail for no reason at all. The fix is to run the job again. Only maintainers can re-run a job, so ask in the pull request if you think it was a fluke.