Leap of faith with Kotlin/Native
This is a 6-month story of the migration of the ScratchMyMap.com mobile iOS & Android applications from the native code-base to Kotlin/Native. All you wanted to know about Kotlin/Native before introducing it in your product.
ScratchMyMap.com (SMM)
ScratchMyMap.com — is a social network for travelers created by my brother and me back in 2017. From the beginning, it was a pet project for us where we tried to acquire new skills, experiment with different technologies while actually delivering a working product to customers.
I have a strong belief that all engineer needs to have a pet project. That is the easiest way to remind yourself that engineering is fun.
Products
In the mobile market, we have 2 applications (2 per platform):
“World Quiz” — geography quiz, is not a part of migration therefore will not be discussed here.
Maintaining 4 applications, considering that I’m the only mobile engineer, was a real hustle for a long time.
Backend
On top of that, since 2017 our backend has been rewritten 3 times. We started with a simple PHP BE that supported only web-page, till the time the first mobile app was ready (android) we had to create separate endpoints for it — V2. At that point, we were far from feature parity on mobile and web. And while I was focusing on the second app — iOS, Volodymyr already started to move to BE to Node.js, creating a 3rd version of the API. The end goal with a new API was to unify BE and FE as much as possible to decrease support overhead. It took a while to release both apps with the old backend and fix all issues. In parallel web already migrated to the new API — V3. As soon as it was tested I started to move mobile apps to it as well.
At that point, I started to investigate what cross-platform solutions are available…
Why Kotlin/Native? (K/N)
Duplication of the code on both platforms was obvious and support was painful. Considering the relatively simple architecture of mobile solutions, pretty much any existing cross-platform solution would fulfill our needs. But I wanted to leverage from the existing code base and keep a user-friendly native UI. Kotlin/Native fits here perfectly.
Flutter
I consider Flutter as well, but after some tests and evaluations, it was obvious that it will take more time to migrate to Flutter. Considering that both apps were already released, and with Flutter, there was little to reuse.
I decided to migrate WorldQuiz applications to Flutter to compare complexity [WIP].
State of things
Before the migration, both applications followed simple MVP architecture. Networking was done via Alamofire and Retrofit. Data was saved in the Realm DB on both platforms. Whoever worked with Realm knows that it doesn’t require multithreading, actually, it doesn’t work well with multithreading. So both applications were relatively simple — load data from the server, save in DB, and propagate to the UI.
Let’s start!
There are a lot of tutorials you can follow to set up a project (i.e.KaMPKit), so I’ll try to focus on some edge cases or interesting facts.
Project setup
Keeping in mind that I want to reuse as much as source code possible, I thought about what is the best way to set up the project. Official examples of the K/N project put shared libraries as well as the iOS project inside the android project (see Calculator), same with other examples like RockDev.
As I still wanted to support the existing version while working on the new one and merge them when required, this approach didn’t really work for me. So I decided to create a new repo, add the existing Android and iOS projects as submodules, and put the shared module inside.
With this approach, original projects were not modified and I used ‘master’ for iterative releases while working on the K/N version. One more benefit from this approach is that I can pretty easily abandon K/N and move back to platform native development.
Gradle
The worst part of my experience with K/N was the Gradle setup. I created an empty K/N project, attached it to Android as a module and as a framework to the iOS project — following official guidelines. It went easy and smooth, I was able to print logs from the K/N project in both Android and iOS.
Since I was convinced that everything works fine, for the K/N development after that I used mostly Android — it was much easier because I could use one IDE and did not need to rebuild the framework to see changes.
At the point when I added almost all dependencies: DI, coroutines, DB, networking and created a few of the view-models and use-cases, only then I decided to test how this all works with iOS. Long story short — I spent around 1 week fixing the build for iOS. Some problems were related to the specific dependencies in Gradle, some were compilation exceptions with unreadable stack: i.e. default parameter in expected class works fine on android but throws compilation error for iOS.
To be fair, I set up the project on 1.3.72, but as soon as 1.4 was released and I migrated-I didn’t face any issues.
Lesson learned: run `packForXcode` after you add a new dependency.
Pack for Xcode
There are two ways to add a K/N library to an iOS project: via a framework or a pod. I tried both ways and should say that at the point when I was integrating, a framework was much more stable. Now, it seems that JetBrains invested a lot of effort to improve pods. One pod benefit is that you can integrate other pods inside your K/N library and use them from Kotlin. I’ll definitely try to migrate to pods this year.
After you integrated ‘packForXcode’ inside of the run phase script in the Xcode you don’t need to call it manually, video above is just for reference.
XCode 11.4 +
Xcode 11.4+ checks whether embedded frameworks are built for a correct platform before the run phase scripts. While switching between iOS devices/simulators this check will cause an error when an existing framework was created for the wrong architecture. In Xcode < 11.4, the `packForXcode` Gradle task replaces the framework with a correct one during the run phase.
There are few possible solutions, most simple — remove the framework from the build folder before switching between devices and simulator, otherwise, you can follow suggestions here, i.e. use different folders for different architectures.
Dependency injection
Years and years of working with Dagger already created a stereotype in my mind that DI can not be easy. I was prepared for the worst, but… surprisingly, I found that there were two nice DI solutions K/N — Kodein-DI & Koin. I went with Kodein-DI — it was supporting K/N longer and seems to be more stable.
DI is separated into modules, in each module, you declare dependencies with different scope: instance, provider. In the end, all modules provided to ‘DIContainer’ and it is wrapped in a ‘DIAware’ component. Looks like this:
Context
One complication here is that Android, unlike iOS, requires `context` for some dependencies like DB. So how to provide Context when it is platform-specific and DI is inside of the X library? Interesting fact, but you can overload the constructor of the expected class. Same as you provide a context in the Dagger module, provide your `platform specific` struct with dependencies to DI-aware components.
Worth mentioning, that in K/N platform-specific classes (expected and actual) are located in the corresponding source directory, by the same package name
On the client side, usage is pretty simple:
Oh no! It is not freezing ❄️
The biggest disadvantage of Kotlin/Native is its memory handling model. You can learn more about it here. While on Android it is native, on iOS object graphs must be frozen to be used in multiple threads. What does it mean for the end-user? That you cannot use multithreading with mutable objects. Is there a way to overcome this? Yes… you can use a special version of coroutines: coroutines native mt (native multithreading coroutines). Basically, they provide you the ability to do concurrent tasks on the main thread, so you don’t violate memory model restrictions.
Obviously, the first question here is what about performance? I’ll be honest with you, I created a few demo projects to test it and have not seen a significant decrease in speed. I also asked a few of my friends to take a look and they admitted that differences could be neglected 🚀.
What is happening here: on the login we create accounts, load a user profile, visits, save world countries' geographical data to DB, etc. When navigating from top travelers to any person, on click we load profile, after that — opening users profile screen and load visits list. When we go to the `stars` tab we load random people who have traveled to this country. Everything, except popular travelers, is saved in DB and populated to the UI via flow.
But what if I need a real thread? ⭐️
There could be cases when you need real multithreading and if you can guarantee immutability or don’t need to share objects between threads you can use a 3rd party solution from Autodesk. It will even work with SQLDelight, but not Ktor.
If you want to use it with Ktor, you can create two coroutine scopes — one for coroutines on the main thread and one for multithreading.
Binary size ⒩
I had some concerns about the iOS binary since K/N does a lot of code generation. But to my surprise, iOS binary becomes even smaller, data from the AppStoreConnect metadata:
K/N iOS application is ~ 3.5 MB smaller than the native one (root cause is probably Realm).
I don’t think it makes much sense to compare binary size for Android before and after, both are native implementations, anyway: before: 11.1 MB; after: 11.4 MB. This is the size of the release, signed APK files.
Line of code 📖
I checked the count of lines of code in all 5 projects: iOS & Android old; K/N powered iOS and Android; K/N shared library and SQL code (for SQLDelight):
Comparison with Swift is not really representative, since 90% of UI in iOS is done in code, while on Android I still majorly rely on the ‘xml’. IMHO, Kotlin is more compact than Swift, just Android SDK enforces a lot of boilerplate code.
The most interesting part, that K/N code with both app total into 24779 lines, while old implementation of iOS & Android, together is 24065 🤷♂️
Reactive programming 👩🔬
I think everybody would agree that modern apps need to be reactive. And first, that came to my mind was to check for RxJava & RxSwift implementation, and I found — Reaktive. I tried it on the demo project, looks fine, even has a way to avoid object freezing (see `isThreadLocal = true’), but… Ktor client requires coroutines. Reaktive has some extensions to move back-and-forth with coroutines, but it is messy. So I decided to use flow. It is built on top of coroutines and provides almost the same functionality as RXJava / RXSwift libraries. As of now, they are still `experimental` but I didn’t face any issues so far, only from time to time API and syntax updates.
I would say, that flows in general has simplex API, and less error prompt compare to Rx*. As well, code is more compact and looks cleaner.
DB first 📚
In the world of reactive apps where most of the data is populated from the BE, one of the high complexity tasks is to ensure data integrity and speed to UI.
So what does “DB first” mean? DB is a single source of truth and all updates on the UI are directly propagated from the DB. I mixed it with an optimistic UI approach, which means that clients, by default, assume that all interactions with BE are successful and update data on the client locally without waiting for BE response, eventually showing only error response.
SQLDelight has something critically important to make the “DB first” approach work on K/N projects — coroutines and flow extension. After you select the `flow` of any data, all updates are getting automatically propagated. So from a VM perspective, all you need is to update DB with new data. Also, flow support on the SQLDelight makes it much easier than observers in Realm change listeners.
Comparing to the Room or Realm — I feel that SQLDelight has some obvious benefits — since it auto-generates methods from raw SQL code, you don’t need to learn any new API.
BTW, Room also supports flow, but unfortunately, it doesn’t work with K/N.
Wrap your flow 🎁
By default flows are not supported in the K/N, which means that you cannot bridge them to iOS, therefore on the Kotlin conference, JetBrains presented a life-hack on how to “fix” it. Eventually, all you can do on the client side is just `watch` a flow, you cannot use any flow operators. This is a limitation, but in my case, since all business logic is in the ViewModel — it didn’t really care.
The only difference between iOS and Android, that for Android, the base view-model extends from the `androidx.lifecycle.ViewModel`, which means that you can use `ViewModel.viewModelScope` for coroutines and don’t care about lifecycle. For iOS, you need to create a view model scope to execute coroutine inside, and don’t forget to cancel jobs when you leave the screen. This is a bit complicated because iOS doesn’t have a `viewDidLoad` counterpart like Android. So I changed the `wrap` method to accept view model scope and expose closable delegates. First resolve issue for Android, second for iOS. Now, in case I need to dispose of flow in `viewWillDisappear` I bind it with `WatchClosable`:
Note that ‘lunchIn(…)’ underneath just call ‘collect()’ , so be careful with back pressure. Preferably modify this to ‘collectLatest(…)’
Cast it all 🔀
The Type system really looks very complicated. If you navigate to any K/N class, you’ll see something like this (automatically generated code):
Here is how the same class looks in the Kotlin/Native code:
sealed class Result<out Success, out Failure>
class Success<out Success>(val value: Success) : Result<Success, Nothing>()
class Failure<out Failure>(val reason: Failure) : Result<Nothing, Failure>()
That is why Xcode doesn’t understand that `Success extends Result` but you still can cast it and successfully use it.
For all arrays and list return type from K/N is always NSArray, so you need to case it as well:
Primitive types can be used as is only if they are not nullable, otherwise, they will be wrapped in the K/N implementation, i.e. KotlinLong, KotlinBoolean; see this for reference. I do not recommend using optional primitive types as a return or params for K/N methods. Save yourself some time and replace it with invalid outputs if possible, i.e. change optional Long default to `-1`.
Date and Time ⏰
Kotlinx-datetime 1.0 was released just a few months ago. Before that, I used `expected` platform-specific implementation for date-time, but it didn’t go well. Mostly because for iOS the only API exposed in K/N is `NSDate().timeIntervalSinceReferenceDate` which is a date referenced from 2001, not 1970.
import platform.Foundation.NSDate
NSDate().timeIntervalSinceReferenceDateNow I’m using K/N date-time for all date-number processing and platform implementation for string formatting.
Testing 💥
Considering that many companies have strict requirements for UT coverage, I’ll share some insights about how it works on K/N platforms (see example). In general, it is not that straightforward because some components can depend on different platforms to inject them — you need to create three folders for testing: common, android, and iOS — same as for the main source code, just change ‘main’ to ‘test’. Another complication is that for DB testing, on Android, you need context, which means you need instrumentation tests. To achieve this, you need to create an expected base test (InstrumentationBaseTest), and for Android annotate it with a runner (@RunWith(AndroidJUnit4::class)).
Last but not least, instrumentation and unit tests, in this case, are in the same module, so you need to have different run configurations to execute them. This is easy to do from Android Studio, otherwise, you’ll need to create some custom Gradle scripts.
By the way, you can run tests for both iOS and Android from AS. But if you are migrating an old project — take a look at this, to make it work with 1.4 need a small change:
Security
No doubt that if you go for a production application — you need encryption. The most obvious use-case, which also applies to SMM, is an encryption of the token. Unfortunately, there is no default encryption in the K/N. I found one good open-source solution — Krypto. There is no user guide so it takes some time to figure out API, but it is definitely working:
Suppress(“EXPERIMENTAL_API_USAGE”)
Both K/N & flow are still experimentation features, despite that, I didn’t face any crashes or inconsistency in the work of API. After the Android and following iOS release, there were a few thousands of user-sessions in total, and the crash-free rate is still high. Maybe it is too early to talk about iOS because it is just released, but so far it is good.
IDE 👩💻
I’m not here to advocate for some specific IDE, but all know that Xcode is the worst (at least that few lucky iOS engineers who had a chance to work with any IntelliJ IDEA products know what I’m talking about).
For Android it is all straightforward — use Android studio for application as well as for K/N part. K/N will be shown as a library module in a file tree. Autocomplete and navigation work fine, obviously, it is not perfect — AS often getting confused and cannot open required files, sometimes cannot resolve some K/N libraries dependencies and need to restart IDE or do other magic but the build will work.
For iOS, it is the situation is worse. You cannot preview the implementation of K/N classes, only auto-generated headers which barely readable, so you still need Android Studio.
Android Studio for iOS 🍏
JetBrain provides a kotlin multi-platform mobile plugin for the AS. It allows you to create a K/N project with all required Gradle tasks, project structure, add an iOS project. The top feature is that it allows you to build and run iOS apps from the AS.
If you are adding this to the existing project, add the path to the iOS app in the ‘gradle.properties’:
xcodeproj=../../scratchmymapios/
Despite I find it interesting, I don’t use this for one simple reason — I cannot modify Swift code from AS, besides that it is cool flex.
VS Code 🆚
I tried VSCode, installed Kotlin and Swift language server (both don’t work well), created a multi workspace environment with the iOS project, K/N, and Android. Eventually, some auto-completion features are working for all 3 projects. Despite that, it cannot replace XCode or AS. i.e. it doesn’t navigate into pods or frameworks, autocomplete seems to show all possible options in the world except the one you need, and doesn’t create proper stubs when overriding methods.
Also, to make Swift work with VSCode, you need to place the `Package.swift` file in the root. After that Xcode will start to show your project as Swift project not iOS in the recent screen, even if you previously open it from xcworkspace. Now each time I need to open a project via file picker, not a big deal, but still annoying.
I continue using VSCode for the VCS across 3 projects.
Kotlin vs Swift 🏃♂️
The great benefit of K/N, that Kotlin and Swift languages have very similar syntax. But unlike Swift, which added support for armv7, x86_64, and aarch64 but did very little after that, JetBrains definitely committed to drive this technology forward and improve it.
At the last KotlinConf they declared that in 2021 they will change the memory model and you’ll be able to share objects between threads seamlessly!
As well, since Koltin added support for the BE, I also see a stronger use case to use it as cross-platform technology. I never tried personally to spinoff BE with Kotlin and can imagine that GO is still better, but who knows… maybe it will replace Java one day?
Trends also look good, considering that now Kotlin is a prioritized language for Android it will definitely grow.
If you interested to learn more about Swift for Android, take a look at this. Honestly, I don’t think complexity is justified as well as binary size (30+ MB), because it needs to include Swift runtime for Android.
Timelines
Migration of two projects in total took me ~6 months, considering that I was working a few hours on weekends and randomly for a few hours on weekdays. It is hard to estimate project complexity, but in terms of components: up to 20 screens, up to 20 tables in DB, a little of business logic.
Join testing
To see it in action you can download apps from respective stores:
To learn more about ScratchMyMap, please visit our web-page.
Other experiments
Besides Kotlin/Native, I did some experiments with Rust and GO, can read more by links below:
