Meta has embarked on a comprehensive initiative to convert its entire Android codebase from Java to Kotlin. This complex transformation, which spans several million lines of code, aims to maximize developer productivity and enhance null safety. The transition, prioritized after developers expressed a preference for Kotlin, started in earnest around 2020 and remains ongoing, with over half the codebase already converted. Meta’s dedication to Kotlin goes beyond merely writing new code in the language. They opted for a complete overhaul, converting existing Java code, a choice driven by the necessity of reaping the full benefits of Kotlin. This ambitious decision signified the requirement of building infrastructure to automate the translation process. Consequently, Meta’s engineers embarked on converting approximately ten million lines of Java code to Kotlin, necessitating the resolution of various issues, including slow build speeds and inadequate linters, to support the process.
The Objective: Maximizing Productivity and Null Safety
The primary objective of this translation effort is to convert almost all actively developed code and code central to the dependency graph to Kotlin. By focusing on actively developed code, Meta ensures significant productivity gains. Converting central code in the dependency graph further aims to mitigate the potential chaos caused by nullability issues present in mixed codebases. This approach also aims to eliminate the drawbacks associated with supporting parallel toolchains and the notably slower build speeds when compiling Kotlin and Java together.
Meta initially approached the transformation by using IntelliJ IDE’s translation tool, J2K, for incremental migration. However, given the enormity of their codebase, this manual method proved unfeasible. Hence, they developed an in-house automated tool called the Kotlinator, built around J2K to manage the translation at scale effectively. This strategic decision allowed Meta to streamline the transition process, paving the way for more substantial productivity and safety improvements across their Android development efforts.
The Kotlinator: Automating the Translation Process
Kotlinator’s conversion process includes six key phases designed to ensure a smooth and systematic transition from Java to Kotlin. The first phase, Deep Build, ensures all code symbols are resolved, particularly when dealing with third-party dependencies or generated code. Preprocessing, the second phase, addresses various issues such as nullability and support for the custom Dependency Injection (DI) framework using a custom tool called Editus, which involves around 50 steps.
The Headless J2K phase utilizes a server-friendly version of J2K, decoupled from the IntelliJ IDE, enabling remote code translation capabilities. This setup allows for efficient code conversion without impeding developers’ local work. The fourth phase, Postprocessing, includes about 150 steps focused on making the converted Kotlin code idiomatic and adjusting Android-specific changes. Integrated linters during the fifth phase ensure perennial issues are systematically rectified with autofixes, benefiting both conversion diffs and regular diffs. The final phase, Build Error-Based Fixes, parses error logs to address build errors by applying necessary adjustments such as adding missing imports or fine-tuning Kotlin syntax.
Custom Tools and Phases for a Smooth Transition
Meta’s approach to the Kotlin transition involved more than just straightforward code translation; it required implementing custom pre- and post-conversion steps due to the extensive nature of their custom framework-utilizing codebase. These steps, numbering over 200, were facilitated by an internal metaprogramming tool leveraging JetBrains’ PSI libraries for both Java and Kotlin. Unlike compiler plugins, this tool analyzes broken code swiftly, which is crucial for handling unbuildable code across numerous files.
These custom steps, developed to enhance the speed and flexibility of the conversion process, included examining Kotlin implementers to translate interface getters into overridden properties efficiently. Though the tool excels at this task, it occasionally faces challenges in resolving type information, particularly from third-party libraries. In such cases, developers must step in with manual fixes. Over time, these custom phases have significantly reduced developer effort while improving reliability by automating delicate transformations. An example of such automation is the condensation of chains of null checks, which helps prevent developers from unintentionally dropping negations.
Ensuring Null Safety and Handling Java’s NPEs
A critical component of Meta’s transition from Java to Kotlin is ensuring null safety, as converting to Kotlin inherently introduces strict runtime validation at the interlanguage boundary. Null safety checks in Kotlin bytecode ensure reliable behavior by invoking checkNotNull automatically, which simplifies the development process by making nullability guarantees transparent across code modifications. However, achieving null safety is complex, as the process involves managing Java’s inherent potential for null-pointer exceptions (NPEs).
Even null-safe Java can introduce NPEs, especially if non-null-safe dependents pass null arguments wrongly annotated as non-nullable. This risk is amplified at central dependency nodes or when dealing with numerous transitive dependents that lack null safeguards. To mitigate these risks, developers exercise caution during translation, prioritizing nullable types based on context clues to prevent unwittingly introducing NPEs. Complementary codemods ensure accuracy by adding appropriate nullability annotations across the codebase. This effort is further supported by a Java compiler plugin that collects runtime nullability data, refining annotations accurately and ensuring consistent null safety throughout the codebase.
Collaborative Efforts and Continuous Refinements
Despite Meta’s best efforts, transitioning their extensive codebase from Java to Kotlin brought challenges that exceeded the capabilities of existing tools. Initially, J2K was approached as a black box; however, the complexities of Meta’s code required direct collaboration with JetBrains. This partnership led to significant enhancements in J2K’s functionality, tailored to meet Meta’s unique needs. This collaboration yielded hooks in the Integrated Development Environment (IDE) that allowed for custom steps, improving symbol resolution and potentially open-sourcing Android-specific steps for community use.
Ensuring null safety has been a top priority throughout the transition. Converting Java to Kotlin introduces stringent runtime validations at the interfaces, mandating careful handling of Java’s susceptibility to null-pointer exceptions (NPEs). To mitigate this, the development team incorporated extensive null safety checks into Kotlin bytecode, automatically invoking checkNotNull for reliable behavior. This change has simplified development by making nullability guarantees clear during code updates.
However, building a null-safe codebase isn’t without difficulties. Even null-safe Java code can introduce NPEs, especially when non-null-safe dependencies incorrectly pass null arguments marked as non-nullable. This risk increases at central dependency nodes and with multiple transitive dependencies that lack null safeguards. Developers manage these risks by focusing on nullable types contextually and applying codemods for accurate nullability annotations. An additional Java compiler plugin gathers runtime nullability data to refine these annotations, ensuring translation accuracy.
Meta’s task of converting millions of Java lines to Kotlin is a complex process balancing automation, risk management, and precision. With more than half of the conversion completed, ongoing refinements and collaborations aim to enhance their Android codebase further. This transition exemplifies the technical and strategic challenges of evolving a large codebase and highlights a strong commitment to future-proofing development practices within the ever-evolving Android ecosystem.
