The Android Open Source Project (AOSP) is a massive undertaking, comprising millions of lines of code across various languages. Building such a complex system efficiently and reliably is a monumental task. For years, the Android build relied heavily on GNU Make, but with the increasing complexity and scale of the platform, a more modern, robust, and performant solution was needed. This led to the introduction of the Android Soong build system, a significant leap forward in how Android is compiled.

From Make to Soong: The Evolution of Android Builds

Prior to Android 7.0 (Nougat), the build process was primarily orchestrated by Android.mk files, which leveraged GNU Make. While Make was a staple for many years, its limitations became apparent as Android grew. Makefiles can become intricate and difficult to maintain, especially with conditional logic and complex dependencies. Incremental builds, crucial for developer productivity, were often unreliable, frequently necessitating a full rebuild to ensure correctness.

Soong was introduced to address these challenges, aiming to provide a faster, more declarative, and maintainable build experience. It doesn't entirely replace all existing tools but rather integrates with them to create a streamlined workflow. At its core, Soong leverages the Kati GNU Make clone tool (for legacy Android.mk files during the transition) and the highly efficient Ninja build system as its backend.

The Heart of Soong: Android.bp Files

The most visible change with Soong is the shift from Android.mk files to Android.bp files. These Android.bp files are declarative descriptions of "modules" to be built, similar in syntax and semantics to Bazel BUILD files. This declarative nature is a key advantage, making build configurations more readable and less prone to errors compared to the imperative style of Makefiles.

Here's what defines an Android.bp file:

Modules: The basic unit of building in Soong is a module. Each module starts with a module type (e.g., cc_binary, java_library, android_app) followed by a set of properties in a name: value format. Every module must have a unique name property.
Declarative Syntax: Android.bp files focus on what needs to be built and its dependencies, rather than how it should be built. The build logic itself is implemented in Go within Soong's internal components.
No Explicit If-Statements (mostly): Unlike Android.mk files that allowed complex conditional logic using ifeq, Android.bp files primarily rely on pre-defined distinctions like processor architecture or debug/release builds. Custom case distinctions are handled by the Go-based Soong modules themselves, promoting a cleaner and more structured build graph.

Globs and Variables: Android.bp supports glob patterns for file lists (e.g., src/**/*.java) and allows for top-level variable assignments, enhancing flexibility and reducing redundancy.
Comments: Both C-style /* */ and C++ style // comments are supported for better readability.

How Soong Works Under the Hood

The Soong build system operates in a multi-stage process:

Parsing Android.bp Files: Soong parses all Android.bp files across the AOSP tree. This involves reading the declarative module definitions and their properties.
Generating Ninja Build Files: Based on the parsed Android.bp files and the internal build logic (written in Go), Soong generates a highly optimized build.ninja file. Ninja is designed for speed and efficiency, particularly for incremental builds, by creating a precise dependency graph and executing build commands in parallel.
Kati for Legacy Android.mk: During the transition period, Kati acts as an intermediary, processing Android.mk files and generating additional .ninja files that integrate with the overall Ninja build graph. This allows for a gradual migration without requiring a complete rewrite of the entire AOSP at once.
Ninja Execution: Finally, the Ninja build system takes the generated .ninja files and executes the actual build commands, compiling source code, linking libraries, and creating the final Android images and artifacts.
soong_ui as the Orchestrator: The soong_ui script acts as the primary driver for the entire build process, orchestrating the different stages and managing auxiliary tools.

Advantages of the Soong Build System

The shift to Soong brings several significant advantages to Android platform development:

Improved Build Performance: By leveraging Ninja, Soong significantly speeds up build times, especially for incremental builds. This is crucial for developers who frequently make small changes and need quick feedback.
Enhanced Reliability of Incremental Builds: Soong's precise dependency tracking and Ninja's efficient execution minimize the need for full rebuilds, making incremental builds more reliable and consistent.
Declarative and Readable Configuration: Android.bp files offer a cleaner, more understandable syntax compared to complex Makefiles, reducing the learning curve for new developers and improving maintainability.
Stronger Type Checking: Soong's Go-based implementation allows for stronger type checking of variables and properties, catching errors earlier in the development cycle.
Better Scalability: The modular and declarative nature of Soong, combined with Ninja's parallelism, makes the build system more scalable to the ever-growing size and complexity of Android.
Pathway to Bazel: The Android.bp file format is intentionally similar to Bazel's BUILD files, indicating a potential future direction for the Android build system to fully embrace Bazel, a highly scalable and reliable build system used by Google.

Challenges and the Road Ahead

While Soong offers substantial improvements, the transition from Make has not been without its challenges. The migration of a massive codebase from one build system to another is a complex undertaking. Developers have had to adapt to the new Android.bp syntax and understand the underlying principles of Soong.

Furthermore, dependencies between legacy Android.mk modules and new Android.bp modules can sometimes be intricate. While Android.mk modules can depend on Android.bp modules, the reverse is not always straightforward.

Looking ahead, Google is actively exploring the integration of Bazel as the ultimate build system for AOSP. Soong serves as a crucial stepping stone in this evolution, providing a modern foundation that is more compatible with a Bazel-like philosophy.

Conclusion

The Android Soong build system represents a pivotal advancement in Android platform development. By moving away from the limitations of Make and embracing a more declarative, performant, and scalable approach with Android.bp files and Ninja, Soong has significantly improved the efficiency and reliability of building the world's most popular mobile operating system. As Android continues to grow and evolve, Soong lays the groundwork for even more sophisticated build capabilities, ultimately contributing to a faster and more seamless development experience for the entire Android ecosystem.

Revolutionizing the Android Build: An In-Depth Look at the Soong Build System