Technology

When it comes to most general purpose operating systems, including Windows, Mac OS X and many Linux desktop distribution, an end user can wipe a device and reinstall that operating system from scratch. So long as the hardware is supported, or has available device drivers, the machine can work with a stock version of the operating system. When it comes to embedded systems, many use firmware, a combination of an operating system and applications, typically stored on read-only storage, and tailored specifically for a device; hardwired for a limited set of functionality.

When Google originally purchased Android Inc in 2005, their development and releases of Android for cellphones was treated more like firmware than a general purpose operating system. As Android has grown, manufactures use the Android Open Source Project (AOSP) as a base, modifying it for each of their individual headsets. The result is that users are now dependent on operating system updates from each manufacturer, leaving many devices with obsolete versions of software or worse, major unpatched security vulnerabilities. This is what’s known as Android fragmentation.

On an Android device, the standard partition layout consists of the following six partitions: boot, system, recovery, data, cache, misc. The following is a partition layout from an Android device:

## ls -al /dev/block/platform/msm_sdcc.1/by-name
... DDR -> /dev/block/mmcblk0p17
... FOTAKernel -> /dev/block/mmcblk0p16
... LTALabel -> /dev/block/mmcblk0p18
... TA -> /dev/block/mmcblk0p1
... aboot -> /dev/block/mmcblk0p5
... alt_aboot -> /dev/block/mmcblk0p11
... alt_dbi -> /dev/block/mmcblk0p10
... alt_rpm -> /dev/block/mmcblk0p12
... alt_s1sbl -> /dev/block/mmcblk0p9
... alt_sbl1 -> /dev/block/mmcblk0p8
... alt_tz -> /dev/block/mmcblk0p13
... apps_log -> /dev/block/mmcblk0p22
... boot -> /dev/block/mmcblk0p14
... cache -> /dev/block/mmcblk0p24
... dbi -> /dev/block/mmcblk0p4
... fsg -> /dev/block/mmcblk0p21
... modemst1 -> /dev/block/mmcblk0p19
... modemst2 -> /dev/block/mmcblk0p20
... ramdump -> /dev/block/mmcblk0p15
... rpm -> /dev/block/mmcblk0p6
... s1sbl -> /dev/block/mmcblk0p3
... sbl1 -> /dev/block/mmcblk0p2
... system -> /dev/block/mmcblk0p23
... tz -> /dev/block/mmcblk0p7
... userdata -> /dev/block/mmcblk0p25

As seen above, the actual partition layout can vary greatly from device to device. There are also additional mount points for internal/external sdcards, but for the sake of simplicity, we’re only going to focus on the system, data and recovery partitions.

The system partition is what contains the core of the Android operating system. All the essential libraries, base user interface and non-removable system apps reside on this partition. In the Linux world, this is the root or / partition. Unlike desktop or server Linux systems, this partition is typically read-only in standard Android operation mode. There are other Linux systems with read only root partitions. Many home Internet routers and satellite navigation systems use Linux as their base and have similar partition layouts with non-writable root file systems.

Package Management

On a standard Linux distribution such as Ubuntu, Fedora, Arch or Gentoo, most of the program files on the root partition are maintained by a package manager. The package manager communicates with central repositories in order to install services, programs and security updates. Shared libraries and subsystems are listed as dependencies. When using a package manager to install a package, typically the dependencies will be installed automatically and can be shared between different packages.

An alternative type of package is a monolithic one, where all the dependencies are included in the package itself. Examples include Ubuntu’s Snappy, the standard .app container in Mac OS X, Java web applications (war/ear files) and Android’s apk packages. Including all necessary dependencies can reduce breakage due to incompatibilities and ensure better support. The trade-offs include wasting space by duplicating dependencies and making each package responsible for security updates of embedded libraries.

There’s another class of package known as an installer. Windows users are most familiar with installers which typically update any shared dependencies that are out of date. Some Mac OS X programs use installers as well; typically those that need to install shared libraries, services or require administrator access to the system.

App Stores

Apple’s app store, released originally with their iPhone product, is basically a crippled version of what had already existed in Linux systems for over a decade. It’s a manager that connects to Apple’s repository that contains monolithic packages. Although the repositories for most Linux distributions have a process to get new packages added, most Linux package managers allow for adding 3rd party repositories.

What that means is if I’m a developer or maintainer for a Linux program and I can’t get my package into Redhat or Ubuntu’s package repository, I can simply create my own repository. Users can be given instructions for adding this repository along with its digital signing key. Even if a project has its packages in official repositories, the project owners may decide to maintain their own repository as well for users who want more up to date versions of their software or bleeding edge beta releases.

Google Play and Microsoft’s app store are similar to Apple’s. They use a single repository, controlled by their respective companies, to distribute packages that go through an approval process. Although Linux distributions may have an approval process for official packages, it’s typically handled using issue trackers and is considerably more open than Apple or Google’s.

The iOS/Android Base

Anroid packages (apk files) are (mostly) monolithic. Although some packages exist on the system partition (packages distributed with the phone by the manufacturer that cannot be uninstalled without root access), most exist on the user’s data partition. However the system partition itself is not managed by any packages.

When an Android update is available, it is downloaded to the data partition and then the phone reboots using the recovery partition. The recovery partition contains a minimal operating system that finds the update, verifies its signature, mounts the system partition as writable and applies that update. This can be a full update for the entire system, but typically manufactures release incremental updates that only contain the files that need to be updated and pre/post install scripts. If you have a rooted device and have changed things on the system partition, these updates may fail or undo customizations such as rooting the device.

The entire system partition can be thought of as a single monolithic package. The update and reboot process is similar to the way Windows updates work (although Windows can write to C:\Windows, it usually defers writes until after a reboot so avoid conflicts with locked files). Unlike Windows, the base of every Android device may be very different. Google simply can’t release updates every second Tuesday like Microsoft. Each manufacture must rebase their work on the releases made by Google and then send out updates for their devices. This allows security exploits to be left out on some devices for weeks or months, if they’re even patched at all.

Although the update process is similar for iOS products, because Apple has such tight control between their hardware and software, they control the complete release system for all their devices. There are no customizations for individual manufactures. There are only a handful of devices they need to support and they are all manufactured by Apple. The trade-off for potentially faster and more comprehensive security updates is lack of device choice and customization.

A General Purpose Base

Last year Microsoft decided to bundle Candy Crush with their Windows 10 operating system¹, the first of a plethora of promoted apps², lock screen ads³ and other bloatware onto their base operating system. Despite the many complaints over the past two decades about Microsoft, at least their base install of Windows use to be fairly bloatware free.

A base Android install can also be bloatware free, but these stock versions are only officially made for Google’s line of Nexus devices as well as some limited Google Play versions of phones. There are many 3rd party mods for Android devices such as CyanogenMod or MIUI which, although they do reduce the overall bloat of most carrier’s Android installs, must be individually ported to each Android device.

So what prevents Android from being more like Windows? Why can’t a user simply install AOSP onto his or her device? One of the major issues in porting involves drivers, and this is another problem where standard packages could come to the rescue. In the Linux world, there are things know as source packages, such as debs on Debain based systems or an SRPM on rpm based systems. Google could create a custom yet standardized format for packaging drivers, as well as a tool chain that could be part of the Android SDK for building those drivers for a specific device and kernel.

Hardware manufactures don’t even have to include the actual source code for their drivers. Nvidia⁴ and ATI both have proprietary video drivers for Linux that contain a layer that links Linux kernel functions with their closed binary blobs. The linux-firmware project is another example where manufactures provide binary blobs of closed source firmware that can be loaded by the Linux operating system as needed for specific devices.

I realize I’m oversimplifying things a bit and there are many other issues involving porting custom Android roms to different phones and devices than just the kernel and drivers. Still Google does have complete control over many of these manufacturers thanks to the Open Handset Alliance (OHA), an exclusive agreement manufactures must sign if they wish to distribute Google Apps with their devices. Google does have the ability to create standards by which an Android base, similar to a base Windows install, is achievable.

Google briefly ventured into creating bloatware free versions of devices by licensing Google Play versions of its phones. These were premium devices by major manufactures that didn’t include branding or carrier customizations, essentially making them similar to stock Nexus devices⁵. Although a step in the right direction, Google killed off this line of phones in 2015⁶.

Conclusions

The current state of Android OS is unmaintainable. Imagine if, for ever Windows update, your device manufacture (e.g Dell, HP, Lenovo) had to integrate that update into their Windows code base and then release a patch for your specific Dell or HP version of Windows. The reality is that many power users wipe devices they purchase to purge their machines of manufacturer installed bloatware.

Google’s preview for Android N does support background security updates⁷, which addresses one of the major issues caused by Android fragmentation. However it still doesn’t address all the existing devices whose manufactures will never bother with an Android N update, and it still doesn’t provide a standard, unified base platform for Android.

Android started out on the G1 (a.k.a HTC Dream), a phone with just 256MB of internal storage. An embedded design approach may have made sense at the time, allowing for a secure device with a minimal footprint. Android devices have become considerably more powerful, some even rivaling consumer laptops, and the embedded model of Android has become one of its biggest limitations. To truly solve the fragmentation issue, Google needs to create a set of standards by which Android is treated as a more general purpose operating system. A unified base system, hardware standards and driver packaging can lead the way to a more streamlined, bloat free and upgradeable device ecosystem with less fragmentation.