Creating a Kivy app for Android 11 on Debian 12 Bookworm

Kivy is a free and open source Python framework for developing mobile apps, with a single codebase, you will be able to deploy apps on Windows, Linux, macOS, iOS and Android. In this page I will focus on builing an Android app using a GNU/Linux workstation running Debian 12.

The Android target will be Android 11 Red Velvet Cake, this is an aspect that should not be overlooked because starting from Android 10 a series of new features have been introduced by Google that have made the life of Android developers much more complicated. With Android 11 the situation has become further exacerbated.

Many pages on the internet that explain how to compile an Android package on Linux are quite dated; they often refer to the old open source Java SDK - OpenJDK version 8 or 9 - while the current Debian stable distribution ships with OpenJDK 17. Here I tried to revamp all the processes using a freshly installed Debian 12 Bookworm workstation.

Actually I used a KVM virtual machine to install Debian 12. I allocated a virtual host with 4 Gb of RAM, 2 CPU, 16 Gb of disk space and 1 Gb of swap. While the CPU and RAM specs are acceptable (I can compile my test app in less than one minute), consider to allocate more hard disk space. The bare operating system required about 4 Gb of space, but the building environment required about 8 Gb of space for a simple 30 kb Kivy script!

During the install I opted for a tex-only workstation (no desktop environment), but with the SSH server so I can login remotely. Once the installation was terminated, I added the following packages:

apt install git zip unzip openjdk-17-jdk python3-pip autoconf libtool pkg-config \
        zlib1g-dev libncurses5-dev libncursesw5-dev libtinfo5 cmake libffi-dev \
        libssl-dev python3-full lld

We opted to leave the overall system as clean as possibile, so all the developing environmente will be installed in user space, as explained in the following paragraphs.

Starting with version 12 Bookworm, Debian adheres to the Python Enhancement Proposal 668 titled Marking Python base environments as “externally managed”, so if an unprivileged user tries to install a Python package in its user space $HOME directory, he gets the following error:

error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
    python3-xyz, where xyz is the package you are trying to
    install.
    
    If you wish to install a non-Debian-packaged Python package,
    create a virtual environment using python3 -m venv path/to/venv.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
    sure you have python3-full installed.
    
    If you wish to install a non-Debian packaged Python application,
    it may be easiest to use pipx install xyz, which will manage a
    virtual environment for you. Make sure you have pipx installed.
    
    See /usr/share/doc/python3.11/README.venv for more information.

note: If you believe this is a mistake, please contact your Python
installation or OS distribution provider. You can override this, at
the risk of breaking your Python installation or OS, by passing
--break-system-packages.
hint: See PEP 668 for the detailed specification.

This means that the Python user space should be remain under the Debian package management system; ading Python packages using pip3 (e.g. downloaded and installed by the user) does not work any more. This is a problem!

Python PEP668 suggests to use the new venv method to install Python libraries in user space, instead of the old plain pip3 method, but this requires you to update all your habits and automation scripts. In particular there is the problem that some scripts downloaded and executed during the setup process will launch the command pip install --user ..., but the --user option is not compatible with the venv method, you will get the error:

ERROR: Can not perform a '--user' install. User site-packages are not visible in this virtualenv.

The packaging tool python-for-android is required in our building framework, but the installation scripts will probably launch pip install python-for-android, thus failing in a default Debian 12 system.

After all the most promising solution seems to be to revert to the older Debian 11 behaviour, disabling the externally managed constraint of Python.

Someone suggests to solve the problem by deleting the file /usr/lib/python3.11/EXTERNALLY-MANAGED, but this is highly discouraged, because you will remove a system file installed by the package libpython3.11-stdlib. If e.g. the package will be upgraded, the file will be installed again. The preferred solution is to override the externally-managed flag by creating the user config file $HOME/.config/pip/pip.conf with the following option:

[global]
break-system-packages = true

WARNING: Till now we did not use this new preferred method, because the package python-for-android uses the old pip install method. See below about the user method install.

Instead of disabling the externally-managed-environment flag or instead of using the pip3 --user install method, it should be possible to use the venv pip3 install method. You need to install the python3-venv Debian package. This recipe did not work for me, but should be investigated further:

python3 -m venv $HOME/python-buildozer
# Activate the venv in this shell session:
source $HOME/python-buildozer/bin/activate
$HOME/python-buildozer/bin/pip3 install --upgrade buildozer
$HOME/python-buildozer/bin/pip3 install --upgrade Cython==0.29.33 virtualenv

Basically a venv is a Python environment which is separated from others you may eventually create and which is separated from the system-wide Python environment. You select in what venv you want to work by just sourcing the activate script at the prompt of your working shell: source python-buildozer/bin/activate. The script will change your PATH so if you launch e.g. python you will get the Python environment with the modules installed only in that venv.

NOTICE: We used the user method to install the Python packages; this require to disable the Debian 12 Python externally-managed-environment. The use of the new venv method shuld be instead investigated. See the above paragraphs.

The entire building framework is composed of several pieces which we will install from direct internet download. This is mandatory because many of them are not included into the Debian stable distribution (or they are obsolete).

• Buildozer - Buildozer is a tool that aim to package mobiles application easily. It automates the entire build process, download the prerequisites like python-for-android, Android SDK, NDK, etc.
• python-for-android - Let you package Python code into standalone android APKs.
• Android NDK - The Android NDK is a toolset that lets you implement parts of your app in native code, using languages such as C and C++.
• Android Commandline Tools for Linux - It is a part of the Android SDK that can be downloaded and installed separately, it contains for example sdkmanager, apkanalyzer, etc.

The firs installation step in user space is to install the buildozer Python package from the internet. We will use the user scheme, i.e. the package will be installed into the user space $HOME/.local/lib/python3.11/site-packages:

pip3 install --user --upgrade buildozer

The package will insall also some executable scripts into the $HOME/.local/bin/ directory, you must add this into your PATH environment variable to have access to them (Debian should do this for you at login, if the directory exists).

Then you must install the Cython and virtualenv Python packages, again from the internet. NOTICE: venv and virtualenv are similar in functionality but differ in implementation; we required a specific version of the Cython module because the Buildozer reccomend that.

pip3 install --user --upgrade Cython==0.29.33 virtualenv

Also the Cython package will install some executables into $HOME/.local/bin/.

All the remaining components of the building framework (Python libraries, Android SDK, etc.) will be automatically downloaded by the Buildozer program, read the following paragraphs.

INFO: This procedure is required only once when we start a new projec.

Now it is time to prepare the Kivy project directory for the build of the Android package. You must check the following:

1. Create a subdirectory where to store all the project files (Python code, images, fonts, json files, etc.).
2. The program entry point must be called main.py and reside into the root directory of the project.
3. You must install all the Python modules required by the program in your environment (user or venv).
4. Your building path must not contain the icc substring , otherwise you will get the error no such file or directory: 'strict'

I created the directory openolyimageshare for my test app, then I execute the following:

cd openolyimageshare
buildozer init

The init process is rather time and space consuming. A good internet connection is required because it needs to download python-for-android, the commandlinetools-linux, the Android NDK and so on. My simple test app required the following space in the following directory:

• $HOME/.buildozer/ - About 2.6 Gb
• $HOME/openolyimageshare/.buildozer/ - About 3.5 Gb

During the buildozer init process you could run into a really sneaky problem caused by your working directory containing the icc sitring (I ran in this problem because my $HOME is /home/niccolo/):

unknown argument: '-fp-model'
clang-14clang-14: : error: error: no such file or directory: 'strict'no such file or directory: 'strict'

Fortunately enough I found the Kivy issue #2329, which explain the problem. I renamed my HOME directory as a workaround.

Now that the builing framework is prepared, it is time to edit the buildozer.spec file that you will find into the project directory. Here you put the information required to compile the Android package. Here I resume the sections that I modified:

[app]
title = Open Oly ImageShare
package.name = openolyimageshare
package.domain = org.rigacci
source.include_exts = py,png,jpg,kv,atlas,ttf,json
source.include_patterns = res/fonts/*,res/img/*,res/layout/*
version = 0.1
requirements = python3,kivy,requests
presplash.filename = %(source.dir)s/data/presplash.png
icon.filename = %(source.dir)s/data/icon.png
android.permissions = android.permission.INTERNET,
    android.permission.READ_EXTERNAL_STORAGE,
    android.permission.WRITE_EXTERNAL_STORAGE,
    android.permission.CAMERA

The title is the app name that will be shown by the Android system; it will be listed under the Settings ⇒ Apps, it will be used as the label below the app icon, etc.

The package.domain and package.name will be concatenated to form a globally unique identifier of your app. You should use an actual internet domain name that you own (reversing the order of the hierarchy, i.e. the Tope Level Domain must come first) and an unique name under your domain. Into the Android system your program will be installed under a directory named [package.domain].[package.name], so that name should be all lowercase, with no spaces, no special chars, etc.

With source.include_exts and source.include_patterns you tell to Buildozer what files must be included into the package. Including a directory does not automatically include all the files within, you still must indicate the extensions.

In requirements you must declare which Python packages are required by your code. Chek all the import statements in your code and list only the ones which are not into the default library (the ones that you installed via Debian package or pip).

You can use presplash.filename and icon.filename to include two artwork in your app. The presplash will be displayed at startup, during the initialization of the environment (unfortunately it is rather time consuming). The icon is instead what you can imagine. Use PNG graphics at least 512 x 512 pixels, you can use transparency too.

In android.permissions you must list all the permissions that your app will require from the operating system. If you forget to declare something your app simply will not be able to do that operation. Beware that starting from Android 10 the access to the external storage (basically the space into the SD card or into the device memory) has undergone a drastic change, see the table below for a basic overview.

Into the buildozer.spec you can define the following:

# (int) Target Android API, should be as high as possible.
#android.api = 31

# (int) Minimum API your APK / AAB will support.
#android.minapi = 21

# (int) Android SDK version to use
#android.sdk = 20

By setting a target API lower than your SDK can support, you can declare what will be the highest Android version that your app is designed to be compatible with. At the moment Buildozer will try to target API 31 (Android 12).

Declaring a minimum API you can tell what is the minimum Android version required by your app. We did not declared the android.minapi, so Buildozer choosed API 21 (Android 5.0.2).

The compile SDK is the environment you want to use to create the app, i.e. the SDK you downloaded from Google (which generally support the higher API available at the moment). This will affects what functions and constructs you can use in your program. If you do not specify a version, Buildozer should detect the highest SDK downloaded and use it. In our case only one SDK was downloaded, and it was SDK API 31.

INFO: This is the only procedure to be executed whenever you want to compile a new version; the version number must be updated into the main.py source code.

Enter the project directory and edit the main.py source code updating the definition of the __version__ variable (the buildozer.spec will refer this value to create the package name). Then choose to make a debug build:

buildozer android debug

The package will be created into the bin/ subdirectory, using a name like packagename-version-arm64-v8a_armeabi-v7a-debug.apk.

When you are ready to publish your package, you must create the release binary:

buildozer android release

In this case the package created into the bin/ subdirectory will be named like packagename-version-arm64-v8a_armeabi-v7a-release.aab, which is not directly installable into the device.

• Buildozer Installation
• python-for-android
• 3 Ways to Convert Python App into APK
• Programming Guide - Create a package for Android

• Python Enhancement Proposal 668
• Troubleshooting ‘Externally Managed Environment’ Error in Debian 12 Pip3 Installation
• pip install -r requirements.txt is failing: "This environment is externally managed"
• How do I solve "error: externally-managed-environment" every time I use pip 3?
• How to solve "error: externally-managed-environment" when installing via pip3

• -fp-model argument not found, directory strict not found

• Storage updates in Android 11
• WRITE_EXTERNAL_STORAGE when targeting Android 10
• How to access non-media & media files on Android 11+
• ASD (app specific directory) vs Private folder
• Access media files from shared storage