jami-docs/extension/tensorflow-extension.md

# TensorFlow extension

```{note}
This page describes how to build TensorFlow and the LiteRT C++ API for Linux, Windows, and Android.
LiteRT (short for Lite Runtime) is the new name for TensorFlow Lite (TFLite).
```

## TensorFlow 2.1.0
A challenge with working with TensorFlow is how to properly build it.
To reduce the building challenges, Docker images have been created with CUDA and TensorFlow libraries available for
GNU/Linux builds [here](https://hub.docker.com/repository/docker/sflagsantos/tensorflow-cuda) and for
Android builds [here](https://hub.docker.com/repository/docker/sflagsantos/tensorflowlite).
Docker can be used to build extensions for GNU/Linux and Android.
However, they are unable to handle Windows.
The following guide describes how to properly build LiteRT Native and the TensorFlow C++ API for the supported platforms.

Requirements:

* Python 3
* Bazel 0.29.1
* TensorFlow 2.1.0 repository:

    ```bash
    git clone https://github.com/tensorflow/tensorflow.git
    cd tensorflow
    git checkout v2.1.0
    ```
TensorFlow headers required to build extensions have been assembled.
Extract the `libs.tar.gz` file found in `jami-project/plugins/contrib` to access the TensorFlow headers.
However, if a different version of TensorFlow is used or assembling TensorFlow from source is required,
instructions to assemble TensorFlow Lite Native and C++ API are shown in the `README_ASSEMBLE` file available at {gitlab-project}`jami-plugins`.

### GNU/Linux
LiteRT does not support desktop GPUs.
Consider using the TensorFlow C++ API if desktop GPU support is required.

If TensorFlow C++ API with GPU support is required, ensure:
a CUDA-capable GPU is available;
all the installation steps for Nvidia drivers, the CUDA Toolkit, CUDNN, and LiteRT are followed;
and the version numbers match and are correct for the TensorFlow version being built.

The following links may be helpful:

* <https://www.tensorflow.org/install/source>
* <https://developer.nvidia.com/cuda-gpus>
* <https://developer.nvidia.com/cuda-toolkit-archive>
* <https://developer.nvidia.com/cudnn>

Set up the build options with `./configure`.

* LiteRT Native

    ```bash
    bazel build //tensorflow/lite:libtensorflowlite.so
    ```
* TensorFlow C++ API

    ```bash
    bazel build --config=v1 --define framework_shared_object=false --define=no_tensorflow_py_deps=true //tensorflow:libtensorflow_cc.so
    ```

### Windows
LiteRT does not support desktop GPUs.
Consider using the TensorFlow C++ API if desktop GPU support is required.

If TensorFlow C++ API with GPU support is required, ensure:
a CUDA-capable GPU is available;
all the installation steps for Nvidia drivers, the CUDA Toolkit, CUDNN, and LiteRT are followed;
and the version numbers match and are correct for the TensorFlow version being built.

The following links may be helpful:

* <https://www.tensorflow.org/install/source>
* <https://developer.nvidia.com/cuda-gpus>
* <https://developer.nvidia.com/cuda-toolkit-archive>
* <https://developer.nvidia.com/cudnn>

Set up the build options with `python3 configure.py`.

* LiteRT Native

    ```bash
    bazel build //tensorflow/lite:tensorflowlite.dll
    ```
* TensorFlow C++ API

    ```bash
    bazel build --config=v1 --define framework_shared_object=false --config=cuda --define=no_tensorflow_py_deps=true //tensorflow:tensorflow_cc.dll
    ```
There may be some missing references while compiling an extension with the TensorFlow C++ API.
If this occurs, rebuild TensorFlow and explicitly export the missing symbols.
Fortunately, TensorFlow now has an easy workaround.
Feed [this]("https://github.com/tensorflow/tensorflow/blob/v2.2.0/tensorflow/tools/def_file_filter/def_file_filter.py.tpl") file with the required symbols.

### Android – LiteRT Native
For mobile applications, it is suggested that LiteRT is the only option to consider to successfully build TensorFlow. Additional requirements are:

* Android NDK 18r

Set up the build options with:

```bash
./configure
        >> Do you want to build TensorFlow with XLA JIT support? [Y/n]: n
        >> Do you want to download a fresh release of Clang? (Experimental) [y/N]: y
        >> Do you want to interactively configure ./WORKSPACE for Android builds? [y/N]: y
        >> Please specify the home path of the Android NDK to use. [Default is /home/<username>/Android/Sdk/ndk-bundle]: put the right path to NDK 18r
```
And build as required:

* armeabi-v7a

    ```bash
    bazel build //tensorflow/lite:libtensorflowlite.so --crosstool_top=//external:android/crosstool --cpu=armeabi-v7a --host_crosstool_top=@bazel_tools//tools/cpp:toolchain --cxxopt="-std=c++11"
    ```
* arm64-v8a

    ```bash
    bazel build //tensorflow/lite:libtensorflowlite.so --crosstool_top=//external:android/crosstool --cpu=arm64-v8a --host_crosstool_top=@bazel_tools//tools/cpp:toolchain --cxxopt="-std=c++11"
    ```