jami
/
jami-docs
mirror of https://git.jami.net/savoirfairelinux/jami-docs.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

Updated Create_Plugin.md (markdown)

This commit is contained in:
agsantos 2020-11-06 13:00:29 -05:00
parent 0ea978e51d
commit 8fb180ca8a
4 changed files with 300 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
.redirects.gollum Normal file
View File

@ -0,0 +1,2 @@
---
technical/Create Plugin.md: technical/Create_Plugin.md

2
technical/7.-Jami-plugins.md
View File

@ -7,7 +7,7 @@ But that is not all, you can also transform your awesome ideas into a brand new
* To properly setup a plugin you must follow the steps in [#How to use it?](#how-to-use-it).
* To build a available plugin, please refer to [#How to build?](#how-to-build) instructions.
* To create your own plugin, please refer to [Create Plugin](http://git.jami.net/savoirfairelinux/ring-project/wikis/technical/create_plugin) instructions.
* To create your own plugin, please refer to [Create Plugin](http://git.jami.net/savoirfairelinux/ring-project/wikis/technical/Create_Plugin) instructions.
## How it works?
Jami can be break down to three main components that interact together: Daemon, LRC and clients.

1
technical/Create Plugin.md
View File

@ -1 +0,0 @@
This page will be available soon.

297
technical/Create_Plugin.md Normal file
View File

@ -0,0 +1,297 @@
**NOTE: this page introduces the Jami Plugins SDK.**
# Jami Plugins
As from September of 2020, Jami team has added plugins as a call feature for GNU/Linux, Windows, and Android users.
This meaning that now you can personalize your call experience by using one of our available plugins.
But that is not all, you can also transform your awesome ideas into a brand new plugin!
Here you will be guided throught the SDK that will help you start your plugin developpment.
The text is organized as:
* A description of our [#SDK](#sdk);
* An example of how to create your own base plugin with our SDK - [#Create my first plugin](#create-my-first-plugin).
## SDK
We developped a Plugin System for Jami and we have a few plugins available to be used.
However as an open source project, we now desire users to be able to create, use, and distribute their own plugins.
To achieve that goal, we also developped a Jami Plugins SDK.
This kit is fully writen in python, and can be invoked running [pluginMainSDK.py]('') from `<jami-plugins>` folder.
To get started you must:
``` bash
mkdir plugins && cd plugins
git clone https://review.jami.net/ring-daemon daemon
git clone https://review.jami.net/jami-plugins plugins
cd plugins
pip3 install -r SDK/requirements.txt
python3 SDK/pluginMainSdk.py
```
You will notice that this script will generate a Jami Plugins SDK shell that allows users to:
* [Create, modify or delete a manifest.json](#create-modify-or-delete-manifest-json);
* [Create a package.json](#create-a-package-json);
* [Create or delete a preference](#create-or-delete-a-preference);
* [Create functionality](#create-functionality);
* [Create main](#create-main);
* [Create full plugin skeleton](#create-full-plugin-skeleton);
* [Assemble files](#assemble-files);
* [Build](#build);
* [Merge jpls](#merge-jpls).
Each one of these functionalities will be detailled next.
We also will explain the importance of the files it generates and any related SDK limitations.
### Create, modify or delete manifest.json
Every plugin must have a manifest. This file contains the official name, a description, and carries the plugin build version as in the example bellow. Without it, the plugin system will not be able to find the plugin library it should load. Due to it's importance, every time Jami Plugin SDK is told to create files to a non existing plugin or to a plugin with no manifest, it will first create a new manifest.
``` bash
{
"name": "HelloWorld",
"description" : "HelloWorld plugin will guide you throught Jami Plugins SDK use!",
"version" : "1.0"
}
```
To create/modify (or to delete) a manifest.json, from inside Jami Plugins SDK shell, the user must call:
```
(Jami Plugins SDK) manifest (-del)
```
The SDK will ask other informations needed to correctly accomplish the task.
### Create a package.json
Jami currently supports plugins for GNU/Linux, Android and Windows. For the latter, the build system used is the same as for Jami, it is, we call the plugin build using a python script. This script will look for a `package.json` file to aquire build informations and commands. Without this file our build pipeline for Windows will not work.
An example package.json file is shown bellow.
* `name` and `version` in the package.json and manifest.json files should match.
* `extractLibs` indicates to the build system if the files under `<jami-plugins>/contrib/libs.tar.gz` will be used. This archive contains header files for **Tensorflow**. Thus, you only need to set `extractLibs` to true if you plan to use this library.
* To play with video, the plugin is dependent of **ffmpeg**. By adding it to `deps`, the build system will automatically compile this library from `<ring-daemon>/contrib` if needed. We also provide **OpenCV** build from inside `<ring-daemon>/contrib` !
If you want to use Tensorflow, we provide built libraries for GNU/Linux and Android with our docker images [here](https://hub.docker.com/repository/docker/sflagsantos/tensorflow-cuda) and [here](https://hub.docker.com/repository/docker/sflagsantos/tensorflowlite).
For more information about OpenCV and Tensorflow build, please refer to [jami-plugins](7.-Jami-plugins.md) technical documentation. There we have a step-by-step!
* If you're using cmake, your can set configuration definition in `defines` property. Exemple: if your configuration line is of the form `cmake -DCPU=TRUE ..` you may set `"defines": ["CPU=TRUE"]`.
* Any command directly related to the plugin build can be defined inside `custom_scripts`. Bellow we create the build folder to with the plugins project will be configured with `mkdir msvc` and we also set the build command as `cmake --build ./msvc --config Release`. Our example CMakeLists.txt may have pre and post build instruction that are not listed here.
``` bash
{
"name": "Hello World",
"version": "1.0",
"extractLibs": false,
"deps": [
"ffmpeg"
],
"defines": [],
"custom_scripts": {
"pre_build": [
"mkdir msvc"
],
"build": [
"cmake --build ./msvc --config Release"
],
"post_build": []
}
}
```
To create a package.json, from inside Jami Plugins SDK shell, the user must call:
```
(Jami Plugins SDK) package
```
The SDK will ask other informations needed to correctly accomplish the task. After the base package.json creation, the user must add or modify any information not previewed by the SDK process.
### Create or delete a preference
A preference is a internal variable that will be used upon loading or while running the plugin.
There is limited types of preferences supported by the plugin system and each of them must contain generic and specific informations. Those informations must be placed under a certain structure that will form one preference and each preference must be listed inside a `preference.json` file.
The generic properties of a preference are those that must be set by any type of preference: `category`, `type`, `key`, `title`, `summary`, `defaultValue`, and `scope`. The specific ones are linked to the type of the preference constructed.
For `PATH` preferences we have: `mimeType`.
For `LIST` preferences we have: `entries` and `entryValues`.
A `LIST` preference must have a list of possible values to be used and a list of 'names' for these values. For example:
If you have two `entriesValues`: '0' and '1', these values may not be understandable by the user. Jami's UI will take the values from `entries` to be shown as 'names' for each one of these `entryValues`.
Then you can call '0' and '1' as 'sent' and 'received'. The UI will show these names that are more user friendly!
It is important to note that `entries` and `entryValues` must have the same number of items.
Another important point to be noted for the preferences is that their values could be modified during runtime if, and only if, two conditions are satisfied:
1. the code that applies the new value is within your functionality implementation and;
2. this functionality is listed in the preference's `scope` property.
To better explain, we have to detail what is a scope and how a preference changement is implemented inside the plugin.
* **Scope**: A scope is a list of functionalities that can modify a preference value even if the functionality is under use. It is, imagine you have a functionality called "Circle" that prints a colored circle to your video. Consider also that the color of that circle is set by a preference and that this preference lists "Circle" as one of it's scopes. In that scenario "Circle" will be able to modify the default circle color to another one.
* **Code implementation**: Continuing our example above, "Circle" also is the implementation of an abstract API class from Daemon (for more details check [#Create functionality](#create-functionality). When a user changes a preference value, the plugin system will call the `setPreferenceAttribute` implementation for each of the functionalities listed by the preference's scope. By it's turn, this function will match the preference unique `key` and call an internal function to apply the new value.
For a pratical example, you can check the 'Foreground Segmentation' functionality of the GreenScreen plugin - [pluginMediaHandler.cpp](https://git.jami.net/savoirfairelinux/jami-plugins/blob/master/GreenScreen/pluginMediaHandler.cpp) and [preferences-tfcc.json](https://git.jami.net/savoirfairelinux/jami-plugins/blob/master/GreenScreen/preferences-tfcc.json). This plugin has both `LIST` and `PATH` preferences and also has one preference that can be modified during runtime. Can you tell wich one?
To create (or delete) a preference, from inside Jami Plugins SDK shell, the user must call:
```
(Jami Plugins SDK) preference (-del)
```
The SDK will ask other informations needed to correctly accomplish the task. If a preference is created from outside a functionality creation pipeline, any API implementation will not be changed in order to avoid overwrittings. Thus, if you want to allow values changement during runtime, you may need to manually modify your functionality implementation to fit running time changements conditions.
### Create functionality
A Jami plugin may wrap one or multiple functionalities. It is, the same plugins may have a functionality to change background and to draw a circle to a video for example.
Each functionality must implement an abstract API class defined by Jami plugins System. That way, we can say that one functionality is one implementation of an API.
Currently the only API available is the `MediaHandler` which allows access to video frames. In the future users will have acess to audio and conversation messages too.
When defining a new a functionality, the SDK will create basic header and cpp files for you work on.
However, it can not be added to the scopes of a previously existing preference. For more information, please refer to [Create or delete a preference](#create-or-delete-a-preference).
To create a functionality, from inside Jami Plugins SDK shell, the user must call:
```
(Jami Plugins SDK) functionality
```
The SDK will ask other informations needed to correctly accomplish the task.
### Create main
This option create plugin's `main.cpp`. A file that implements the plugin external loading function that Jami Plugin System
will call to initialize and register all functionalities for latter use.
The SDK is set to rewrite the main file every time you create a new functionality.
However if you want to manually create or delete a functionality, we recomend calling this option.
To create a `main.cpp`, from inside Jami Plugins SDK shell, the user must call:
```
(Jami Plugins SDK) main
```
The SDK will ask other informations needed to correctly accomplish the task.
### Create full plugin skeleton
This option performs a sequence of actions to properly create all base files needed to start a plugin development. The steps are:
1. gather authorship information;
2. define a plugin name;
3. create manifest;
4. create functionalities and preferences;
5. create main file;
6. define basic package.json;
7. define basic build files (build.sh and CMakeLists.txt).
If all is completed successfully, the plugin may be build, installed, and loaded. Also the functionallities may be toggled, however, since their data processes are not implemented, they will perform no action. In our `HelloWorld` plugin, we implement a simple process using **OpenCV**. For more complex options, you can refer to our available plugins at [jami-plugins](https://git.jami.net/savoirfairelinux/jami-plugins). Feel free to implement any ideas you may have, but you should respect those constrains:
* use ffmpeg and opencv from ring-daemon project.
* if using tensorflow, choose version 2.1.0. Why?
* We have all needed header files of tensorflow 2.1.0 in <jami-plugins>/contrib/libs.tar.gz;
* We disponibilize docker images with libraries for Tensorflow C++ API and Tensorflow
Lite, for both Android and Linux development.
* if you need other libraries, check if we support it's build with ring-daemon project,
otherwise, you will have to build it and ensure correct link to the plugin libraries.
To fully create a basic plugin with pre-implementation of desired functionalities APIs, preferences, package, manifest, main file, and basic build related files, from inside Jami Plugins SDK shell, the user must call:
```
(Jami Plugins SDK) plugin
```
The SDK will ask other informations needed to correctly accomplish the task.
### Assemble files
The final plugin file is an archive compressed to a `JPL` format. This archive contains libraries, manifest.json, preferences.json, icon.png and other custom important files for your plugin.
OBS.: files added by the developper must be organized under the `data/` folder.
The SDK assemble option has two different behaviours:
* it creates a build folder and copies there all files that will be compressed to the final plugin archive. For linux host: `<jami-plugins>/<HelloWorld>/build-local/jpl/` and for a windows host: `<jami-plugins>/<HelloWorld>/msvc/jpl/`;
* it compresses all files inside the build folder to the jpl archive wich is output under `<jami-plugins>/build`.
Both process should be called from inside the CMakeLists.txt as POST_BUILD and PRE_BUILD commands, respectively. Also, the build.sh script should call them. For more information about CMakeLists.txt and build.sh files, please refere to [build](#build) and to our available plugins at [jami-plugins](https://git.jami.net/savoirfairelinux/jami-plugins).
To create a build folder and copy all important files there, from inside Jami Plugins SDK shell, the user must call:
```
(Jami Plugins SDK) assemble -pre
```
To compress all assembled to a jpl archive, from inside Jami Plugins SDK shell, the user must call:
```
(Jami Plugins SDK) assemble
```
The SDK will ask other informations needed to correctly accomplish the task.
### Build
The SDK build option has two different behaviours:
* it creates basic CMakeLists.txt and buils.sh files;
* it build the plugin library with default options defined at the files mentioned above.
A description of thes CMakeLists.txt and buils.sh files are found further in this section.
To create basic CMakeLists.txt and buils.sh files, from inside Jami Plugins SDK shell, the user must call:
```
(Jami Plugins SDK) build -def
```
To build plugin library with default configuration, from inside Jami Plugins SDK shell, the user must call:
```
(Jami Plugins SDK) build
```
The SDK will ask other informations needed to correctly accomplish the task. For the moment, the build option does not support cross-compilation neither non default builds. If you have build variables to be set differently than the default option, please directly use the `<jami-plugins>/build-plugin.py` script.
#### CMakeLists.txt
This file is used only by Jami's Windows build pipeline.
If you need to pass any defines to cmake generation, your definitions can be in `package.json` as explained in the package section. The package.json created specifies the default configuration that calling the build from Jami Plugins SDK will consider.
If you want to build with no default configuration, you can: directly use the script mentioned above and pass non-default definitions as an argument or; you also can manually change your package.json file. For examples, you can refer to our available plugins at [jami-plugins](https://git.jami.net/savoirfairelinux/jami-plugins).
Another important information about CMakeLists.txt is that it has to add custom commands. For PRE_BUILD, it must call the pre-assemble functionality process. For POST_BUILD, it must copy the built library to the build folder and call the assemble functionality process. In the end, your jpl archive may be found under `<jami-plugins>/build`. The CMakeLists.txt file automatically created by our SDK, already respects these constraints.
#### build.sh
This file is used by Jami's to build plugins for GNU/Linux and Android platforms.
The basic script consider the environment variable `DAEMON` that must point to the ring-daemon folder. Besides, you can pass an argument for the platform used like `-t android` if you want to cross-compile for Android. Further custom definitions and environment variables should be handled by the plugin developper.
If you want to build with no default configuration, you can modify the environment variables values and then call the build.
Ie: for android, you can set which ABI you want to build with `export ANDROID_ABI="arm64-v8a armeabi-v7a`.
For other examples, you can refer to our [technical documentation](https://git.jami.net/savoirfairelinux/ring-project/wikis/technical/7.-Jami-plugins) and to our [available plugins](https://git.jami.net/savoirfairelinux/jami-plugins).
Another important information about build.sh is that it has to call pre and post assemble. Before the build, it must call the pre-assemble functionality process. After it, it must copy the built library to the build folder and call the assemble functionality process. In the end, your jpl archive may be found under `<jami-plugins>/build`. The buil.sh file automatically created by our SDK, already respects these constraints.
### Merge jpls
If you have more than one jpl archives, like one build for Android and anothe for GNU/Linux platforms, you can merge then into one to easy it's distribution. However, you should know that merging two or more jpls may inccur orverwritting some of the files inside them if they are not equal for all archives. The only files that may not present conflicting contents are the ones that do not repeate themselves. If conflicts occur, files from the first jpl in the arguments will prevail over the others.
To merge two or more jpls, from inside Jami Plugins SDK shell, the user must simple call:
```
(Jami Plugins SDK) merge foo.jpl bar.jpl baz.jpl
```
The SDK will ask other informations needed to correctly accomplish the task.
## Create my first plugin
Through this section we will present a step-by-step construction of a HelloWorld plugin using our SDK.
Our goal is to print a coloured circle in the midle of the video frames using OpenCV.
The color of that circle will be defined by a preference wich will be changeable during runtime.
Also we can set a stream preferece to define if the plugin will modify the video sent or the one received, this time we don't want to allow a changement during runtime.
We can define a second functionality that will aways draw a circle in the right top corner, with the color defined by the same preference as the previous functionality but that cannot be changed during runtime.
At the end we will exemplify how to build your plugin with and without the SDK.
### Step 1 - prepare developpment environment
THe first step towards plugin development is to properly prepare the environment.
``` bash
mkdir plugins && cd plugins
git clone https://review.jami.net/ring-daemon daemon
git clone https://review.jami.net/jami-plugins plugins
cd plugins
pip3 install -r SDK/requirements.txt
```
### Step 2 - create HelloWorld with one functionality
``` bash
python3 SDK/pluginMainSdk.py
(Jami Plugins SDK) plugin
```
### Step 3 - build and show results
### Step 4 - create anothe functionality for HelloWorld
### Step 5 - build and show results
Now you can start your own creations!
Do not forget to tag Jami or Savoir-Faire Linux in yout repository, this way we can get to know how the community is developping their own plugins!