Plugin documentation: refactor plugin documentation
Change-Id: I3380cb715f1c0e0951dd02bccab132aafc1b0806
This commit is contained in:
Xavier Jouslin de Noray
Sébastien Blin
parent
a48997de84
commit
e921bf8ca8
|
|
@ -0,0 +1,58 @@
|
|||
**NOTE: this page describes what is a Jami Plugin and how to install and use them.**
|
||||
|
||||
# Jami plugins
|
||||
As from September of 2020, Jami team has added plugins as a call and chat feature for Linux, Windows, and Android users.
|
||||
This meaning that now you can personalize your call/chat experience by using one of our available plugins.
|
||||
But that is not all, you can also transform your awesome ideas into a brand new plugin!
|
||||
|
||||
# How it works?
|
||||
Jami can be break down to three main components that interact together: Daemon, LRC and clients.
|
||||
Daemon is the core of Jami, and although it does not interact with users, it is involved in every
|
||||
command. Therefore, Daemon has a `JamiPluginManager` class that among other actions perfoms install/uninstall, load/unload, edit preferences and control plugins' usage.
|
||||
Despite Daemon importance, what a plugin effectivelly does to a call video/audio or to a chat message is unknown to it the same way Daemon does not know what is effectivelly done by LRC or the clients interfaces.
|
||||
Plugins then can be seen as a forth interacting component in Jami.
|
||||
|
||||
The plugin system inside Jami exposes different APIs that can be used by the plugins. For instance, the ChatHandler and the Media Handler API. The latter enables the plugins to modify audio and video streams from Jami calls and is used by our GreenScreen plugin but could also be used to build a YouTube streaming system, various instagram-style filters, a real time translation service, etc.
|
||||
|
||||
Plugins can be composed by one or multiple media and chat handlers that are responsible for attaching/detaching a data stream from Jami and a data process. Each handler represents a functionality that can be totally different between them or can be a modified versions of the same core process. In our example, we have only one functionality, it being, the GreenScreen plugin has one media handler which data process is responsible for segmenting the foreground from a video frame and applying another image to the background, just like it is done with the green screens in movies!
|
||||
|
||||
To use one custom functionality, it is necessary that Jami knows all plugins' handlers, which one is going to be used and the data that should be processed. Plugin's handlers are created once a plugin is loaded and are shared with Daemon's Plugin Manager. The data is inside Jami flow (for a call plugin, in the event of a new call, Jami creates and stores the corresponding media stream subjects). Finally, once a user puts a plugin functionality in action Jami tells this handler to attach the available data. When deactivated, Jami tells the handler to dettach.
|
||||
|
||||
# How to use it?
|
||||
|
||||
## Setup
|
||||
A Jami plugin is a `pluginname.jpl` file, and it must be installed to your Jami.
|
||||
Once installed, Jami will add your new plugin to the available plugins list but they will not be ready for use yet. Plugins are libraries and must be loaded if you want to expose them.
|
||||
Moreover, plugin may have preferences and besides install/uninstall and load/unload actions, it is possible to modify those preferences. For example, our GreenScreen plugin allows the user to change the background image displayed.
|
||||
|
||||
### Android
|
||||
To setup a plugin for Android you must go under Setting, enable plugins if they are disabled, and select a plugin file from your phone.
|
||||
After installed it is automaticaly loaded.
|
||||
Optionally, you can manually perform load/unload using the checkbox button on the plugin list.
|
||||
|
||||
|
||||
|
||||
For Android uninstall, you must click on the plugin and a uninstall option will appear allong with the preferences and a reset preferences option.
|
||||
In order to a preference modification can take effect the plugin has to be reloaded.
|
||||
|
||||
|
||||
|
||||
### Linux/Windows
|
||||
Similarly, for the client-qt available on Linux and Windows and for the client-gnome available only on Linux, you must go to Prefences, enable plugins if it is disabled, and select a plugins file from your computer.
|
||||
Each plugin in the shown list is linked to two buttons beeing:
|
||||
|
||||
- Client-qt: a load/unload button and a preferences button;
|
||||
- Client-gnome: a load/unload button and a uninstall button;
|
||||
For client-gnome it is not possible to change plugin's preferences.
|
||||
|
||||
|
||||
|
||||
|
||||
## Use!
|
||||
A media handler functionality only takes place if you turn them on during a call.
|
||||
For either Android, Linux or Windows you can do so by clicking on the plugins icon on your call screen.
|
||||
|
||||
|
||||
|
||||
|
||||
Similarly, for chat handler functionality, you will see a plugin icon in the chat window as in the images bellow.
|
||||
|
|
@ -1,71 +1,10 @@
|
|||
**NOTE: this page describes what is a Jami Plugin and how to install and use them.**
|
||||
|
||||
# Jami plugins
|
||||
As from September of 2020, Jami team has added plugins as a call and chat feature for Linux, Windows, and Android users.
|
||||
This meaning that now you can personalize your call/chat experience by using one of our available plugins.
|
||||
But that is not all, you can also transform your awesome ideas into a brand new plugin!
|
||||
|
||||
* To properly setup a plugin you must follow the steps in [#How to use it?](#how-to-use-it).
|
||||
* To build a available plugin, please refer to [#How to build?](#how-to-build) instructions.
|
||||
* To create your own plugin, please refer to [Create Plugin](7.1 - Create Plugin) instructions.
|
||||
|
||||
## How it works?
|
||||
Jami can be break down to three main components that interact together: Daemon, LRC and clients.
|
||||
Daemon is the core of Jami, and although it does not interact with users, it is involved in every
|
||||
command. Therefore, Daemon has a `JamiPluginManager` class that among other actions perfoms install/uninstall, load/unload, edit preferences and control plugins' usage.
|
||||
Despite Daemon importance, what a plugin effectivelly does to a call video/audio or to a chat message is unknown to it the same way Daemon does not know what is effectivelly done by LRC or the clients interfaces.
|
||||
Plugins then can be seen as a forth interacting component in Jami.
|
||||
|
||||
The plugin system inside Jami exposes different APIs that can be used by the plugins. For instance, the ChatHandler and the Media Handler API. The latter enables the plugins to modify audio and video streams from Jami calls and is used by our GreenScreen plugin but could also be used to build a YouTube streaming system, various instagram-style filters, a real time translation service, etc.
|
||||
|
||||
Plugins can be composed by one or multiple media and chat handlers that are responsible for attaching/detaching a data stream from Jami and a data process. Each handler represents a functionality that can be totally different between them or can be a modified versions of the same core process. In our example, we have only one functionality, it being, the GreenScreen plugin has one media handler which data process is responsible for segmenting the foreground from a video frame and applying another image to the background, just like it is done with the green screens in movies!
|
||||
|
||||
To use one custom functionality, it is necessary that Jami knows all plugins' handlers, which one is going to be used and the data that should be processed. Plugin's handlers are created once a plugin is loaded and are shared with Daemon's Plugin Manager. The data is inside Jami flow (for a call plugin, in the event of a new call, Jami creates and stores the corresponding media stream subjects). Finally, once a user puts a plugin functionality in action Jami tells this handler to attach the available data. When deactivated, Jami tells the handler to dettach.
|
||||
|
||||
## How to use it?
|
||||
|
||||
### Setup
|
||||
A Jami plugin is a `pluginname.jpl` file, and it must be installed to your Jami.
|
||||
Once installed, Jami will add your new plugin to the available plugins list but they will not be ready for use yet. Plugins are libraries and must be loaded if you want to expose them.
|
||||
Moreover, plugin may have preferences and besides install/uninstall and load/unload actions, it is possible to modify those preferences. For example, our GreenScreen plugin allows the user to change the background image displayed.
|
||||
|
||||
#### Android
|
||||
To setup a plugin for Android you must go under Setting, enable plugins if they are disabled, and select a plugin file from your phone.
|
||||
After installed it is automaticaly loaded.
|
||||
Optionally, you can manually perform load/unload using the checkbox button on the plugin list.
|
||||
|
||||
|
||||
|
||||
For Android uninstall, you must click on the plugin and a uninstall option will appear allong with the preferences and a reset preferences option.
|
||||
In order to a preference modification can take effect the plugin has to be reloaded.
|
||||
|
||||
|
||||
|
||||
#### Linux/Windows
|
||||
Similarly, for the client-qt available on Linux and Windows and for the client-gnome available only on Linux, you must go to Prefences, enable plugins if it is disabled, and select a plugins file from your computer.
|
||||
Each plugin in the shown list is linked to two buttons beeing:
|
||||
|
||||
- Client-qt: a load/unload button and a preferences button;
|
||||
- Client-gnome: a load/unload button and a uninstall button;
|
||||
For client-gnome it is not possible to change plugin's preferences.
|
||||
|
||||
|
||||
|
||||
|
||||
### Use!
|
||||
A media handler functionality only takes place if you turn them on during a call.
|
||||
For either Android, Linux or Windows you can do so by clicking on the plugins icon on your call screen.
|
||||
|
||||
|
||||
|
||||
|
||||
Similarly, for chat handler functionality, you will see a plugin icon in the chat window as in the images bellow.
|
||||
|
||||
## How to build?
|
||||
# How to build?
|
||||
If you want to make something with your video call, it is possible that you will do so with OpenCV and/or deep learning models (Tensorflow, PyTorch, etc).
|
||||
So, before going to the plugin, it is necessary to build plugin's dependencies.
|
||||
|
||||
### Dependencies
|
||||
## Dependencies
|
||||
Here we give you the steps to build [OpenCV](https://opencv.org/) and [ONNX](https://www.onnxruntime.ai/) but do not feel limited to these libraries. We offer a [page](7.2 - Tensorflow Plugin) with detailled explanation of how to build tensorflow C++ API for Windows, Linux and Android.
|
||||
Other libraries should work as long they and the plugin are correctly built!
|
||||
|
||||
|
|
@ -73,14 +12,14 @@ Other libraries should work as long they and the plugin are correctly built!
|
|||
We kindly added OpenCV 4.1.1 as a contrib in [daemon](https://git.jami.net/savoirfairelinux/jami-daemon/tree/master/contrib).
|
||||
This way you can easily build OpenCV for Android, Linux, and Windows. You only have to follow the corresponding instructions.
|
||||
|
||||
##### Windows
|
||||
#### Windows
|
||||
```bash
|
||||
set DAEMON=<path/to/daemon>
|
||||
cd ${DAEMON}/compat/msvc
|
||||
python3 winmake.py -fb opencv
|
||||
```
|
||||
|
||||
##### Linux
|
||||
#### Linux
|
||||
With Docker (recommended):
|
||||
```bash
|
||||
export DAEMON=<path/to/daemon>
|
||||
|
|
@ -107,17 +46,19 @@ make fetch opencv opencv_contrib
|
|||
make
|
||||
```
|
||||
|
||||
##### Android
|
||||
#### Android
|
||||
Using Docker (recommended):
|
||||
```bash
|
||||
export DAEMON=<path/to/daemon>
|
||||
cd ${DAEMON}/../
|
||||
docker build -f plugins/docker/Dockerfile_android_onnxruntime -t plugins-android .
|
||||
docker run --rm -it ${DAEMON}/:/home/gradle/src:rw plugins-android:latest /bin/bash
|
||||
export DAEMON=<path/to/jami-plugins>
|
||||
cd <path-to-plugins
|
||||
docker build -f docker/Dockerfile_android_onnxruntime -t plugins-android .
|
||||
docker run --rm -it -v ${DAEMON}/:/home/gradle/plugins:rw plugins-android:latest /bin/bash
|
||||
cd plugins/contrib
|
||||
ANDROID_ABI="arm64-v8a" sh build-dependencies.sh
|
||||
```
|
||||
|
||||
**NOTE: if onnx makes error on root permissions: --allow_running_as_root**
|
||||
|
||||
Using your own system:
|
||||
```bash
|
||||
export DAEMON=<path/to/daemon>
|
||||
|
|
@ -141,7 +82,7 @@ cd native-${TARGET}
|
|||
make
|
||||
```
|
||||
|
||||
#### Onnxruntime 1.6.0
|
||||
### Onnxruntime 1.6.0
|
||||
A difficulty for a lot of people working with deep learning models is how to deploy them.
|
||||
With that in mind we provide the user the possibility of using onnxruntime.
|
||||
There are several development libraries to train and test but, they are usually too heavy to deploy. Tensorflow with cuda support, for instance, can easily surpass 400MB. In our GreenScreen plugin We chose to use onnxruntime because it's lighter (library size of 140Mb for cuda support) and supports model convertion from several development libraries (Tensorflow, PyTorch, Caffe, etc.).
|
||||
|
|
@ -160,7 +101,7 @@ The following links may be very helpfull:
|
|||
* https://developer.nvidia.com/cuda-toolkit-archive
|
||||
* https://developer.nvidia.com/cudnn
|
||||
|
||||
#### Linux and Android
|
||||
### Linux and Android
|
||||
We kindly added onnxruntime as a contrib in [daemon](https://git.jami.net/savoirfairelinux/jami-daemon/tree/master/contrib).
|
||||
This way you can easily build onnxruntime for Android, and Linux.
|
||||
|
||||
|
|
@ -275,7 +216,7 @@ export ANDROID_SDK=${ANDROID_HOME}
|
|||
export ANDROID_NDK=${ANDROID_HOME}/ndk/21.1.6352462
|
||||
export ANDROID_NDK_ROOT=${ANDROID_NDK}
|
||||
export PATH=${PATH}:${ANDROID_HOME}/tools:${ANDROID_HOME}/platform-tools:${ANDROID_NDK}:${JAVA_HOME}/bin
|
||||
ANDROID_ABI="arm64-v8a armeabi-v7a x86_64" python3 build-plugin.py --projects=GreenScreen --distribution=android
|
||||
ANDROID_ABI="<android-architecture-separate-by-space>" python3 build-plugin.py --projects=GreenScreen --distribution=android
|
||||
```
|
||||
|
||||
The GreenScreen.jpl file will be available under `<ring-project/plugins/build/>`.
|
||||
|
|
@ -0,0 +1,25 @@
|
|||
# How to publish?
|
||||
The Plugin Store is a system to distribute plugins over a server. The organisation needs to be eligible to publish a plugin( more details [here](jami-plugins-certificate.md)). To upload a plugin, the new plugin must pass a verification process.
|
||||
|
||||
## Plugins Store Requests
|
||||
|
||||
All the requests from an organization that wants to be elegible to or to upload a new plugin in
|
||||
the plugins store is done with an Issue request template mechanism on [Github](https://github.com/savoirfairelinux/jami-plugins-store-requests).
|
||||
Two types of request are possible: requests to authorize a new organization to upload new plugins and requests to upload plugins.
|
||||
|
||||
### Authorize a new organization
|
||||
|
||||
The organization must deliver a Certificate Signing Request (CSR).
|
||||
The CSR can be generated with our tool from the [SDK](https://review.jami.net/plugins/gitiles/jami-plugins) or
|
||||
with openssl tool.
|
||||
The certificate authority must review the organization request and sign the certificate with the SDK tool.
|
||||
The certificate authority must send the certificate to the organization.
|
||||
Before sending it, a job runs to verify that the certificate is correctly signed.
|
||||
The certificate authority reserves the right to refuse the certificate request or revoke the certificate. Upon sending a certificate signing request, the organization agrees to the terms of
|
||||
use of the certificate authority. If the private key of the organization is compromised, the organization must revoke the certificate and generate a new certificate and notify the certificate authority.
|
||||
|
||||
### Upload a new plugin
|
||||
|
||||
The organization must deliver the plugin signed correctly with the certificate of the organization.
|
||||
A job is triggered to check the signature of the plugin and the certificate are correct.
|
||||
The Jami team is not responsible for the plugin content.
|
||||
|
Before Width: | Height: | Size: 146 KiB After Width: | Height: | Size: 146 KiB |
|
Before Width: | Height: | Size: 31 KiB After Width: | Height: | Size: 31 KiB |
|
Before Width: | Height: | Size: 93 KiB After Width: | Height: | Size: 93 KiB |
|
Before Width: | Height: | Size: 274 KiB After Width: | Height: | Size: 274 KiB |
|
Before Width: | Height: | Size: 234 KiB After Width: | Height: | Size: 234 KiB |
|
Before Width: | Height: | Size: 254 KiB After Width: | Height: | Size: 254 KiB |
|
Before Width: | Height: | Size: 833 KiB After Width: | Height: | Size: 833 KiB |
|
Before Width: | Height: | Size: 436 KiB After Width: | Height: | Size: 436 KiB |
|
Before Width: | Height: | Size: 347 KiB After Width: | Height: | Size: 347 KiB |
|
Before Width: | Height: | Size: 31 KiB After Width: | Height: | Size: 31 KiB |
|
|
@ -0,0 +1,15 @@
|
|||
###########
|
||||
Jami Extensions
|
||||
###########
|
||||
|
||||
The Jami Extensions manual is a reference for Jami Extensions developers, documenting
|
||||
the various aspects of the Jami Extensions API and architecture.
|
||||
|
||||
.. tooctree::
|
||||
:maxdepth: 1
|
||||
|
||||
how-it-works
|
||||
how-to-build
|
||||
how-to-publish
|
||||
jami-plugins-certificates
|
||||
jami-store-jami-client
|
||||
|
|
@ -20,29 +20,6 @@ Every file and archive is signed by a plugin certificate. The certificate is abl
|
|||
The certificate lifecycle management is designed in the Opendht library.
|
||||
The certification revocation is also managed by the Opendht library. If you want to know more about the Opendht library check [OpenDHT Page](https://opendht.net).
|
||||
|
||||
## Plugins Store Requests
|
||||
|
||||
All the requests from an organization that wants to be elegible to or to upload a new plugin in
|
||||
the plugins store is done with an Issue request template mechanism on [Github](https://github.com/savoirfairelinux/jami-plugins-store-requests).
|
||||
Two types of request are possible: requests to authorize a new organization to upload new plugins and requests to upload plugins.
|
||||
|
||||
### Authorize a new organization
|
||||
|
||||
The organization must deliver a Certificate Signing Request (CSR).
|
||||
The CSR can be generated with our tool from the [SDK](https://git.jami.net/savoirfairelinux/jami-plugins) or
|
||||
with openssl tool.
|
||||
The certificate authority must review the organization request and sign the certificate with the SDK tool.
|
||||
The certificate authority must send the certificate to the organization.
|
||||
Before sending it, a job runs to verify that the certificate is correctly signed.
|
||||
The certificate authority reserves the right to refuse the certificate request or revoke the certificate. Upon sending a certificate signing request, the organization agrees to the terms of
|
||||
use of the certificate authority. If the private key of the organization is compromised, the organization must revoke the certificate and generate a new certificate and notify the certificate authority.
|
||||
|
||||
### Upload a new plugin
|
||||
|
||||
The organization must deliver the plugin signed correctly with the certificate of the organization.
|
||||
A job is triggered to check the signature of the plugin and the certificate are correct.
|
||||
The Jami team is not responsible for the plugin content.
|
||||
|
||||
## Signature mechanism
|
||||
|
||||
A signature is a hash generate by a subject certificate private key. This signature can be verified with the public key of the subject. As an organization, a certificate has to be signed by the root certificate
|
||||
|
|
@ -78,7 +55,7 @@ The signature file contains the signature of the signatures file.
|
|||
The Store Root C.A. is the certificate authority. This certificate has the responsibility to sign the Organization Certificate. The Organization Certificate is the certificate of the organization.
|
||||
This certificate has the responsibility to sign the Plugins Certificate. The Plugins Certificate is the certificate of the plugin. This certificate has the responsibility to sign the plugin.
|
||||
|
||||
|
||||
|
||||
|
||||
In this example, the certificate authority trust SFL and Foo organizations.
|
||||
Each organization trusts theirs plugin certificate.
|
||||
|
|
@ -1,9 +1,9 @@
|
|||
**NOTE: This page describes how the Jami plugin store works and how to use it.**
|
||||
|
||||
# Jami Plugin Store in the Client
|
||||
The Jami team has introduced plugins as a call/chat feature for GNU/Linux, Windows, and Android, providing users with the ability to personalize their call/chat experience. If you want to learn more about Jami plugins, please refer to the [Jami Plugins Page](./jami-plugins.md).
|
||||
The Jami team has introduced plugins as a call/chat feature for GNU/Linux, Windows, and Android, providing users with the ability to personalize their call/chat experience. If you want to learn more about Jami plugins, please refer to the [Jami Plugins Page](how-it-work.md).
|
||||
|
||||
To enhance the plugin ecosystem, the Jami team has decided to include a Jami plugin store where they can distribute both their own plugins and third-party plugins. However, before a third party can add a plugin to the store, they must request a certificate signed by the certificate authority. For more information about plugin store certificates, please see [Jami Plugins Certification](./jami-plugins-certificate.md).
|
||||
To enhance the plugin ecosystem, the Jami team has decided to include a Jami plugin store where they can distribute both their own plugins and third-party plugins. However, before a third party can add a plugin to the store, they must request a certificate signed by the certificate authority. For more information about plugin store certificates, please see [Jami Plugins Certification](jami-plugins-certificate.md).
|
||||
|
||||
The Jami daemon and client are responsible for verifying, installing, and managing new plugins in the device's environment. Both the daemon and the client fetch new plugins, display them, verify their authenticity, and install them. Jami already has the capability to display and install plugins.
|
||||
|
||||
|
|
@ -35,7 +35,7 @@ If the client decides to download a plugin, it sends a request to the plugin sto
|
|||
|
||||
If the certificate and file signatures are verified correctly, the daemon proceeds with the installation. If not, the plugin archive is removed, and an error message is returned to the client.
|
||||
|
||||
|
||||
|
||||
|
||||
### Plugin State
|
||||
|
||||
|
|
@ -44,8 +44,8 @@ When a user wants to install a plugin, a visual trigger should be displayed to s
|
|||
The plugin can be in one of three states: installed, installing, or downloading. To handle all of these cases, a state machine is used to represent all the states of the plugin and the transitions between them.
|
||||
The network manager and update manager for Windows have been refactored to handle all the states of the plugin during the download process.
|
||||
|
||||
|
||||
|
||||
|
||||
A routine is triggered each days to check if a plugin is not up to date. The plugin version manager allow us to check and update if necessary plugins.
|
||||
|
||||
|
||||
|
||||
Loading…
Reference in New Issue