A *swarm* is a group able to discuss without any central authority in a resilient way. Indeed, if two person doesn't have any connectivity with the rest of the group (ie Internet outage) but they can contact each other (in a LAN for example or in a subnetwork), they will be able to send messages to each other and then, will be able to sync with the rest of the group when it's possible.
4. Bob announces to his other devices that he creates a new conversation. This is done via an invite to join the swarm sent through the DHT to other devices linked to that account.
5. To validate that Alice is a member, she removes the invite from `/invited` directory, then adds her certificate into the `/members` directory
6. Once all commits validated and on her device, other members of the group are discovered by Alice. with these peers, she will construct the **DRT** (explained below) with Bob as a bootstrap.
and adds her device and CRL to the repository if missing (others must be able to verify the commit). Merge conflicts are avoided because we are mostly based on commit messages, not files (unless CRLS + certificates but they are located). then she announces the new commit via the **DRT** with a service message (explained later) and pings the DHT for mobile devices (they must receive a push notification).
For pinging other devices, the sender sends to other members a SIP message with mimetype = "application/im-gitmessage-id" containing a JSON with the "deviceId" which sends the message, the "id" of the conversation related, and the "commit"
To avoid users pushing some unwanted commits (with conflicts, false messages, etc), this is how each commit (from the oldest to the newest one) MUST be validated before merging a remote branch:
+ For each commits, check that the device that tries to send the commit is authorized at this moment and that the certificates are present (in /devices for the device, and in /members or /admins for the issuer).
2. Conflicts with banned devices. If multiple admin devices are present and if Alice can speak with Bob but not Denys and Carla; Carla can speak with Denys; Denys bans Alice, Alice bans Denys, what will be the state when the 4 members will merge the conversations.
3. A device can be compromised, stolen or its certificate can expire. We should be able to ban a device and avoid that it lies about its expiration or send messages in the past (by changing its certificate or the timestamp of its commit).
This is the only part that MUST have a consensus to avoid conversation's split, like if two members kick each other from the conversation, what will see the third one?
This is needed to detect revoked devices, or simply avoid getting unwanted people present in a public room. The process is pretty similar between a member and a device:
+ First, she votes for banning Bob. To do that, she creates the file in /votes/ban/members/uri_bob/uri_alice (members can be replaced by devices for a device, or invited for invites or admins for admins) and commits
+ If the vote is resolved, files into /votes/ban can be removed, all files for Bob in /members, /admins, /invited, /CRLs, /devices can be removed (or only in /devices if it's a device that is banned) and Bob's certificate can be placed into /banned/members/bob_uri.crt (or /banned/devices/uri.crt if a device is banned) and committed to the repo
+ Fir she votes for unbanning Bob. To do that, she creates the file in /votes/unban/members/uri_bob/uri_alice (members can be replaced by devices for a device, or invited for invites or admins for admins) and commits
+ Then she checks if the vote is resolved. This means that >50% of the admins agree to ban Bob (if she is alone, it's sure it's more than 50%).
+ If the vote is resolved, files into /votes/unban can be removed, all files for Bob in /members, /admins, /invited, /CRLs, can be re-added (or only in /devices if it's a device that is unbanned) and committed to the repo
a. If no other member in the conversation we can immediately remove the repository
b. If still other members, commit that we leave the conversation, and now wait that at least another device sync this message. This avoids the fact that other members will still detect the user as a valid member and still sends new message notifications.
The goal here is to keep the old API (addContact/removeContact, sendTrustRequest/acceptTrustRequest/discardTrustRequest) to generate swarm with a peer and its contact. This still implies some changes that we cannot ignore:
2. TrustRequest are retried when contact come backs online. It's not the case today (as we don't want to generate a new TrustRequest if the peer discard the first). So, if an account receives a trust request, it will be automatically ignored if the request with a related conversation is declined (as convRequests are synched)
removeContact() will remove the contact and related 1:1 conversations (with the same process as "Remove a conversation"). The only note here is that if we ban a contact, we don't wait for sync, we just remove all related files.
In this case, two conversations are generated. We don't want to remove messages from users or choose one conversation here. So, sometimes two 1:1 swarm between the same members will be shown. It will generate some bugs during the transition time (as we don't want to break API, the inferred conversation will be one of the two shown conversations, but for now it's "ok-ish", will be fixed when clients will fully handle conversationId for all APIs (calls, file transfer, etc)).
After accepting a conversation's request, there is a time the daemon needs to retrieve the distant repository. During this time, clients MUST show a syncing view to give informations to the user.
Note, while syncing:
+ ConfigurationManager::getConversations() will return the conversation's id even while syncing
+ ConfigurationManager::conversationInfos() will return {{"syncing": "true"}} if syncing.
+ ConfigurationManager::getConversationMembers() will return a map of two URIs (the current account and the peer who sent the request)
To be identifiable, a conversation generally needs some metadata, like a title (eg: Jami), a description (eg: some links, what is the project, etc), and an image (the logo of the project). Those metadata are optional but shared across all members, so need to be synced and incorporated in the requests.
To update the vCard, a user with enough permissions (by default: =ADMIN) needs to edit `/profile.vcf`. and will commit the file with the mimetype `application/update-profile`. The new message is sent via the same mechanism and all peers will receive the **MessageReceived** signal from the daemon. The branch is dropped if the commit contains other files or too big or if done by a non-authorized member (by default: <ADMIN).
In the synchronized data, each devices sends to other devices the state of the conversations. In this state, the last displayed is sent. However, because each device can have its own state for each conversation, and probably without the same last commit at some point, there is several scenarios to take into account:
5 scenarios are supported:
+ if the last displayed sent by other devices is the same as the current one, there is nothing to do.
+ if there is no last displayed for the current device, the remote displayed message is used.
+ if the remote last displayed is not present in the repo, it means that the commit will be fetched later, so cache the result
+ if the remote is already fetched, we check that the local last displayed is before in the history to replace it
+ Finally if a message is announced from the same author, it means that we need to update the last displayed.
Because two admins can change the description at the same time, a merge conflict can occur on `profile.vcf`. In this case, the commit with the higher hash (eg ffffff > 000000) will be chosen.
The archive MUST contain conversationId to be able to retrieve conversations on new commits after a re-import (because there is no invite at this point). If a commit comes for a conversation not present there are two possibilities:
+ The conversationId is missing, so the daemon asks (via a message `{{"application/invite", conversationId}}`) a new invite that the user needs to (re)accepts
1. We need to sync and order messages. The Merkle Tree is the perfect structure to do that and can be linearized by merging branches. Moreover, because it's massively used by Git, it's easy to sync between devices.
However, non-permanent messages (like messages readable only for some minutes) can be sent via a special message via the DRT (like Typing or Read notifications).
Swarm massively changes file transfer. Now, all the history is syncing, allowing all devices in the conversation to easily retrieve old files. This changes allow us to move from a logic where the sender pushed the file on other devices, via trying to connect to their devices (This was bad because not really resistant to connections changes/failures and needed a manual retry) to a logic where the sender allow other devices to download. Moreover, any device having the file can be the host for other devices, allowing to retrieve files even if the sender is not there.
The sender adds a new commit in the conversation with the following format:
```
value["tid"] = "RANDOMID";
value["displayName"] = "DISPLAYNAME";
value["totalSize"] = "SIZE OF THE FILE";
value["sha3sum"] = "SHA3SUM OF THE FILE";
value["type"] = "application/data-transfer+json";
```
and creates a link in `${data_path}/conversation_data/${conversation_id}/${file_id}` where `file_id=${commitid}_${value["tid"]}.${extension}`
Then, the receiver can now download the files by contacting the devices hosting the file by opening a channel with `name="data-transfer://" + conversationId + "/" + currentDeviceId() + "/" + fileId` and store the info that the file is waiting in `${data_path}/conversation_data/${conversation_id}/waiting`
The device receiving the connection will accepts the channel by verifying if the file can be sent (if sha3sum is correct and if file exists). The receiver will keep the first opened channel, close the others and write into a file (with the same path as the sender: `${data_path}/conversation_data/${conversation_id}/${file_id}`) all incoming data.
When the transfer is finished or the channel closed, the sha3sum is verified to validate that the file is correct (else it's deleted). If valid, the file will be removed from the waiting.
In case of failure, when a device of the conversation will be back online, we will ask for all waiting files by the same way.
Git operations, control messages, files, and other things will use a p2p TLS v1.3 link with only ciphers which guaranty PFS. So each key is renegotiated for each new connexion.
The DRT is a new concept used in swarm to maintain p2p connections. Indeed, group members define a graph of nodes (identified by a hash) en must be connected.
1. Each node has a connection to the next node. So we only need $N$ connections, but it's not effective to transmit a message, because the message will go through all peers, one by one.
2. Each node is connected to all other nodes $N\timesN$ connections. Effective to transmit, but need more resources **WILL BE CHOSEN FOR THE FIRST VERSION**
Note: to optimize the socket numbers, a socket will be given by a **ConnectionManager** to get multiplexed sockets with a given hash. This means that if we need to transmit several files and chat with someone, only one socket will be used.
b. Not connected, so will never receive the request cause Alice must not know if Bob just ignored or blocked Alice. The only way is to regenerate a new invite via a new message (cf next scenario)
2. Alice gets a message received (from herself) if successful
3. Two possibilities, alice and bob are connected, or not. In both case a message is crafted: { "application/im-gitmessage-id" : "{"id":"$convId", "commit":"$commitId", "deviceId": "$alice_device_hash"}"}.
c. Bob doesn't know that conversation. Ask through the DHT to get an invite first to be able to accept that conversation ({"application/invite", conversationId})
For a serious group chat feature, we also need serious crypto. With the current design, if a certificate is stolen as the previous DHT values of a conversation, the conversation can be decrypted. Maybe we need to go to something like **Double ratchet**.
1. Something like a Mattermost in a company, with private channels, and some roles (admin/spectator/bot/etc) or for educations (where only a few are active).
2. Horizontal conversations like a conversation between friends.
+ Persistence: Actually, a message on the DHT lives only 10 minutes. Because it's the best timing calculated for this kind of DHT. To persist data, the node must re-put the value on the DHT every 10 minutes. Another way to do when the node is offline is to let nodes re-put the data. But, if after 10 minutes, 8 nodes are still here, they will do 64 requests (and it's exponential). The current way to avoid spamming for that is queried. This will still do 64 requests but limit the max redundancy to 8 nodes.
Currently, the file transfer algorithm is based on a TURN connection (See https://git.ring.cx/savoirfairelinux/ring-project/wikis/tutorials/File-transfer). In the case of a big group, this will be bad. We first need a p2p implement for the file transfer. Implement the RFC for p2p transfer.
