Congregation P2P network
A decentralised, trust-less P2P network.
Abstract This document outlines the decentralised "Congregation" peer to peer (P2P) network (CNET) and its protocol. The Congregation network is a decentralised, trust-less peer-to-peer network which acts as a base for more complex peer-to-peer applications. It makes up a minimal base protocol that all applications building on top of it have to speak. It is intended to serve as a communication layer for different kinds of decentralised applications. All platforms running atop of it are part of the same larger network, but reside in smaller sub-networks for the individual platforms. It natively supports the creation of sub-networks within platform networks so that applications can control their network topology for efficient communication.
Introduction
Free speech Currently, there is a world-wide, satanic crusade against freedom of speech, truth, and justice. The synagogue of Satan is thoroughly entrenched in all governments, newspapers, and universities. Through their positions of power, they enforced a culture of "political correctness", which allows only a single, "approved" satanic standpoint on all controversial topics. It mandates new ways of speech and anyone who behaves politically incorrectly might have his life and reputation destroyed by a sub-human SJW outrage mob. Many platforms that are aimed to provide free speech are routinely shut down by their providers. Decentralised platforms cannot easily be shut down, which makes them more reliable.
Trust Centrally operated platforms are usually at the control of a single person or small group of people. Users have to trust the owners to faithfully operate the site and not censor them or shut them down. Decentralised, trust-less systems are not perated by a single entity, and no specific party is required to operate faithfully for such a platform to function. As CNET is truly decentralised, there is no hierarchy of nodes, and no node in the system has any special authority.
Motivation As I was planning one of my future decentralised software projects, I realised a powerful, custom P2P network would be needed. Instead of writing a specialised P2P network that is tailored to only fit that individual project, I came up with a P2P network that fulfills all requirements while still keeping it as general as possible. This way, it can be reused for future projects.
Goal CNET is designed to be highly customisable, so that it can be reused for future projects which will also require a custom P2P network, but also interoperable, so that different platforms running atop of it can easily connect to each other. The aim of the platform is to create a versatile P2P network that can run many platforms which require trust-less decentralisation. These platforms should all run on a single global network to make them more robust and allow them to easily connect to each other. The name "Congregation" is referring to the Bible: the congregation/church is also a decentralised entity.
For where two or three are gathered together in my name, there am I in the midst of them.
— Matthew 18:20 (KJV)
It is intended to allow God's children to freely gather online, where the synagogue of Satan cannot shut their platform down.
Network architecture
CNET forms a mesh network where every node is directly or indirectly connected to every other node. However, this mesh network is organised in such a way that it forms a tree of sub-networks at the same time. It combines the flat hierarchy model of a normal peer to peer network for connectivity and simplicity with the tree hierarchy model for efficiency and isolation and scalability. These two models exist simultaneously.
Topic sub-networks A topic is, as the name suggests, a specific topic that nodes can talk about. An example for a topic would be a service (i.e., forum), or a topic within a service (i.e., board). In practice, a topic is represented as a hash digest. For each topic, a sub-network is created, which is formed by all nodes subscribed to it. Communication concerning a topic is entirely contained within the topic's sub-network. This makes topic networks separate from each other, which improves communication efficiency and scalability.
Network tree The network tree is a tree of networks within the CNET's global network, whose leaves are the topic sub-networks. It is a fully expanded prefix tree where the keys are the topic hashes, so each topic network has a deterministic position in the tree based on its name. Each node in the topic tree is a separate sub-network which contains all its children's networks, but not its siblings'. The topic tree provides an efficient mechanism for looking up and connecting to individual topics. Via the tree's root, which contains all topics, each peer is indirectly connected to each other peer, and can look up every topic sub-network. CNET's global network forms the root network of the network tree. Non-leaf networks in the network tree belong to CNET only, and no platform running atop CNET can send messages over them. They exist only to allow nodes to find and connect to the (topic) sub-networks they contain.
Connectivity Every node in a sub-network is connected to at least two other peers within the same sub-network, if possible. This means that within every sub-network, all participants are directly or indirectly interconnected. This allows logarithmic time broadcasts through all networks in CNET and prevents network partitions, which would cause separate, parallel instances of a sub-network to exist, which must never happen, as platforms running on CNET would effectively branch into inconsistent versions.
Peer identities
To achieve strong anonymity, in CNET, peers have no native identities. Platforms can, however, introduce identities very easily within their sub-networks, and even make them reusable in compatible platforms. Natively, a peer is only distinguished by its network address (i.e., IP address).
For spam protection purposes, however, some kind of identity might be added, where each identity is bound to some expensive to generate Proof of Space plot. In that case, identities would be semi-transient and only exist to keep track of bad actors. Nodes would not publish their identity to each other, but only sign broadcast messages with their identity to make it easier to filter out spam (since generating valid identities is not something that can be done frequently due to the expensiveness of the chosen Proof of Space algorithm). So, even if it were to be added, this spam protection identity would not be a strong identity in the sense of permanence, and could regularly be replaced to prevent finger printing. However, in the beginning, this will not be supported.
Communication
In more complex decentralised P2P systems, it is very common to have many asynchronous communication protocol executions at the same time. Because all messages that are sent are part of some protocol execution (or start a new protocol), the CNET protocol dedicates a virtual connection to each context. These virtual connections are multiplexed over the single physical connection that exists between two peers. This guarantees liveness even when many protocols are executed at once in parallel. All communication belongs into some context. When a new context (i.e., new instance of some protocol) is created, a virtual connection can be created for that context. Contexts are always bound to a single topic sub-network. This feature allows the multiplexing of parallel protocol executions over a single physical connection without interference.
Multiplexing/Virtual connections Over a single physical connection, multiple virtual connections are multiplexed. For each sub-network that two peers share there is a virtual connection between them. All messages that are sent to another peer always belong to a single topic sub-network. Additionally, messages themselves are also multiplexed over their sub-network's virtual connection, so that large messages do not stop all other messages from coming through.
Rate limiting Different message categories can be configured to have different priorities, and nodes can be set to allocate certain amounts of their bandwidth to certain categories of messages. This allows that even if there is spam over certain channels, other channels will still achieve a minimum threshold of throughput. Of course, if there are less messages of a certain category than is allotted for them, then that space is made available for other messages.
Packets In CNET, physical connections transport packets (encoded into a byte stream) which have raw data as payload. A packet is different from a message since it does not know anything about the semantic structure of its payload. Packets are needed for multiplexing and controlling virtual connections. There are multiple types of packets:
OPEN
packets create a new virtual connection. They contain no payload other than the new virtual connection's ID.DATA
packets contain a block of raw data for a specific virtual connection.CLOSE
packets close a virtual connection. This is used to terminate the execution of an ongoing protocol instance.MESSAGE
packets open a new virtual connection, send data, and close the connection afterwards.
Messages
Generally, there are two types of messages:
Broadcast messages (BMs) are sent to all peers within a topic network. A peer who receives a broadcast message will relay it to all his other peers in the same sub-network. Peers do not relay the same message twice. Broadcast messages are public and concern everyone in the network. All the broadcast protocols can be used for broadcast messages.
Probabilistic message routing To increase the efficiency of broadcast message routing, broadcast messages start to get relayed before they are even completely received and checked for validity. This is important for large broadcasts, such as streams, and reduces the latency especially. The probability with which this behaviour is chosen depends on whether the message's sender has a good track record of sending valid messages and the current rate of spam in the network. In case of high levels of spam in the network, all messages are checked before they are relayed. This limits performance, but reduces bandwidth.
Direct messages (DMs) are not relayed by the recipient to other peers. Like BMs, they belong to a topic. Direct messages are private and only concern the recipient. DMs can only be sent if there is a direct connection to the recipient.
Whether a message is a broadcast message or direct message is defined by its type.
Furthermore, messages can be divided into:
Domain-specific messages are message types that are implemented by the individual platforms that run on the network. They are only valid within their sub-networks.
Native messages are special messages that belong to the CNET protocol. Some of them are bound to a single sub-network, while others affect all sub-networks at once. Examples for native messages are those that are part of the peer discovery or sub-network discovery protocols.
Anonymity
CNET, by default, has no native anonymity layer, and this is done for several reasons:
- There are many anonymity services which are incompatible with each other, so choosing any specific one would result in a technological lock-in.
- Anonymity layers are always limited in reliability, throughput and latency and greatly limit the scope of possible platforms that can run on CNET.
- Custom anonymity services can be built using CNET, and then be used to run anonymised platforms.
Protocols
Primitives
As CNET aims to be a communication platform to build platforms on, it provides communication primitives that can be used to form custom protocols. Ideally, platforms will not have to implement any communication primitives on their own.
Broadcast This is a simple broadcast protocol which floods the whole network. Peers relay the message to all their peers, but detect and ignore duplicates.
ON msg FROM sender:
IF msg IN this.cache
RETURN
ADD(msg) TO(this.cache)
FOR p IN this.peers - sender
SEND(msg) TO(p)
Abortable Broadcast This is a two-phase protocol which performs a slowed down broadcast which can be limited in reach by a follow-up signal. It is used to communicate with a specific subset of nodes which are at unknown locations in the network. It is intended for interactive protocols (i.e., the intended recipients interact with the original sender). It reduces the network's load compared to a normal broadcast by locally containing the message and stopping its propagation after the intended recipients have been reached and the original sender has been contacted. First, the creator of a message sends it to all his peers. The peers react to the message, but do not relay it until after some timeout (i.e. 1.5s) which guarantees that another message sent afterwards will arrive before they finish waiting. The creator can send an abort message at any time, which will prevent the peers from relaying the message. If a peer already relayed the message, then he will also relay the abort message. Since at every hop, the message is artificially delayed for some time, the abort message can quickly catch up.
ON msg = (id, msg') FROM sender:
IF id IN this.broadcastBuffer OR msg IN this.cache
RETURN
ADD(id) TO(this.broadcastBuffer)
ADD(msg) TO(this.cache)
WAIT(1.5s) // Or any other "long" time.
IF id IN this.broadcastBuffer
ADD(msg) TO(this.sentBroadcasts)
FOR p IN this.peers - sender
SEND(msg) TO(p)
ON msg = (id) FROM sender:
IF id IN this.broadcastBuffer
REMOVE(id) FROM(this.broadcastBuffer)
IF id IN this.sentBroadcasts
FOR p IN this.peers - sender
SEND(msg) TO(p)
Fixed broadcast This is a special broadcast that sends messages to a random set of up to N peers that are at most M hops away. When a normal or even abortable broadcast are not optimal due to the number of potential recipients, but it is still intended to reach random unknown peers, this type of broadcast is preferred. The below pseudocode is simplified to not include duplicate route handling.
ON msg = (msg', peers, hops) FROM sender:
IF msg.hops == 0 OR this.peers - sender IS_EMPTY
HANDLE(msg')
ELSE
split = MIN(SIZE(this.peers - sender), req.peers)
FOR p IN RANDOM_SAMPLE (split) OUT_OF(this.peers - sender)
SEND((msg', req.peers/split, req.hops-1)) TO(p) // adjust for modulo.
Backtracing The broadcast algorithms can easily be expanded so that each relaying peer remembers from which peer he received the message first. The recipient can then send a reply message that refers to the broadcast message, and sends it back to the first (or first couple) peer he received it from. The peer then relays the message backwards along the path he received the initial message from. This could be used to create tunneled communication channels in case the recipient cannot create a direct connection to the sender or other reasons.
Native protocols
Since CNET has a complex network structure, it requires native protocols for organising itself.
Joining the CNET/discovering peers When a node joins the network, it connects to one or two previously known entry points, which are just nodes whose IP is known outside the network. The new node requests a number of peers, and the entry points refers it to other peers it is connected to, as well as peers it is only indirectly connected to. It is recommended that every node connects to at least 4 peers that come from at least two random entry nodes for strong interconnectedness. The peer discovery protocol uses a fixed broadcast of the sender's IP address, so that the recipients can connect to the sender. The limited broadcast limits the scope and the number of recipients, so that the sender does not receive an exponential amount of connection requests.
Joining a topic sub-network When joining a sub-network, the node sends a connection request via an abortable broadcast within the parent network. The node tries to connecto to multiple peers in each network, if there are any. After connecting to enough peers, the broadcast of the connection request is aborted. This means that when joining a leaf network in the topic network tree, a node has to connect to all parent networks first. Although this does not seem necessary at first, it is required for the network to stay highly interconnected and organised and prevent network partitions.