Fedor Indutny's Blog

HyperBloom

Over this weekend I got not so original (but definitely a fun one) idea to build fully distributed and decentralized Twitter. At the time it was inspired by the DAT Project and Hypercore, neither of which could support public replies to user feeds.

Hence, the most natural thing was to write a new protocol! Say hello to HyperBloom!

Protocol #

It is crucial to understand the needs for the protocol before discussing the protocol itself. Let me list few requirements for it:

How could it combine all these three qualities into one protocol? By combining existing solutions, of course:

Trust Network #

Having grow-only set that is writable by anyone on the web is the best way to introduce enormous amount of spam into social network. This can be tackled by accepting writes only from your friends. However, this kills the virality of the platform.

Perhaps friends-of-friends should be allowed to append to that Set? Better!

The way HyperBloom addresses this is by letting author's issue so called Trust Links. Each Trust Link acts like an edge in the Graph: A -> B or A trusts B. When two peers connect to synchronize the values in a Set, they each present a collection of links from the author of the Set to the peers themselves. Each successive link in such collection is a continuation of the previous link: A -> B, B -> C, C -> D. With a total limit of 5 links in one collection (chain).

The limit is imposed to save the bandwidth. To further save it peers help each other by automatically issuing links that create shorter path to the author.

Example:

  1. Peer B has following chain: A -> B
  2. Peer C has following chain: A -> D, D -> E, E -> C
  3. Peer B sends B -> C to C to minimize the route
  4. Peer C uses A -> B, B -> C later on

Each link has an expiration time, and such automatic link as in the example will have the expiration time set to: minimum(A -> D, D -> E, E -> C). Essentially, A always controls how often it wants to refresh its peers' trust.

Set #

Set is not particularly interesting. It borrows some design decisions from Bitcoin SPV client. In particular the use of Bloom filters to optimally compute the difference between the peers and set the missing values.

Usage #

I'm combining both Hypercore and HyperBloom into a project called (for no particular reason. May be Hyper-uni-corn?) HyperCorn. HyperCorn uses HyperCore logs for JSON message fields, and HyperBloom for notifying authors about replies to their feeds.

HyperBloom Trust Links are stored and distributed through HyperCore. So far it appears to be working, but it has pretty long way to go still. Mainly it needs an UI. Contact me, if you are interested!

Open Questions #

I'd love to hear your opinion.

(You can reply on twitter)

Thanks for reading.