Take a look at pubsub on IPFS


news image

by Jeromy Johnson on 2017-05-17

We recently merged a simple, experimental pubsub implementation into IPFS. This
implementation is just a beginning. It is far from the performance and security goals
we will achieve in our long-term target. However, even this early implementation opens
the doors to several useful and interesting new applications.

In this post, I will point out some applications of this technology, show how to
get started started using ipfs pubsub, and discuss upcoming improvements.

Why pubsub?

called ‘pubsub’ for short, is a pattern often used to handle events in
large-scale networks. ‘Publishers’ send messages classified by topic or content and
‘subscribers’ receive only the messages they are interested in, all without direct
connections between publishers and subscribers. This approach offers much greater
network scalability and flexibility.

Some applications include collaborative document editing, “dynamic” website
content, chat applications, multiplayer games, continuously evolving datasets,
and webservice workers passing around messages. It gives us ways to make IPFS fast
for large-scale networks such as datacenters, local area networks, and large p2p
applications. In the near future, IPNS records will be pushed over pubsub, allowing
lightning fast updates of peers’ IPNS entries. Peers could use pubsub to track the
head of a merkle-linked global log.

Getting started with pubsub for go-ipfs

Note: There is also a js-ipfs implementation of pubsub. Documentation will come soon.

First, you’ll need to enable the pubsub code. Make sure you’re running go-ipfs 0.4.5 or
above. Once you have that version of ipfs installed, start the daemon with:

> ipfs daemon --enable-pubsub-experiment

This will tell ipfs to create and enable the pubsub service. It also implies
that you will only be able to use pubsub with other peers who choose to enable

To subscribe to the topic foo, run:

> ipfs pubsub sub foo

Now, any messages for the topic foo will print to your console.

To publish a message to the topic foo, open up another terminal and run:

> ipfs pubsub pub foo "hello world"

You should see “hello world” printed out in the first terminal. You can also
run the pub command on any other connected ipfs node and your node will
receive the message. Messages are routed through connected, subscribed peers.
This means that if peers A, B, and C are all subscribed to foo, A is connected
to B, and B is connected to C, but A is not directly connected to C, A will
still receive messages that C published to foo through B. This can be very
useful for routing messages in networks with poor NAT traversal or otherwise
suboptimal connectivity.

To see all peers with pubsub enabled, check the output of:

> ipfs pubsub peers

To see all the topics you are currently subscribed to, run:

> ipfs pubsub ls

Pubsub in the wild

As an example, we have integrated pubsub into Orbit.
This allows Orbit to provide a fully distributed, peer-to-peer chat without any
server anywhere. We are also actively working to put Conflict-Free Replicated Data
Types (CRDTs) on IPFS pubsub using libraries like Y.js and swarm.js.

Together, pubsub and CRDTs open new doors for collaborative editing of distributed
content. We are working with @edsilv and
@aeschylus to prepare a demo of IPFS in two
IIIF image viewers
#240. This will showcase how to
collaboratively annotate images from repositories dispersed around the world.
The demo will take place at the IIIF 2017 Conference in The Vatican.
We will publish a video of the demo, along with all of the code.

What’s next?

The next two areas of focus for IPFS pubsub are authentication and message routing.

Currently, any peer can publish to any pubsub topic. We plan to implement an
authenticated mode for pubsub topics, where only authorized peers — those given a
cryptographic key or capability — can publish messages. We are still working out
the sharing and capability granting model.

After that, we plan to improve message routing. The current routing algorithm
floods messages to every subscriber, resulting in some peers receiving the same
message multiple times. We affectionately call this approach “floodsub”. We plan
to replace it with a more efficient routing algorithm, which will go a long way
towards reducing overhead and improving scalability.

Please note that this is a simple first-blush implementation of the technology.
It has known limitations that we will address in future iterations. As it is
today, the pubsub implementation can be quite bandwidth intensive. It works well
for apps with few peers in the group, but does not scale. We have designed a more
robust underlying algorithm that will scale to much larger use cases but we wanted
to ship this simple implementation so you can begin using it for your applications.


All that said, we hope you give ipfs pubsub a try. You can head over to the
Discussion Forum to ask questions, get help,
or simply let us know how it goes.

Read More

This site uses Akismet to reduce spam. Learn how your comment data is processed.