Hacker News new | past | comments | ask | show | jobs | submit login

> It’d be a lot nicer place if projects like this would be open, so people could use it on their own systems

Check Feathers.js if you want realtime on your own system.




I have, sadly it doesn’t really scale well enough.

I work on Quasseldroid, an android client for the quassel IRC bouncer, and the idea is that you can always access all messages instantly, while the client only buffers a few of them.

This works quite well, and is easy to do with Feathers.js and others, even Firebase...

...if there wasn’t the issue that the average user is in hundreds of channels and gets tenthousands of messages per hour, in some cases, even thousands of messages a second (if the user is in a channel with > 10k other users, for example, and everyone is chatting – such as during eurovision on Quakenet’s #eurovision).

Although users self-host, so we don’t have infrastructure costs, they usually do so on a raspberry pi – and Feathers.js can’t easily handle thousands of changes per minute to its database while running on a raspberry pi, and accurately sync them to multiple clients.


Oh wow.

So how did you/are you solving this?


Well, while I only work on the Android client, the solution currently in use is a large codebase of code specifically written for this single purpose. In C++. Compiled natively. That’s the main reason why good performance with the system is even possible.

A huge performance boost (reconnection times down from ~2 minutes to ~12 seconds on 64kbps 2G network, using a Nexus 5X) could be accomplished by using Java NIO with non-copying IO for parsing the custom binary protocol, but we’re looking into using flat protobufs for improving performance further.

But then the database layer becomes the bottleneck. SQLite is far too slow to be normally usable, most users use PostgreSQL for the backing database.

Basically, the trick is in "don’t use JSON and JS, do stuff with native code and binary protocols to reduce overhead".

But this makes maintenance basically impossible, and isn’t really ideal either.

DISCLAIMER: I do not speak for the Quassel project, all opinions represented here are solely my own.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: