Hacker News new | past | comments | ask | show | jobs | submit login
Building a Distributed Log from Scratch: Scaling Message Delivery (bravenewgeek.com)
130 points by tylertreat on Jan 10, 2018 | hide | past | favorite | 1 comment



> In Kafka and NATS Streaming, reads (and writes) are only served by the leader. Similarly, Amazon Kinesis supports up to only five reads per second per shard (a shard is Kinesis’ equivalent of a partition). Thus, if we have five consumers reading from the same shard, we’ve already hit our fan-out limit.

Long live DistributedLog. One of the great aspects of DistributedLog is the read scaling. Start reading from any replica. Even read proxy node caches records for all consumers. Launch as many read proxies as you need, since read proxies are stateless.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: