Hacker News new | past | comments | ask | show | jobs | submit login
Kafka in Pieces (thebaffler.com)
86 points by apollinaire on Oct 12, 2020 | hide | past | favorite | 33 comments



Although it’s not considered his best writing by most, I enjoyed the castle more than any of his other books. I find its surrealistic depiction of bureaucracy so close to how one experiences bureaucracy in every day life. He must have been such an interesting person to chat with.


What a coincidence. When I was making game inspired by The Castle I also didn't finish it ;) On a more serious note, while the intro is epic, the rest is mostly internal monologues and very tedious dialogs so I don't think it can be done.


A game inspired by The Castle? Quite interesting! Could you share a bit more? I imagine it to be a kind of rogue-like but I might be totally wrong here :-)


3 screenshots: https://de.catbox.moe/61vfy6.jpg

(currently I'm rewriting engine and when I'm done, I spent time on something easier to do than Castle, probably Slayers)


Oh, an RPG, of course! Absolutely loving that sketch by the way.


I wonder if the reason why Kafka wanted his work to be burned was because he didn't want to give people in power the idea that the world could be taken in such a dark direction.

It certainly feels like the people who run the world these days read a lot of Kafka... What reads like a horror story to one person reads like a fairy tale to another.


One of the most powerful pieces of art I've ever personally experienced was just a stark sign with black-on-white lettering that read:

WHILE THE PEOPLE SLEEP, THE STATE DREAMS


Gives me the chills just hearing about it.


Kafka is best read like anything in the Cyberpunk genre - as a warning, not a suggestion.


The collective mind lacks imagination. In the absence of good ideas to choose from, it will gravitate towards bad ideas.


I've wondered if Kafka would have the same reputation had he finished his writings. I read Amerika, and though I enjoyed the book my strongest memory of it was the sudden shock I felt when I realized it was done and the last 40 pages were an afterword. It's hard to imagine an actual ending that produced such a lasting effect on me.


I got tricked :) This is not about Apache Kafka but about Bohemian novelist Franz Kafka. Nothing wrong, just saying my mind.


Not the first HN thread to have this happen: https://news.ycombinator.com/item?id=23599427

Come for the distributed, replicated, partitioned log. Stay for the existential angst, sinister bureaucracy, and paranoia in the face of the absurd.


Realistically, aren't they the same thing in the end? Can you have the former without truly feeling the latter?


The defining characteristics of distributed systems are random failure and unreliable communication. So yes. Sounds pretty Kafkaesque to me. (Or Byzantine.) Distributed consensus in particular strikes me as a problem quite worthy of The Castle or The Trial.


I consider it a fun little game to guess for each mention of "Kafka" in a HN title, whether it means the author or the software. Definitely not trivial, since this time, I would have guessed "Kafka in Pieces" to be an introduction to the software, component by component.


If there is something out there which is this introduction, would love to read it!


back in the day we combined the two and called it Usenet.


Does https://olduse.net still work for anyone else?


Sure. I have heard this now so many time here. To me however Kafka the software does resemble a Kafkaesque way: the literary style found in Kafka novels. Apache kafka is nightmarishly complex software which barely works. And I never fully grasped what I would use it for.


I think Kafka inherits too much of this "heinously complex" reputation from Zookeeper (which itself is also not heinously complex, just some people have an allergy to either JVM, ZAB or both).

Internally it's one of the simpler pieces of software I rely on. It's base API is small and effective and works "as you would expect". Replication uses the same fetch API consumers use, because why wouldn't it? Controller election uses tried and true ZK patterns, it's log storage is pretty simple, even the compaction logic is understandable.

To be fair I have spent a lot of time with it, patched it and it's pretty much the core of my toolbox but I don't think this is a controversial opinion among data infrastructure engineers. Compared to other software we work with Kafka is some of the simpler, dumber stuff which is refreshing.

On the other hand I really want to learn Pulsar but it definitely is more complex, my hope is that complexity pays off with big architectural advantages but we will see. :)

On the topic of what to use it for chances are if you don't know what it's for you probably don't have the problems it's meant to solve. I don't mean that in a derogatory way just that it's designed for large distributed architectures where many interested parties want to consume the same data. Or where a small number of very high throughput applications need a buffered transport that can take the load and spill to disk etc. I.e it is inherently niche, most companies don't have these problems.


We came for the gems but all we got is shards :)


It's named Kafka because LinkedIn's data access story was Kafkaesque and they wanted a way to wrangle it, so they wrote a tool to do that and named it Kafka.

It's good if you have a use for it.


Me too. Maybe this can be solved by updating the title as "Franz Kafka in Pieces".

I know, respecting the original title is a nice thing. On the other hand, HN is a tech oriented web site and +90% of the times if there is a Kafka in the title it is related to the stream processing framework, not the writer. So, just writing the writer's full name (in those rare occurances where the writer is mentioned here in this news site) can solve that tiny problem.

Those are my two cents...


That is one way. I think may be Kafka the software need to be called Apache Kafka here. Not sure if kafka the software will remain popular in next few years as cool developers seem already moving to Apache Pulsar, NATS and so on.

> +90% of the times if there is a Kafka in the title it is related to the stream processing framework, not the writer

This makes me think why these crappily implemented software choose such pretentious literary names. 'Linkedin Sprof' would be a suitable name for this below average software.


"The state in which Kafka left The Castle is representative of the condition of his entire oeuvre."


I wonder how Kafka would have felt about Apache Kafka preserving his namesake. What specifically about an event pipeline is kafkaesque?


Snippet from "Kafka: The Definitive Guide"

  People often ask how Kafka got its name and if it signifies anything specific about the application itself. Jay Kreps offered the following insight:
    I thought that since Kafka was a system optimized for writing, using a writer’s name would make sense. I had taken a lot of lit classes in college and liked Franz Kafka. Plus the name sounded cool for an open source project.
    So basically there is not much of a relationship.


Nice. A real answer. Always surprising whats gets dug up on hackernews.


Kafka famously left several novels unfinished... like the log, man... Just keeps going


I feel like it's origins in LinkedIn, the performances people are expected to put in there, and the fact that it runs on a virtual machine, often on a cloud machine make the irony too great to be believable.


May be event processing is stuck for hours/days with single log message "Processing Event..." with no further details :-)


"No other event could have been processed here, since this queue was created only for you. Now it will be closed."




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: