Can't watch the video right away, but I'll just say that every time I read the ZeroMQ guide (http://zguide.zeromq.org/page:all) I am frustrated by the lack of any clear technical explanation of ZeroMQ's basic architecture.
Today I gave it another shot, and after half an hour of trying to glean technical detail and skipping any parts that read like a like a comic book ("zap-pow-kaboom satori paradigm-shift moment"), I think I've learned the following:
* ZMQ works by spawning a background thread that runs a
poll() (or equivalent)-based async I/O loop. This thread
is created when you create a ZMQ "context."
* On top of a context you can create a ZMQ "socket", which
may map to N underlying UNIX sockets (or a comparable
transport), and which will transparently queue data to slow
or unavailable receivers.
* Over a socket you can send or receive "messages", which
are length-delimited strings which are always delivered
in full with the original length. Send/receive can be
either blocking or non-blocking.
* Contexts can be shared across threads, but sockets are
not thread-safe. Communication between application threads
and the ZMQ I/O thread is via a lock-free queue.
The main features seem to be topology-agnostic programming (since you don't have to know what a socket is connected to to send/receive over it) and transparent queuing. In my opinion transparent queues can be problematic because they transparently use up memory that can be hard to account for. Topology-agnostic programming certainly seems interesting, but in my experience of distributed systems programming I never seem to need or want complex messaging topologies. I guess I don't really get what all the hype is about. Maybe I should watch the video.
I don't think there's much to gain (at least right now) by using zmq in a simple single-client -> server setup.
I read the testimonials, which got me excited, so I then tried to make use of zmq in a system I was working on. Using plain old sockets, I could whip up a connection in a few lines (OK, python helps here), and deal with all failure modes associated with them. Mapping this robust setup to zmq, required multiple layers, async requests, and a polling mechanism. I don't recall now, but I may have had to throw a router in there as well. I'm not dissing zmq, as it's a great technology, and solves some complex problems well, but it's not for everyone, and people without complex problems will more likely be confused by it than helped.
The documentation is OK, but I think tries to abstract too much of what's going on, and just leaves you with a bunch of less-than-descriptive names for things that you have to take for granted. All the mamas, papas, and dealers analogies didn't help me at all.
I started using ZeroMQ at work a few weeks ago after reading the guide.
The part that I like about ZeroMQ is that it's a simplifying abstraction, e.g. it's a fairly simple wrapper about sockets that makes life easier for me. I guess it's mostly like a library of patterns for socket usage, so it makes a request-response loop and pub-sub scenarios really easy to implement. The other thing that's great is that it's pretty much programming language agnostic, meaning that C++ code and Python code look fairly similar and can trivially be used together.
As for the transparent queueing, there are simple options to control the size of the queue; it's even possible to spool the queue to disk as soon as the memory limit is reached (see setsockopt docs and look for HWM, for high water mark).
All of your reasons make sense to me, it's just that all the advocacy docs set ZMQ up to be this game-changing system. I can totally buy "a fairly simple wrapper about sockets that makes life easier for me." "world-saving superheros of the networking world" not so much.
I don't know, the simple building blocks it provides can be combined in a bunch of ways that are really pretty cool. I think apenwarr would might call it a "simplifying assumption". It's hardly ever the complex things that are game-changing, right?
It is challenging to explain anything subtle, when your viewpoint depends so much on what you already know, your preconceptions, and assumptions.
Sorry for the comic book style. It's how my mind works. Feel free to send a patch. :)
The benefits of 0MQ to me, as a programmer, are:
* Really easy to write utterly solid multithreading code, in a language (C) that has zero support for this;
* Really easy to take this same code and make it multiprocess, or multibox, with few changes.
* Built-in handling of asynchronous I/O, which I need in any real application.
* Speed, meaning I can be lazy and my code still runs very quicky. This is less obvious than it seems. 0MQ's critical path is extremely evil.
It's so hard to explain this with a diagram or a slideshow. The only way to understand is to use it, to write code with 0MQ. Certainly for me, when I started programming 0MQ apps, my understanding of what this library was about, and what it could do for me, totally changed.
To me the headline feature of ZMQ is that I don't need to run an independent server to pass messages around. It's basically a socket with queuing. That reduces my administrative complexity, which is a clear win.
I guess it would've been better if the title were "Advanced Network Communication Architectures (or Patterns)" with ZeroMQ. Network Architectures sounds more hardware-ish and (at least to me) sounds like it's talking about the actual network topology... :-)
Today I gave it another shot, and after half an hour of trying to glean technical detail and skipping any parts that read like a like a comic book ("zap-pow-kaboom satori paradigm-shift moment"), I think I've learned the following:
The main features seem to be topology-agnostic programming (since you don't have to know what a socket is connected to to send/receive over it) and transparent queuing. In my opinion transparent queues can be problematic because they transparently use up memory that can be hard to account for. Topology-agnostic programming certainly seems interesting, but in my experience of distributed systems programming I never seem to need or want complex messaging topologies. I guess I don't really get what all the hype is about. Maybe I should watch the video.