It seems to partially duplicate the functionality of Mesos, as they are writing their own task scheduling framework [0] based on Google's Omega [1].
The Omega authors claim that Mesos' system of application specific schedulers accepting resource offers from the Mesos master is well suited toward short-lived jobs (think ephemeral Map/Reduce or MPI type workloads) but is not well suited for long lived 'service' jobs (like a Rails app or DB server). As this seems to be an important use-case for Flynn, it would seem like a valid architecture decision to not use Mesos.
Mesos is exceptionally good at managing long-running service and that use case represents about 50% of the workloads I've seen on large production clusters.
"Scheduling" long-running services is straightforward, as they typically only need to be run "once." It's trivial to use something like Marathon [0] to do that, and you then immediately benefit from Mesos' fault-tolerance and self-healing. Marathon also makes it easy to elastically scale the long-running processes (e.g., start more Rails servers when traffic increases).
Mesos achieves fairness by alternately offering all
available cluster resources to different schedulers,
predicated on assumptions that resources become available
frequently and scheduler decisions are quick. As a result,
a long scheduler decision time means that nearly all
cluster resources are locked down for a long time, inaccessible
to other schedulers. The only resources available for other
schedulers in this situation are the few becoming available
while the slow scheduler is busy. These are often insufficient
to schedule an above-average size batch job, meaning that
the batch scheduler cannot make progress while the service
scheduler holds an offer. It nonetheless keeps trying, and
as a consequence, we find that a number of jobs are abandoned
because they did not finish scheduling their tasks by
the 1,000-attempt retry limit in the Mesos case (Figure 7c).
This pathology occurs because of Mesos’s assumption
of quick scheduling decisions, small jobs and high re-
source churn, which do not hold for our service jobs. Mesos
could be extended to make only fair-share offers, although
this would complicate the resource allocator logic, and the
quality of the placement decisions for big or picky jobs
would likely decrease, since each scheduler could only see
a smaller fraction of the available resources. We have raised
this point with the Mesos team; they agree about the
limitation and are considering to address it in future work.
Its worth noting that Andy Konwinski was a coauthor on both Mesos & Omega, so I'd hope they (Omega authors) represented Mesos' capabilties accurately. I don't have any personal experience running Mesos in production, I'm just going of what was written.
Ah, interesting. My personal experience with Mesos has included clusters that are basically all transient services (like map reduce jobs) or all long running services, but not both. I can see how that might lead to pathological scheduling decisions.
The Omega authors claim that Mesos' system of application specific schedulers accepting resource offers from the Mesos master is well suited toward short-lived jobs (think ephemeral Map/Reduce or MPI type workloads) but is not well suited for long lived 'service' jobs (like a Rails app or DB server). As this seems to be an important use-case for Flynn, it would seem like a valid architecture decision to not use Mesos.
[0] - https://github.com/flynn/flynn-host/tree/master/sampi
[1] - http://eurosys2013.tudos.org/wp-content/uploads/2013/paper/S...