This. I randomly picked Bellingham as a quiet place to go to for a week and do some self-teaching plus vacationing.
Decent food, friendly people, nice coffee shops, great areas to go trail running in. I'm not sure I'd go there for an actual vacation, but for just getting away and trying to learn some new things, it was a great place.
I had a hell of a time getting PgPool II w/ PG Streaming Replication set up right. The trivial cases seemed to be fine, but when I started triggering failovers back and forth, I ran into a lot of cases where PgPool II would go stale due to a data file left around in /tmp.
I eventually got it working, but there were way too many informally created scripts that PgPool and PG had to know to trigger failovers, initiate resyncs from WALs, etc. I didn't like it at all, and around about then AWS started offering PG on RDS, so I just moved to that.
So, my advice would be, unless you've got someone on team for who that isn't that much work, you get a lot of benefit from going w/ hosted. RDS Postgres has been pretty great - not exceptional, but for my use cases, okay. Hoping they add cross region read replicas for PG sometime soon, as that would make a lot of expansion opportunities really easy.
I don't have enough background knowledge to ascertain if this is the correct understanding of the problem, but this explanation reads very well, and is the best 'mechanical' explanation I've seen in this thread so far.
Cool library, can imagine how moving away from boxing / unboxing can be a huge boost for them.
I've been looking for something that gave SIMD intrinsics to Java programmers - does anyone know if such a thing exists? Could be a nice addition to this lib.
I've been on the fence about trying this on Instrumental; we've been looking at moving to DynamoDB, but I wanted to at least see if it would give drop-in magic performance benefits, so I spun off a new SQS queue of our incoming data this evening and did a write throughput test.
These are only initial impressions, but:
* It's definitely faster. Our write behavior is largely upserts against integers and doubles, and I'm seeing roughly 100% improvement against stock Mongo 2.4. The machine in question is an m2.2xlarge with a 1000 piops EBS volume attached, and it's doing about 7000 update operations a second. ( safe mode )
* I'm seeing consistently lower IO util than stock Mongo. Stock tends to vary wildly between 200-750 write ops, while under sustained write traffic, I see about 250 write ops.
* CPU usage is pretty well balanced against all cores, as opposed to stock's behavior.
* It's too early to say whether or not the storage savings will be as good as claimed, but at this point it seems that the TokuMX reprs are about 40% of the stock reprs. Like I said, most of our data is ints and doubles tho.
VERY LARGE CAVEAT I'm not running the database in a replica set because I'm lazy. So, the write throughput numbers are likely the best case scenario of what you'd actually be running in production.
(edit: line spacing)
If you've got to use MongoDB, it seems pretty nice. If it came with a tiny person that maintained the database for you as well, it'd be a no brainer.
I've definitely taken a look at Object Rocket, and it's a really nice looking service. However, our dataset size would take us right into your custom quote plan, and based on the rate of increase between the different plans, we'd paying significantly more w/ ObjectRocket than we would w/ DynamoDB (if we do indeed move to DynamoDB).
Awesome, great to see Instrumental (https://instrumentalapp.com/) in your list. I'm one of the folks working on it, would love to hear your thoughts on us.
Some quick reviews of some of the products listed:
* Lighthouse (http://lighthouseapp.com) - Bug Tracker - good for keeping track of simple stuff last I used it (~2 years ago), but Github Issues obviated its use for me.
* Pivotal (http://www.pivotaltracker.com/) - Project Management - great tool, not trivial to keep well managed tho. Easy to let your project get out of hand with tons of tickets, requires some discipline in its use.
* Trello (https://trello.com/) - Project Management - simple, fast. Really great for keeping tasks focused on a small team, I'm not sure how it would suit a larger team though.
* Airbrake (http://airbrake.io/) - Error Handling - You didn't have this in your list, but it deserved a mention. It's okay for server side error handling, its client side stuff leaves something to be desired though. More often than not their hosted JS lags on load, causes your page load times to go up as well. Doesn't currently offer a supported hosted version.
* Stripe - (https://stripe.com/) - Billing & Payment Processing - Does just about everything right imo. Great documentation, great interface, website is well engineered. Analytics / reporting would be awesome tho.
* Intercom - (http://intercom.io/) - Support/Help Desks - I seriously love Intercom. For managing a team of people doing outreach to users, it is awesome. I view it as a fantastic tool for triaging retention.
* Uservoice - (http://uservoice.com/) - Support/Help Desks - You didn't mention them either, but I thought I'd add. They are pretty great, even for small companies. I think their sweet spot is a larger support team tho. Great interface.
Pivotal: Horrible, abysmal tool. Hate it with a passion. No clear overview at all, UI is full of shiny colors but is messy as hell. I'm really glad we've switched to Jira[1] after fighting with pivotal for a couple of months. YMMV ofcourse :)
Pivotal is an agile planning tool; JIRA is an issue tracking and classical project management tool. While each can be coerced to the other's function, it's really comparing apples and oranges. I'll admit that JIRA and Pivotal continue to muddy the differentiation with afterthought additions like JIRA's Greenhopper and Pivotal's time tracking. But JIRA is a good tool for issue tracking and Pivotal is a good tool for agile planning. If you're trying to use either tool for the other's purpose, you'll hate it.
I still can't read "tool for agile planning" without crying inside my head. We really need "tools" to "implement" "processes" in the spirit of a 6-line manifesto, do we? Sigh.
Aww. Don't cry. It's not a process tool. It's a prioritized to-do list that can track your velocity. Despite being opinionated, it doesn't enforce a process. Though you'll find that if you don't embrace those 6 lines, you'll quickly make a mess inside Pivotal and it will happily let you do so.
I definitely agree with the YMMV bit; I have found this category of tools to be incredibly divisive. I know a lot of folks who thought that going in the opposite direction of yours, Jira to Pivotal, was one of the best decisions they've made for a project.
I've only ever casually used Jira, however, so am ill qualified to speak to its strengths.
I'm in the category you're describing. (Switched from Jira/Greenhopper to Pivotal and happy about it).
I think the divisive bit may, in part, be that Pivotal is a strongly opinionated tool while Jira is a fairly customizable and open ended tool.
For projects & teams committed to the process Pivotal champions, it's highly optimized. For teams using a different process, or who need to customize views for different people etc, Jira can provide more options, and be a better fit.
For me personally... at the moment, I really appreciate Pivotal's relative simplicity and the way it encourages folks to focus on the somewhat nearer term.
I love Pivotal. While I haven't used Jira, I've gotten the feeling that other members of my team have used it in the past, and don't like it at all. On another note, Siebel time tracking is a pit of hell.
Percentiles is a feature we're planning on adding at some point in the future, though we have customers calculating it themselves right now using things like metriks-instrumental ( https://github.com/netshade/metriks-instrumental ).
Alerts is actually in beta right now :) - we're testing it out with a few customers. It's based on our query language, and alerts you with a graph of the problem when the event occurs. ( or updates an HTTP endpoint you control, if you wish )
edit: I forgot to mention, my apologies you encountered an error. We've been living on the edge of browser support land for the time being, and could do a better job letting you know that you're not meeting the minimum reqs.
Our agent (for in process collection) is Ruby only right now. That said, we've got a large number of customers who really like our graphs and query language, and so send us metrics using a Statsd backend[1] written by one of our other customers. (edit: which is to say that we really support anything that has a Statsd client)
Speaking to the first portion of your complaint, I remember coming across that particular feature of YAML years ago, and being surprised at how odd it was that it was present. I shrugged it off because (at the time), I considered YAML to be something that simply would not enter the serialize / deserialize phase of incoming requests. Ignorance of all the frameworks actions on my part, and further ignorance to not think of the security effects of that particular feature, certainly. I would presume that I am not the only in that situation.
You're definitely right that the security reports should be handled better. I hope that this whole situation results in a better security culture in the Ruby community.
Regarding your tone ("intellectually dishonest", "trained monkey", "systemic engineering incompetence pervades an entire language community"), it's a bit of hyperbole and active trolling. You are certainly right in many of your points, and you are certainly coming off as a jerk. It may not be as cathartic for you, but I'd suggest toning it down to "reasonable human being" level in the future.
> It may not be as cathartic for you, but I'd suggest toning it down to "reasonable human being" level in the future.
The Rails community has exhibited such self-assured, self-promotional exuberance for so long (and continues to do so here), it feels necessary to rely on equivalently forceful and bellicose language to have a hope of countering the spin and marketing messaging.
Case in point, the article seriously says, with a straight face:
"They’re being found at breakneck pace right now precisely because they required substantial new security technology to actually exploit, and that new technology has unlocked an exciting new frontier in vulnerability research."
Substantial new security technology? To claim that a well known vulnerability source -- parsers executing code -- involves not only substantial new technology, but is a new frontier in vulnerability research?
This is pure marketing drivel intended to spin responsibility away from Ruby/Rails, because the problems are somehow advanced and new. This is not coming from some unknown corner of the community, but from a well-known entity with a significant voice.
I can understand your frustration with the community. I share this frustration at many times, because I feel that Rails / Ruby tends to value style over substance.
I'll also raise an eyebrow at that particular sentence, though without spending much time looking into what's backing it I can only add that I too find that slightly incredulous.
I definitely question your stated intent. Were you to "counter the spin and marketing messaging", would that reduce the number of vulnerable machines? Overall, reduce the number of people that use Ruby/Rails, if that is your intent? Given the number of comments you've made to that effect versus the number of folks using Ruby/Rails, I'd suggest you have a very long battle in front of you.
Put another way, I perceive your tone as an exasperated, reactionary tone to a group that you happen not to like. If you are indeed trying to achieve some greater good here, I believe there's more effective ways you could achieve it.
Otherwise, just tone it down in the future. You had good points, there's no need to insult people from an effectively unassailable position.
> Overall, reduce the number of people that use Ruby/Rails, if that is your intent? Given the number of comments you've made to that effect versus the number of folks using Ruby/Rails, I'd suggest you have a very long battle in front of you.
I'd like it to be 'cool' in the Ruby community to apply serious care towards security, API stability, code maintainability, and all the other things that aren't necessarily fun, but are very much necessary to avoid both huge aggregate overhead over time, and huge expensive failures like this one.
I'd like to see a shift towards an engineering culture where taking the time to consider things seriously is considered 'cooler' than spinning funny project names, promoting swearing in presentations, and posting ironic videos.
It seems increasingly obvious to me that for this to occur, one can succeed in pushing back against emotive marketing with a similar approach, and thus shift the conversation.
