Hacker News new | past | comments | ask | show | jobs | submit login
Digg v4's Problems are not Technical (thebuzzmedia.com)
65 points by rkalla on Sept 9, 2010 | hide | past | favorite | 41 comments



"TC had an interesting story on the reaction from Digg employees who found out that John Quinn was being fired via TechCrunch and not from their internal team members. That is insulting and alienating, but it also says a lot about Digg and how that team is running right now."

Damn.


Two Digg engineers have said on Quora that Cassandra in particular is not the problem (http://www.quora.com/Is-Cassandra-to-blame-for-Digg-v4s-tech...)

"The whole 'Cassandra to blame' thing is 100% a result of folks clinging on to the NoSQL vs SQL thing. It's a red herring.

"The new version of Digg has a whole new architecture with a bunch of technologies involved. Problem is, over the last few month or so the only technological change we mentioned (blogged about etc) was Cassandra. That made it pretty easy for folks to cling on to it as the 'problem.'


It may not only be Cassandra, but it would not surprise me to find that the reason the VP was let go was because he oversold some architecture or combination of architectures as a panacea. When everyone else found out that Cassandra + whatever else wasn't going to be the magic bullet after all, someone's head needed to roll.


The story of Digg is yet another cautionary tale about the "get big fast" approach.

+ Digg has 500 servers in two private datacenters. Reddit has dozens on EC2.

+ Digg has ~100 employees. Reddit has a half dozen.

+ Digg took $40M. Reddit took less than $1M.

Everything we're seeing now the result of the decision to take this path. Half-hearted attempts to resurrect the dying beast are predictable, as is the drama and finger pointing.

It's very unlikely anyone could have saved Digg. Kevin just walked in a situation in which he was setup for failure. Even if he had picked the perfect strategy there's no way he would have the time and support to prove it.


As of 10 months ago, reddit had 218 EC2 instances (no clue what size). See http://www.reddit.com/r/IAmA/comments/a2zte/i_run_reddits_se... Not trying to argue with your overall point though-reddit has definitely run much leaner.

edit: I stand corrected, thats just 218 virtual processors.


218 virtual processors

Actual numbers: http://news.ycombinator.com/item?id=1454369

  25 c1.xlarge
  26 m1.large
  22 m1.xlarge


Reddit also likely makes a miniscule fraction of the revenue of Digg...


Reddit is spending $10k a month or something silly on servers, and asking for donations to be able to keep going.

I wouldn't hold Reddit up as a shining example of a success story just yet.


$10k per month is peanuts, considering how much money they could make if they knew how to monetize their site. It's pretty clear, from their pathetic advertising strategy and their mis-handling of "Reddit Gold", that they have no idea how to do it. OTOH they don't really have much of an incentive, since they are owned outright by Conde Nast.


It's pretty clear, from their pathetic advertising strategy and their mis-handling of "Reddit Gold", that they have no idea how to do it.

Having talked directly with the Reddits directly, it's a bit more complicated than that. The Reddit's hands are tied in a number of ways because of their relationship with Conde Nast.

They are quite motivated to make the site profitable, but they have been busy managing the server farm and trying to manage the community with a hand full of guys for quite a while. Conde Nast didn't want them to hire any one else.

IMHO, those guys deserve an insane amount of credit for keeping the site up and running rather than contempt because they haven't monetized properly.


huh. All this time I've been assuming they're incompetent for not being able to monetize the site, without ever considering they might not have any intention to do so. A good ad strategy wouldn't see the reddit people any richer, but it would increase the risk of reddit implosion and the loss of their jobs. In this light the Reddit Gold strategy is brilliant. Reddit Gold gives them just enough revenue so they aren't a liability for Conde Nast, and Conde Nast can't fire them for not producing real ad revenue without cutting off the Reddit Gold revenue. That, or they really are idiots.


If you read the top comment of this thread, each of the top 20 digg users made ~10k a week or more for merely submitting to digg.

Considering reddit has 7 times the traffic of digg (On the day of the digg debacle) and no data centers itself, the server charges are reasonable.

In the words of Raldi, "You can run it in 2 servers, we know; but we are just not that good enough"


It's a reasonable amount if they made anywhere near that in revenue. But I'm betting they don't.


Reddit took less than $100k.


"As one commenter quipped, if Twitter had fired engineers because of rocky launches and questionable code in the first few months of getting popular, there would be no one working at the company. (Ha!)"

I'm of mixed feelings about that bit (there was questionable code, but the launches were pretty smooth given the lack of hardware), but it's pretty obvious the person doesn't know the history of the site. But mostly that's because the PR team worked really hard to make sure people didn't.

The first 2 years were amazing. Digg is now six years old, Kevin has taken 7 digit figures "off the table," there's no question that this launch was unacceptable from so many perspectives. They just fired the wrong person.


I'm not sure I agree. Clearly Digg was having some kind of monetization issue. This whole episode might be pretty calculated really. They had to know those changes where going to cause general outrage. They had to think that the benefit would outweigh that.

In the end they might add tremendous value.

What they didn't count on was the site taking a huge step back in terms of reliability.


I'm not sure which part of my comment you disagree with. But the monetization issue is (in my opinion) due to Kevin and Jay taking money off the table, and spending much of their time drawing digg salaries while building other businesses (revision3, pownce, wefollow, fflick). If they'd devoted 100% of their time to digg there wouldn't be a monetization issue.


I took "off the table" to imply that they had decreased the companies value with Digg V4 in the order of 7 figures. I misunderstood.


ojbyrne,

No doubt about it I don't know the details of the technical impls of Twitter in the early days -- I only became aware of it when it's usage started exploding and the "fail whale" became such a common occurrence.

No knock to the Twitter team, if that was just hardware scaling issues from the get-go, not much they can do about that. The reference to the comment was more for the cute anecdote more than anything.


Digg's trouble illustrates how the size of a site's audience isn't as important to the bottom line as their quality. (From a marketeer's point-of-view) Digg, reddit, even Facebook have trouble turning a profit despite receiving a huge number of visitors. It's a very significant change from the size-uber-alles orientation of the traditional media.


Where to you get the idea that FB isn't profitable ?

2010 at least $1bn in revenue - http://techcrunch.com/2010/03/03/facebook-revenue-2010/

2009 $700m in revenue - http://www.theregister.co.uk/2010/06/21/sources_say_facebook...


I didn't say that they weren't profitable. I said they made very little profit considering that they have an enormous audience. A substantial chunk of the planet frequents Facebook, yet people still argue whether they turn a profit. That says something about the value of large audiences.


Revenue numbers say nothing about profitability.


It looks to me like digg has essentially collapsed. Have you ever seen a front page there with so few comments per post? Each is <50.


The author mentions users leaving because Digg has become "publisher dominated". I haven't dived in yet, but based on the descriptions, it sounds much less publisher dominated to me, especially if you consider MrBabyMan & Co. to be publishers where Digg is their outlet. I'm really looking forward to finding Digg friends that are more like the people here on HN so that they might have more influence on which stories I see. I'd like to see Digg become an effective source of interesting links again instead of the incrediby base thing it has become.


Obvious throwaway account.

As a former digg "power user" this is my take on the recent change and why I think is the only thing that will save digg.

I have been using digg since almost the beginning. At first I was using it for the same reason almost everyone way using digg at that time, for tech news and discussions. It might be very had to believe now, but once upon a time digg used to have very engaged group of users with thoughtful and knowledgeable comments, a lot like HN and some smaller sub-reddits on reddit. Digg was truly powered by the users with very little if any influence. It was a lot of fun and very addictive.

I think it was during v2 that they made some questionable choices by adding friends feature (I could be wrong, it could have been there before that) and "fudging" with the frontpage algo where instead of stories hitting the frontpage because of the interest on stories, they soft-limited on the number of stories-per-category could hit the frontpage. The idea was to "diversify" the site. Which in the business sense is not a bad idea, you obviously want more people to come to your site not only people who are interested in technology, but the implementation was a sham.

This had 2 negative effects:

- People who are passionate about technology and also the core majority of the digg users lost interest and left or at least started using digg less. You could really tell by the quality of the comments.

- It opened the flood gates for other non-tech sites who are now interested to get their site on digg frontpage. Getting on digg FP is profitable and also good for SEO, the amount of quality link-back you get after hitting the FP is crazy. Since the majority of the blogosphere is everyone quoting/rehashing everyone else with little original content. If you could get your original content to digg FP other related site/blog would pick it up and link back at you. Before you had to work your ass off to get high quality link back, now all you had to do was get on digg frontpage, which wasn't very difficult because of the half-assed "community" feature they added where you could friend people and they would get an RSS feed of your submission and you could blindly digg their stuff. People where literally blindly digging there stuff left and right.

The fact that this was going on alone should piss people off at Digg HQ. But they let this go on for as long as they did is, to me, mind-boggling. Either they were incompetent or they really consciously wanted the site to die.

But that's not even the worst part. But this was _very_ profitable for people who can fine tune the process of getting to the digg frontpage. ("very profitable" is subjective if you are making millions of $ a month. But if you are making thousands of $ in your spare time with very little effort I think it was profitable).

Digg users who were power users at that time (and still are) but was doing it only for fun, now saw that they can actually have fun and make money at the same time!

As it happens in most internet community (even here at HN) there are some community celebrities and people will support them no matter what. They could possibly do no harm. "babyman and co" (a la digg power-users) had such mindless followers who would support them and mindlessly digg anything they submit. And these power-users had relationships with publishers who would pay them good money to submit their stuff. You get FP you make money, if you don't get FP you still make (somewhat less) money for the "effort". But with power-users with their mindless followers, they had a very very high probability of hitting FP. So they were most sought after.

How do I know all these? Apart from being one of the top 20 digg users (until recently, in terms of FP count) I knew 12 of the 20 top digg users personally and met them often and some of the rest I spoke to regularly by phone and IM.

So how I personally profited from this?

As someone who is fairly tech savvy and half-decent writing ability. I setup 3 separate tech blogs/sites and with the co-ordination with other power users (they didn't ask for a cut-off or money if you ask them to submit your own sites) I was hitting the FP left and right. At one point I was getting 75-80% on the FP stories and during christmas holiday with high CPMs premium ads I was raking in ~8-10k a week from those 3 sites by spending less than 4 hours a day (I am also counting my time on working on the sites). I don't know about you guys, but to me, thats big money. After that, I was doing it full-time for about one year.

(Since then I sold my site, and left digg. If you must know why, and I don't have a non-cheesy answer, I felt like I was cheating, which I was, and doing something very wrong. It hit my conscious, as hard as it might be to believe. I wanted to wash myself out of this, so I sold the site and just left everything that has to do with digg.)

But systematic manipulation was not limited to power users, there were automatic bots, browsers plugins, and userscripts that could be systematically used to digg and bury submissions. How I found out about this is I was once hanging out with one of the powerusers and while waiting for our food I was checking out digg (I know I am lame), and I saw his account was digging stuff even though he was obviously not using digg himself at this very moment. So I pressed him about it and he let me in his small secret group and now almost everyone who matters does it. Sometimes around last year they did update their site and did some mass banning on users who were using bots and scripts (I was banned too but they gave me 2nd chance), but that didn't stop shit. Everyone was back to the old game after few weeks.

At this point I must point out how ironic is that people, "babyman and co" (a la power-users), who are leading the "digg revolt" this time around was the subject of all the previous "digg revolts" (except for the first one). All these time everyone was crying about getting rid of power-users, or influence of a group of people. But now that they have "fixed" it, in the sense that power-users don't have as much influence on it as they used to, people are crying about the same features that gave the power-users the ability to manipulate the site. Digg4 pretty much kills the concept of power users, yes you can still game it, as we have seen with reddit feed being on Digg frontpage, but that was according to digg a glitch in their algo, but its nothing like before.

The concept of "individual power-users" is killed or diminished. Now the publishers are the power-users, the more followers they have they more likely their site will hit the FP and they get submitted automatically, but with proper algo they can also let the regular user level out the field with user generated contents (at least that what I understand from the chatter).

Digg4, in my opinion, is the only thing that will save digg. There was NEVER people-powered-digg since v2, there was the perception of it; now it is just obvious out in open.

Sorry for the wall of text and possible spell/gramm error. I felt like I needed to say something about this. I don't use digg much anymore I usually hang out here or at reddit.


You seem perturbed. Would you ever consider setting up your own Digg competitor, given Digg is going down?


I wouldn't say I am perturbed. I would say I am amused by all the "guess-timations" related to digg whenever the subject of "Digg is dead" comes up. I feel that there are very few people outside the digg power-users who understands how systematic the gaming was going on. I am sure people knows that gaming was going, but I don't think a lot of people understands or knows the fine details.

Hopefully, by sharing my thought I could influence the widely held idea that "Digg finally screwed up bad this time" to "Digg finally did something right for the first time".

Honestly I feel that digg management is incompetent at best. Jay was more interested in selling Digg to the highest bidder as soon as he joined digg and after that didn't work out too well, he was more interested on generating revenue (nothing wrong with that) at the expense of taking care of the site and the community.

To answer your question. No I don't think I would do something like Digg. Digg in its current form, and even when it started, is a glorified RSS feed; which doesn't even reflect the choice of the hive-mind or your interest. The real meat is in the community (just look at reddit). If you get the software right, the user will take you from there. Besides who else wants another source of distraction to get news?


I actually have. We announced the site here http://news.ycombinator.com/item?id=1454934 as well.

The novel idea of quippd is to allow users to add/correct links/headlines/content in an effort to cut down the gaming -- no story is just about a single provider of content (optimally).

Check out our coverage of the burning of qur'ans story for instance -- many articles being referenced, not a single source.

http://quippd.com/show/2994/reply/4389/Florida_Church_plans_...


Surely the whole idea is flawed. It's as useful as internet polls are.

Once any site like digg/reddit/etc gets big enough to be worth gaming, it will be gamed.


Yes, and welcome to planet Earth. HN is moderated manually and automatically. Paypal is ruthlessly policed. Both have resulting unpleasant stories. Security is vigilance and striking a balance between many things. On HN, what interests me are Top Comments, how about you?


Even top comments are often just appealing to lowest common denominator, current fashion, etc

Democracy, and 1 vote per person is just a flawed system. Thankfully there is a fair bit of dictatorship on HN which keeps things fairly sane.


>Democracy, and 1 vote per person is just a flawed system. Thankfully there is a fair bit of dictatorship on HN which keeps things fairly sane.

I agree, and this is where reddit, for instance fails.

Once quippd gains a userbase that is interested in trolling and posting bad comments, we plan to implement a slashdot style moderation system (still the best moderation system out there, imo).


One of the aspects not mentioned is that Digg was a tech website at the beginning. The quality of the audience, and subsequently, the quality of the content, decreased as they opened the flood gates to other categories.

It seems like their only option was to either stagnate with an increasingly unappealing demographic, or reinvent themselves.


It's the very nature of the beast (social news, that is)

You can't have a site to aggregate the "best" news without restricting your user base. With fewer users, you can't make as much money.


zbanks,

Interesting point, and that might explain why sites like HN and reddit (primarily technical and occasionally quirky) are still maintaining relatively high quality content.

I suppose it's the jack of all, master of none curse.

I would agree with the assessment of the previous poster that the quality of Digg did seem to be effected to the site opening it's categories up to the world, and not just to the tech community... that did seem like a milestone in me noticing I wasn't quite as happy with what I was reading.

Also see reddit in the situation where with 6 people, it is barely scraping by... sort of a crappy rock-and-a-hard-place aspect of social news.

Then there is the Facebook or Twitter approach which is the polar opposite end of something like HN, you have ALL content from everyone's brain, and the real money is in figuring out algorithms to surface it accordingly.

I don't know that I'm holding my breath for that algorithm though.


I remember reading some interview with the digg-staff around the Cassandra switch. I remember that the "problems" they were facing with the relational databases were mostly bad queries designed to exhaust even a decent DB.

Instead of just writing proper SQL, they wanted to replace their entire server architecture. To me that looked like a pretty bad decision and I would halfway label that a "technical problem", although it may be more related to the technical competence of the staff than anything else.

Now ofcourse I can't remember where I saw it, or what the specific issues were and I'd love to revisit it. Anyone got a good memory or a link handy? :)


That's a pretty absolute statement based upon, it seems, absolutely nothing.


And it's pretty ironic to have a writer criticize a rushed release by saying, Digg v4 was also suffering from a surprisingly amount of down time. Kettle, meet pot.


blurry,

I'm not clear on what what ironic -- that the release was rushed, so down time should be excused or something else. Could you clarify?


>I'm not clear on what what ironic

Brilliant.

(The irony is that you're missing a word in the sentence blurry quoted. And you did it again.)




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: