Same logic with communism - all the ones that ended in oppressive dictatorships, mass executions or starvation weren't "real communism" or weren't "implemented right"...
Politics is just deciding what to do with shared resources, an ideology informs people’s choices about what to do with shared resources, managing people doesn’t really have anything to do with scum or politics, I think you’re missing the mark.
More to the point, you’re picking something more divisive than scrum and adding your thoughts about the divisive topic to add to a discussion about scrum, it doesn’t add to the conversation it just ratchets up the conflict. Take it to Facebook.
People aren’t a resource, they are the ones sharing the resources. You’re conflating a capitalist business structure with politics, they aren’t related.
Ah - I've used S2 in the past. Great work. It scales really well for larger datasets.
Interesting that you mention the use of multiple hilbert curves as well. We also experimented with two Hilbert Curves, rotated by 90 degrees. This helps to get around what we've dubbed the "Hilbert Equator" problem where two objects are quite far on the curve because they are placed close to one of the major fault lines in the fractal (for lack of a better word)
There are always caveats with these types of indexing methods, especially if you require dynamic high-performance indexing and support for rectangles/polygons. You can only move the edge cases around, there is no way to eliminate them. This allows you to tailor an indexing scheme for specific workload assumptions but this obviously breaks down if you need an algorithm that generalizes to many workloads and data models. There isn't just one type of edge case with this type of indexing, there are several which may or may not be relevant depending on what you are trying to do.
Some research from the 1980s showed it is only possible to mitigate bounded categories of edge case, thereby improving generality, by indexing on complex higher-dimensionality embeddings. Mitigating more categories requires more dimensions and more complexity. However, no one could figure out how to construct these embeddings for even basic cases or deal with more practical curse of dimensionality issues, so that is largely forgotten (the researchers themselves made comments to the effect that they didn't think a tractable solution was possible). I've never even been able to find that literature in electronic form, unfortunately.
Like AI, it is an interesting open-ended problem space. You can prove that an elegant optimal solution is not tractable so it ends up being a search for asymptotically optimal algorithms that become exponentially more complex the closer you get to optimal.
Tackling these boundary problems are the literal "edge cases" in geospatial indexing and they exist everywhere, so this i a good reason for using an existing library as the authors have already solved them.
Hexagons are cool, but they are not necessarily the bestagon for a spherical geometry since you cannot break a hexagon into smaller hexagons, whereas an S2 cell is a "cube" with spherical sides or HEALPix uses a rhombic dodecahedron [0] both of which can be split into smaller divisions of themselves.
Not to discourage you from your experimentation, it's all trade offs and you might find a good one. Good luck!
> you cannot break a hexagon into smaller hexagons
Actually you can.
1. Think of hexagons as six equilateral triangles sharing a center point.
2. Place one smaller vertically flipped equilateral triangle, in each original triangle.
3. Each original hexagon center point is now the center of a smaller (1/2 linear dimension, 1/4 area) hexagon.
4. New small hexagons replace each of the six original hexagon's edges. Since edges are shared, this is an increase a 3x increase in number of hexagons.
So each new hexagon has 1/4 the area of the original ones (and 1/2 the linear dimensions). This results in 2x the linear dimension resolution, 4x the area resolution.
Grids could also be increased in scale the same way. By retaining a half-sized (in linear terms) square at the center of each original square, and turning each original edge and corner into new squares. With the same 1/2 and 1/4 ratios of linear and area scaling.
Does that end up with specific points that have that same difficulty at intersections of equators? I suppose even if that is the case, a point is something that's easier to position over a dead area than a line would be.
What's the reason behind the sudden, violent drop off from the front page for a few posts? They seem to be on a fairly linear trajectory and then go down by 30+ positions within a 10 min window.
Post that have more comments than points, for example, get massively deranked rather quickly.
The reason this is done is to remove from the front page hot-button topics (i.e. controversial political news), but often this also includes friendly, niche posts with vibrant conversation, which is very sad to see. These tend to have < 1 upvote/comment ratio than most "Generic tech-adjacent news" like "Apple releases new iPhone".
For example, the Ask HN I posted a few hours ago suffered the same fate: https://hnrankings.info/39788649/: from #13 on the frontpage, to 3rd page in minutes.
Last time I saw it happen on a lovely post about Forth and its various implementations over the decades — not really at risk of flame wars, but alas.
Lately, my gut feel (not based on data analysis) is that there are more people commenting without upvoting than there used to be. I wonder whether that's true.
HN has worked this way for over a decade. It’s a combination of user flags, flamewar detector, and manual moderator classification. The goal is to have a front page that consistently surprises or delights anyone bookish. So the stories that get bumped tend to be ones that do the opposite.
I think you underestimate the number of flags from users and overestimate the amount of direct moderator intervention.
There have been countless times where I've been the flag that tips a comment or a post over into the dead (or for posts, deranked) territory. I flag comments that are rude, abusive, or deliberately filled with misinformation. I flag posts that are likely to start flame wars, that are highly off topic, or that are clearly low-effort blogspam marketing with no valuable content.
If any of that sounds subjective to you, that's because it is. Most of the favoritism and bias you observe can be attributed to users like me who have an idea of what the HN culture should be and subjectively flag things that don't fit.
I imagine also, that plenty of HN readers use “flag” as a mega-downvote, for articles they don’t like or are offended by or whatever. Since there is no downvote for articles (only for comments), the “flag” is the only way for readers to try to bury one.
It’s the other way around, I think. We probably underestimate the amount of direct moderator intervention, both for stories and comments. It’s their full time job.
Ask yourself: what do you think Dan’s team does all day, every day, if not make decisions that influence the site? His comments are all that are publicly visible, so most people assume he just writes comments — an assumption as far from the truth as can be. And most people have no idea that there are even other members of the team, let alone know what they do.
This is by design. HN works best when people are focused on the content, not the site. It’s one of the most influential newspapers in the world, precisely because it focuses on the content. And you’d be kidding yourself if you feel they don’t decide which stories should stay on the front page, every day, a dozen times. Again, it’s their whole job.
Note that them seeing a story and allowing it to stay is also a choice, even though it’s implicit. So them doing nothing because they chose to let a story stay is not the same thing as them doing nothing because the community chose to put it there. Frankly, you wouldn’t want to visit HN if it was community run, because the community is terrible about choosing which stories should be on the front page. This has been proven true for over a decade, and the main reason Dan doesn’t come out and say so is because it’s one of those things that shouldn’t be said out loud. There’s no reason to call attention to how utterly awful the majority of users’ decisions are about what should be on the front page.
Remember, millions of people visit the site now. You wouldn’t want that many people to try to decide. The voting system is an indicator to the mods of potential community interest, but it’s ultimately up to the editors what stays and what goes.
People call this favoritism or censorship or other frankly silly terms, but there’s nothing sinister about it. The way onto the front page is simple: make something that someone bookish would find interesting or surprising. The "surprising" criteria is the trickiest part, because it covers what most people mean by "newsworthy" (e.g. https://news.ycombinator.com/item?id=39778999). But HN has a particular kind of newsworthiness that filters out the majority of stories that the majority of users want to put on the front page.
It’s a tricky subject, and it’s long past time someone should write an essay to explain it all. I’m not quite sure how to do that without angering the community xor mods, but it would be interesting if it were possible. One reason this idea appeals to me is because I’d like to at least thank all of the dozens of people who help run HN behind the scenes; they rarely get any kind of recognition, because they don’t need it, let alone want it. But that makes them worth thanking all the more.
I recognise the huge amounts of effort involved in this and I applaud the moderators for keeping HN an interesting place to be.
That said, I think it's reasonable for us to have visibility on their manual interventions, and this could be easily surfaced via the Hacker News API (https://github.com/HackerNews/API), if the Story JSON included the values of "contro," "bury," and "gag" fields, which are currently opaque to users of the API
One other positive thing to mention about HN: the cost of running HN is non-zero, yet it remains ad-free. I'm grateful for this. So many websites have become unusable due to monetisation dark patterns.
This sort of system just encourages more meta debates about what is and isn't on the front page and why. Most of them are repetitive and boring. This is analogous to the 'receipts for downvotes' perennial - if you think about what it would mean in practice, it's not hard to see the result would be a crappier site.
That doesn't fix the boringness problem and people have meta-discussions about HN on other forums all the time so there's that for those who are into that sort of thing.
What makes you believe Dan has a team? And what do you mean by "team"? My presumption is that there are a number of unpaid volunteers who have been delegated small pieces of the moderation effort, but that there are no employees except Dan. Am I wrong?
Personally, I'd feel better about the future of the site if I thought there was a strong moderation team making numerous decisions behind the scenes. Instead, I think most stays/goes content decisions are made by one vastly overworked Dan and by a bunch of flagging users who frequently have ulterior motives.
I reverse-engineered the HN ranking algorithm a few years ago [1]. To answer your question, posts have a sudden drop due to several types of penalties. Many posts are penalized from the start due to their title or domain. A "controversy" penalty kicks in if an article has more comments than upvotes and at least 40 comments; this penalty is usually the reason why an article catastrophically drops. Other penalties are applied by the moderators. If the voting looks suspicious, a "voting ring" penalty is applied. I found that penalties were very common: 20% of front page articles were penalized and 38% of second page articles. There have been changes to the ranking algorithm since I looked at it, but it is generally the same.
+1, and thanks for taking the time to work on this! I've always wondered how HN ranking works until this moment where every question I had regarding this is being answered in the article.
Also, the article is more than a decade ago, and I'm wondering if there have been any considerable updates to the HN ranking algorithm since then?
I've also tried reverse engineering the algorithm recently to plot a scattergraph of story ranks vs age-adjusted score. I'm looking for outliers in order to work out where HN moderators have intervened.
It turned out to be very difficult as there are so many arbitrary factors, and the HN API doesn't expose those factors.
It's good to see the OP's graphs over time, showing the large jumps which clearly indicate manual intervention. That's a much more effective approach than my own analysis.
My motivation is to surface any patterns in those interventions. Annecdotally I notice that positive news about Elon Musk and his companies has been suppressed more than once.
Obviously, the moderators are responding to things like user flags, but in my opinion there should be more transparency around this, in order to prevent users brigading certain topics, using flags inappropriately, to manipulate the narrative.
I understand the fundamental motivation for this sort of thing. The web is a static document display that over many decades of augmentation and extension has morphed into an app development platform. And arguably, a lot of the mechanics of it are less elegant than in purpose build app-development environments.
But, I don't think another language that tries to abstract the fundamentals is the solution to that. I think this is true for Hyperscript - as much as it is for JSX and Co. JavaScript is the most widely known programming language, has the most extensive knowledge base and the biggest ecosystem. And there's no shortage of frameworks that make using it easier without reinventing the wheel.
I believe using anything else for a serious project severely limits the talent pool from which you can hire, the ecosystem you can draw from and the maintainability of your project in the long run.
I believe folders (or groups or similar) are the right solution to this, just not in the way that Google is implementing it. Basically, you group resources into folders and then users have read or write access to that folder.
This way, your database guys can access one big folder with all the database stuff, your server guys can access server stuff and your frontend guys can deploy to an S3 bucket, but not much more.
This is the level of granularity you need in the real world. i don't believe that any organisation, no matter how big or sophisticated has an employee that can have roles/datastore.backupsAdmin, but not roles/datastore.backupSchedulesAdmin.
> i don't believe that any organisation, no matter how big or sophisticated has an employee that can have roles/datastore.backupsAdmin, but not roles/datastore.backupSchedulesAdmin.
Believe it. Based on past experience as CTO and head of Security Engineering at one of the biggest orgs, this split is used and necessary, unless you want to inject yet another approval loop somewhere.
The first one lets someone get, list, or delete the backups, the second one lets someone make backups happen or not happen. I can absolutely see forcing regular backups to happen (a regulatory requirement) being a different person than whoever is using the backups, even different from the admin who can delete those backups.
(Delete means the backup admin can make it as if a backup didn't happen by deleting, but that's not what the compliance regulation covers, it has to happen in the first place, which is what the scheduleadmin covers.)
Absolutely! I'm grateful that I don't have to worry about code formatting any more. But I remember in one of my earliest job the company used style checkers as a pre-commit hook that rejected your commit if they found trailing whitespace. That was before code formatting was part of your IDE. (Especially for us front end devs who used Notepad++ rather than an IDE at the time).
And notepad++ had no easy way of showing trailing whitespace. So every commit was a dance of commit -> read rejection log -> remove trailing whitespace -> commit again.
I once worked on a project that would not even compile if a function had a param without a comment explaining the purpose of that param written with a specific format. Every comment was validated on compile time, you couldn't even comment code just to test something. Life was hell.
I have memories at my post grad school where any deviations from the expected code source formatting led to a -1 on the mark. On the first few projects it was not rare to see students getting marks well into two or three digits negatives.
Code reviewers that made me type in trailing whitespace that editor stripped: "this change is on lines you are not modifying". Still makes my blood boil many years later.
