node.js is different. It's not a language, it's a small/extensible set of tools built on top of a solid language, and VM (V8) that are very mature.
Node.js is in great hands with Isaac. Unlike many other web/network platforms node doesn't go around gobbling up everything into core. The community behind node is stronger and contributes more than I've seen before. Focusing on that third party module system experience is a good move to further grow the community, one that Isaac has been heading for a while.
While many people will see the BDFL stepping down from a project as a bad sign, I think its the opposite. It has a dedicated and invested company behind it (Joyent), it's core doesn't try to do too much, it has legs of its own strong enough and it's community fault tolerant enough to grow quickly without him.
Ryan spawned a project that grew a community that in the long run isn't reliant on him, a rare achievement of platform/language creators.
This change is a sign of Node's shift from improving the core to making modules work better. I see it that way, at least. Isaac is the author of npm after all.
Unfortunately the core is missing some significant features, whereas modules work pretty well already (though I can see room for doing pre-compiled binary modules like "PPM" does for Perl on Windows).
Honest question: have you actually tried working with modules and dependencies in Node? It's better now, but it's still not pleasant and often installing a module starts with debugging.
And what's the core missing, would you say? Personally, I thought the cluster stuff was already getting heavy.
> Honest question: have you actually tried working with modules and dependencies in Node? It's better now, but it's still not pleasant and often installing a module starts with debugging.
Yes. And you're right, but then it's not often that much better than other languages. Personally I'd rather they get the core language finished first (though their concept of "finished" and mine seem to differ).
> And what's the core missing, would you say? Personally, I thought the cluster stuff was already getting heavy.
There's no way to lock a file (making file writes completely fragile). There's no way to seek() in a file. There's no way to open a file with O_EXCL set (meaning lockless file writing is impossible). There's no temp file functionality in core, and without O_EXCL the solutions available for this on npm are utterly broken. There's still major bugs in the Crypto routines (they take Strings instead of Buffers in various places)... Honestly I could go on.
You can open a file with O_EXCL if you pass in the open flags as a number. (You can find them on require("constants"), and they need to be binary-OR'ed together.) This isn't documented. It should be. It should probably also be exposed in a cleaner way.
Most of the rest of what you describe is APIs that need to be polished and refined a bit. The boundaries are well defined at this point, though. We probably won't add another builtin module at this point, or dramatically expand what any of them can do. (I don't consider seek() dramatic, it's just tricky to get right given JavaScript's annoying Number problems.)
> You can open a file with O_EXCL if you pass in the open flags as a number. (You can find them on require("constants"), and they need to be binary-OR'ed together.) This isn't documented. It should be. It should probably also be exposed in a cleaner way.
That's great to know. Obviously I'm just following the docs.
> Most of the rest of what you describe is APIs that need to be polished and refined a bit
My concern is merely that there have been a number of statements put out saying "we won't be adding anything more to the API", and that we are basically almost at 1.0, at which point there won't ever be anything added to the core API. Lack of flock() is huge (you can't write to an existing file safely without it - and Node developers are doing that all the time, including your own NPM). Lack of an ability to create temporary files safely seems a fundamental weakness - especially when so many Node apps are dealing with file uploads - that's a disaster waiting to happen. We are going to be dealing with Node.js security bugs because of these issues for a VERY long time.
flock() is not trivial to do in a portable way. For a unix-only flock(), check out the fs-ext addon. Same for mktemp.
I wouldn't be opposed to either being in core if it could be done in a clean way, but this is just adding another knob that can be done with an addon easily enough. If you care more about having flock() than about writing portable programs, then that's what the fs-ext addon is for.
> We are going to be dealing with Node.js security bugs because of these issues for a VERY long time.
Of course we'll be "dealing with Node.js security bugs for a VERY long time", because we'll be using Node.js for a very long time. Software is buggy, and many bugs are security hazards. We'll be dealing with "Unix security bugs" and "C security bugs" and "Java security bugs" forever as well.
Please do not make vague suggestions about security issues. Either you've found an issue, and should be submitting it, or you haven't, and are just spreading fud.
> For a unix-only flock(), check out the fs-ext addon
Which I wrote.
> Of course we'll be "dealing with Node.js security bugs for a VERY long time", because we'll be using Node.js for a very long time.
That's not quite what I meant - I mean that people right now are writing temp files in LOTS of Node.js applications in an insecure way. It's good that O_EXCL is available, I'll try and submit a patch to node-temp, but really temp file creation should be in core (amongst other things).
This isn't a vague suggestion. There are COUNTLESS security bugs created every day by insecure temp file creation. Let's see, from npm these packages have security bugs because they rely on the insecure node-temp: ShipItJS assetgraph-builder confy filerepl gracie joose js-loader muffin nerve redisfs.
I disagree that it is "huge" that flock() is missing; if you are relying on advisory (emphasis on advisory) file and record locking to implement coherence, then your architecture is likely an accident waiting to happen -- it's a highly problematic interface for many reasons. Of course, if you really need it, use fs-ext as Isaac mentioned -- NS more generally, if you need some system functionality that isn't present in core, develop an add-on and knock yourself out. It's not entirely pleasant, of course -- but don't pretend that an interface not being in core means that some bit of system functionality is somehow off limits...
Huge might be the wrong word, but I've definitely been frustrated that it's missing. Java has a standard library class that works well enough for whole-file advisory (or mandatory if you're on Windows, I suppose) locks on a local filesystem. It's eminently helpful for excluding a second process from starting with a lock that the OS will clean up for you if/when your process terminates for whatever reason.
smf(5) can get you part of the way there, but not if you're (say) writing some system bits that you need to deploy on platforms in addition to SunOS. Plus, if you can avoid depending on any not-just-pure-JS modules then you can deploy one tree onto all of your platforms without additional build steps, C++ compile/ABI issues, etc.
THANK YOU. This addresses a question I've had on stack overflow for a couple of days. It also addresses at least two issues in Node.js's github tracker under the 'fs' flag.
I don't know that anyone is suggesting that it's "finished". In fact, it's not even to a 1.x release yet. I consider anything below that a work-in-progress at at minimum. Generally anything under a 2.x release has lots of room for improvement.
The fact that they're shifting focus isn't a big concern. I'd at least wait until Isaac comes out and says what the future roadmap is before criticizing it.
You have to remember, developers close to a platform always live 6-12 months in the future, at least. (Talk to Brendan Eich about JavaScript, he's like 5 years in the future.)
What "finished" means in this context is that we've established which problems we're going to solve, and the solutions to those are well-understood. There aren't new problems that node-core is setting out to address by making changes to the binary itself.
The new hard things to tackle are above the core layer - CI, userland module documentation and discoverability, binary deployments, etc. There's quite a bit of polish and refinement to be done at the lower level, but the paths are clear at this point, as far as that goes.
Downvotes, really? I don't hate on Node - I develop a very popular open source server in it. But it has some fundamental weaknesses in terms of how finished it is, and most of the core developers don't think those missing things need adding before calling it 1.0 and the API gets set in stone.
Of course :) You stepping in and writing NPM was HUGE! Along with Ryan, you and guys like Tim, Guillermo, TJ, Felix, now Bert and others really got the ship pointed in the right direction
It makes a lot of sense to move forward at some point, but this time the timing is a bit suspicious: now that node.js started to have a lot of traction the lead developer stepping down is a bit unexpected. My guess is that there is some very exciting new adventure in the Ryan's future, or that sometimes three years are simply enough from a "feelings" point of view. Anyway best fishes for the future.
EDIT: edited a bit in the form to make my thought more clear.
What does this mean for the Node community at large?
I recently had the pleasure of checking out the NodeUp podcasts. A large portion of the first episode is dedicated to enforcing the notion that Node is a new language, and that it will take decades to catch up to the maturity that Python and Perl provide as platforms.
Opening up a subprocess, or having an `os` module which works in Windows, for example, all seemed like daunting tasks that took other languages decades to perfect.
While I'm sure Node can catch up quickly, partly because Javascript is fast to write, and partly because of the trailblazing that other languages have already done (among other reasons, I'm sure), I can't help but think that the loss of the original author's momentum behind the project will cause Node to suffer.
Python has had Guido as the BDFL for over 20 years. Larry Wall is firmly entrenched in the church of Perl. Having one voice shepherding the direction of the language has had tremendous influence on both languages. I can't help but think that node.js is in a similar situation.
> Having one voice shepherding the direction of the language has had tremendous influence on both languages. I can't help but think that node.js is in a similar situation.
Not really. Node.js isn't a language that needs direction; ECMAScript is a separate standard with its own drivers. Node isn't even an implementation of a language, and V8 has its own community directing it.
Node is a small core with pretty simple principles. The rest (the most important part, I'd argue) is a thriving third-party community, and that's something the loss of any single developer won't change.
The difference is that node.js is just an environment for an already established language to run on. Ryan accomplished a lot of great stuff with node, and Isaac will do the same. JavaScript != Node.
Pure conjecture, but didn't he start complaining/ranting about programming around the time Node.js for Windows was being developed? Maybe it burned him out.
Weren't those complaints about nix's complexity though? The complaints about things like the nix filesystem were specific enough that it's a stretch to point toward Windows.
negative: It would have been nice to see a 1.0 release before this happened.
positive: Without the originator in place perhaps a more formal product infrastructure can be put in place by Joyent. I'm thinking in particular about a structure for the documentation of primary third party modules like Connect and Express. At present the readme's and examples on Git, Sencha, and ExpressJs are all done by the original module developers. They're good but like most documentation done by the original developers make too many assumptions about what readers know. I don't mean to be too critical but if users are going to Stackoverflow for answers then there is a problem.
> It would have been nice to see a 1.0 release before this happened.
I dunno, a lot of big players are using node in production, and the back-compat's getting reasonable. Why does the number when you type -v matter?
> Without the originator in place perhaps a more formal product infrastructure can be put in place by Joyent. I'm thinking in particular about a structure for the documentation of primary third party modules...
I think that's the opposite of what Node needs. All of these third party modules came about precisely because there's no blockers or infrastructure to getting started with extending Node. The documentation for some of these modules could definitely be better, but I don't think having a central authority imposing upon the community is a good solution.
> I don't mean to be too critical but if users are going to Stackoverflow for answers then there is a problem.
The reason SO is so huge is precisely because users of pretty much anything will always have problems. Node is not, and never will be, an exception.
"Now that the rewrite on top of libuv is largely complete, I am ceding
my position as gatekeeper to Isaac Schlueter. Our energy will now be
largely focused over the next few months on improving the third party
module system experience including a website for browsing modules, a
new addon build system, and binary installations from npm. Isaac is in
the unique position to bridge the gap between core and external
modules to ensure a pleasant experience. After three years of working
on Node, this frees me up to work on research projects. I am still an
employee at Joyent and will advise from the sidelines but I won't be
involved in the day-to-day bug fixes. Isaac has final say over what
makes it into the releases. Appeals for new features, changes, and bug
fixes should now be directed at him. "
I sort of had a feeling this was coming for a while. Isaac has been doing more and more of the release updates on the blog, not to mention the increasing shift in commits on GitHub from Ryan to Isaac.
Hm. I am just wondering if it wouldn't be better if Ryan could make living of Node and therefore could stay to work on it. How does one make living of Node ... em, charge for it, maybe? Is it so radical to suggest that Node could be a paid product.
Ryan and Isaac were both hired by Joyent after becoming established in the node community. Everyone seems happy with this being an open-source project.
This is very surprising. Node is not mature yet and the guy with "the vision" is stepping down. Makes you wonder if node is heading in a direction different than Ryan envisioned and it's no longer a labor of love.
Note: I actually rather like Matador. I just find this particular style--high density, no semicolons--much harder on my eyes. I'm sure many others can read it better than I can, however.
Leaving aside the global variable, I think this example's biggest issue is the complete lack of whitespace and the big stack of chained calls (which I'm not personally a fan of, but your house, your rules :)) - this is it with a few reader-sympathetic newlines added: https://gist.github.com/1710254
Personally, I find using comma-first and omitting semicolons except where required for ASI reduces the amount of noise down the right-hand side of the code and the amount of scanning I have to do there (for variable definition, object literals and nested calls in particular).
> Leaving aside the global variable, I think this example's biggest issue is the complete lack of whitespace and the big stack of chained calls (which I'm not personally a fan of, but your house, your rules :))
IMO, the whitespace isn't so much an issue as the chained calls. Once you exceed about 5-8 lines of method chaining where your eyes now have some difficulty lining up the indentation (although Sublime really helps in this regard), the lack of visible line terminators can be somewhat obnoxious to someone who comes from a background of C/C-inspired languages.
I agree with Isaac's assertions that it's important to understand line terminating, and I also agree with his other statements regarding pants. I prefer to wear pants unless I'm hanging out in my Python room. It's just easier on my eyes.
I am used to looking for punctuation on the right hand side because I am a native reader of the English language. So, I don't think I've ever had to exert myself or do any extra scanning to see right-hand side punctuation in Javascript or English.
Commas first cause more noise on the left side; they distract me because I'm used to reading English and I'm used to reading code that's written in English based programming languages where people generally agree that punctuation belongs on the right side.
Honestly though, I care more about comments than coding style, something that the Joyent guys don't seem too fond of despite the complexity of the Node.js code. I just don't get it.
Wow, what was the last language whose creator stepped down so quickly? Either way, Isaac has been a huge source of energy since the beginning, so im glad its going to him.
Node.js is in great hands with Isaac. Unlike many other web/network platforms node doesn't go around gobbling up everything into core. The community behind node is stronger and contributes more than I've seen before. Focusing on that third party module system experience is a good move to further grow the community, one that Isaac has been heading for a while.
While many people will see the BDFL stepping down from a project as a bad sign, I think its the opposite. It has a dedicated and invested company behind it (Joyent), it's core doesn't try to do too much, it has legs of its own strong enough and it's community fault tolerant enough to grow quickly without him.
Ryan spawned a project that grew a community that in the long run isn't reliant on him, a rare achievement of platform/language creators.