> Dependencies of your dependencies! Imagine installing a library that depends on a couple dozen libraries in its own right. You may not have time to inspect every single node in the dependency graph.
Excuse me, are you, the developer, telling me, a user, that you don't know what you're giving me to run on my computer?
Hi my name is node_modules have you met me? If you invite me to a party I bring my 500 closest friends as my plus one!
I just looked at the node_modules folder for a project that runs user configured code when your imap server gets a new mail via imap idle. It's called imapnotify and inside its node_modules dir I see it pulls in 549 js files.
The majority are under 100 lines per file dozens are between 1 and 30.
Cutting just the last segment so that /foo/bar/baz.js and /bam/baz.js both appear as baz.js and removing dupes reduced the number of unique filenames to 383.
Basically every js dev is either telling you or not telling you that they don't know what they are giving you to run on your computer.
There are reasons forums known for being less diplomatic than HN often take delight in heaping scorn and ridicule on js devs. This is one of those reasons.
That is every javascript package ever incidentally. Do you have an alternative suggestion as far as connecting to an imap server and responding to events? Not a library to write such a thing but an actual implementation.
No, I don't, but even if there are no such alternate packages, that doesn't mean it's OK to use one that you can't sufficiently understand.
If you can't find an acceptable package, then I'd absolutely recommend writing that functionality yourself. connecting to an IMAP server and responding to events aren't complicated tasks, after all.
You misunderstand what I'm saying. This isn't about disagreeing with how anything is written, this is about the responsibility of developers to know what they're shipping.
If devs can't do things like be able to tell what dependencies a module/library has, then they shouldn't use that module/library.
With NPM as an example to fully know every dependency is a highly time-intensive and complicated task due to following:
* you can't be sure code in package is same you see on github as npm doesn't do signatures
* code already in node_modules is usually minified and obfuscated
So only real way to be sure is to make own repository, copy every dependency, check code, build own packages and only use packages from your repo. Repeat process when versions are updated.
So to answer your question, yes, 99% of apps, they don't know.
I think that ship sailed a long time ago. Right now I'm building an app with Google's mobile toolkit, Flutter. Flutter itself is a few hundred kloc. And on top of that, I'm using a bunch of libraries; I wouldn't be shocked if it were the same amount of code again. Assuming I can properly code review a few hundred lines per hour, reviewing all that will take me a year or so. Which is maybe 5x what it will take me to ship an MVP.
You are missing the fact that dependency tree is constantly changing as dependencies get bug fixes and security patches, so in the middle of your review you will be forced to re-check.
Unless you are willing to freeze whole tree until review is done but then you don't get bug fixes and security updates for period of initial review + review of updated packages.
Is that a joke? Surely you aren't suggesting that developers should have full knowledge of all transitive dependencies that compose their application stack.
> Is that a joke? Surely you aren't suggesting that developers should have full knowledge of all transitive dependencies that compose their application stack.
This will be borderline impossible for a nodejs developer, but a perfectly realistic expectation for e.g. a Django project.
Honestly, it boggles the mind to think that a responsible developer would just pull in some 3rd party dependency without auditing what it does/calls and what other sub-dependencies it has, and then ship it as their product that their business relies on.
Everywhere I’ve ever worked, adopting a dependent library was a HUGE DEAL. You don’t do it lightly. You have to know what’s in it, know what it calls, know what the license is, understand the increased security attack surface, measure how much bigger it makes your binary, measure any performance deltas, and so on. In bigger companies, you need to get all sorts of approvals... Who the heck are these companies where you just hook up the git submodule and wing it?
Can you even know every transitive dependency in an average Node.js project? I just looked at a simple web service we have, which doesn't use any framework or such, and it's 5 dependencies pull in 75 dependencies altogether.
Since most applications nowadays are server side, you would usually already pull 1 GB+ of Docker layers, before you even add application + libraries.
Facebook on iOS seems to be around 400 MB of size (first result from quick googling, might be wrong), and average Electron app is 120 MB+, so I don't think the size is a huge concern.
For licensing reasons alone, almost got hit by one dependency switching from LGPL 2 to GPL 3 and one project we bought dropped dead once someone went over its dependency tree and noticed the mess of highly incompatible licenses, it would have taken years to get to a point where we could legally publish anything. Not knowing about all of your dependencies is a liability.
Not full knowledge. But you should know about every single dependency, know what they do and if they're from a trustworthy source. Being able to say with confidence that you aren't installing malware on your customers machines is the least you can do.
On the developer side there's more though, you should know what their release cycles are like, how they manage compatibility, how long they support older releases etc. You don't what your app stuck in a 3 month upgrade cycle just because a transitive dependency needed an update.
There's a reason dependencies are there own form of technical debt and need to be minimized.
Knowing every transitive dependency in your application stack is totally unrealistic and is pretty much impossible for any modern commercial software outside of specialized/embedded systems.
Having knowledge of every dependency explicitly linked into your application is a basic tenet of software development that literally every software professional would endorse as a necessary best practice, but it's not what we're talking about here. I am 100% certain that every developer with a commercial Electron app is aware that they are relying on Electron as a dependency and Github as a trustworthy software vendor.
I think you're conflating platforms and dependencies. The OS it runs on is a platform that the the user already trusts, left-pad is a dependency the developers are responsible for. There's some grey area like electron or the JVM but even there electron sticks out as a huge binary blob the user isn't explicitly asked to install.
I never brought up operating systems in the context of this discussion on dependencies. Electron is not an operating system, so I am not sure what your point is.
Chromium and also Electron are trustable sources. Besides that any Mac OS X App from 10.15.1 (Catalina) has to be notarized. I got the dmg notarized so I know that the malware is not the issue. Now asking every ElectronJS developer to know every single part of Chromium or even electron itself is a bit unreasonable.
as CTO I know all the dependencies in the application stack of our company and I require developers to have sufficient understanding of their purpose, licensing model etc.
There are tools to simplify this task. It’s much harder to do for TypeScript than for Java, mostly because NPM ecosystem is a huge pile of junk, but it’s still important part of developer's job.
> as CTO I know all the dependencies in the application stack of our company
You know all of the transitive dependencies in the application stack of your company? Out of curiosity, what is the ballpark figure on the total number of transitive dependencies in your application?
as CTO I know all the dependencies in the application stack of our company
I'm a few years out of the Java world these days, but you've vetted every line in every one of those Apache commons projects that gets pulled in? Because that's a lot of reading...
Over 10 years ago I went through every single Java RSS parsing library and all their forks (I think there were 5) to fix Xml eXternal Entity (XXE) attacks (eg[1]) and submitted fixes for them all.
That was a huge amount of work, because they all used random version of different XML parsers, all of which required external entity processing to be turned off in different ways.
None of the authors of the packages were aware of the issue, and as far I could determine at the time I was the only person in the entire Java ecosystem who knew all the different parsers arguments.
This was a critical, remotely exploitable vulnerability. I'd never claim to know about it now, and it was only for a few weeks back in 2006 that I really knew it then.
So when you say you "know" all the dependencies in your stack, what do you mean exactly?
We barely have any of the Apache commons dependencies - in most cases use of them is not justified in modern Java. If there’s anything like that we are ready to support such dependency on source code level, patching and setting up own build pipeline if necessary. Free software cannot be trusted if you are not ready to own it. Commercial software cannot be trusted either, but there are other ways to manage the risks there.
I can't believe you check all the source code, dependencies source code and the rest of the shit show that npm brings to the party, that's very cool, all CTOs should be like you. As a CTO should you have to do this though? I'm not sure.
We are growing startup and it’s important to have this culture of understanding what and how you build from the very beginning. This responsibility is now transferred to technical leads, but I’m still aware of what’s going on and spend some time on the code reviews.
Usually you don't write a project and then sit down to read through all dependencies, but it's a process that happens when you do add a dependency.
So, the process would be something like: have a problem that a library might solve, find a library that could solve the problem, verify it can solve the problem, look through the implementation, look through it's dependencies, then actually include it in the build. If anything doesn't look nice at each step, rollback and find something else or start writing your own implementation.
Do this from beginning of the project and you'll always have a full understanding of the system.
I think that's not an unreasonable thing to expect at all. If the libraries you're using make that impossible, then you shouldn't use those libraries. You should know what you're shipping.
As a developer, I'm honestly and utterly stunned to see other developers arguing otherwise.
Excuse me, are you, the developer, telling me, a user, that you don't know what you're giving me to run on my computer?