Hacker News new | past | comments | ask | show | jobs | submit login
Uutils: an attempt at writing cross-platform CLI utilities in Rust (github.com/uutils)
243 points by GhettoComputers on Dec 6, 2021 | hide | past | favorite | 314 comments




What is the progress over years?


One fun recent development in 2021: https://sylvestre.ledru.info/blog/2021/03/09/debian-running-...

> tldr: Rust/coreutils ( https://github.com/uutils/coreutils/ ) is now available in Debian, good enough to boot a Debian with GNOME, install the top 1000 packages, build Firefox, the Linux Kernel and LLVM/Clang.


Thanks, I heard a lot of people saying its not complete but I haven't had any issues with it so far, pretty huge endorsement to see its even a debian package, that's a huge sign of maturity.


As a GNU coreutils maintainer, I'm happy to see another implementation. A lot of the coreutils development effort goes into testing, and new implementations should be able to leverage the existing GNU coreutils test suite, as it just uses whatever utilities are in the $PATH

BTW some notes on how the GNU coreutils are tested are at: https://www.pixelbeat.org/docs/coreutils-testing.html


> A lot of the coreutils development effort goes into testing, and new implementations should be able to leverage the existing GNU coreutils test suite, as it just uses whatever utilities are in the $PATH

This makes me really happy.

Separately from this, I've been wondering for a long time if there's a way for standards (and de facto standards) to share test suites for other implementers to re-use. Sort of a npm but only for test suites. Does such a thing exist? I wrote a TOML parser recently and had to re-derive the test suite from the specs.


I've started thinking about this last year and wrote down some thoughts in this draft post: https://ttm.sh/2l3.md

> Could we get the best of both worlds by treating specification and compliance (testing) as a single problem? This hypothetical approach i call specification-driven development, whereby a specification document is intended both for human and machine consumption. In that case, the specification contains a written presentation of concepts, in addition to a machine-readable test suite that follows a certain format to programmatically ensure that the concepts and behavior described in the specification are implemented properly.

I've centered the document on my personal usecases (CLI and sysadmin checks) but i don't see a reason it couldn't be employed for API/ABI checks.


I've been thinking about that when working with recurrence rule libraries. It's a standard about calendaring and there's millions of edge-cases: developping your own library for prod purposes is a huge risk and most of the effort would probably be in tests. If all these projects shared a test suite, they could:

- Fix their own existing bugs - Help other projects to fix their bugs - Be explicit about what they support and what they don't - Help writing new libraries (eg in a faster language, or for another ecosystem)


OpenGL, for example, has an official test suite as well as one or more open source ones (I know of piglit).


Indeed, I'm picturing sort of a canonical repository for test cases (with a common interface). I would love to have a place I can obtain a test suite for a given standard, execute it, and even potentially publish my conformance. Potentially even have a badge to post on GitHub repos indicating conformance to a certain version of the standard.


Public tests are good but relying on public tests only encourages kludges to make tools pass the tests. A form of overfiting. That's why some compression tests do not publish their test corpora.


But compression tests only have that issue because there are degrees of success for those tests, even for the same compression algorithm. I don't think they're hiding the test vectors because they're worried about gaming the tests by purposely failing to process valid inputs to achieve better metrics, just writing overly specific heuristics.

For straightforward corner case acceptance tests (which I would assume covers most of the coreutils test suite) there's not really a danger of overfitting unless the developers are literally writing if statements that match a single input from the test and provide the correct output.


That feels like an issue to be solved by the tests. If a program conforms to the public test suite the program "works" or the tests aren't covering the specification.


For compression tests it's a little different, since the problem is often underdefined even for a fixed algorithm and different implementations may produce encodings with different efficiencies (space, compression time, decompression time) for different inputs. Compression implementations can overfit on some inputs and produce subpar results on average even if they produce valid outputs for all inputs.

However I doubt this applies to coreutils' tests, which I suspect are more about conformance.


That makes sense if the public tests are "compress these bytes, expect this output" but I'd expect instead to have a lot of specific components with their own individual tests.


Do you think this applies if a corpus contains both affirmative and negative tests? As in, including not just conforming JSON but also a set of JSON that should be rejected due to non-conformance? I agree it could be challenging for instance for compression - where there is a more challenging definition of 'wrong.' I'm just wondering if this idea has legs, and appreciate your thoughts.


I can't answer with certainty, but I think any fixed set of tests can be cheated with overfiting.


I do think that's a material risk - however broadly, do you think that such a scheme would make ecosystem better or worse? If you made a hypothetical JSON implementation in your language of choice, would you use it?


> I do think that's a material risk - however broadly, do you think that such a scheme would make ecosystem better or worse?

I don't think it would be easy to cheat if:

  - tested implementation is open source: it would make cheating too obvious,

  - tests are constantly updated: it would make cheating too cumbersome and

  - tests include a randomization: it would not always work.
So, satisfying these points would drastically increase trust on the test corpus and tested program.

> If you made a hypothetical JSON implementation in your language of choice, would you use it?

On my machine? I use my own hacked kernel on my machine! In production? Only if tests indicate my implementation is as good as the best ones available.


Gave it a whirl with `cp`:

    > ./target/debug/cp /dev/null /dev/zero
    thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" }', src/uu/cp/src/cp.rs:1295:54
Good advice is "don't panic." Trying `mv`:

    > ./target/debug/mv . .
    ./target/debug/mv: cannot stat '.': No such file or directory
oof

Anyways the point is that there's lots of legitimate institutional knowledge baked into coreutils and a naive RiiR effort will re-introduce previously fixed bugs.


Both are quite naive Errors I must say. The good thing about Rust is that it doesn't hide the complexities of the file system from you — it requires you to explicitly handle the edge cases. OR you know, you could just write a cp that panics when there is a PermissionDenied because you decided that never happens and happily unwrapped the Result.

The last one is a different kind. Rusts std lib indeed cannot stat . Which means they would have to translate . into the coreutils meaning manually. Their tests should have cought that one tho.


> Rusts std lib indeed cannot stat .

But the kernel can! I mean, stat() isn't a standard library function, it's a system call with (reasonably) well-defined semantics. And "." is a valid path, which returns a valid struct stat block representing a guaranteed-valid (cwd is always live; you're always "somewhere" even if the filesystem has removed the directory) directory.

Coming at this from the perspective of "translate . into the coreutils meaning" is almost certainly the wrong way to think about it. It's rust that has the broken picture of the filesystem environment, coreutils is just doing unix.


On Linux.

This isn't a Linux project, it is a cross-platform project.


No, stat() of "." predates linux by almost two decades. And coreutils is a set of unix tools. You can build them on other systems, but only via emulation layers that are reponsible for handling the friction (and those layers would likewise be subject to this same logic).

Or I guess what you're saying is that uutils is a cross platform project? Which is true enough, but it's still responsible for faithfully representing the behavior of the underlying system. And when running uutils on linux, stating "." (in this case to detect that src and dst are the same file) is legal and valid.

It's true there are other ways to solve the same problem than emulating a linux syscall layer, but you have to pick one. You don't get to use "the standard library can't stat ." or "we have to run on windows" as an excuse.


Nobody is saying this is not a bug. I am saying that "It's rust that has the broken picture of the filesystem environment" is not true in any way. Rust implements a cross-platform filesystem API. The fact that it does not work the same way as the Linux kernel does not make it "broken".


That's a little out of context. The situation being described[1] is that "rust cannot stat the path '.'". A core requirement of coreutils is to work with a unix filesystem where "." is a valid path. Ergo, from the point of view of the requirements at hand, the rust filesystem layer is "broken". It doesn't do what uutils needs it to do.

[1] And I'll admit that I don't have any understanding of the actual low level issue here or why native pathnames are being interpreted by the runtime and not passed through to the OS layer.


Because if you just pass them through, you will not have consistent behaviour across platforms. That was the original point: Rust implements a consistent, cross-platform filesystem API. It does not necessarily match whatever your platform happens to do.


But uutils needs to necessarily match whatever your platform happens to do, or else it breaks[1] in silly ways like this when it turns out that "." stops being a path.

[1] Maybe this is all just a semantic argument about this use of the word "broken"? This is a long-standing usage. It doesn't mean that Rust's standard library isn't useful for anything, it means that it isn't useful here because of design choices that don't match the problem at hand. If it were something that can be fixed, it would just be "buggy". But it can't (for the reasons you mention!). Therefore it's "broken", not "buggy".


What do you mean with "Rusts std lib indeed cannot stat ."? I wasn't able to find something related to this on the web.


I'm not sure what they mean either, this seems to succeed: https://play.rust-lang.org/?version=stable&mode=debug&editio...


Note that in the readme `cp` is listed as semi-done.


Wtf is "semi-done"?

There is "done" and there is "not done"...


I imagine "semi-done" means "works enough that it can be dogfooded by the devs and tried out by the curious among us, but does not exactly mirror behaviour of GNU cp in terms of corner cases and error output".

I think we need to be more flexible than classifying things as done/not-done. Even GNU coreutils has open issues[0] - is it "done" or "not done"?

[0] = https://debbugs.gnu.org/cgi/pkgreport.cgi?pkg=coreutils


If that's how you categorize software, then no software has ever been "done". There are always new cases, bugs, features, and performance improvements. You will never run out of any one of those things.

In this case, semi-done probably means what it usually means: some common cases are handled, while others aren't yet.


if you believe that, there is not a single piece of done software in the world, besides perhaps tex.


"semi-done" means:

1. Not done. :-(

2. A lot has been done already, i.e. not just-started.

3. Not almost-done, i.e. there's significant work still to be done


In that case at least one of "done" and "not done" needs to be a spectrum; semi-done is a point on one of those spectra


You think that's confusing? Wait until you hear about half "A" presses.


Portions of code could be completed and others are not? See semifinished.


WIP? Dude, chill… you’re not paying for it


Could not get mv to fail as you did on the Mac or Linux. Would you re-run with the latest bits and let me know what your platform is? I am going to fix these two bugs, if they still exist.


Are you running an irregular version of libc? Something else unusual?


Same with cp. Could not reproduce on the Mac or Linux.


So it doesn't work for cases where it shouldn't work. And it errors out but doesn't do anything stupid

I don't see the problem for something that was just released as a very preliminary version

(Though I don't know where cp is getting "PermissionDenied" from)


> I don't see the problem for something that was just released as a very preliminary version

Fwiw the project os half a decade old.

> (Though I don't know where cp is getting "PermissionDenied" from)

Have not looked at the problematic code but e.g. copy_file_range(2) says

> EPERM fd_out refers to an immutable file.


While that is a good point, it is quite unsightly to panic from unwrapping. Instead it should propagate that error back to the top level where a nicer error message can be generated.


More importantly cp should continue copying other inputs even if one of the source files fails to be readable. Handling errors by panic doesn't suggest that will be the case.


I don’t know if that is _more_ important, but I agree that it is also important :)

The reason I say this is that while users will expect this behavior, doing it well (so that it can be parallelized, for example) requires careful thinking about how errors should flow, how lines are printed to stdout, etc.


There is much to love about this!

GNU coreutils is one of the pillars of our civilization. Re-implementing it in several new languages can only benefit everybody, as it will lead to a higher understanding of all the implementations, including the original one.

So far, the tests of this Rust version seem to be used to compare against GNU coreutils, with a goal to attain feature parity. I hope in the near future, the careful memory management of Rust will motivate new tests that will be passed by the Rust version but failed bu the C one (e.g., some memory leak, or an input that triggers a segfault in the C version). Then, you will see a new curve in the graph concerning the number of tests failed by the C version, which at some point will cross the decreasing curve of the Rust version! The coverage of the rust implementation is still a bit sparse for that, but hopefully it will improve.

There is one feature, though, that is unfortunately missing in this Rust re-implementation: the fact that users of coreutils will always be able to examine the source code of the programs that they run. This is guaranteed by the original copyleft license. Βut this Rust version uses a "stealable" license that allows distributors of these programs to strip users of that right. This is really sad.


I'm writing a book "Rust From the Ground Up" which teaches Rust by rewriting a coreutil per chapter, from the original BSD sources. The first 3 chapters (true/false, yes, and head) are released. wc is next, then cat, cut, rev, and uniq.

https://rftgu.rs/


The book looks very interesting. By when do you plan to complete the next set of chapters?


Thanks! I've written the whole book - now I'm going through and editing and typesetting each chapter and trying to publish one a month.


That's wonderful. Do you have twitter? I just want to be notified once the book is complete.


Yes I’m posting updates here: https://mobile.twitter.com/rustftgu

Thanks for the interest!


Nixpkgs has a PR for replacing the build environment of packages with uutils-coreutils[0]. From testing by building various packages over a dozen bugs were reported upstream, but a lot of software does build successfully with uutils[1][2], after upstream fixes, including chromium, vlc, nix, rustc, emacs, vim, uutils itself etc.

It's an interesting experiment to replace such a fundamental dependency with a rewrite. A lot of the build failures are due to uutils not implementing GNU extensions, some of which can get pretty involved.

[0] https://github.com/NixOS/nixpkgs/pull/116274

[1] https://github.com/NixOS/nixpkgs/pull/116274#issuecomment-85...

[2] https://github.com/NixOS/nixpkgs/pull/116274#issuecomment-86...


I'm not quite sure I follow; is this a proposal to actually use this in nix, or just a PoC that it could work?


The latter! If merged the package set that uses uutils for stdenv would be accessible via an overlay (similar to the cross-compilation infra we have), so one could build pkgsUutils.hello to build GNU Hello, for instance.


The comments complaining about the choice of license reminded me that a FSF licensing intern did the same, and demanded it be changed due to some false claim about it being a derivative work[1]. Let me remind everyone that there was probably a good reason for using a permissive license, and no-one has the right to demand changes based on their definition of freedom - unless they stepped up and contributed.

If this truly injures you, you can either do your own rewrite or, like Muse Group, outright buy the rights to the software and relicense.

[1] https://github.com/uutils/coreutils/issues/834


There's also this issue by FSF director Ian Kelling:

https://github.com/uutils/coreutils/issues/1781

Honestly, it sounds like they're deeply scared of long-standing FSF-backed software having to compete (or even be replaced) with alternatives that just happen not to be GPL (e.g. they have similar thoughts on LLVM/clang), to the point where they decide to troll the issue trackers. It feels really childish.


> with alternatives that just happen not to be GPL (e.g. they have similar thoughts on LLVM/clang)

LLVM/clang is worse than just GPL, as far as I understand the design of GCC is intentionally horribly bad to make it painful to integrate with a proprietary tool chain. Of course this means that things like refactoring support end up being basically impossible to build on GCC, so people move to clang out of necessity.


It's both a technical issue and a political issue; RMS himself has repeatedly advocated against interfaces that would expose internal representations that GCC uses because they could be used to build proprietary add-ons.

These days GCC does have a plugin interface, but they came up with a really funny hack to try to stop people from using proprietary plugins. Programs compiled with gcc can require linking with libgcc, and the libgcc license is "GPL, except it doesn't apply to software compiled with GCC and all-Free plugins". So their idea is that if you add a proprietary plug-in to GCC, that makes all code compiled like that have to be GPLed instead.

It is, of course, questionable whether this hack would hold up in court. It is also rather useless, because it's not hard to reimplement enough of libgcc to make it work. For example, the Linux kernel only links with libgcc for a few architectures.


Something something those who give up software freedom for software's usefulness deserve neither something something.


> Honestly, it sounds like they're deeply scared of long-standing FSF-backed software having to compete (or even be replaced) with alternatives that just happen not to be GPL

They are, for good reason. Big companies hate open source, despite their claims to the contrary. Look at Apple's proprietary forks of all the BSD stuff, and handset manufacturers' proprietary forks of Android. The FSF should absolutely be doing everything in their power to minimize how often this happens.


That's an argument you can certainly make. Unfortunately for them, as it turns out, deciding licenses for other projects is not in their power, and trolling other projects' issue trackers is counter-productive to the goal of convincing them change their license.


> Look at Apple's proprietary forks of all the BSD stuff

What forks of what stuff?


OS X, Darwin off the top of my head


https://opensource.apple.com/release/macos-115.html

Here's the kernel, including all the BSD stuff:

https://opensource.apple.com/source/xnu/xnu-7195.141.2/

You can compile it yourself and run macOS with your own kernel build. Here's a blog by Apple's head of XNU development explaining how that works:

https://kernelshaman.blogspot.com/2021/02/building-xnu-for-m...


Yeah and the open source people gave up, it’s like giving you a free recipe for a beef Wellington in a world they control of food. Nothing uses Darwin besides Apple


And? The source is there. There is no requirement to do extra work to make it useful for anyone but yourself. They did not even need to release the source, but they did.

You made a claim about "proprietary forks of all the BSD stuff". The kernel is open source. The rest of the OS is written from scratch by Apple, and originally by NeXT, and is not a fork of anything.

Are you standing by your original claim?


I never claimed that.



If anyone else wants to do this as a learning exercise I found https://www.maizure.org/projects/decoded-gnu-coreutils/ which goes into design and code of most of the utilities from the original coreutils.


Should be "uutils is an *attempt* at rewriting coreutils in Rust"

Not to shit on the project, just that the title made it seem like something ready to use .. (most of the tests are still failing)


Title changed to that from "Coreutils Rewritten in Rust". Thanks for the heads-up!


This will probably sound completely insane, not least of all to people who prefer non-GNU coreutils, but I kinda hope that as uutils matures, some shells will offer an option to compile in the uutils rewrites as builtins. Especially for newer shells and shells that want to target Windows in a first-class way, that could be a real portability win.


Why is WSL[1] insufficient for Windows portability?

1. https://docs.microsoft.com/en-us/windows/wsl/about


WSL is great if you want/need a full Linux environment. It is quite heavy and probably overkill if all you want is to use some coreutil commands from the Windows terminal.


Gitbash is a relatively nice solution for this.


WSL1 is very light in my experience.


I wasn't primarily concerned with Windows portability, but with the portability of scripts altogether, and no longer having to concern oneself with variations in coreutils versions for at least some scripts, even if they run on multiple platforms.

But to answer your question about Windows:

Because requiring an emulation layer is a huge step up in complexity from being able to distribute your shell as a single static binary which is just a few MB. Right now Elvish and Nushell, for example, can be deployed on Windows as binaries that don't require any external dependencies, any installation procedure, or any configuration. If they could link against uutils statically, they could also come with basic utilities without having an environment to manage.

WSL (along with MSYS2 and Cygwin) is deeply stateful. Now to make sure everything is working you have to check the compatibility of your whole system, and the versions of coreutils can still vary in addition to the version of the shell interpreter.

WSL2 also requires virtualization, which can complicate using it under virtual machines.

You may not want everything you run under your shell to depend on WSL, which is also happens with WSL2.

Even with WSL1, a script that requires a chroot environment to deploy and run it is way less portable than one that doesn't.


Because WSL is not Windows. It's a fantastic tool but it's not actual native Windows support. I use Windows to develop software on a daily basis, and I don't have WSL installed at all.


More interesting than rewriting the GNU coreutils are the projects that are reinventing them. Like ripgrep (which built on the example of ack) is a better grep, by discarding backwards compatibility. Or fd, which abandons find's arcane syntax. We've learned a thing or two since the 70s when most of these tools were first invented.


Agreed, it would be an interesting idea to make a new coreutils with everything modernised/cleaned up to be easier to use and more consistent.



Tried it too bad no shell compares to fish for ease of use. Every new shell gets asked "can it do the thing in fish" and the answer is always kind of or no, zsh did not function as well even with the configs and it seemed much slower.


I was reminded yet again how terribly non intuitive the core file-system commands are for end user on the command line. Maybe having Rust implementations readily available would allow people to fork them and improve the interface in some new shell+utils combo package.

(rm, cp require -r, mv doesn’t, no auto-directory creation for mv - if you don’t believe me look at git/hg which don’t copy the semantics + I’d like reliable dry run and by default confirm every destructive action)


> rm, cp require -r, mv doesn’t, no auto-directory creation for mv

These are all like that on purpose.

mv doesn't have -r because it's a safety feature for rm and cp, since deleting or copying an entire directory is expensive so you should have to specify it explicitly. Moving a directory is really just renaming it, which isn't expensive when it's on the same filesystem (and it usually is).

The most common case for the destination directory not existing when moving something is a typo in the path. Then you'd end up creating a directory you didn't intend to, possibly on a filesystem you didn't intend to, and moving all the other arguments over there. It could be useful to make it possible to explicitly specify that you want this, similar to mkdir -p, but you could add that to the existing mv without breaking anything.


An inspectable and reliable dry run option and/or an interactive confirmation would solve some of these problems.


Not really. Dry run is useful for complex commands you're not sure the effects of. These options are so that when you intend to type:

rm /home/john/some/file.txt

But you accidentally type:

rm /home/john some/file.txt

It says:

rm: cannot remove '/home/john': Is a directory

Instead of john not having a home directory anymore.

And people aren't going to use a dry run option or interactive confirmation every time they run a simple command like that.


I was thinking more of

- listing how many files/directories would be moved

- listing how many files/directories would be overwritten

- telling whether the transfer will cross drives (especially for mv)

reasonably speaking this should not be included in the base posix cp and mv, but could be maybe provided as intrinsics in bash for example


The expensiveness is a bit naïve nowadays. Same for assuming typos by default.


Deleting entire directories by accident is expensive in more than computing resources.

And copying them is still actually expensive. Data expands to consume all available space. Accidentally copy a 16TB directory structure and you're going to max out the I/O on your machine for hours and maybe run it out of space. It's not a big ask to type two characters to confirm that you want to do that.


It’s just inconsistent. There are a infinite possible typos that range from expensive to destructive.


> rm, cp require -r, mv doesn’t, no auto-directory creation for mv

These make total sense for me.


why MIT license ? that is bound to be turnoff for a lot of folks.

also, a minor nit: there are still a lot of work [1] that needs to be done, and imho, it is a bit premature to title this article as is presented here.

[1] https://github.com/uutils/coreutils#utilities


Who is MIT a turnoff for? It’s strictly more permissive on the consumer side than GPL is.

Which isn’t to say anything about how a project ought to be licensed; just that MIT enjoys overwhelming popularity with newer projects.


> Who is MIT a turnoff for?

For me, for example. I personally prefer my foundations on xGPL (preferably V3 and later), because some company or set of companies just can't fork and run away with it.

I personally consider computing utilities and compilers essential infrastructure and their sustainability while being completely transparent are critical for me.


I'd argue the opposite. xGPL makes it easier for the founding company to just run away with it. We saw it with MongoDB, where it being AGPL means one company can control and unilaterally relicense it. Other examples I'd see of being GPLed doing little to nothing to prevent such shenanigans are any Oracle owned GPLed properties -- Java, MySQL, and VirtualBox all have user-hostile projects and misfeatures added even with the GPL. Conversely, permissive Free projects like LLVM and Postgres have had a lot harder time with one company controlling, because its non-copyleft nature means that everyone has a fair footing in controlling the direction.


The problem is not xGPL, it's the copyright transfer. If you don't force copyright transfer on the patches you accept, you can't relicense a code overnight.

All "xGPL to shenanigans" incidents have underlying copyright transfer problems. Recently, an emulator had gone the same way. They asked for copyright transfer to be able to relicense from GPL, and some folks here have used derogatory adjectives for people who didn't want to transfer their copyrights.


Java -> OpenJDK

MySQL -> MariaDB

OpenOffice -> LibreOffice

Chrome -> Chromium

VSCode -> VSCodium

and so on.

GPL protects my rights.


Ah that is why clang in now loosing the race to being C++20, while two of its major founders are more than happy with C++17 for their OS stacks and main languages on those stacks.

So where are the others stepping in to fill the void reaching C++20 compliance and catching up to GCC and VC++?


But I work for a company, and I want to run off with these things.


Why obeying GPL and sharing your improvements is bad?

RedHat built a company on that model? Provide value and GPL won't threaten your business model.


It’s not bad, it’s just that I’m not allowed to. I don’t like to inflict that on others.

Seeing something useful and then having to do it myself anyway because I cannot use it due to the license is painful.


What I know is if projects use permissive licenses, the software end users getting is usually proprietary. Copyleft is designed to prevent that from happening.


It doesn't matter if some other project is proprietary. The proprietary company's proprietary product was always going to be proprietary.

What matters is whether they contribute back anything to open source. And it's a hell of a lot easier to get Legal to sign off on contributing back bug fixes and enhancements on a piecemeal basis rather than adding a recurring obligation to the books.


> The proprietary company's proprietary product was always going to be proprietary.

The point of copyleft is to make this not true. Most companies don't have the resources to reimplement Linux from scratch, so they won't be allowed to make their drivers/kernel modifications proprietary.


I know that's the theory, but I don't see that happening in practice. See, e.g., TiVo, Android, NVIDIA drivers...


> The proprietary company's proprietary product was always going to be proprietary.

A lot is companies would have preferred to have proprietary operating systems, but the Linux license prevents them.


Plenty of companies aren't hindered by the Linux license as they are not required to give you the source most of the time, they just point you to their SoC vendor. Or worse, they just don't use Linux and use BSD or Windows Embedded instead.

I don't see this as a valid point.


> Who is MIT a turnoff for?

For me. I want my software to be as much gplv3 as possible. Not trolling. Also not arguing, it's just the way I roll.


I would honestly like to understand why the downvotes. I am giving a blatantly honest answer of a case of people like me who would be turned off by a licence. I don't expect anyone to necesserily agree but I don't get why my opinion is being downvoted. Is it insulting or irrelevant to the issue or what?


I think it is a valid answer to a question that has been posed. I think the reason for the down-votes is that you did not elaborate on as to why MIT is a turnoff for you, and/or why you would rather prefer GPLv3 over MIT.

...or perhaps they did not like the last part of your comment, the "it's just the way I roll" one.


Maybe you're right. I thought it is understood why pro-gplv3 people like the licence.

Just to elaborate then, I personally want my software to use gplv3 because I side with the ideological/ethical aspects of free software. I want to support projects/teams/orgs that build this future. An mit licence would allow a company to capitalise on the community's effort withought giving nothing back. Or even worse, make modifications and deliver closed source blobs to users. I personally do not like that. Thus, I wouldn't support/use this library.

To be clear, I am not running 100% gpl software atm, but I see it as a journey going there, slowly transitioning.


> Who is MIT a turnoff for? It’s strictly more permissive on the consumer side than GPL is.

well, ever wondered why router vendors include GPL license paper in the box? this [1] is why...

[1] https://www.crn.com/news/applications-os/205100091/busybox-s...


Didn’t they just switch to toybox? https://en.wikipedia.org/wiki/Toybox


Apple, Sony and Nintendo are quite happy with it.

FreeBSD and clang communities, maybe not so much.


MIT is a turnoff for potential contributors, not for users. A lot of people don't want to see their work that was done without compensation end up in some company's proprietary fork.


Is it much better that they just credit them in some obscure notes nobody reads?


The major issue is that the companies don't want to, in CC terms, "share alike". Attribution is a lesser concern.


Wouldn't you run into the same problem with a company that runs linux computers and never contributes back?


> It’s strictly more permissive on the consumer side than GPL is.

You have been misinformed. The GPL is a much better license for users. MIT is arguably better for the authors because they can deny certain freedoms to their users (the ability to change the code on a system, for instance).


Again, not interested in arguing the actual ethics of the licenses. But I think most people would consider a license that allows modification without demanding redistribution to be more individually permissive than one that requires redistribution. Companies love MIT and it’s ilk, because it’s more permissive of their (arguably poor) behavior.


It's more permissive, true, but it's not more free. As an analogy, consider that taking away people's permission to own slaves didn't make society less free.


The fact that you need to add a specifier for that 'individually' permissive says everything about the difference in the licenses' goals.


GPL3 or GPL2? After what github did with copilot, Chinese companies that don’t release source code, or western companies that do the same I don’t see any ways to enforce them, you just need to trust the authors or the users. Has there been a landmark case that solidifies any of this? Chinese companies don’t follow US IP either so I find it hard to believe they can do anything.


How does MIT prevent users changing the code? They have explicit permission to take the code and change it however they want.


Not if you don't get the source with the binary.


It doesn't prevent that directly. What it does is allow proprietary forks, and those proprietary forks prevent that.


MIT vs GPL brought up again?

Flame war! Flame war! Flame war!


Ok, let's go!

You're stupid and probably ugly for wanting a flamewar!


No, he's right. We're rubber and you're glue!


Is it? I thought it was as standard as and on pretty much equal footing with BSD and Apache licenses. Is it substantially different?


Who, exactly?

I think most don’t care, and big corps love the MIT license


Don’t they love them all? Free software in general is nice, PS3 ran Linux, was based on BSD, and I’m sure they used MIT software somewhere there too.


Big corps love permissive licenses; they do not love all open source licenses, such as AGPL.


I never heard of it, and I never seen it being used, whats a big project that uses it?



>Big corps love permissive licenses; they do not love all open source licenses, such as AGPL.

Doesn't Microsoft love using Linux for azure, and contributes to it heavily in code and money? Using binary blobs is how they can get around others using their contributions, and it seems like copilot has made a mockery of all the licenses anyway.

>Mastodon

Oh interesting, looks like it was forked by corporations though including Trump's Truth Media, and Gab, and Truth Media just shows the boilerplate github source code. To me it still seems they love all free code.


What would be a better license ? (If you want GPL you can keep using GNU...)


Obviously a few of these need work, but I've been in the unfortunate position of recovering a Linux system with a busted .so that broke almost all of coreutils, but cargo worked and so did the coreutils alternatives. Static linking is an absolute godsend.


In the old days, /sbin guarded statically linked binaries to used before dynamic link is available. Would surely help in this situation. Don't know exactly why that changed.

EDIT: Just checked, it is still there! And... It doesn't contains sudo. Can't see a good reason why.


Use busybox instead.


Doesn't help when `sudo` is broken ...


How do these utils help when sudo is broken?


What I wound up doing was getting a working shared library and editing the LD_PRELOAD, then getting sudo back to use normal coreutils.


I thought you would just end up doing everything as root lol


sudo and coreutils being broken are separate things though.


Glad that this is MIT. Copyleft is great but it's important to have non-viral alternatives.


"non-viral"? Sounds like late 90s Balmer talking.

(I understand being worried about the license of libs you're going to deploy, but the license of cli utils isn't something you need to worry about "infecting" your proprietary code.)


MS contributed to Linux because their code was "infected" by GPL code from linux.


Why is it important?


This is very cool.

The problem with POSIX is that, while it's possible to implement the bare minimum, it's hard to not have a few extensions. There are some truly braindead ideas that GNU coreutils have absolutely improved upon.

Sadly, a new coreutils collection simply brings further incompatibility. There's already significant switching logic out there to handle BSD vs GNU coreutils (have you ever used sed -i?), adding another flavor makes this sort of thing dead on arrival, at least to me. I'm not retrofitting my scripts to support these implementations too.

Aside, Nix is the only real solution to this problem, as it can replace these tools wholesale, hermetically. Being able to depend on a specific implementation in a portable manner is rather compelling.


> Many GNU, Linux and other utilities are useful, and obviously some effort has been spent in the past to port them to Windows. However, those projects are written in platform-specific C, a language considered unsafe compared to Rust, and have other issues.

I wonder why some people in the Rust community seem to remind us every few sentences how Rust is superior to C because it's safer.

/s And now it's C, yesterday it was C++... what's next, ASM ?


Because they're justifying a project whose sole difference to the other implementation is the language its written in.


So it's more like a "Look, it's possible to do this in Rust!"

than

"We did it in Rust, and we believe it's better because ..."

?


It literally says why it is better and you even quoted it:

> However, those projects are written in platform-specific C, a language considered unsafe compared to Rust

Or to paraphrase it using your wording: "We did it in Rust and we believe it is better because it does not have memory unsafety issues that are the number #1 reason for security issues every year"


So without playing dumb or anything, you say they say:

"We have security issues every year because we program in unsafe languages like C, and to solve this, we should start rewriting things in Rust because it's 'memory safe'."

Amazing!


If you fancy living on the edge, there's already an AUR package 'coreutils-hybrid' that uses stable uutils (with unprefixed names) where possible, falling back on GNU coreutils:

https://aur.archlinux.org/packages/coreutils-hybrid


I wonder if distros (on the maintainers side) really want to move to more Rust in such core pieces? But maybe, as the readme here suggests, this is more for non-Linux platforms?

I know in GNU Guix the more Rust is used has lead to struggles in packaging and building from source for non-x86-64 architectures (demands of building the Rust toolchain, bootstrapping), e.g. [0]. With something like librsvg [1] being a very low level dependency now, Rust has become rather integral to many parts, with any changes to these libraries requiring massive rebuilds for a source-based reproducible distro like GNU Guix.

[0] https://lists.gnu.org/r/guix-devel/2021-11/msg00197.html

[1] https://gitlab.gnome.org/GNOME/librsvg


I've contributed a few small bits to uutils and in the testing I did, on those few small bits, the performance case is pretty exciting. Much easier to do concurrency with rayon, etc.


I’ve spoken to my friend about this and why it’s so much faster, he said it could be done in c as well but rust is much easier to write. Pretty impressed with rust, I just heard many people insulting it at first, throwing around “memory safe” and it seemed like rust programmers were trying to ingress into established programs and there was a great deal of hatred for it.

I fell in love with the performance of rust CLI tools and mainly see its utility as a fast performing binary, memory safety is a bonus but if I had known about its performance earlier I would have ignored anti rust propaganda like quoting Linus saying “Nothing better than C”.


Yeah, the hate for this project is ridiculous. I'm not a professional programmer. I did it just for fun, because I wanted to learn Rust. Some of these tools need someone to fill out some basic functionality, and you probably use some of these tools everyday, so you already know the spec. (FWIW I'd recommend contributing to anyone who is looking for a basic Rust project. It has a very welcoming project leadership.)

And, yes, so-called "free concurrency" ended up being an incredible performance story, although, full disclosure, I/we were just able to use rayon in many places because Rust is just that composable.


Sorry if that came off as "hate" or being dismissive, did not intend it that way! Just genuinely curious given the difficulties I'm seeing on the distro side around Rust packaging, at least for Guix that goes from source in a reproducible/bootstrapable way (which seems very difficult beyond x86-64).


Oh, not at all! I'm sorry I gave that impression. I was referring to uutils, and, more particularly, some folks (GPL advocates) who are disappointed about the choice of license (MIT).

See, for example: https://github.com/uutils/coreutils/issues/2757


> he said it could be done in c as well but rust is much easier to write

This article is a good example

http://dtrace.org/blogs/bmc/2018/09/28/the-relative-performa...


Thank you so much for this article, I knew benchmarks were usually nonsense in real world performance when I used it, but rust wasn't ever seen, and these real world implementations show that it really works well.


I believe that coreutils should be rewritten in Go. Much better option.


I've seem a few attempts. This is just one example: https://github.com/u-root/u-root/tree/master/cmds/core

EDIT: You may also be interested in https://github.com/mvdan/sh


Why would Go be a better option than Rust here?

Go and Rust are optimized for different types of environments. There are a lot of situations when selecting Go over Rust would be a more pragmatic choice. I don't see how coreutils is one of these.


It seems like a reasonable opinion to me. Looking at the list of what's in coreutils, I don't see anything where performance would be a huge issue other than perhaps dd, so any performance edge Rust might provide seems moot. And I would guess that Golang would be less code and faster to write/implement.

I would feel differently if it were something else, like gawk, for example. Rust would be a better fit there.



Last commit was 2015, is it functional?


I have filed an official issue with the uutils repository, asking the developers/maintainers to consider switching to GPL:

https://news.ycombinator.com/item?id=29456115

I've tried making non-confrontational arguments in favor of the switch. Consider adding to them if you agree (or counter-arguing if you oppose, I guess) - but please keep the tone there less argumentative than here.


If we replace GNU coreutils[1] with this, could we finally call it "Linux" instead of "GNU/Linux"?

[1] And gcc with clang when bootstrapping


Is getting rid of GNU that important?


I’m not sure if this answers the “how important” question, but: a large part of the GNU ecosystem is functionally unmaintained. The various tools under binutils receive so few patches that they’ve been the standard punching bag for academic fuzzing research for the last 20 years, and the coreutils aren’t much better.


I think the point of this project is less about getting rid of GNU, and more about promoting Rust.

There's nothing wrong with GNU coreutils.


https://www.cvedetails.com/vulnerability-list.php?vendor_id=.... 4 vulns in the past decade that rust wouldn't have had.


4 vulnerabilities in 10 years sounds like a fantastic record, considering that the coreutils as a singular package has been around for 20 years (and the original tools for far, far longer). That's twice as long as Rust has been around.

Not to mention that memory safety is just one aspect of security. Rust's ownership model might have prevented those 4 vulnerabilities, but that doesn't mean that a whole host of others couldn't have slipped through.


If 4 vulns in 10 years for 1 package is acceptable, you are accepting that your OS will constantly have bugs. Sure there are other security problems than memory safety, but of the exploitable bugs, buffer overflows of some sort account for a majority.


My OS does constantly have bugs. It would be pretty unreasonable to say that my OS should never have bugs because of the practical impossibility of delivering perfectly bug-free software. So if people, after 10 years of trying, only found 4 critical vulnerabilities in all of coreutils, then that seems pretty good to me.

Rust itself has had 9 CVEs relating to memory safety in 2021 alone[0], which you can justify because Rust's development is highly active.

https://www.cvedetails.com/vulnerability-list/vendor_id-1902...


Recently there's been some folks "backfilling" CVEs. For example, the last one on that list was filed in 2021, but was fixed in 2015. The second to last one was filed in 2021, but was fixed in 2020.


What would motivate someone to do that? Sounds misleading to file a bunch of CVEs years after the fact.


The idea is, a lot of tooling relies on the CVE system to determine if a system is vulnerable, and so making sure that there is an actual CVE filed for every security bug is a good idea for the robustness of said tooling.

At least, that's my understanding.


It would be nice to have a totally memory safe userland, but it's at least worth noting that most of cureutils is generally not going to be working with attacker controlled data in a way that really matters.

Again, would be nice, just not where I'd start. But kudos to anyone doing the work.


From a performance standpoint I love love trying them and eagerly seeing if I can replace the GNU tools. Its like using musl over glibc.


What kind of performance issues are you running into with GNU tools?


I installed rust implementations and they’re all just much faster


About as important as keeping GNU, empirically.


Yes:) In fact, if you're into that, I'd argue that Alpine is already not GNU/Linux, since it uses busybox and musl.


You’d have to call it “Rust/Linux”


Most distros still use GNU libc but I guess you could build a distro with Musl and alt coreutils.


I've done this in go. It's fun to do, but the real reason I did it was memory safety issues in C. We need to implement as much code as we can in memory safe languages.


This is so much better than the previous, buggy, unsafe and dangerous C implementation.


It's unfortunate that FSF director Ian Kelling doesn't see it the same way, and thinks this project has little merit other than to work around the GPL license of coreutils...

https://github.com/uutils/coreutils/issues/1781

I'm curious as to what extent GNU coreutils governance is tied to the FSF these days.


Isn't that a little rich, considering a lot of GNU software itself started as a reimplementation of proprietary UNIX applications under a different license.


If you have a proprietary program, reimplementing it under a FOSS license is a good thing. If you have a copyleft program, reimplementing it under a license that allows proprietary modifications is a bad thing.


From the OpenBSD Copyright Policy (https://www.openbsd.org/policy.html) just for a different perspective from another FOSS project:

"While this may superficially look like a noble strategy, it is a condition that is typically unacceptable for commercial use of software. So in practice, it usually ends up hindering free sharing and reuse of code and ideas rather than encouraging it..."


I find this claim hard to believe since Linux is GPL, and it's used commercially way more than OpenBSD is.


Linux is something of a special case, as the vast majority of the users can "use it" without thinking about the implications of the GPL. Because, for example, there's a specific exception for software that uses Linux syscalls, and that glibc is licensed LGPL.

That's not true for most uses of other GPL licensed libraries and software, where the license makes your code a "derivative work".


If you're just using Linux on your own servers, or writing userland software for it, then sure, the GPL is easy to comply with. But if you distribute hardware with Linux preinstalled on it, then you do have to provide all the source code. How is this meaningfully different from basic userland tools?


Not that I'm agreeing with them, but many companies are wary of sharing their code because it opens up a path to replicate their services without paying for them. Vendors that distribute devices with Linux, though, are able to avoid that because the hardware is often still proprietary. Binary blobs still exist despite the Linux GPL license too. Even in projects that seem lauded by the "open source" community, like the Rpi.

I just don't think Linux is a great example to explain to a company why they should be fine with licensing their software under the GPL. It's not a good direct comparison in most cases.


Of course, Linux is special case, KHTML/WebKit/Blink is special case, Android is special case, etc. Blame special cases.


KHTML/Webkit/Blink are all LGPL and/or BSD, which is specifically not restrictive for linking to.

Many people assume all of Linux is GPL. I say "special case" because not everyone knows the most-linked-to stuff isn't.


? Android is, AFAIK, completely permissively licensed except for the kernel, and KHTML/WebKit/Blink are BSD/LGPL; what would be special about them in this context?


> used commercially way more

Depends by what metric and the definition of "commercially". I'm not sure this is the case for (physical) devices sold with software pre-installed.

I mainly see BSD licensed binaries on network appliances (routers, firewalls and other security appliances), apple hardware, automotive headunits, ...


There are some that are Linux based. F5 load balancers is a good example. I don't know what strategy they use to avoid it, but as far as I know, you can't replicate your own F5 with whatever GPL compliance code releases they do.

But, for some manufacturers, you can. Synology is a good example...see Xpenology.


  > it is a condition that is typically unacceptable for commercial use of software
whats the reason for that?


Because the GPL, specifically v3 is viral. The point being made in this statement is that the requirement to give back is potentially onerous for many commercial entities, and that this hinders open sharing rather than promoting it. It's a differing point of view. Notice that, unlike the majority of pro GPL comments, it doesn't use pejoratives or appeals to emotion to make it's point.


“Viral” is a term with a fairly pejorative emotional connotation and is not a very accurate way of describing copyleft.

https://en.m.wikipedia.org/wiki/Viral_license#Criticism_of_t...


The original statement doesn’t use pejoratives. Viral is the best way that I can describe it, I suppose insidious is better, but that too could be considered a pejorative. I’m struggling to find a word because I view it negatively.


Maybe "self-propagating", or "self-preserving"? The term "insidious" would not be appropriate for sure, as it means "dangerous".


Self-preserving is the best. Thank you. I personally don't like self-propagating, when the truth is that it's free, but with a catch. Forcing people to do the right thing, even if the intent is good, is still forcing someone to do something the don't want to, or even can't.


I like the viral word personally, I see it positively as popular. Free software is pretty popular its so popular that even non free software is turned into "free beer" free software! ;) If its free and open source, free to use (legal or not) people will use it virally, like Windows was virally pirated because it was free, and linux is virally used in servers since they don't have to pay per core to Microsoft.


I don't see it as pejorative, its used positively in social media for popular. Making GPL licenses popular is good.


How is that assertion not a double standard?


Not really. It has to be possible in practise to make proprietary modifications, get them accepted into common use, somehow prevent other people just cloning them, and steal the momentum of the open source project.

If someone makes proprietary extensions to "ls", they are just going to be stuck maintaining a fork. There is no realistic threat here.


"good" according to whom?


Why? You're allowed to build whatever you want, however you want? Why police people's licenses? Are they misrepresenting something, misleading people?


It's thanks to open source that you are allowed to build what whatever you want. You don't have to give back if you don't want to.


Besides the core aspect of your comment, you are aware that this entire subthread is about:

* GNU Coreutils: GPL licensed

* uutils: MIT licensed

right?

The MIT license is not only an Open Source license, but it's even a Free Software license according to the FSF itself.

So this is nitpicking of the 100th degree at this point.


Not if you dust out your history books :-)

"Were all Unix commands re-written in Linux?"' https://unix.stackexchange.com/questions/85189/were-all-unix...

"Is the GNU Coreutils copied from Unix?" https://unix.stackexchange.com/questions/81302/is-the-gnu-co...


Perhaps I'm misunderstanding, but both of your links seem to disagree with you.


The point is they re-implemented Posix, an open standard. Not copied a proprietary system.


Yeah, but POSIX was just codifying the proprietary UNIX standards. UNIX got bootlegged in a era where license enforcement wasn't strict because people didn't even know what licenses where. These days UNIX would have been a strictly proprietary AT&T product, from day 1.

Heck, POSIX stands for Portable Operating System (POS) + IX because X is a cool letter and IX because POSIX is cooler than POSIX plus, you know, UNIX.


MQTT was created by IBM

https://mqtt.org/

If I implement it in Rust now, dont think I will be accused of copying IBM?


Would Ecmascript[1] be another example?

[1]: https://en.wikipedia.org/wiki/ECMAScript#History


The GNU project chose a much better license, which lended itself towards free use and further public development continuing. The Rust re-implementation uses what many - including the FSF and myself - consider to be an inferior license with respect to public interest. So it's going back a step from the direction of the GNU project.

Personally, most of the software I write is under a 3-BSD or MIT license - unfortunately. I wish I were in a position to write more GPL code.


I really don't get why you're being downvote. Your explanation is perfectly rational and demonstrates a political stance. People can disagree with it but its expression is most definitely correct.

GNU coreutils are protected against a hypothetical proprietary ownership and I value that protection a lot.


The issue isn't the FSF's opinion on this matter, it's that it is incredibly childish of them to go voice that opinion on other projects' issue trackers. You don't go to someone else's house to tell them their beliefs are wrong.


they are rewriting GNU project under a less free license. i find it pretty funny that they did do that :)


Yeah stop with that, MIT/BSD/ISC is definitely more free then GNU...you know in the normal world "less restrictions = more freedom" just someone from the FSF would think otherwise.


More free for developers, not users. MIT code can be used in a closed source system a lot easier than a GPL one can. Developers love to talk about the freedoms they have with licenses, while ignoring that the whole point of the GPL family is disregarding that in favor of the user's rights.


> More free for developers, not users.

I protest this dichotomy. But even beyond the dichotomy, you're atomizing in the way you discuss people. A community or large group of users have among them both developers and resources to recruit developers to further their collective needs. When MIT/BSD-licensed software ends up being developed beyond the original freely-accessible base within commercial entities, as in:

> MIT code can be used in a closed source system a lot easier than a GPL one can.

then those communities and groups are stuck with a thorn in their side they can't remove, which warps technical fields to accommodate it and is utterly frustrating.

This has been my personal experience in more than one field. As an example it's like that with the CUDA ecosystem for GPU use, much of it is closed and hidden while employing technologies like LLVM, and likely a lot of other FOSS and free academic publications regarding effective computing. I really wish their choice was either much crappier closed-source commercial stuff, or, say, a GPL'ed LLVM, and having their drivers and user-mode toolchains open. If they were to choose the latter, than great; if they were to choose the former, than they would be at a competitive disadvantage and hopefully others (e.g. AMD) would use that to gain more traction with GPL'ed software. I'm not saying that this would _necessarily_ happen, but it might very well.

> Developers love to talk about the freedoms they have with licenses, while ignoring that the whole point of the GPL family is disregarding that in favor of the user's rights.

I think you meant MIT and BSD? Anyway, the point of the GPL and Free Software (using FSF/RMS terms) is to get enough software to have GPL-style license that it becomes unreasonable to license software any other way, with the end result being that essentially all released software will be free for use, dissemination and modification and we will be able to forget about software licenses altogether since people will stop trying to limit that and what is now the GPL would essentially be the legal default.


Being a user of BSD if feel more free too. Know why? Because i can do nearly everything with it, and i don't even need a Lawyer to read and understand the license.


That makes no sense. You can still, as a user, do whatever you want with GPL software. The GPL is long because it has to be in order to protect the user's rights; It's an actual contract between the source provider and anyone who uses it. It also doesn't require a lawyer to read. Just sit down for 10 minutes and read it yourself; It's pretty straightforward.


Even as a user, you cannot do "whatever you want" with GPL software. Specifically, you can't distribute the GPL unless you also adhere to the GPL's requirements for distribution. (It's even worse for the LGPL variant, because that imposes requirements on how the software is built in the first place.)

What counts as a "distribution"? Well, that's a contract interpretation question, and if you are not asking that question of a lawyer, then you have a fool for a client. I suspect that there's actually a lot of technical GPL violations going on, but since the open source community is not litigious in general, there's little realization of those violations and even less care that those are going on.

Personally, I am not a fan of the GPL licenses for this reason. If you are trying to use legal contracts to enforce norms, it is disingenuous to argue that you don't need lawyers to be involved. Using social contracts and pressure instead would truly allow everybody to avoid lawyers and achieve much the same goals.


>It's pretty straightforward.

Really?

https://www.gnu.org/licenses/gpl-3.0.en.html

>It's an actual contract between the source provider and anyone who uses it.

I don't want a contract, especially NOT with the FSF and their shady GPLv3 introduction...not thanks. If i want a contract i go to Oracle.


Why can't a company just use GPL code, if I started a company that used linux servers, I could contribute 0 code and get all the benefits.


:) it is less free in a sense that unlike MIT GNU ensures continuation of free software. for me if someone really cares about free software then choosing MIT is like being a hippy handing out flowers hoping that the world is nice


>for me if someone really cares about free software then choosing MIT is like being a hippy handing out flowers hoping that the world is nice.

I like that, and i like free flovers. What i don't like are lawyers and stuff like the GPLv3, AGPL and all the licenses who try to put some political view on me.

Look it's like that. If you fork and close down a BSD/MIT code base, you loose all those devs (no free flowers from hippies anymore), instead you have to pay your own dev's, juniper can sing a song about that problem. So without any pressure, just with the logic of economy you try to stay as near as possible to the free codes base with your product (and integrate your changes), netflix can sing a song about that too.

Continuation has nothing todo with the license but with interest of developers see -> Hurd


what the FSF hopes for is that if there is enough GPL-type licensed critical software which self-perpetuates free software then free software will become by far the most dominant paradigm. in this sense it is really not hard to see why the license change in the rust rewrite of GNU coreutils is going to become the most dominant issue for FSF. screaming about technical merrits makes no sense

>Continuation has nothing todo with the license but with interest of developers see -> Hurd

i think developer interest has far more to do with technical viablity of a project than license choice. besides, as in all software some projects fail, some succeed, and some are just beginning


>why the license change in the rust rewrite of GNU coreutils is going to become the most dominant issue for FSF

Why is that a problem? There was already toybox (BSD), muslc (MIT), llvm (UIUC (BSD-style)). The dominant issue the FSF has is that they are no technically interested anymore but politically, and so no one need's them, no one wants them.

And listening to Stallman is really boring, the last time he was kind of self-aware was with the Religion of Emacs, since then just repetition and being rude to interviewers and trying to trick devs (Linus and the GPLv3)

BTW: I am NOT talking about the GNU project, just the FSF


>Why is that a problem?

simple. imagine: you value free software; you invest a lot of effort to make a really important critical piece of software; you license it under MIT; some company uses your software as a critical component, develops a lot of software on top of it, keeps the source closed, makes massive profit for itself, dominates the market and becomes a monopoly, all thanks to your critical component.

now the question is: are you pissed off? if you are not, choose MIT. if you are pissed off choose a self-perpetuating copyleft license


>all thanks to your critical component

Like to give a real world example for that? Or is it just theory?


sure. i know hedgefunds and banks that critically relly on a lot of permissive-licensed software and develop heavily on that code base. conversely, they wouldn't touch copyleft software with a 10 ft pole

one of the other posters also mentioned CUDA's reliance on LLVM. i think that is a very good example also


So you think they would choose GPL-Code because that one component is so immensely good that you can take over the world?


take the example of nvidia and their use of llvm in nvcc[0]. if llvm used a copyleft license, nvidia would either need to invest into an effort to replace llvm (a huge undertaking), or have cuda released under a free license. given the importance of cuda in parallel data computation, as well as nvidia's monopoly in the machine learning community, i think a copyleft license would have made a world of difference

[0] https://en.wikipedia.org/wiki/Nvidia_CUDA_Compiler


>nvidia would either need to invest into an effort to replace llvm

No they would buy a license for one of the hundreds proprietary compilers, and you will eat their dogfood like you do today with cuda itself.

>or have cuda released under a free license.

Dreamer...it's nvidia....you know the ones with the "driver" that runs with the GPLv2 kernel.


> Dreamer ...

i thought you said you are a hippy

anyway at this point everything becomes hypothetical. however what is certain is that copyleft licenses could make life much more difficult for monopoly companies than permissive licenses, and i am ok with that, just as i am ok with FSF being agressive about matters pertaining to software freedom. could they have a better and more effective approach, i dont know. maybe. i am not involved in any shape or form with them, but i guess that right now they have my support


I said i like free flowers...flowers without a contract attached to it.

FSF can be as aggressive as they want, they will not change anything...quite the opposite actually.


> they will not change anything

they exist for over 35 years now. are you sure they haven't achieved anything? :)


MIT is not a less free license than GPL. They're both equally completely free.


yeah i wasn't being very serious by saying it is "less free". but it is funny that so many people are so touchy about this

MIT license basically allows you to do whatever you want. however it should be well noted that if you really belive in software freedom and write your code for sake of furthering free software, choosing MIT is a naive choice since you let any derivates of your work to be closed source


How can you be so certain they're not interested in knowing what the consequences of their chosen license are? By your logic, It would be childish to ring your bell and point out you have a water leak. I'm glad my neighbors don't think like that.


I think the insinuation is more that the GNU project probably was also breaking license terms in order to do the rewrites. (Not commenting on the veracity of that claim though)


I don't think anyone is accusing either project of breaking license terms. In fact, I find the GP's argument tenuous, since the Kelling says "you're making less free coreutils than their predecessors" and the GP says "that's rich, considering that the original coreutils were more free than their predecessors".


That would be quite a feat, given that the proprietary unices didn't have source available for their utils. Where would the GNU project get it?


While I don't for a moment believe that they were ripping off code or otherwise violating copyright, unix source code wasn't that hard to get if you wanted it; sure, it wasn't freely available on the net, but plenty of companies, research groups, and even schools were working with it.


The FSF director, per the post you linked to, made no comments on the merit of the project. Instead, commented on the project choice of license.

As anybody can also see, from the reference you provided, the way the discussion was closed, with not a hint of an effort to reply, already raises a small red light. Had the project replied along the lines of: "Its the one we choose for the project" even without a rationale attached, would turn it less into the interesting event I think it is.


The issue title alludes to the project having little merit other than its non-copyleft license.

I think closing the discussion like that was entirely the right choice; this project already had to deal with a prior issue from FSF-affiliated individuals complaining about the license choice. It isn't the FSF's job to go around telling people their license choice is wrong on their issue trackers. This is just a waste of everyone's time.


To quote a famous event with the group ABBA. When they started singing they realized too late there was already a canned tuna fish company with the same name. The story says the owner told them: "I wont make you any problems as long as you do not to plan to sell fish."

As the project is called Coreutils...Its not to the FSF to tell others what license to use but its reasonable to ask for a clarification the project uses a different license.

As we seem to try to save time for everybody...Do you know if the project has a public reference, with the rationale for the current choice of license? ( Note: No criticism implied concerning their current choice)


https://en.wikipedia.org/wiki/ABBA reports something different:

> Fred Bronson reported for Billboard that Fältskog told him in a 1988 interview that "[ABBA] had to ask permission and the factory said, 'O.K., as long as you don't make us feel ashamed for what you're doing'


I am fairly certain I saw Benny Andersson telling the story like this in a documentary many eons ago, but my ABBA fu fails me now and cant find anything to back me up.

This is an interesting take on the story but now have to get back to work... https://youtu.be/gC09BnFmQ30?t=118


Here's what I found using a Google Book search.

https://books.google.com/books?id=CILUDgAAQBAJ&pg=PT297&dq=%... has:

> Stig phone up the managing director who enthusiastically approved, the only condition being that the group wouldn't discredit his company.

https://books.google.com/books?id=6lu_BgAAQBAJ&pg=PT57&dq=%2... says:

> There was just one tiny problem with formalising the anagram; ABBA was also the name of Sweden's premier brand of canned fish. Fortunately, after some brief negotiations, Stig was able to placate the fish firm and they gave ABBA, the pop group, their blessing.

https://books.google.com/books?id=-hkbqflhfKoC&pg=PT7&dq=%22...

> ... the name was also that of a Swedish company which sold canned fish; whose directors were worried lest ABBA the quartet should bring the name into disrepute, but after Stig Anderson had assured them that ABBA were aiming to publicise the name in a very positive way, the convern abated.


All this stuff happened before Google come to existence. I recall a somewhat humorous conversation with a friend about an exotic Portuguese rock band. He insistent could not have existed, because he could not find anything about them via a Google search!

What become of the ABBA Seafood brand has a take on the history:

https://www.orkla.se/brands/abba/

"The Abba brand and the pop group ABBA have been the subject of some confusion over the years. During the 1970s, Abbas' switchboard usually received calls from Swedish and foreign fans who, if they were not allowed to talk to any of the group's members, could settle for a signed idol card.

Before the group Abba broke through in 1974, they actually called from the record company Polar and asked for the green light to use the name. Per Brolund, then HR manager at Abba, gave his consent to a reservation. "That the young people behaved and did not damage Abbas' good reputation."


I don't understand your point about this being "before Google come to existence."

If your memory predates Google's ~1998 search services offering, it's surely possible that you've misremembered what Benny said?

And the first reference I gave was to a copy of "Bright Lights, Dark Shadows: The Real Story of ABBA" from 2001, only a few years after Google. I don't see how Google search per se is relevant.

The Orkla link you gave is in Swedish(!). Good thing there's Google Translate. It essentially supports the point that Abba was concerned about behavior, not jocular opposition to becoming fishmongers as you suggested.

I do find it interesting that the first source I gave says "managing director" while the one you gave says "HR director". I expected "managing director" to mean CEO.

Ha! There's an ABBA fandom entry about it (because of course there is), at https://abba.fandom.com/wiki/Abba_seafood , which references the same page and translates the term to "staff manager".

Perhaps there are organizational differences between Sweden and US/UK which make direct translation difficult?


I think the response was incredibly muted. It gave it exactly the amount of attention it deserves.


it is the FSF's role to comment on software licences first, and the technical merrits second (even that is a maybe). FSF is about software ethics and in that case ian's comment is appropriate since all he asks is that at the very least license differences be acknowledged


It's not just the FSF's role to "comment" on software licenses, but also to advocate for the use of free software licenses. That involves convincing developers to use them. To go into a project and dictate to them that you don't find their work notable, while still demanding respect for your own project, isn't likely to spark good-will towards the FSF amongst developers. If I were a director at FSF, I would cringe at this kind of interaction, and lean much more towards the "honey" approach for catching flies.


>To go into a project and dictate to them that you don't find their work notable

a key person from FSF commented on the project. he didnt say it is not notable

on the other hand this is a rewrite of GNU coreutils, and as such FSF has every right to comment in the project


> this is a rewrite of GNU coreutils, and as such FSF has every right to comment in the project

Sure, and FSF also has every right to tell anyone who writes proprietary software that they hate their guts, if that was how they felt. My point is that this is not a particularly effective strategy for advertising the GPL to developers.


>My point is that this is not a particularly effective strategy for advertising the GPL to developers

maybe the strategy sucks, maybe not. but maybe speak for yourself. i actually like their quirkiness and lack of shyness


> he didnt say it is not notable

He did say that the most notable thing about the work was the license, which effectively dismisses the technical work as not notable.


>which effectively dismisses the technical work as not notable

that being your interpretation and deduction. i am a bit more empirical and judge only what he actually said. moreover i am also not that much of a rust-lang fanboy, at least not to the extent that i see every rust rewrite of a c-lang code base as a technological marvel


I am sure the authors of the uutils utlities did actually look at the GPL source code. If that is the case, shouldn't this project be GPL as well?

disclaimer: I did actually make the first commit on one of these tools, but I didn't look at the source of the original C code.


Looking at source code does not make everything you subsequently write a derivative work of that code.

Not looking is a strong legal defense for whatever you wrote not being a derivative work. Looking just means you can't use that defense, it doesn't mean you actually copied code.

Given the significantly different paradigms of Rust and C, and that most coreutils don't really do that much algorithmically, it would be pretty easy to avoid accidental copying with this kind of project.


No, that's not how copyright works.


Then why are people who look at leaked Windows source code not allowed to contribute to Wine or ReactOS?


Because by rejecting work from people who've seen the leaked code, you can be sure that your project doesn't become a derived work. That doesn't imply the reverse; that if you accept code from people who've looked, your project must be derived a work.

All cows are animals, but not all animals are cows.



Pretty sure they did. Does it require proving? How would one prove such a thing if so?


You would have to prove that the Rust version is substantially similar to the C version, to an extent that wouldn't happen if someone who hadn't looked at the C code had written the Rust version purely from the spec/behavior.

For example, significantly identical code flow, variable naming, or function structure would be red flags, especially if it's not the "obvious" implementation.


I'm reasonably sure that idiomatic Rust code can't really infringe C copyright. I'm not saying that it's impossible, especially as you say, regarding overall code flow or variable naming, but the languages are quite different in how they approach things.

Unlike for example, Java 1.5 and C# 2. I think at the time you could almost copy paste Java code into a C# and have it compile after tinkering with it for 2 minutes.


Copyright doesn't just protect the single implementation, but also any derivative works. If I take some C code and copy it (with a rewrite) into C# (or whatever), the C version's copyright still applies because mine is a derivative work. That's where the "clean room reverse engineering" idea comes from: Implementing a spec isn't infringing, so just have the person reversing and the person implementing communicate using a spec.

In this case, the "spec" of the coreutils would probably the man pages.

Changes to the C code to be idiomatic Rust does throw a wrench into the whole thing, and it would have to be sorted out by a court, but it's not as simple as people like to think.

As usual: IANAL


> If I take some C code and copy it (with a rewrite) into C# (or whatever), the C version's copyright still applies because mine is a derivative work.

Copyright protects creative output. That means if you copy creative aspects of the code - things like the structure, naming, algorithms down to small details of the implementation - then you are creating a derivative work (this is what happens if you translate code from one language to another verbatim - not trying to be idiomatic - most of the time). If you merely read the original code to understand what it does and create an implementation that produces the same behavior, but is otherwise not substantially related to the original, then you have not copied any creative aspects and you have not created a derivative work.

"Clean room reverse engineering" (which, done properly, is extremely rare and the vast majority of open source projects related to reverse engineering do not do it to a proper standard) is a strong legal defense to show that creative aspects could not have possibly been copied. However, it is neither water-tight (the spec could've conveyed unnecessary creative aspects accidentally), nor is it required to show non-infringement. It's merely a defense; not doing clean-room RE doesn't mean you are doing anything infringing.

> Changes to the C code to be idiomatic Rust does throw a wrench into the whole thing, and it would have to be sorted out by a court, but it's not as simple as people like to think.

Practically speaking, given the significant differences between what is idiomatic in both languages, and the relative simplicity of what most coreutils actually do, there's a good chance that even a careless "translation" into idiomatic Rust would erase most creative aspects and put you well on the way to being able to claim it's not a derivative work. Effectively, the translation would involve a round-trip through an abstraction to the level of a spec, simply because what's idiomatic is so different. This is, of course, not a hard guarantee, nor would I bet the legality of my project on this aspect alone, but rather just an observation of what is likely to happen in many cases. In other words, you'd have to be really careless to end up creating a derivative work of coreutils here; the language difference works significantly in your favor.

IANAL, but I've been doing reverse engineering projects for over 15 years and my main job these days is leading an open source project to support an undocumented platform via reverse engineering. So I have a bit of experience with this matter :-)


TIL. Thank you for correcting me! BTW, your work on Asahi Linux is incredible!


I don't read that he thinks the project has little merit other than changing the license.


The issue title literally says the most notable thing about the project is the license. That reads to me like a clear dismissal of any technical merits; if the most important thing about your project is its license, it's not a very interesting project.


I see your point. I also think the license is the most important thing, and probably even more for someone working on a GNU project at the FSF, but the project is still very interesting in many ways.


The stance is probably not surprising, given the point of FSF. Whether you agree with them or not, the FSF has always been very clear that the most important thing about software is its license. From their point of view, a free program is always superior than a non-free one, regardless of the technical merits.


[flagged]


> Giving up your original rights seems to be the worst kind of payment

That's only an issue if you think that payment is more important to you. FSF are fighting for freedom. Money is secondary.

Now, you can argue about what is the best option to achieve freedom. FSF has chosen one (which I happen to agree with). There are others.


I'm just using FSF terminology when describing a FSF point of view. I myself don't agree with the FSF stance on that issue, but the point still stands - the FSF is an advocacy organization (not a tech organization) that is at its very core about promoting any free (per their definition) software as superior.


Are you really free without the right to enslave other people?


I release all my code under MIT license. Do I still have the right to enslave other people? If yes, could you please describe the details, I'd be rather interested to learn how I can force whoever uses my code to use MIT license as well? Thank you.


The point is that the people who use your software have that awkward right, which you have carefully granted them (as that is the only right removed by the GPL). This is frankly sufficiently obvious that I can only imagine you are arguing in bad faith :/. Like, I definitely have seen good-faith arguments against the GPL, but this is not one of them.


So I can't force other people myself, but I still am a (potential) accomplice in a completely different kind of "forcing other people" stuff because I didn't explicitly prohibited others from doing it. I... understand this argument, thank you.

P.S. "Carefully granted"? The words "to deal in the Software without restriction, including without limitation the rights to [rather long list of verbs]" is anything but "carefully granted", it's a blanket permission.


By taking away one freedom (the right to relicense software under other licenses or exploit the software, analogical to slavery), you grant more people more freedom (the freedom to modify and use *their* software as they please, analogical to abolishing slavery).


And yet including the additional restriction "this software may not be used to enslave people" makes the software no longer "free". It's odd how often slavery is mentioned in examples about how free copyleft software is, while the first of the so called four freedoms insists that using software to literally enslave people is a freedom that must be protected.


Maybe this should be read as "I consider the license a no-go, so any technical merits the project has are not relevant (for me)"? If you already dismiss the project because of its license, you probably wouldn't let technical merits redeem that.


since FSF is about software licence ethics they will probably find technical merrits irrelevant if the project uses a licence that is less appealing to their standards than the already existing solution. i dont see why that should raise your eyebrows


I raise my eyebrows at them deciding to voice these concerns on a project's issue tracker. Regardless of how strong their political stance is, it is ultimately none of their business what a given project they are not affiliated with chooses as its license.


It's a rewrite of one of their main project. Of course they can have some comments about it.


We detached this subthread from https://news.ycombinator.com/item?id=29457121.


i dont see anywhere in that link that Ian Kelling said anything about the technical merits (or lack thereof) for this project


MIT license is a big no. Folks still don't understand why copyleft is so important nowadays.


The MIT license is better than any proprietary software license, yet MIT-licensed software attracts so many more negative license-related comments than proprietary software.

Seems a bit counterproductive.


I think its because those who license software with MIT where GPL could be used are considered "traitors" and as some people believe "traitors are worse than enemies.


False equivalency. I'm comparing permissive licenses with copyleft licenses. I'm not comparing MIT license with any proprietary licenses. And if uutils is proprietary, you won't see this discussion happn at all.


Or maybe they do understand the licence and simply decided they did not agree with its ethics.


And you're not educating them.


> Folks still don't understand why copyleft is so important nowadays.

True, it's incredible how badly the FSF has done in terms of actually marketing their licensing structure. At this point I doubt most developers care much.


>Many GNU, Linux and other utilities are useful, and obviously some effort has been spent in the past to port them to Windows. However, those projects are written in platform-specific C, a language considered unsafe compared to Rust, and have other issues.

>Rust provides a good, platform-agnostic way of writing systems utilities that are easy to compile anywhere, and this is as good a way as any to try and learn it.

If you want some software to be platform agnostic, you can write it in C in a way that is not platform specific. So rewriting some software in Rust to be platform agnostic doesn't seem a good reason for a rewrite. Also, even if the C software to be rewritten isn't platform-agnostic, at least some part of it is, so by using C you don't have to rewrite everything.

I'm sure that Rust can have a lot of valid use cases but the trend of "Hey, let's rewrite X in Rust just because ..." I don't agree with.




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: