> Overall, pretty weird stuff. I am not sure why the Go team went down this route. Maybe it simplifies the compiler by having this bespoke assembly format?
Rob Pike spoke on the design of Go's assembler at a talk in 2016 [1][2]. I think it basically came down to the observation that most assembly language is roughly the same, so why not build a common assembly language that "lets you talk to the machine at the lowest level and yet not have to learn a new syntax." It also enables them to automatically generate a working assembler given an instruction manual PDF for a new architecture as input.
Yes, this is a great leap forward in my opinion. I had to do a project at a previous job where I wrote an agent that ran on x86, MIPS and ARM, and doing it in Go was a no-brainer. The other teams who had a bunch of C code that was a nightmare to cross-compile were so jealous they eventually moved a lot of things to Go.
I've been doing this for 35 years and cross compiling anything nontrivial was always a toolchain nightmare. Discovering a world where all I had to do was set GOARCH=mips64 (and possibly GOOS=darwin if I wanted mac binaries) before invoking the compiler is so magical I was extremely skeptical when I first read about it.
It's still pretty slow, but overall correct. There's tricks, like reader connections and a single writer connection to reduce contention. There was a blog post on here detailing some speedups in general.
A fair enough assessment, it be that way, however I will note that a large reason that C exists in the first place was to have a machine independent language to write programs in.
> however I will note that a large reason that C exists in the first place was to have a machine independent language to write programs in.
That's fair, but what we call a monstrosity by modern standards is much simpler than porting the assembly
There were cross plaform languages before C, but they never really took off for system development the wat C did (OSs, for example were commonly written in pure assembly)
A side effect of C not having a price tag associated with it, anyone with UNIX source tapes got a C compiler for free, until commercial UNIX became a thing, and splitted into user/developer SKUs, and thus GCC largely ignored until then became a thing worth supporting.
mips64!? That's a blast from the past. It must be some kind of legacy hw that's getting current software updates in some kind of really niche use case. Or academia. :)
Like previous you, I have to admit I'm skeptical but would be happy to be wrong.
> mips64 .. must be some kind of legacy hw that's getting current software updates
Hundreds of thousands of linux-based smartnic cards, actually. Fun stuff. Those particular ones were EOLd and have been replaced with ARM but the MIPS based ones will live on in the datacenters until they die, I'm sure.
> Like previous you, I have to admit I'm skeptical but would be happy to be wrong
Seriously, you are going to be delighted to be wrong. On your linux machine, go write a go program and write "GOOS=darwin GOARCH=arm64 go build ..." and you will have yourself an ARM mac binary. Or for going the other way, use GOOS=linux GOARCH=amd64. It really is that simple.
Ah I found this https://ctrl-c.us/posts/test-goarch I guess it's qemu-user-binfmt registering the alternate bin formats to automatically run under QEMU, that's pretty neat
The Go build system runs under your current architecture, cross-compiling tests to your target architecture.
Then, the Go test runner also runs under your current architecture, orchestrating running your cross compiled test binaries.
Since you registered to run cross-compiled binaries under QEMU, those test binaries magically run through QEMU.
The Go test runner collects test results, and reports back to you.
The first run might be slowish, as the Go compiler needs to cross compile the standard library and all your dependencies to your target platform. But once that's done and cached, and if your tests are fast, the edit-test cycle becomes pretty quick.
"EdgeOS" is based on Linux, and people run vanilla Linux distributions on those boxes, as well as OpenBSD and NetBSD.
I wonder how long Marvell will continue selling those Octeon MIPS64 chips, though. Marvell (then Cavium) switched to ARM nearly a decade ago (2016) for newer chips in the Octeon series. I think Loongson sells more modern MIPS64 (or at least MIPS64-like) chips, but they don't seem to be commercially available outside China.
Go essentially copied the design from Plan9 compilers, which it was originally based on. It's one of the many things it inherited from Plan9 environment.
I would love to see a deep dive on what features / architectural paradigms the Golang runtime shares with Plan9. Has anything like that been written?
One that always sticks out to me personally is the use in Go of the term "dial" instead of "connect" for network connection establishment. This is, AFAICT, another Pike+Thompson-ism, as it can be seen previously in the form of the Plan9 dial(3) syscall — https://9fans.github.io/plan9port/man/man3/dial.html .
---
A tangent: I have wondered before whether Pike and Thompson drafted the design for the language that would become Golang long before working at Google, initially to replace C specifically in the context of being the lingua-franca for systems programming on Plan 9. And that, therefore — at least in the designer's minds — Golang would have always had Plan9 as its secret "flagship target" that it should have a 1:1 zero-impedance abstraction mapping onto. Even if they never bothered to actually make a Plan9 Golang runtime.
You could test this hypothesis by implementing an actual Golang runtime for Plan9†, and then comparing it to the Golang runtimes for other OSes — if Plan9 were the "intended home" for Golang programs, then you'd expect the Golang runtime to be very "thin" on Plan9.
(To put that another way: imagine the Golang runtime as something like WINE — a virtualization layer that implements things that could be syscalls / OS library code, in the form of client-side runtime shim code. A "WINE implementation for Windows" would be an extremely thin shim, as every shim call would just point to a 1:1-API-matched piece of OS-provided code. My hypothesis here is that "Golang for Plan9" is the same kind of thing as "WINE for Windows.")
† I was saying this as a thought experiment, not thinking there would actually be a Plan9 implementation of the Golang runtime... but there is! (https://go.dev/wiki/Plan9) So someone can actually check this :)
> I would love to see a deep dive on what features / architectural paradigms the Golang runtime shares with Plan9. Has anything like that been written?
If it has, then it's most likely available on https://cat-v.org/. Even if it hasn't, cat-v.org is a great starting point.
Besides, close to your line of thought, and assuming you didn't knew about this already, Pike & al previously worked on Limbo[0], a "predecessor" of Go, used to wrote Inferno[1], a Plan9-like OS, which could be hosted on arbitrary OSes via a bespoke virtual machine called "Dis".
So there were indeed a few previous "drafts" for Go. I'd doubt that Go has been designed "for" Plan9 though.
There’s also libthread[1] which implements concurrency model similar to goroutines (at least earlier, as they appear to be no longer just cooperatively scheduled?). That manual also mentions Alef and Newsqueak as influential predecessors.
While good cross-compilation is one of Go's strengths... I don't really think a unified assembly language contributed to that? Any assembly code can't be shared across multiple architectures in general, so the only potential benefit would be the easiness of parser development, which doesn't sound like a big thing. Rob Pike himself even noted that it can be offputting to outsiders but considered it's a worthy trade-off nevertheless [1], the conclusion I don't really think justified.
This is by far one of the best parts of Go. Its all around simple and painless to use. anyone designing a language should study what Go did well and what they didn't.
Pretty much, yes. If you imagine “Bob” has a policy that he can only converse with numbers in his address book, then you could think of it as:
1. -> Alice calls Bob
1.a. Bob does not pick up the call, but adds the number shown from caller ID to his address book
2. <- Bob calls the number (Alice) back
3. -> Alice picks up and they talk happily
My interpretation is he's lamenting that certain Unix OS abstractions (e.g. open, close, read, and write) do not lend themselves well to building distributed systems, like a distributed filesystem. Plan9, for example, designed it's API with such possibilities in mind.
The thing is that Unix grew up w/o a network. Disk was slow though, so perhaps it could have had async I/O from the beginning, but... everything was slow, so maybe not. Most importantly the key is that when one is building something new, often one will build the easy stuff first, and synchronous metadata operations is definitely easier than async -- this is a lot easier to forgive 50 years ago than now because now we know that async matters a great deal, but 50 years ago it was a lot less clear.
Agreed. I avoid reader-writer locks unless absolutely required and benchmarks prove it worthwhile.
Their usage often fails to outperform a regular lock due to additional overhead. They seem to make sense only in specific high-contention scenarios where arrival rate is high and/or the critical section has a long duration [1].
> When io.Copy copies from a TCPConn to a UnixConn, it will now use Linux's splice(2) system call if possible, using the new method TCPConn.WriteTo.
Interface upgrades, yet again, transparently giving us more zero-copy IO. Love how much mileage they’re able to get out of this pattern in the io package.
Go standard library is absolutely littered with these kinds of tweaks. Not necessarily bad, but it does make an integration with non-standard library less satisfactory because such special casing is not scalable. Still within the expectation though.
I'm curious to hear others' answers to this question as well. I've been looking into building my own OpenBSD based home router and so far thinking a Protectli Vault [1] would fit the bill.
Do you have any privacy/ad-blocking extensions installed? I used to have the same issue and realized it was due to the DuckDuckGo Privacy Essentials extension. When I turn that extension's protections off for cs.opensource.google then it works.
Hey, DDG engineer here. Sorry about the issue, that’s super annoying. I want to get this fixed, but I’m having trouble reproducing the issue - navigating to the link in the parent comment seems to work as expected for me. Any chance you’d be willing to provide a few repro steps so I can track down what’s going on? Namely:
- which browser you’re using and version
- extension version
- when it’s happening. Does it only happen when you click a link from another site?
Thanks so much for your time, and for sharing the issue.
Well, turns out I'm no longer able to reproduce the issue either. I just turned DDG Privacy Essentials back on for https://cs.opensource.google and I was able to view the site just fine.
Back when it was happening (maybe ~1 year ago?), I was using the latest versions of Firefox and DDG Privacy Essentials and it ocurred even if I went directly to https://cs.opensource.google. I had confirmed back then that when I turned DDG Privacy Essentials on I got "Permission denied", but with it off I was able to view the page.
I'm on the latest verisons of Firefox and DDG Privacy Essentials now. Seems it is no longer an issue in the latest version(s).
Thanks for following up! Appreciate it. We’ve been investing quite a bit of effort over the past few months into tracking down and understanding issues like this that are caused by our various privacy protections, so it’s nice to hear that things are improving.
Yes, you can find the documentation in other places and that is what I usually do, but it sucks because this is the official documentation of the programming language.
Having those links to quickly take a look at the implementation is what made this documentation so good.
Rob Pike spoke on the design of Go's assembler at a talk in 2016 [1][2]. I think it basically came down to the observation that most assembly language is roughly the same, so why not build a common assembly language that "lets you talk to the machine at the lowest level and yet not have to learn a new syntax." It also enables them to automatically generate a working assembler given an instruction manual PDF for a new architecture as input.
[1]: https://www.youtube.com/watch?v=KINIAgRpkDA [2]: https://go.dev/talks/2016/asm.slide#1