I can agree with most of the article, but for some it looks like we have not been using the same language ecosystem at all.
Go really feels opinionated around the wrong things, in order to claim "simplicity" as a feature:
- No generics just mean you're going to be handing interface{} all the way in your stack, it makes things more complex and less readable for no reason (and less safe)
- error handling which is essentially string-based, in 2017? Give me a type system, please
- gofmt is a good idea (just like clang-format or yapf), and having it is great, but the maintainers specifically refuse to add simple features, saying "running it in an automated process is not supported", despite all github projects already having it in their automated CI suite
- go test is good, but in the end it is found lacking
- dep, well, if you don't mind dumping the code for all your deps in the source tree, it's probably ok, but it solves a deficiency that the language should not even have (see next point)
- stability: the language itself is fine on that, but the ecosystem really isn't. The fact that they don't even have the package management quality of python (which I find awful already) is baffling. I don't even know how they could think "yeah, github-importing is a good idea, let's do it". Go is eight years old tomorrow and you still have to rely on third-party tooling to do proper imports that don't burn your house.
>- No generics just mean you're going to be handing interface{} all the way in your stack, it makes things more complex and less readable for no reason (and less safe)
Depends on what you do. I've written a good chunk of go code and interface{} is the rare exception rather than the rule, usually employed where a user might supply arbitrary types (ie a unmarshaljson like function)
But if you do a website or webapp, 99% of your code is not using interface{} in it's methods.
>- error handling which is essentially string-based, in 2017? Give me a type system, please
You can put anything behind an error:
type IntErr uint64
func (i IntErr) Error() string {
return strvonc.Atoi(i)
}
The string is only present when you either use the stdlib errors or you output an error.
Additionally, nothing in Go forces you to actually use the error interface (sans stdlib).
You can invent your own Error interface that uses ints. (Won't work with most external libraries but nothing is stopping you)
Panic and Recover can to my knowledge both handle things that aren't errors; recover returns a interface{} that you check for value and type. You can put anything in there.
>stability: the language itself is fine on that, but the ecosystem really isn't.
A lot of projects focus on stability. If the API is breaking a lot they usually use Gopherpit or gopkg.in
to ensure compatibility in the future.
In my experience, breaking the API is not something well used libraries do lightly (libraries with no or little use do break it sometimes since they lack the usage to refine it)
> But if you do a website or webapp, 99% of your code is not using interface{} in it's methods.
That's not really true. If you want an SQL database to back your website, then you'll deal with interface{} with Go's sql package. If you want to gzip your output, then you'll use interface{} with your writer.
To be fair, I think interface{} gets an almost unfair amount of hate. While it is fscking annoying when passing around core types (I hate having to use switch clauses just to inspect the type of an interface{}!!!) I do love how interface{} is used for complex structures. In fact there are a few areas of Go's standard library which I wish used interface{} more in that respect (eg Go's `file` struct should be an interface{} so I can create custom methods for os.Stdin/out/err)
No, they are interfaces, but they are not interface{}s, the interface that is met by having no methods and is therefore met by everything. Nobody is complaining about the use of interfaces that declare a useful set of methods, and are meant to be used when the set of methods is all the target code cares about. They may complain about some of the edges around them (no support for contra- or covariance), but that's not the complaint that is being discussed here.
The complaint is specifically around interface{}, which is a type that means nothing, and so everything fits into it. It means that if you are a function that is receiving an interface{}, you know nothing about the argument to start with, and will have to either pass it along to something else, or examine it via type assertions or in the worst case, the reflect library.
As a multi-year Go programmer, I agree that A: the "true" interface{} that is the meaningless type appears less often than people think, and that if you program is shot through with it, you are either working in a domain where Go is not suitable (complicated math processing, IMHO, for instance), or you are programming some other language in Go, and need to learn how to use Go properly, but also, B: it does happen that things that would be well-typed in other languages will have interface{} show up, which means you're going to sacrifice runtime safety and some performance, though the exact impact depends on what you are doing.
(I say the "true" interface{} because there's also some places where interface{} shows up, but it doesn't hurt you. For instance, the standard JSON library accepts an interface{} to say what data structure to parse the JSON into, and the code that implements that is somewhat complicated, but for the most part, that interface{} doesn't bother you, the Go user. In theory you can encounter run time errors if you pass in something that doesn't work, but in practice, if you pass in the same type every time, which is the natural case that is going to result if you just program normally, it's not a problem. In general, `interface{}` showing up in the incoming parameters of a library is much less concerning than an interface{} showing up in the return parameters for a library. In the former case, even though the language does not enforce type safety, you can enforce type safety by how you use the method; in the latter, you are stuck with an interface{} in your code with all that implies.)
"io.Reader is literally an interface{} though. I think what you're referring to is the other packages that define their own Reader's, ie using structs with methods, those are referred to as interfaces despite not literally being an interface{}."
That appears to be a definition unique to you that I have never seen from anyone else before, including the Go designers. io.Reader is an interface, no braces. Its full expansion would be interface { Read(p []byte) (n int, err error) }, and it is incorrect to substitute that for interface{} as that is a very different thing. Structs that implement the interface are, well, structs that implement the interface. interface{} is specifically reserved for discussion about the empty interface.
For instance, do a browser find for "interface" in https://golang.org/doc/effective_go.html ; you will find the documentation frequently referring to "interfaces" and that interface{} is reserved for the empty interface.
For the final proof of this, note how confusing it is for you to be reading the Go criticism as being about the use of interfaces-in-general in Go. Why would that provoke such a reaction? The answer is that people aren't talking about the feature, they are specifically referring to the number of places interface{} appears, the "Any" type, the "I don't know what's in there at all" type. It's not the use of interfaces in general, it's the way that suddenly one goes from a decent enough statically-typed language to a not-all-that-great dynamically-typed language when interface{} appears. (I disagree with the HN gestalt about the severity of this problem, but I understand it and still agree it's a non-zero problem.)
This is correct. I think part of what you're saying, in fewer words, is that there are interface method signatures, and there are interface expressions, which are ultimately different.
With this statement, you've shown you have close to 0 Go experience.
* interace{} - a place-holder for a value of "any type". It means "a value that satisfies the empty interface", i.e. an interface with no methods. All types implement it. To do anything with the actual data it encapsulates, you have to do a type assertion to unbox the value. The type assertion will panic or return an error if the type assertion is incorrect. This is something like C's void , or Java's Object.
io.Reader, io.Writer - these are interface types that specify a set of methods that must be implemented by another type in order to satisfy that interface. These are akin to C#/Java's interfaces, but are less restrictive in the sense that any type in Go can implement them, not just "classes". E.g., http.HandlerFunc - a function type that implements the http.Handler interface.
Yeah, it is unfortunate. My ranting was born from frustration that HN used to be a refuge from the usual pointless egotistical jibes as seen on other forums and social platforms. But over the years the quality of counterarguments has slowly been lowering to the level where it's now commonplace for people to make judgemental remarks about a persons ability based solely on a solitary semantical error (I obviously get the distinction between the two - I just wasn't aware they were described differently in literature).
I have amassed a large body of real world projects under my belt over the last 3 decades and have used Go for 6 of those years. I've written some pretty interesting stuff in Go like file systems compiled against FUSE and compilers. Yet I by no means consider myself an expert in anything - least of all Go - as to do so would imply I'm not open to learning something new (even basic things I somehow missed during my initial studies). Despite that I still find it highly offensive and unnecessary for someone who knows nothing about me to say "you have close to 0 Go experience." when I've clearly demonstrated and described various Go-isms prior to his or her remark.
Edit: weirdly I've got more down votes for this comment than my last one. I guess some people think rudeness is acceptable. Sad times.
> That's not really true. If you want an SQL database to back your website, then you'll deal with interface{} with Go's sql package.
True. But for most real world scenarios I only ever use interfaces at the boundaries between APIs and they quickly get asserted into a type after being passed.
Conceptually they're a bit like passing marshalled data between APIs in that sense.
> Conceptually they're a bit like passing marshalled data between APIs in that sense.
That's IMHO a great way to think about when it's Good™ to use them. Yes, there are times when you just have to use the empty interface type but it's really not as bad as everyone makes it out to be.
I love Go because it is explicit and because explicit types make everything easier in the long run. I get tired of seeing Go written like Python/Ruby/$otherDynamicLang with overuse of interface{} and/or reflect.
> But if you do a website or webapp, 99% of your code is not using interface{} in it's methods.
How do you deal with the lack of sets and trees? I believe those two data structures are frequently almost indispensable - even in the most boring mostly-CRUD web projects. And it's either type assertions on {}interfaces or go-generate based specialized implementations. Both feel odd to me. (Am I missing something?)
For Sets, you can get away with the language slices and maps, which work sufficiently well.
I haven't found a need for trees in most of my projects and where I did I implemented it when necessary.
You can however, if you need to, separate the code and data of a tree by using a data adapter. The tree only stores a unique ID for an object which you can receive using the adapter, which would be a simple map or slice.
But as mentioned, I very very rarely needed them and not yet in any CRUD web project.
Context: I've been programming in Go full-time for the last 2 years and come from a Python and C# background.
> No Generics
The lack of generics was something I was hung up on, too, but TBH I haven't found myself reaching for them for a while now.
> Error handling
This is not a problem for me, either. I can easily handle different `error` types and if I want to provide more context I pull in github.com/pkg/errors
> gofmt
gofmt is great
> go test
go test has been totally sufficient for me and I was quickly writing and running tests. The testing package itself is builtin and writing tests is dead simple.
> dep / dependency stability
Dependency management has been a huge PITA with Go and is my largest complaint. Personally, I've used everything go GB vendor, godeps, glide, and now dep. dep has been Just Working for me and is decent enough working on a team. As for dumping the dependency code in your source tree-- it lives in a folder called ./vendor so it's easy enough to not look at it.
> The lack of generics was something I was hung up on, too, but TBH I haven't found myself reaching for them for a while now.
I mean, if you lose an arm you probably stop trying to pick stuff up with your missing hand after a while too, doesn't mean it's not better to have two arms.
Are generics as useful as an arm? I think that's a slight exaggeration. I'm sure they're useful in a lot of situations but there's a lot of programs you can write where their absence is neither here nor there.
> The fact that they don't even have the package management quality of python (which I find awful already) is baffling. I don't even know how they could think "yeah, github-importing is a good idea, let's do it". Go is eight years old tomorrow and you still have to rely on third-party tooling to do proper imports that don't burn your house.
Python packages are installed via a separate tool so I'm not sure what you're getting at here.
Including github in the import path is a good idea. It makes it so you always know where the source code for any package is. This consistent approach empowers sites like godoc.org. I don't know how many times I've tried fruitlessly to find documentation for libraries in other languages, but in Go the developer who created the library gets it for free. I wish more languages did this.
Do you remember the left-pad fiasco in npm? Out of the box you'll have similar problems in Go, but at least you'll get them when you work locally, not when you spin up a new node. And to fix the problems you end up having to do the same thing regardless. Either you vendor your dependencies or you setup some sort of caching middle-man proxy.
Go's dependency headaches aren't the fault of the language or the ecosystem - Go actually has a really well crafted module system, specifically designed to fix major performance issues in languages like C++ and Scala. Rather they're a reaction to the problems latent in the "easy" solutions and simply switching is just trading one set of problems for another.
> It makes it so you always know where the source code for any package is.
Where is a useful question to answer, and nice to have consistent across an ecosystem. But What is also a very important question to answer, and tying What to be mutable and dependent on When is... highly unfortunate.
The opinions expressed in Go are solely those of Google about the productivity of Google programmers. Nothing else is driving design. It is by Google employees for Google employees. Making it widely available and open source is a strategy for extracting value in the form of bug reports and patches and Google friendly upgrades from a world wide community for the price of a few conference talks and some sticker swag.
That Go solves some problems for programmers outside Google is a side effect of being a general purpose programming language. It is not a primary business goal except in so far as it raises Google's bottom line, creates a hiring pipeline, and keeps its employee moral off the floor.
I think if you're replacing "Google" with "those Bell Labs guys", you're closer to the truth. A lot of the design decisions are quite familiar if you look at the past work of e.g. Rob Pike.
It's not necessarily all about peak efficiency for specific enterprise applications.
I agree completely. Go is opinionated about the wrong things, in the wrong way. Which in the end make it a worse language than it could have been.
One thing you did not mention is very varied quality of the standard library. I’m still baffled about ”standard date” crazyness. There are huge warts in the sql interface etc
My collegues used to joke golang is a great solution to google’s problems (for others, not as good as it arguably could have been to put it mildly)
- Go's opinion about generics has always been that it should have them, but finding the right expertise to implement them in a non-ridiculous way has been a struggle. They are marked for inclusion in coming versions (2.0).
- error is an interface. If you are returning strings most of the time, you're probably doing it wrong.
- Wasn't gofmt originally executed during compile time? I'm not sure why it changed, but it suggests that such automation fell apart in practice.
> Go's opinion about generics has always been that it should have them, but finding the right expertise to implement them in a non-ridiculous way has been a struggle. They are marked for inclusion in coming versions (2.0).
The fact that people can claim that C#, Java, Scala, Kotlin, Haskell, Rust, Swift have "ridiculous" implementations of generics sounds like a very bad excuse.
The Go team has, until recently, been quite open about not being confident enough in their own abilities to implement it well in Go. Maybe to the point of absurdity, but they were explicitly warned by designers of other popular languages (specifically some of those on your list, I believe) to not be cavalier in how they add generics.
That doesn't mean nobody on earth can implement generics well. I'm sure if someone from the C#, Java, Scala, etc. teams, with the necessary expertise, wanted to step in and contribute generics for Go, it would have been a welcome addition. At very least for the community who would have jumped all over a fork with generics. However, I do not see that code anywhere. You cannot force people to do things they do not willingly want to do.
Now that the Go team has had enough time to put for the effort to learn more about generics and the research related to them, they feel more confident about being able to add them in a somewhat reasonable way and have officially announced that they will be added in the near-ish future.
> gofmt is a good idea (just like clang-format or yapf), and having it is great, but the maintainers specifically refuse to add simple features, saying "running it in an automated process is not supported", despite all github projects already having it in their automated CI suite
A Golang noob here. Can you (or anyone else) expand more on this? Are there any links that I can read more about this?
Actually, the mistake is mine, I meant to write golint (but too late to edit). I have no issue with gofmt. As for golint, https://github.com/golang/lint/pull/100 is the kind of thing I'm thinking about.
The Go team aren't the most respectful, but I think Dsymonds is uncharacteristically rude. Most of my experiences with the rest of the Go team have ranged from "sorta terse, but I'll give the benefit of the doubt" to very positive.
I've been writing Go for quite a while, and probably used everything it's got to offer as a language. Go is opinionated, yes, however, for some crazy combination of reasons, it's amazing at getting out of your way for the most part.
Using interface{} instead of generics is a bad idea, you lose type safety. It's messy, but code generation is your other alternative, sort of like your own templating system. I've used go's built in templating code to generate type safe Go code from things like SQL table schemas. I'd highly recommend that route over naked interface{}
Error handling is string or type based, that's up to you. I declare specific error types for my functions, like FooError or BarError, the caller can then switch on err.(type) and respond appropriately. Alternately, you can define concrete error objects and return those, and make comparisons like "if err == FooError".
I'm with you that dependency management is terrible, vendoring is a poor workaround.
> - stability: the language itself is fine on that, but the ecosystem really isn't. The fact that they don't even have the package management quality of python (which I find awful already) is baffling. I don't even know how they could think "yeah, github-importing is a good idea, let's do it". Go is eight years old tomorrow and you still have to rely on third-party tooling to do proper imports that don't burn your house.
Google operates on a mono-repo basis, as I understand it (from speaking to ex-googlers.) All libraries etc. etc. all under one big repo. If you think about how that would appear to end tooling consuming it, the approach taken with imports makes sense. It's lousy for the rest of us outside of the google mono-repo ecosystem, but it does make sense.
I am new to go and am especially curious about what you find lacking on go test. Could you elaborate on that point? I've found it nice to work with so far and wonder what kinds of issues others have had/what I can watch out for. Thanks!
Why? I'm not saying go is a bad language at all, it has a lot of good points, as outlined in the source of this discussion. It is however not a perfect language and as such, it never hurts to criticize it, if only in the hope that it will improve in the future. I personally find the mental model required for go quite abrasive, but that's only me, and I won't hold it as valid criticism.
The alternatives vary according to the issue you want to tackle, so I would be hard pressed to recommend a particular one. And go does fill some vacant spots in language space.
That's the problem, I see the following often suggested instead of Go:
Java - maybe if you use Spring Boot you can get around some of the boilerplate, but there is a ton of baggage in the ecosystem.
Clojure - This is really popular where I work but LISP + Java Ecosystem turn a lot of people off. Also, some people really want static typing (maybe Spec is good enough?)
Node/TypeScript - terrible concurrency, good for throwing something together that works rather quickly, good for cutting corners. Terrible memory, cpu profiling. As a professional node developer in the day job I could go on and on, but at least the type system is better than Go (too bad the concurrency story is so abysmal)
Elixir - not popular, but at least it's on BEAM VM and has lots of libraries. Not statically typed (yes you can use dialyzer). Never done anything with it, but apparently deployment is a pain.
I personally write JavaScript mostly because its ubiquity. It's good enough, until it's not.
What I've noticed, and personally experienced, about these porting projects is that they're generally a premature optimization but are a great way to learn a new language. The gains from a port are largely intrinsic in nature, residing with the programmers. However, systems and operations largely carry on just fine with the original language chosen.
Only one story comes to mind where a programmer had a legitimate problem using a high-level language and solved it with a lower-level language, and that was Armin Roacher, who solved a significant performance issue using Rust: https://blog.sentry.io/2016/10/19/fixing-python-performance-...
Can you think of other stories? Please, share.
I encourage all to be more skeptical about potential gains and realistic about who is actually gaining.
Developers have the gain, company suffers, developers move on for higher salary with more skills, CTO is in it, doesn't fullfil his duty towards the company or investors, CEO has no clue and doesn't manage the CTO.
The story I see over and over again as a startup CEO consultant.
Developers learn a new language that is faster to develop, safer and faster to run than existing solutions (which work but are of increased technical debt) and successfully & quickly port the existing solution to the new tool.
The company doesn't really benefit from it apparently because they never scale enough or because they lacked a target so they had their developers working on sth small instead on working on the company's products.
This skill is highly valued and the developers leave for an other company while the original one struggles to continue.
Do you think that it may be the case that the problem in all these is not within the developers or tool? Furthermore, shouldn't the CEOs and the CTOs know better and to a better job?
> Developers learn a new language that is faster to develop, safer and faster to run than existing solutions
Would that it was that simple. Most of the time I've seen this it's been more like “writing new code is fun, fixing a bad design in our current code is too much like work” without factoring in the cost of rewriting, testing, optimizing all of the code which wasn't a problem relative to simply replacing the hotspot which was.
Yes, sometimes a codebase is so bad that there's nothing of value to be preserved but that's pretty rare and unless it's some turd from an acquisition or consultants the odds are extremely high that the broken business culture responsible for the first bad codebase will cause just as many problems the next time around.
That does happen but in my experience it's far less common than fanboyism or poor architectural/analytic skills — things like spending months rewriting a Python program in Go for a single-digit percentage gain because someone on HN said the magic pixie dust would make it faster and nobody checked whether it was I/O bound, spending most of its time in OpenSSL, etc. I've seen people propose rewriting code in C rather than optimize a database query because Real Programmers use hard things like C whereas SQL is beneath them. (I wish I was making that up)
To be clear: I'm not saying that Go isn't a perfectly fine choice, only that our field is more prone to fads than we like to admit and most of the things which cause bugs or make people wait are decisions higher in the stack.
I understand the situations you are describing and I still have the impression that the main problem behind this is inadequate leadership (and probably at several layers...).
There are more than enough Real Programmers or lacking soft eng who are not experienced or knowledgeable enough to know when to use C/python/go/sql/etc but again for me the problem is either whover hired them and whoever is managing them.
I agree that the main problem is leadership — and arguably mentorship as well. As a field we're too predisposed to view all problems as technical and ignore the social factors which lead to the visible technical symptoms.
Well, the "in the eye of ... with no proper evaluation" argument can be had for a lot of other aspects of software development and product management, doesn't it?
Hell, I've seen this as a lead/senior engineer. I'm seeing it right now, in fact, as a person on our team wants to write some stuff in language X, which nobody else on the team has experience with, and which isn't used anywhere else in the company, and to solve a problem that is perfectly well solved (in both development time and performance) with the language and tooling we already use.
As far as I can tell this engineer just wants to stick "built stuff in X" on his resume. He doesn't have a good reason to not use our existing stack. If I were the manager I'd demand more technical justification.
But this also might be a problem of a company not providing any improvement / career prospects for your programmers.
If working as a programmer means sticking to C# or Visual Basic or Python codebase and just working on adding new features or new business logic to your application for years and years, many engineers will not find that fulfilling. The best engineers will always want to learn about new tech and approaches (dev ops, containers, PaaS, automation, new languages and tools, AI / machine learning etc).
You cannot blame them for leaving. What you should instead do is provide interesting challenges for them, if they want to use some new technology, work on finding a justifiable use case that management will approve for it.
I understand that for a company that treats tech as a cost centre this approach makes sense but then they can't wonder when best engineers keep leaving to proper tech companies that do actual R&D and where they can become best engineers they can be.
You need to allow your best programmers to learn new tech and progress in their career or else they'll eventually leave for greener pastures and you'll be left with mediocre people who are content and don't want to learn new things.
It's like that analogy I have heard over and over again.
CFO asking CTO: What happens if we invest in developing / training our people and they leave?
CEO replies: What happens if we don't and they stay?
This has nothing to do with whether engineering is a "cost center" or not. It's about fundamentally what is important about the practice of engineering. Whether to use Python or Go or Java or Rust or whatever is rarely anything other than bike shedding. In those cases where it's not, the key problems are rarely which language is technically better: more often the problems are associated with labor supply and how well the proposed language' ecosystem plays with existing implementations'.
To be fair I've been on the underling side of that where I wanted to build some new server we needed in a new language (coincidentally, Golang), but it was honestly just for curiosity/shits&giggles. It became somewhat of a running gag for ~5 years where every new service we needed I'd suggest Golang.
I'm a senior engineer and I'm also the person that wants to use the shiny new thing.
I'm just learning Go and I'm itching to find a project to use it on. It's super hard to justify it though when C# or Python would work just as well and those are languages that we already use.
The best time to do it is on a new project or component of an existing system, in my opinion. It doesn't make it a good idea, but it can be a nice way to introduce something new and possibly more productive.
How about this: once you write code in Go and test it well, you can deploy it and forget about it unless there is a hardware issue? I recently ported some C++ code to Go which processes 8 Billion events per day flawlessly.
Once you write a well designed, well tested, and feature complete piece of software, you can deploy it and forget about it unless the server it's running on breaks.
Go isn't special in that regard, unless there's something that makes Go easier to write, test, or deploy, which might be the case, but you haven't supported that.
Our team was able to develop/deploy about 20 Microservices in Go in the past year or so which is really awesome. I can say this after having worked with several other languages.
I ported a Groovy/Grails app to Java/SpringMVC once.
My technical motivations were more about moving away from Hibernate. Arguably there were plenty of reasons for leaving it as it was, though -- if I had known enough to be able to solve certain data access complexities back then, Grails was in many ways a better fit for the problem. The experience in a (more-or-less) functional language wouldn't have hurt any either.
The business motivation was simple: Java/SpringMVC programmers are (or were then) far more plentiful, so adding/replacing tech resources was made much simpler.
This example actually seems to be the opposite of an optimization. There's 50% more code to maintain and there appears to be almost no appreciable benefit to compensate for that.
LOC is a bad measure for the complexity of code. Quite the opposite, expressive code can be faster to understand and easier to maintain than the same functions with half the code size.
In any case, switching to a compiled language and static types should bring some benefit, in addition to the gain in speed.
IME 95% of the time SLOC approximates code complexity just fine. Just because you can write horrendous 'clever' one liners in perl does not mean that most code out in the wild is like that. I certainly don't think the same author writing the same app in two different languages will be writing like that.
Zero mention was made of bugs caught or customer-noticeable speed improvements so I'm inclined to think that there was actually no benefit to having static typing.
No, there was no appreciable user benefit, and that was intended. To respond to others: it wasn't a "premature" optimization, because it wasn't any kind of optimization. The sole benefit in this case was for me to learn Go programming -- and I noted that in the article title and intro. :-)
No-one goes out of their way to choose a slower language. People who choose to write something in Python are getting value out of it, or at least believe they are; if writing the program in Go will be more costly (in development time, defect rate, library availability or something else) than writing it in Python then writing it in Go is premature optimization, and if writing it in Go wouldn't be more costly than writing it in Python then why were you thinking of writing it in Python at all?
IME thinking about language performance at all is premature optimization; language performance almost never makes the difference, other between-language differences will swamp any gains from language performance.
> Are you serious? Python code is about 10 times slower than equivalent code in Go.
And 98% of the time it doesn't matter. For most business problems that you'd want to solve with a computer, for modern hardware, Python's performance is more than adequate. It's really not worth worrying about.
No. This is only true if you have a script you run once an hour or so.
If you have a server side application that you hope to be successful then the code will be running all the time and you will pay the price literally because cloud hosting services charge for CPU and RAM usage, which will be off the charts if your server is in Python.
Not to mention all the time you will spend firefighting the performance issues that will inevitably arise. Which also literally costs money in terms of developer salaries and opportunity cost (you can't spend that time to develop new features).
> If you have a server side application that you hope to be successful then the code will be running all the time and you will pay the price literally because cloud hosting services charge for CPU and RAM usage, which will be off the charts if your server is in Python.
Your hosting fees are a tiny fraction of your expenses. Development time is worth a lot more.
> Not to mention all the time you will spend firefighting the performance issues that will inevitably arise. Which also literally costs money in terms of developer salaries and opportunity cost (you can't spend that time to develop new features).
Scaling is a nice problem to have. If you get to the point where you actually need to improve performance, you'll be able to afford to spend time on it.
> Your hosting fees are a tiny fraction of your expenses.
Only when your requirements are low.
Things get very expensive when you start requiring real performance.
> Scaling is a nice problem to have. If you get to the point where you actually need to improve performance, you'll be able to afford to spend time on it.
You don't know that. If you're a startup, it's very likely that you're still not profitable at that point.
But the code doesn't have to become more complicated for it to be premature optimization. It literally just means improving something (generally performance) before there's a clear need.
> Writing the same code in a fast compiled language is just the default thing that you should be doing when you are not doing premature optimization.
I assume you mean "rewriting" here, since we're talking about porting existing code. Sometimes it's the best use of time, sometimes it's premature.
Remember the context here. The idea is "premature optimization is evil". A lot of people often abuse the quote and then conclude "Caring about performance is bad" which is a complete non-sequiter and not the originally intended meaning of the quote.
The reason premature optimization is evil is that it consumes a lot of time and makes the coder more difficult to read and reason about.
If there's no clear benefit to this, then all the work done to optimize it resulted in _negative_ value. There was no gain, only time wasting and more complicated code.
An example of where performance doesn't matter: a piece of code is only executed once every 10 seconds and it finishes executing in 5ms. Spending time to reduce its execution time to 1ms does not produce any tangible benefits.
An example of where performance does matter: a piece of code is running all the time and it takes 500ms. If you can reduce it to 20ms, it's totally worth it.
In Go it's impossible to make for...range support user-defined types, and you have to write out every loop the hard way (and rewrite any loop that was using a builtin type). In Python you just define __iter__. That's what's meant by expressive, you can write what you mean instead of having to explain yet again how it should be implemented.
Of course they're different on that level of detail. In go, if you want programmable iteration behavior, you would create a channel that is fed values by a coroutine. Then you can range over it. It's not a new type, but that distinction seems a bit picky.
What I am talking about is the static type system. You can't express that in Python, despite how much more convenient some of its features are.
I agree that writing in a fast, compiled, statically typed language is not necessarily premature optimization. It is my first choice for a new project in the absence of any other mitigating factors.
There are often mitigating factors, though.
And switching from an existing language to use a new one that is possibly more optimized carries costs that have to be considered. If the new implementation shaves a few percent off some time or other resource scale, and the sum of that savings over the life of the program is more than the cost to do development and maintenance it's worth it. But that often isn't the case.
Let's say that C produces 100% of the performance available. How much does Go produce? Maybe 90%? How much does Python, say, produce? 25%? So going from Python to a compiled language produces a huge gain, and going from Go to C produces a little bit more.
But why not get all the gain? Well, sure, if performance is the only thing you care about. But it usually isn't. You might also care about networking, or multithreading. Would I rather write that code in C, or in Go? Is it enough better in Go to be worth that last 10% of performance? Arguably, yes.
When someone says "X matters", it doesn't mean that they are saying "X is the only thing that matters". It is not correct to conclude that they should, for consistency, be saying "turn the X knob as far as you can". Instead, interpret it as saying "given your other constraints, look for a sweet spot where X is somewhat toward this end of its range."
Yes, that's my point. It is premature optimization, because you do not have a use case for such performance. Instead you might want Python's extensive numerical libraries, or you enjoy its quick prototyping capacities. The same way that you do not always go for C, and sometimes prefer Go, because of non performance related requirements.
That's why saying a compiled performant language should be default is not logic I understand. Default should be whatever you need for your requirements. If you don't need anything special, just go with what you already know well.
Fine, but if that's your point, don't say "Why not use C all the time by default then?" Actually make that point, rather than leaving us all guessing as to what your point is, and more important, why it's your point.
And, as I said, "Why not use C all the time by default then?" does not actually follow as the logical endpoint of the argument that your trying to answer, so making that your response really doesn't contribute much to the advancement of the conversation.
Using a compiled language is not a premature optimization if you're not sacrificing expressiveness.
With that said, Go is not the best choice. Like, for anything.
That was just a harmless (I hope) hyperbole.
I've actually used Go a couple of times for image processing workers and such. And that's when I developed my distaste for this language.
I mean, I get it, it's easy to pick up and use. Coroutines and channels are useful, and make easy to write async stuff.
But god damn, it's so dull and boring, there's no joy in it. We're programmers, we're supposed to extract abstractions and express them in the code we write. And IMO your language should definitely try to help with that.
Instead, Go wants you to do err, value = ..., if err == nil {
... } on almost every single line.
You can't adequately compose stuff. Well, you can try, but you quickly run into its ugly, verbose syntax for anonymous functions or lack of generics.
Maybe I'm not getting something, maybe it's just not for me, I dunno.
But next time I need high performance compiled language, I'll use Swift or maybe even Rust. Also I think Scala Native is somewhat stable already, gotta look into that.
OP was saying porting a project to Go sounds like premature optimization. The alternative is obviously to continue to use the original language.
The question to be answered is why are you porting it? Fun and giggles? Hoping to achieve better performance? Scale? Portability? Correctness? Simplicity? Support?
I'd say most of the time its for fun and giggles. Which I think is as fine a reason as all the others, but when its costing someone's elses money, maybe you should be more considerate.
Ya, maybe for some. I think for businesses part of it is familiarity, as its now a lot of people's first language. I also think its a choice made often when you start, because it allows you to go faster in the beginning. Which is where a lot of new projects need to be the fastest. Its still a great choice for projects that are short lived, that's why ML works so well on it, most of ML is just quick iterations on prototypes until you find a solid model.
Depends. If we're talking about high performance, yet expressive language without a steep learning curve, I say Swift.
The only major drawback right now is lack of pure Swift AWS SDK. And unfortunately, AWS team has no plans to write one, yet.
The most general answer is OCaml, though in practice I'd probably use something else depending on the details. For most cases there's a better pick than OCaml, but I don't think OCaml is ever a worse choice than Go.
Recently I have had the opportunity to write a microservice in Go. It was a very refreshing experience switching from Scala. Go is a lot faster to compile and runs with a far smaller footprint. The channels are nice. On the other hand the testing feels wrong because its so annoying to do mocks. And don't even get me started on the dependency management.
Overall Go feels to me like some version compiled PHP with better concurrency support.
I really wish that there was a language which had these features:
- simple (so not Scala, Haskell, Perl 6, etc.; no Rust either, unfortunately)
- clean (nice syntax, preferably Python inspired, but consistent)
- modern (generics, some functional features, string interpolation, etc.)
- decent concurrency/parallelism story
- good IDE, preferably supported by the core dev team
- compilation to a (possibly static) native binary
- decently big ecosystem
- and a big enough community of contributors or commercial backing from at least 1 stable entity
- also preferably didn't have a lot of historical baggage
From what I gather C# might actually be close to this, 2-3 years from now, if the .Net Core transition happens smoothly and the OSS community adopts it... Who knows, maybe even Kotlin 2019 or so?
I like Kotlin too, but in which way is it "much more expressive" in your opinion? What can you express in Kotlin that you can't express in Java? Scalas type system is much more expressive than the one in Java as an example. Only thing that comes to mind in Kotlin is non-nullable references.
Most of the stuff in Kotlin looks more like less boiler plate (data classes) than "more expressive".
I do not argue that Kotlin is a nicer and a language with more features that are useful.
What of these would you call "more expressive"? I'd agree with sealed classes, they express a limited set of possible classes. Perhaps tail recursion as I can express problems in way of recursion without stack overflows. I don't think coroutines are more expressive than Futures/get, only a runtime optimization for many concurrent executions.
Async/await coroutines (C#, python, ECMAscript 7) are much nicer than futures, at least ime. I think that in terms of expressiveness of async code it goes
callbacks -> futures/promises -> coroutines
Because callbacks are a completely different style of code, your async looks totally different. Futures and promises are better, but still make it difficult to ever break out of the async context, coroutines make the difference between async and normal code nonexistant, and allow clean escape from an async context.
Interesting. I thought the lack of proper tail recursion is a JVM limitation. Hence why Clojure uses loop/recur to handle it by translating it to a loop rather then as an actual tail call. Which means things like mutually recursive functions are not possible in Clojure without blowing the stack.
How does Kotlin handle that? Is it internally rewriting to a loop or trampoline, or did they find some way to actually make proper tail calls?
Yes. You add a `tailrec` modifier to your function. The compiler then checks if the last op in the function is a call to itself. If it is, it gets replaced with a loop. If not, you get a warning that `tailrec` is ignored.
It's not immediately obvious, but when you write some Kotlin you'll see.
Little things that are a apin in Java and elsewhere are quick and natural in Kotlin:
- nullable types are smoother than Options
- null checks are relatively painless
- the stdlib works with you, not against
- named parameters and optional parameters means no need for builders
- etc. Etc.
I have written some Kotlin, and perhaps it's me, but I don't see.
"nullable types are smoother than Options - null checks are relatively painless"
Might be the case, but Options are more expressive in that I can combine them and restrict them with other type classes to express larger constraints. Reusabilty is higher e.g. with Monad transformers like OptionT.
So nullable types are less code, but also less expressive as building blocks of larger constructs.
They might be also less expressive because they conflate two concepts: Not initialized and not-there, whereas Option expresses not-there and _ (in Scala) expresses not-initialized.
Nim has everything nailed down except the ecosystem/commercial backing; It does have some of those, but not to the extent that I would blindly tell you they are there.
It's as fun as writing Python (with a similar syntax), but it has essentially all the goodies you want from Lisp when you need them, runs as fast as equally optimized C, produces standalone native binaries, _or_ standalone JavaScript if that's your thing.
Cheers to Nim. I'm learning it in my spare time for fun. However, I will not consider using it for anything more than toy projects until it hits 1.0. Hopefully soon.
> I will not consider using it for anything more than toy projects until it hits 1.0. Hopefully soon.
It should happen soon (TM), but from what I understand it will be just another version, just this time renamed to 1.0 - this basically means you could start using it more seriously even now.
Yeah. I understand that it will be just another version with not that many changes. Nim as it currently already feels stable and solid. But, I believe v1.0 release will carry a commitment to not introduce breaking changes at least for reasonable period of time after.
A lot of people I know are bailing on .net because it doesn’t live up to the promises. Everyone else gets the cool stuff first, the vendor that owns it is a dick and there has been so much schizophrenia over the last few years that it’s a business risk adopting it in case you have to throw your product under a bus again.
Go is showing itself to be a good compromise especially with the simplified nature of deployment and runtime stuff. It’s like a safer, higher level C, with a decent runtime library. That has so much mileage.
I’ve learned one thing as well over the last 15 years of using .net as well. An IDE is a monolithic brain damager. If you have to do something outside the scope of it, you’re shot. Give me an extensible text editor and shell.
What about Java? Its easy, modern in the way you describe, has a good concurrency story, multiple great IDEs, can compile to a static binary with commercial JVMs, or experimentally in OpenJDK9. Really big ecosystem. Huge commercial backing. It just maybe fails at historical baggage and syntax.
Kotlin fixes those last two. And Java 10 will also fix a good chunk of the syntax complaints.
In all fairness though, I think you're focused on nitpicks that have no practical benefits. You're asking for a language with a better user experience. Which is great, as it can make programming more pleasant, and I'm all for it.
That said, I'd focus on language changes that provide functional value. Haskell adds a powerful correctness layer at a low expressiveness cost. Go adds a useful CSP concurrency mechanism with green threads, as well as being friendly to containerisation. Erlang and co add a great deal of reliability features and massive parallelism with first class actors. Clojure gives you powerful metaprogramming, state of the art immutability and a similar CSP layer as Go, while maintaining the access to Java's large ecosystem. Kotlin adds null type checking, and better functional primitives. Rust gives you safe memory access without the performance overhead of a GC. Chapel allows for super parallel computing. Scala gives you OOP with the safety layer of Haskell on it. Nim gives you faster python, with type safety. Crystal gives you faster Ruby with type safety. Etc.
Those are the languages that challenge the status quo, and have a chance at bringing real value in terms of the average output they result in when used to program with.
A slightly better UX for Java or Python is a hard sell. Its like the famous Unix quote: "the most dangerous enemy of a better solution is an existing codebase that is just good enough." - Eric S. Raymond
Java, C#, Python, Ruby, C++ are all good enough, even UX wise, to be displaced by something that doesn't offer compeling functional value. At least, in my opinion.
C# actually already has most of that except for "compilation to a (possibly static) native binary", which is really just a convenience for shipping. It's also not really very cross-platform, it's definitely a "Windows First" language.
I'd recommend continuing to look into .NET Core. It's really quite stable these days and has all of the features you've mentioned, as well as C# having a lengthy history as a language. Knowledge is relatively transferrable amongst JS (especially TypeScript) developers, as well as Java developers. This puts you into an area where finding capable people is easy and cost-conscious (vs. niche languages.) Historical baggage has also been trimmed with .NET Core, and Microsoft has done well maintaining the language over time (adding new concepts and removing old ones.)
In a couple years, .NET Core will be even more stable and will surely have convinced the stragglers in the community to adopt it. (I'm seeing this is already mostly the case, because everyone likes not being locked into Windows.)
I'm a pretty big fan of .NET Core (coming from Node.JS) and I really do expect it to grow a lot in the next few years as people discover how easy and sane it is.
OCaml seems to match a majority of the criteria, though AFAIK no "good IDE" (tooling like merlin is available, but if you're looking for e.g. a refactoring IDE you're probably SOL), the ecosystem is small, and it does have some historical baggage.
I guess beauty is in the eye of the beholder, though your eye seems questionable, the only "shortened keywords" I can find in the wikipedia page are "rec" (which is a long-standing shortening of "recursive" in recursive lets) and "fun" (for functions).
As another commenter mentioned, you might want to try out Reason [0]. It's a new-ish programming language from Facebook that uses OCaml as its backend.
SML is what Ocaml should have been. With a few small syntactic changes (mostly to add some nice-to-have features), SML syntax would be better in every way instead of almost every way.
Pretty much everything compiles to bytecode (e.g. Java does, Python does, C# does, …)
I expect they want a native binary (possibly even statically linked) so they can trivially distribute the artefact without requiring that the VM be installed or bundled which is a PITA, in which case "compiles to bytecode" is useless.
- arrays with slices (that could also be used for strings) and Cilk Plus array notation [1]
- maybe actual string type with separate concatenation operator (possibly space as awk uses and C for literal strings) it could probably be folded to improved arrays
- standard SIMD vector types so that most use for operator overload disappears
- cleaned up standard library
- actual macros so the remaining use for operator overload disappears
- module system (so you would generally not need make and could easily compile to static binary)
- already you can build with static musl libc [2]
- something like libdill [3] or libmill [4] in stdlib
- there are multiple good IDEs, debuggers, analyzers
Yeah, but Kotlin Native loses all the Java libraries. The more interesting thing would be Java native compilation coupled with Kotlin. So you'd write a Kotlin Java app and pass that through the Java native compiler introduced in Java 9. Unfortunately that's a beta feature right now, I think it only supports Linux x64, that's why I said Kotlin 2019, I assume that by then the Java native compiler will support all Java platforms.
I'm probably too conservative, but I'd rather use these kinds of things from the actual platform developers. I'm a bit more liberal with libraries, but in general I want the compilers/interpreters from the upstream source. Software has enough bugs as is :)
I think my dream language would be C# but with a slightly more Pythonesque syntax (significant indentation, no semicolons or braces...). I wonder if a transpiler like that exists.
I haven't had a problem with testing in Go, but I also don't mock many things. Most things that should be mocked are already interfaces in my code. My biggest grievances with Go are its limited type system and dependency management. Unlike most[^1], I don't need semantic versions--I just need deterministic, reproducible builds, and for that, vendoring is fine in theory, but Go's tooling around it needs work.
I love that Go is a language that can be mastered in a matter of months; that it has such a simple "get up and running" story; that its tools mostly do the right thing by default; and that almost everything is straightforward. In most other languages, I have to figure out what build system to use, how to script it, how to specify and download dependencies, what testing library to use, how to actually run the tests, how (if it's possible at all) to get a static binary, etc--and then I usually have to learn a complex language on top of all of that. For the most part, Go just works out of the box.
I think folks also underappreciate Go's runtime--it has a fast garbage collector and an awesome goroutine scheduler that all get statically compiled into your application binary. Mostly, I want a functional language that compiles to Go so I can still leverage Go's awesome tooling, runtime, and deployment model, but with things like generics and algebraic data types. I've started prototyping it too, and so far it's coming along smoothly (modulo all the things I'm learning about building a programming language). On that note, there is some effort going on in the Go community to build a VM for Go, which would probably make an even more attractive compilation target, at least from a toolchain perspective. Even if my project doesn't take off (and it likely won't), I think this is a low-effort way to address Go's most significant complaints.
[^1]: Well, "most" judging by the significance many of Go's critics place on semantically versioned dependencies
I have, but early on and not much since. I forget my particular issues; I just remember at the time there didn't seem to be a way to do what I wanted (I think it was adding docker as a dependency).
YMMV, but my experiences with SBCL were way better than with Go. Using Hunchentoot and the other Quicklisp libraries made developing fun again. With Go I had to constantly worry about errors, repeat the same code snippets over and over again, had to comment out "unused" variables just to try something... not fun. That said, let's see how Lisp works out once I cross the 10'000 LOC barrier. But by then it won't be a microservice anymore, anyway...
Chances are, if you are using mocks in Go, you are doing it wrong. Functions should take interfaces (not interface{}, but interfaces that list methods) and return structs. In your tests, you create a fake that meets the interface and pass that to your function under test.
If you have to test file io or DB interaction, you can either do the adapter pattern or, as I prefer, just actually use the filesystem and DB but with some test infrastructure around it.
Might be splitting hairs. There are fakes, mocks, and stubs. Generally, mocks assert, stubs do not. Some folks define both stubs and mocks as fakes. This does not fit perfectly for Go. The "fake" in Go is just a struct that matches an interface but is purely for testing. It allows for dependency injection where you pass a fake designed for your test. It can have its own properties and helper methods, but the key thing is that it matches your interface. You don't make assertions against the fake like you would a mock. You assert against responses from your functions under test.
Example:
> type Bar interface{
> Walk() (int, error)
> }
>
> func Visit(b Bar) (string, error) { ... some code ...}
In the production code, you call Visit with some struct that matches the interface (has a Walk method). In tests, you create a new struct like fakeBar that also matches the interface. You can have your fakeBar return any string or error you like and you can verify that Walk behaves the way you need it to.
I'm at the same spot myself, trying to learn Go by building a simple web app. Your article was a refreshing read to me.
One extra thing that is the major motivation for me to go to Go is the ability to deploy as a single God damn binary and not deal with dependancies and pip. pip and counting on OS repositories may be the right thing to do, but it is just too cumbersome. I have an air gapped setup at work, and there is no easy way for me to deploy Python applications - because all projects just say pip instal this and pip install that. It would make my life so much easier building something on my laptop, scp the binary to servers and expect them to "just work".
Coming from a data science background, don't anaconda and environment description files [1] solve this problem with python? Not even sudo is needed to install anaconda and setup / clone environments, also it can manage python versions independently from the system's python.
I have just started using Conda locally to learn before taking it to work. I'm a new sysadmin and my predecessor maintained a restricted list of python packages on all cluster systems like a dictator. I don't want to be one.
Does conda cloning the environment mean I can just copy the project directory to another server and expect my program to run without it looking for the packages on the internet?
No, the intended way of cloning works by exporting the exact specifications to a text file and re-creating the environment from the text file.
If you use packages from conda channels (which are repositories of packages), then it should work flawlessly. Usually basically everything is available from either the official channel (anaconda) or some community channel (like conda-forge). At least in the data-science domain. I'm not really familiar with the python web-development scene, so not sure about that.
Edit: actually once I had to copy an environment from one machine to another (because of connection problems), and I seem to recall that worked too, but I don't think this method is officially supported.
Well, as my edit says, I've had success with installing anaconda on an other machine and copying the environment folder to the proper place inside the installation, but AFAIK that's not an officially supported way of doing it.
It does work, AFAIK, but you can also move it around on a single machine if you rerun "virtualenv <dir>" on the env (it fixes the links). It's been a while since I've done that, so I'm not sure if it still works, but it should.
EDIT: Yeah, just tried it again, it rewrites the shell files so everything works again. It even uses the same python version as you used.
I don’t disagree, but FYI pip has a —cache flag to use packages from file, and the files could be on a usb stick of course. Same approach can be used to speed up CI setups.
Surely there's tools to bundle python apps? I'm not as familiarized with the python ecosystem, but with node you can use browserify or webpack to bundle an app into a single JS file, or pkg [0] to generate a single executable.
Some languages have even more powerful tools for that sort of deployment. One of the best examples I've seen is distillery [0], an Elixir package which allows for hot upgrades. That means you can just copy a single a file and upgrade without having to take anything down.
I am doing some python to go conversions for some performance gains in parts of my stack - I really like it so far because of how they brought some of the perks of languages like python into a compiled language. One thing I really miss though is ipdb.
The Delve debugger still doesn't let you run functions and I miss all the autocomplete fun with ipdb. I hope they can make go a little more friendly to debugging in general.
>However, it was only 1900 lines of Go (about 50% more than the 1300 lines of Python).
>However, I’d seriously consider using Go for larger projects (static typing makes refactoring easier)
Assuming a fully tested codebase and a relatively sane runtime type system, I'd much, much rather maintain the shorter codebase than the statically typed one.
Assuming a fully tested codebase, there is an overhead in writing and maintaining unit tests that has to be considered as well, and using a static type system obviates a whole category of them. You move the type specification out of tests and manual documentation directly into the code where you'll find out at compile time what you did wrong. Not at run-time, or during testing because you diligently wrote a test that considered the case.
I agree that brevity sometimes aids refactoring, but there are many parameters that have to be weighed in, and often I find that the benefit of static typing outweigh whatever ignoring the type or checking it at run-time might offer.
>Assuming a fully tested codebase, there is an overhead in writing and maintaining unit tests that has to be considered as well, and using a static type system obviates a whole category of them.
I've not found this to be true, and tests that do type checking are a waste of time in any language.
Bugs that are caught by static type systems are almost always caught by a combination of behavior tests and a sprinkling of sanity/type checks on border code (e.g. a module's publicly facing API).
>you'll find out at compile time what you did wrong
In a fully tested code base that is regularly tested the distinction between compile time and runtime does not matter a lot.
By contrast n code bases in dire need of tests (which is more common than generally assumed), the distinction is absolutely vital.
I'm not sure about Javascript, but why do you have to have two APIs when you have async? At least in C# you write the async version of the API and that's it.
Then you go asyncStuff.Request on it, and it blocks. There are some quirks but I remember that it worked all right.
I really liked to read the article. You gave your opinion and didn't try to sound that this was the way, it was clear that it was YOUR opinion. Thanks for it.
Go really feels opinionated around the wrong things, in order to claim "simplicity" as a feature:
- No generics just mean you're going to be handing interface{} all the way in your stack, it makes things more complex and less readable for no reason (and less safe)
- error handling which is essentially string-based, in 2017? Give me a type system, please
- gofmt is a good idea (just like clang-format or yapf), and having it is great, but the maintainers specifically refuse to add simple features, saying "running it in an automated process is not supported", despite all github projects already having it in their automated CI suite
- go test is good, but in the end it is found lacking
- dep, well, if you don't mind dumping the code for all your deps in the source tree, it's probably ok, but it solves a deficiency that the language should not even have (see next point)
- stability: the language itself is fine on that, but the ecosystem really isn't. The fact that they don't even have the package management quality of python (which I find awful already) is baffling. I don't even know how they could think "yeah, github-importing is a good idea, let's do it". Go is eight years old tomorrow and you still have to rely on third-party tooling to do proper imports that don't burn your house.