Hacker News new | past | comments | ask | show | jobs | submit login
Deconstructing Go Type Parameters (go.dev)
140 points by psxuaw on Sept 27, 2023 | hide | past | favorite | 58 comments



It really doesn't help that the major (perhaps only?) official resources on generics in golang are these blog posts [1][2] and the spec. And now this blog post.

The whole "what type am I getting"/make()ing is really tricky (as outlined in this doc) especially when its a pointer/interface/slice/etc. And a lot of feels like it doesn't need to be as much of a complex decision tree as it is. Is there any other documentation on this stuff that I'm missing?

Theres a lot of complication buried in golang people don't talk about that much. nil vs empty slices, interface{} and any behavior differences, make() and what it would do for various type scenarios, impossible to remember channel semantics (which ones panic again?). Of course, theres always a good explanation for why it is the way it is, but for a language so opinionated, stronger opinions on better DX in the deeper parts would be great.

[1] https://go.dev/blog/intro-generics [2] https://go.dev/doc/tutorial/generics


Pointing people at the spec is probably the best decision the Go team ever made. Why wonder about things when you can have true authority in an honestly very short piece of text? No guessing, just an exact description of what's going to happen.

(I was in there recently reading about operator precedence recently and discovered an operator I didn't know existed, "bit clear (AND NOT)", &^. Amusingly, I needed to do that operation but wasn't sure if I needed parentheses for X & (^Y) or not. I still don't know why it's a dedicated operator, however. The spec rarely says WHY, just WHAT.)


It does help that the spec (and memory model!) is quite good and reasonably concise. I send people there as often as possible when they start getting interested in learning things For Real, blogspam is rarely anywhere near as useful despite being multiple times longer.

(Which is not meant to claim this article is blogspam - making recommendations with examples is quite different from a spec, and we definitely need recommendations)


Yeah. Effective Go feels a little dated now. My most frequent link is to Code Review Comments: https://github.com/golang/go/wiki/CodeReviewComments


> Why wonder about things when you can have true authority in an honestly very short piece of text?

There is no such short piece of text because

> The spec rarely says WHY, just WHAT.


The one that got me was the implicit wrapping of structs into interface types. As an example, the below code segfaults with `fatal error: panic while printing panic value: type runtime.errorString`:

    package main
    
    type SomeError struct {
        SomeMessage   string
    }
    
    func (se *SomeError) Error() string {
        return se.SomeMessage
    }
    
    func doSomethingSomeWay() *SomeError {
        return nil
    }
    
    func DoSomething() error {
        return doSomethingSomeWay()
    }
    
    func main() {
        err := DoSomething()
        if err != nil {
            panic(err)
        }
    }


      func doSomethingSomeWay() *SomeError {
Yeah, you'd never return a concrete error like this -- clear red flag in any code review.

For precisely the reason you're demonstrating here! (Among several others.)


100% agree.

But do we have linting tools that raise this as an issue?


"Linting" this isn't really possible, because it's really beyond what a linter can handle. It would take a full static analysis job to determine that there's a problem, because there isn't any individual line of code that's wrong.

In this code, the error lies in the combination of all three of:

1. doSomethingSomeWay unconditionally returns a nil pointer.

2. That nil pointer claims to implement the error interface, but it is lying. If you actually call error on it, it will panic no matter what. But since the nil is not removable (no non-nil pointer types) this sometimes happens in real code, even though in this case it's obviously a faked up problem.

3. The main function packs the *SomeError into an error, then calls the method on it that will crash.

As this code. I would call it wrong if it had the type signature "func () error"; then it is definitively the part of the code that is lying and packing a value into an interface that does not implement that interface. But the example as written doesn't quite have any one line or function that a linter, generally simpler than static analysis, can pick up on.

(You may be surprised that I don't simply blame the code in main. I can explain why: https://www.jerf.org/iri/post/2957/ and a followup https://jerf.org/iri/post/2023/value_validity/ . It's actually an entry point into a very important thing for high-level programmers to understand, even well beyond Go.)


As jerf says, not really easily, no. But code review is a perfectly reasonable mechanism for catching this kind of thing.


I can't answer that, but the poster mentions it segfaults so it's identified that way for sure.


nil interfaces are not equal to nil values in Golang. Interfaces are fat pointers with a type and value pair and if the type is populated then the pointer as a whole will not compare to the `nil` value as it requires both elements to be nil. Frankly, I think this was a design mistake.


Yes, this has to do with interface values and how interfaces are implemented in Go. It's a bit weird.


Go101 is a jewel of a resource. It’s a spec written by an impartial observer not worried about trying to make golang look good but instead giving you the understanding to avoid all the edge cases

https://go101.org/


Eh, I can't agree. I'm sure there is useful signal in there, but the author has a highly idiosyncratic view of the language, recommends tons of "optimizations" based on benchmarks that often aren't sound, and has several bits of code that are simply incorrect.

As one example, the IsClosed function in https://go101.org/article/channel-closing.html is definitely not correct. The author doesn't seem to be aware of the comma-ok form of a channel receive as documented in https://go101.org/article/channel-closing.html -- which would be the correct way.

Many others. Reader beware.


Never seen this before but that’s awesome!


> interface{} and any behavior differences,

I'm curious now. They are type aliases. In which situations there is a difference in behavior?


There are no differences. You can always substitute one for the other without change in behaviour.

These two are interchangeable:

   func f[T any]() {}
   func f[T interface{}]() {}
and these two are interchangeable:

   func f(x any) {}
   func f(x interface{}) {}
Maybe OP means the difference between using an interface as type parameter constraint and using an interface as a function parameter / variable type.


No difference as any := interface{}


I was firing that list off the top of my head and I think that one was a mistake. My bad.

But, I did recently have some weird typing issues around "satisfies" x vs "is" x and using new() constructors of the T in a generic functions (particularly de-alternative serialization of proto.Message types) and had quite a bit of frustration and confusion. That may have been on me, but I did open my statement with "[wish it had better docs]".


I am a Go fan and have been coding in it for years, but this crap:

func Clone[S ~[]E, E any](s S) S { return append(s[:0:0], s...) }

looks just like Rust, which has the fugliest syntax I have ever seen. Personally I use maybe 3 or 4 generic functions to work with arrays(oh, sorry SLICES), otherwise I do not touch them. Could not care less about them and all that noise they caused.


In go, the stdlib impl is actually:

    func Clone[S ~[]E, E any](s S) S {
      if s == nil {
        return nil
      }
      return append(S([]E{}), s...)
    }
    
For comparison, `vec.clone()` in the rust stdlib is:

    pub trait Clone: Sized {
      fn clone(&self) -> Self;
    }

    impl<T: Clone> Clone for Vec<T> {
      fn clone(&self) -> Self {
        <[T]>::to_vec(&**self)
      }
    }
I think the rust one is much easier to read. The go one has an if statement, which means the go one has higher cyclomatic complexity, and is thus harder to understand and reason about.

The rust one does have "&**self", which looks a little strange perhaps, but overall seems simpler than the go one.


What is “&**self” called?


It is so simple it does not have a name. It is simply a double deref + borrow.

Since "deref" can be implemented differently for each type, you cannot know what it does in general, but in this case '*Vec<T>' turns it into a '[T]'.

If you compile in release mode, all of the following implementations of 'clone' will emit identical assembly:

    <[T]>::to_vec(&**self)

    <[T]>::to_vec(&self[..])

    <[T]>::to_vec(self.as_slice())

    self.as_slice().to_vec()
The stdlib picked the coolest looking one. Can't fault them for that.


Honestly I'd have preferred `.as_slice().to_vec()`


It reborrows `*self` als a slice. I prefer The Methode `as_slice` which does the same.


Yeah, that one if statement really melts my brain. /s


I'm okay with code like this being tucked away in the std lib (or a 3rd party library) IF it makes my own code easier, and in this case it does since calling it is just

    c := Clone(ms)
(which is highlighted in the article)


I suppose this is necessary because this:

  func clone[S ~[]any)(s S) S
would only allow things with an underlying type of []interface{}, not "any type" as an inferred type... and that applies to the final example too:

  // allows any collection of stringable things
  func WithStrings[S ~[]E, E interface { String() string }](
  
  // allows only things like:
  // []interface { String() string }{...}
  // and named types like that, but not:
  // []strings.Builder{...}
  // because that isn't the same collection type,
  // it's just a collection of compatible elements
  func WithStrings[S ~[]interface { String() string }](...)
I guess this is the price to pay to avoid introducing co/contra variance? It may be worth it, and it seems likely that it would be a thing you can improve without breaking compatibility.


Wow, I'm not super happy about the syntax of this language. I'm familiar with what each paragraph is describing from multiple other languages, but I can't even guess how some of the syntax here maps onto those other languages, even with the explanations.


You have to click through to read the assignability rules. A very short summary is that people often create new types based on core types. For example:

   type Name string
Now you can define methods on Name:

   func (n Name) Foo() { ... }
   ...
   x := Name("Me")
   x.Foo()
But, you can't use Name and string interchangeably:

   func StringFoo(x string) { ... }
   StringFoo(Name("Me")) // does not compile
In the case of generics, maybe you want to write a function that can handle any string:

   func Bar[T string](x T) { ... }
   Bar(Name("Me"))
   Bar("Me")
This doesn't compile, because Name isn't assignable to string. The fix is to declare the type parameter as [T ~string]. (The compile error suggests this, in fact. You can also write string(Name("Me")) but if T were being used as the type of the return value, the returned value would be type string, not type Name.)

In the case of slices, it can be more complicated. These types all seem similar, but aren't the same:

  []string
  []Name
  type StringSlice []string
  type NameSlice []Name
The idea of the article is figuring out how to write a generic type signature that would accept any of these and return the right type.

Finally, you can rename types and use them interchangeably if you don't want the "safety" of making a new type:

  type Name = string
Now you can't write methods on Name, but you can use Name and string interchangably. (This, incidentally, is how "any" works. The package builtin contains "type any = interface{}".)

If the complaint about syntax is not using <T type> to denote type parameters like Java and C++, [] simplifies the parser. You can read the original generics proposal for all the details.


Awesome explanation!


All that instead of type classes eh?


Indeed. Pains me to see what they are doing while not learning from existing and well established language patterns.

I understand that Go wants (or wanted) to stay "simple" but now it seems to become the worst of two worlds: it's neither simple anymore but also doesn't benefit from high level language features like typeclasses because it's too late to add them now.


It's more similar to semantic subtyping as a set interpretation of some types. That avoids having different constructs for similar things.

Personally I find this elegant.


Go has type classes, they're called interfaces


Not the same, because with an interface, the methods defined for the struct must match the interface.

With type classes you can bridge that gap because the implementation of the struct for a specific type class can be defined anywhere.


> [] simplifies the parser

Strong PHP vibes. They also went for weird syntaxes that don't exist or rarely exist in other languages to simplify the parser.

It's 2023. How is parsing anything is a problem?


Go 1.18 had to run programs written for Go 1.17. What if this is your program:

    a, b = w < x, y > (z)
Is that "a, b = w[x, y](z)" (call function w of with type parameters x and y against z), or is it "a = w < x; b = w > (z)" (assign true to a if w is less than x, ...).

With type information, this is possible to disambiguate, but the goal of the parser is to not require type information. Remember, now if you want correct syntax highlighting your editor has to have the type information, but you haven't typed that in yet!

As always, I think they made the right choice here.


> With type information, this is possible to disambiguate, but the goal of the parser is to not require type information.

Why?

It looks like a pretense at purity for the sake of purity.


Do you want to parsing your language to require implementing the typechecker and using shotgun heuristics, all so you can use <> instead of []? Compiler writers die for your sins in code. Let them do things that can drastically simplify things everywhere, instead of suggesting that they're intellectually jerking themselves off. Not that they're immune to that.

EDIT: Apologies, after a bit I realized the above is a bad comment.

Separation of concerns is a common pattern in programming. It allows for things to be testable and changes to be more localized. This is an example of that.


Separating parsing from semantic analysis simplifies the parser considerably.

If one of Go's aims was to make different trade-offs than [C++][1] did, then the choice not to use "<" as a grouping operator was a good one.

[1]: https://youtu.be/WfIr7lKT4Sk?t=204


lots of things in Go are very annoying, but this isn't one of them. being able to parse the language without type information makes parsing way easier and faster, which you definitely care about in at least one case - syntax highlighting in your editor.


> It looks like a pretense at purity for the sake of purity.

if you go (ah!) far enough on the other direction, you end up with 'template typename' soup.


So you don't have to wait as long for obvious errors. Anywhere near as long, like a couple orders of magnitude in many cases - milliseconds vs minutes.

Allowing ambiguity isn't just a style decision, it pushes error detection later, where you have enough info that it's no longer ambiguous.


I’m almost a grey beard in typed functional programming and I’m actually confused by this.


The language and way of writing feels very weird. The concepts are simple genetics but for some reason I don’t get the idea of underlying type. Wish they used a more formal language


it's not doing anything particularly interesting from a formal perspective. If you're familiar with Haskell's newtype deriving, the tilde is solving the same problem—just at the use-sites of the type instead of at the declaration site. The types it tends to be useful for are the types that Go allows as constants (https://go.dev/ref/spec#Constants), which tend to be the type classes that modern Preludes will overload the literals for.


That’s exactly it… every time I’ve tried to find detailed docs about go generics I just find really confusing prose with examples


I'm not sure I'm following the preamble about the nuances of a slice with a zero capacity allocating a new backing array, given the fact that if I follow the link to the docs and then to the source, the implementation is exactly how I would have expected it to be done: append(S([]E{}), s...) - which of course is different and would make the preamble redundant.


My guess is that originally, the post didn't have "// body omitted" below.

The actual implementation requires having the `S` type to refer to, and the point of this post is to explain why the `S` type has to be named. By writing it as they did, the "// body omitted" one _could_ have had the same body, even without an 'S' type to refer to.

I bet the "// body omitted" bit of the post got refactored, and the reason for making the first one different from the stdlib impl got lost.


The generic function is cloning a slice so you need to allocate a new backing array. (think deep-copy)

Here that's just a one-liner that works.

I need to check but I think that if you need to keep the same capacity, you may want to preallocate manually.


I understand that, my point solely rest around defining the clone function from the slices package in the article, explaining why, yet the actual implementation is different. The source code I found via the docs doesn't consider capacity, it merely uses a empty construct and appends.


Ah you're right... It's rather a shallow copy.

Good point. I have no clue either.

Maybe you can open an issue?


I have a question that this post doesn't answer: is the order of type parameters significant? Is there a canonical way, an idiomatic style for the order?

Example:

  func Clone1[S ~[]E, E any](s S) S {
    return append(s[:0:0], s...)
  }
vs

  func Clone2[E any, S ~[]E](s S) S {
    return append(s[:0:0], s...)
  }


The logic for inferring types plays out better for the first. Go limits the depth of searching for type inferences, to keep compilation fast/small/simple. It’s always possible to be more explicit but nice to infer when calling generic code.


Is the order of function parameters significant?


Fair. Type parameters is significant when you call the function with explicit type parameters. Ex: clone1 := Clone1[[]string, string]

But when type parameters values are infered, the order looks much less important for the API designer.

Do we have rules (idioms) somewhere about a recommended order for such parameter types?




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: