I want to see an “Effective Go” about unit testing functions that call other functions, without devolving into meaningless mocking and error propagation boilerplate.
I love testing in Go because I avoid meaningless mocking. Test structs that match interface signatures are meaningfully used to validate any state handling and/or error generation, and ensures error paths are properly exercised. In unit tests, we validate logs and metrics when appropriate in addition to return values. However, if you are mocking db, http or other net packages, you are likely doing it wrong. You want to know you handle an error from the db, you don't have to mock a db: you have a struct whose interface has 'GetUser(username) (*User, error)', and your test version returns an error when needed. The fact the real code will use a db is an implementation detail. You should be able to refactor the real implementation to change from a db call to an http api call and still have valid, useful tests. Elsewise, your unit tests are too coupled and hinder refactoring. Anyway, I love testing in Go; it is one of my favorite parts of working with it.
Disclaimer: I’ve never worked with Go, there may be some nuance here I’m missing.
This sounds like dependency injection of test behavior as a substitute for the “real” IO implementations injected in whatever glue code accesses the unit under test. If that’s a correct understanding, it sounds like mocking by another name?
Not that I think there’s anything necessarily wrong with that. And I think it that kind of inversion of control can often produce more robust designs/systems/tests. But I think it’s a good idea to recognize that’s what it is, and that it has similar limitations to other mocking techniques.
There's basically two ways I have seen this work. In one case, you have a "IO thing" stored somewhere in a struct.
type example struct {
backend db.DB // Never actually used the database library, so this MAY not be the right type name
...
}
Then your code uses that to call the methods on var.backend, and you can replace it with a test instance. This feels approximately like dependency injection, maybe? Or maybe just encapsulation?
And the other is that you pass your "IO" as a parameter. A pretty typical example would be fmt.Fprintf.
func Fprintf(out io.Writer, format string, args ...interface{}) (n int, e error)
It is of course limited, but all testing is limited. But, most "mocking" I have seen replaces the original type while the test is ongoing, and that specifically is pretty hard in Go. It is trivial to pass in a parameter, or set a struct slot, to something that fulfils a specific interface. And it is often quite useful. And, again, of course something that you may have to write tests for, to ensure that it does what you intend it to (especially if it is complex and needs to hold some state, which unfortunately sometimes happens).
Mocks are simply auto-generated and consistent test structs. And you still need to test the code which implements GetUser based on DB or HTTP or whatever else.
I am okay with the level of abstraction available in production code. Where it gets ridiculous is the tests. Unit testing the simplest, most obvious Go code is a huge chore. Each error-returning function call costs 15 seconds to type but 15 minutes to work into the tests.
There is unnecessary friction, which doesn’t really seem to fit with the Go ethos. I would have assumed from afar that testing gets a lot of consideration in a language like Go.
I am not being sarcastic when I say I want an "Effective Go" for this. If you have examples of high quality tests in an MVC-style service codebase, I would love to see them!
Others have recommended Advanced Testing With Go [1]. I haven't personally watched it, but Mitchell Hashimoto writes clean code.
But if you'd like I'd be happy to take a look at some of the problems you're encountering. I'm @alecthomas on the Gophers Slack or Twitter, feel free to DM me. No guarantees, but it sounds very unusual for an additional test to consume 15 minutes of setup.
"MVC" is at least a half dozen different patterns at this point, and at least one - the one the closeparen is talking about, rendering responses to incoming requests out parts of a data store - is pretty common in Go.
What's different is that the "model" is often e.g. a bare sql.DB and not an object repository, and the "view" is a template called directly by the controller, but "MVC" - even this kind, which isn't the original kind - doesn't have to be an auto-wired DI interface-laden mess. PHP and Java just made it that way.
Maybe it would pay to be more specific, since "MVC" can mean a lot of things. We have requests come into handlers, which map between wire and internal representations. They invoke controllers, which implement the business logic and call gateways and repositories. Repositories wrap storage clients and gateways wrap HTTP or gRPC clients.
Do you not do this? What do you do instead? I guess I can see how testing would be less painful with fewer layers, but at the cost of the production code becoming more entangled.
When I read MVC I guess I take it pretty literally, in that you would have packages named model/s, view/s, and controller/s. That's what I'm speaking to. Of course, abstractly, many programs tend to be structured in some kind of layering scheme that's MVC-ish :)
I don't understand why you would reply with this comment. If the question is "how do I do X?" and X isn't something that you should be doing, why is it not constructive to point that out?
Re error propagation: look at standard library’s JSON encoding source code. They use panic internally to bubble up errors from recursive calls and have a recover at the API boundary which turns it into an error value. I thought it was a neat trick, until error handling matures further.
What do you mean? If you compose functions (and objects) together then you have to mock things out one way or another, inject them in, and assert that your code interacts correctly with its dependencies.
This doesn't seem like a go specific issue but an engineering one.
In Java or Python you do not normally need write a test case for “if this dependency fails, it will be propagated to the caller.” In Go you have to do this or your line/branch coverage will be abysmal.
Java and Python mocking is also a lot more lightweight; you don’t have to generate mocks ahead of using them, regenerate them when the interface changes or work generation into your build process, etc. Richer reflection APIs make it a pretty casual handful of characters to mock something out.
It would be interesting to know what errors we might get, but unit testing an error return branch doesn't tell us that. It tells us that any non-nil error will be returned.
I make a sequence of calls during the service of a request. If any one of them fails I want to stop and return the error to the caller.
Having to check this in the particular case of every call at every later underlying every handler, does not make my software more reliable in than when it is simply guaranteed in general.
The language's exception facility does what it says. We don't need a unit test to prove that in every particular case, any more than we need a unit test to establish that the language correctly carries out our assignments or function calls.
You picked two languages (Java and Python) with extremely weak guarantees about disposal of resources when exceptions are thrown. Any time I acquire a non-trivial resource I must make sure it's disposed of properly, via a `close` etc. method. (And these are most cases of interest; acquiring trivial resources e.g. memory shouldn't need error handling in Go either.)
This isn't theoretical. I review a lot of Python code and I would say in over 20% of cases I see a try block with a `finally` longer than two lines, it mistakenly uses a variable that might not be set when an exception is thrown earlier than the writer expected.
When a controller calls a couple of gateways and a database access layer, it is almost never holding resources. I would agree that testing error handling is more interesting when there are resources to clean up or fallback logic to implement. That's just very rarely the case. Mostly I just need to bail out of the request.
You don't need to do this in Go either. Embed the interface type in your mock type. You only need to implement functions that will be called in the function under test.
- Don't unit test so much (or expand your definition of "unit", or whatever semantic difference you prefer). Test functionality. (Unit tests can still be appropriate for large classes of complicated pure functions, e.g. parsers, but these don't require mocks.)
- Don't mock so much; rather, stub (or mock if absolutely necessary) at lower levels. Use the httptest server; use the sqlmock or sqlite drivers; use a net.Conn with canned data; etc.
I would appreciate any resources you could point me towards to help make this argument against the Staff+ engineering leaders at my company who are pushing standards that say exactly the opposite.
This sounds a lot more like company politics than a technical issue, but I would probably start with Mitchell Hashimoto's talk "Advanced Testing With Go" - along with the just, like, reading the tests / testing tools in stdlib. They didn't include httptest so you could spend time mocking away http.Client usage behind an interface!
(I should add that this is explicitly contra to e.g sethammons's suggestion above, which seems to be relatively common in the part of the Go community that come from PHP. I inherited a couple large projects that did this. Today they use sqlite instead, and both the program and test code is ~50% the size it used to be.)
For us, stub injection points come naturally out of 12-factor-style application design; the program can already configure the address of the 2-3 other things it needs to talk to or files it needs to output, etc, just out of our need for manual testing or staging vs. production environments. If you have technical leadership encouraging Spring-but-in-Go, you'll probably hit a wall here too though.
It's also possible you're simply writing too many functions that can return errors. Over-complex code makes over-complex tests; always think about whether you're handling an error or a programming mistake - if the latter, panic instead of returning.
Thanks for the suggestion. I watched the talk and found some new information, as well as confirmation of some things I had been starting to adopt. I don't find the stdlib very informative about my problem, since most stdlib packages are "leaf nodes" - not layers that call out to lower layers. I'll check out more of Hashicorp's tests as I suspect their code might be more similar to the kind of code I work on. From a quick glance, in all of Consul I see only a handful of Mockery mocks, suggesting they are doing something very differently.
You're giving a baby-level introduction to mocking in a thread about how this approach leads to low-quality, meaningless tests in some cases, and right beneath concrete suggestions about how to make tests better by deviating from this pattern.
I'm sorry if you've had bad experiences with this approach in the past, but it emphatically does not lead to low quality and/or meaningless tests. It's the essential foundation of well-abstracted and maintainable software.
Here's a test I wrote recently, in the style expected at my company. Tell me what exactly you think this is contributing to maintainability, or what you think could be done better. I spent an hour on this and found it pure drudgery. I half suspect I could have written a code generator for it in that hour instead. I had no idea whether the code really worked until I ran it against the real upstream.
The unit under test is a gateway to a configuration service.
It's hard to give solid advice based on a view of this single layer, but at a glance unless this gateway client is itself something to be extended by other projects, this is probably not something I would write test cases for per se. If "apipb" stands for protobuf, I definitely wouldn't inject a mock here but would make a real pb server listening with expectations and canned responses. (Our protobuf services have something like this available in the same package as the client, i.e. anyone using the client also has the tools to write the application-specific tests they need.)
The resulting code probably wouldn't be shorter, but it would exercise a lot more of the real code paths. The availability of a test server with expectation methods could also (IMO) improve readability. Instead of trying to model multiple variants of behavior via a single test case table, using a suite setup + methods (e.g. `s.ExpectStartTransaction(...); s.ExpectUpsert(...)`) would make clearer test bodies. Check sqlmock for something I think is a good example of a fluent expectation API in Go.
When you want to exercise this code, you want to construct an instance with mock/deterministic dependencies, so that you have predictable results when you apply input and receive output. That's the model: give input, assert output.
But your linked code is kind of different! Each subtest varies not the input but the behavior of the mocked dependencies. I understand the point: you want to run through all the codepaths in the gateway method. But is that worth testing? Do the tests meaningfully reduce risk? I dunno. It's not obvious to me that they do.
The use of gomock is also a big smell. Generating mocks kind of defeats the purpose of using them. I would definitely write a bespoke client:
