This is not the same as Go's defer, as this honors scope; this is similar to D's scope guards (and hence the name of the C++ class...). In Go, when you use defer, the code in question is deferred until the end of the function, no matter in which scope you used the feature: this means that if you have a loop in which you use defer, everything is deferred until well after the entire loop finishes, not during each execution of the loop. This also means that Go has extremely... interesting?... behavior with relation to argument capture for the expressions used in a defer statement to make this even remotely sane. There are reasons the people who work on Go think this makes sense, but whether or not you agree with those semantics, the semantics implemented in this blog post are very different.
This is a neat C++ exercise, but when you're working daily with a language and intend to write maintainable code that's going to be around for a while, it's best to stay within common, boring conventions of that language. In C++'s case, just stick to explicit RAII-based guard wrappers and your life will be a lot easier. Macro-based emulations of other language features can lead to lots of unexpected behavior.
Sorry for pooh-poohing someone sharing some ideas, but being someone who has to maintain long lived code in quite a few languages, code like this immediately sets off my alarm bells.
Don't get me started on the dude who use #define magic to make C look like Pascal.
If you tell people that the only way they can do something "on scope exit" is to write a whole class with a destructor and manually capturing variables, the result is they won't bother. They'll just put the code at the end of the function, and say "I'm too lazy to make this exception-safe."
This defer() macro is almost exactly what I've been using for the last 5 years or so:
I've seen a lot of C++, and there's no consensus on what "boring conventions" are.
Some places ban templates, some ban RAII. Some ban operator overloading. Some ban lambdas or C++11 features. Some ban macros.
There's a lot of variety, and just because you use something like this in your library or project doesn't mean you can't come out and play. C++ is the most "be yourself" language I've encountered. There is no idiomatic code.
Now, I get where you are coming from, and I'd love C++ a lot more if there was less feature abuse, but I've accepted that the ship has sailed.
> Don't get me started on the dude who use #define magic to make C look like Pascal.
I did it for memory allocation, because C designers weren't able to create something as simple as new, rather requiring us to keep using sizeof on malloc calls.
That was only for an year, until I got hold of Turbo C++ and only dropped pure C when my teachers required me to deliver assignments in C.
The best solution to this problem seems to be Python's "with".
with open(oldfile, 'r', encoding='utf-8') as infile:
with open(newfile, 'w') as outfile:
....
The implied close will be executed on exiting the "with". "with" calls .__enter__ on the "open" object at entrance, and .__exit__ at exit. This works even if exit is via return or an exception.
If the implied close generates an exception, everything still works reasonably. __exit__ has a "traceback" parameter, which indicates an exception is in progress. An exit operation should do its closeout, then re-raise the exception sent as the traceback, if any. This works even when a destructor raises an exception. (A typical case is closing a network connection. If the other end dies, close will generate an error, which should not be lost.) The default use of "with" in Python thus comes with correct single and nested error handling.
That's not true of RAII or Go's "defer". In C++'s destructors, you can't return an error code, and an exception in a destructor is usually a big problem.
It's hard, but not impossible, for a deferred call in Go to report an error.
The deferred function can muck with the parameters of the function which called it. This only works
if the deferred function was written specifically for the context in which it was called; just deferring a "close" won't do this. So you don't get proper error handling by default.
(Languages seem to need exceptions. Both Go and Rust started with an always-aborts panic mechanism. That was too drastic in practice, and in both languages, panicking was made recoverable and provided with unwinding machinery. So both languages now have the heavy machinery for exceptions, without the language support for them. This results in kludges to simulate exceptions.)
> Languages seem to need exceptions. Both Go and Rust started with an always-aborts panic mechanism. That was too drastic in practice, and in both languages, panicking was made recoverable and provided with unwinding machinery. So both languages now have the heavy machinery for exceptions, without the language support for them. This results in kludges to simulate exceptions.
This is completely incorrect with respect to Rust, both historically and practically. Rust started with unwinding, and then grew opt-in abort machinery for those use cases where it matters. And, for what seems like the millionth time, unwinding does not equal exceptions; this is like saying that C has exceptions because of setjmp/longjmp. Unwinding exists in Rust solely to give one the ability to fail gracefully, running destructors that might have valuable side effects and allowing Rust-written subsystems to avoid taking down the entire system. For an example of the latter, Skylight, who ships a Ruby gem written in Rust, "catches" any unwinding so that an errant failure in their gem does not bring down a client's entire Ruby process. I have been writing Rust for years and, without exaggeration or (heh) exception, have never once seen the ability to "catch" unwinding used as a basis for constructing ad-hoc recoverable exception-handling, especially since abort-on-panic is an entirely valid mode for Rust applications (this is how Dropbox's backend Rust is compiled, for example) and completely obviates any sane attempt to use panics for recoverable error handling.
And the same goes for Go. The only legitimate use of panic is still always-abort; recover merely helps one define what part of the program has aborted, and is a noop except within a deferred call — you'd be hard pressed to use recover as casually as exceptions in Java, Python or Ruby. Typically, this is the case when a master/workers scenario is implemented[0], where the master has to survive a worker's death, because they're different "processes", but the latter definitely dies from panicking (possibly even caused by a call to a third party lib such as described by kibwen)
The situation is actually a bit more nuanced in Go; an idiomatic Go library may use panic/recover internally as a means of error handling, but is supposed to convert any panics to error codes for external consumption (IOW, panics should not be a part of any public API). You can see this pattern endorsed in this blog post: https://blog.golang.org/defer-panic-and-recover .
> The best solution to this problem seems to be Python's "with".
Ehhh... `with` is a crude substitute for destructors. C++ is a whole language designed around RAII. Python really is not.
You do make a good point that the handling of exceptions in destructors in C++ is problematic. But you can actually throw from destructors, as long as the destructor wasn't called during unwind for another throw. I'm of the opinion that the inability to throw again during unwind is a glaring bug in the standard, since it presumes people can know if their code is going to throw, which is like presuming people know where their bugs are. But the reasonable way to work around it is to squelch exceptions from destructors when another exception is already in-flight. This is usually fine, as the "secondary exceptions" thrown during an unwind are usually side-effects of the "primary exception" anyway.
Surely ruby's blocks are much better here. For one they're a much more general construct than with, and second exception handling is done with normal language constructs like rescue and ensure. Implementing one is simple and obvious and is nicely contained in a single method.
For example
File.open(args) do |file|
# use file
end
And the implementation is simply
def self.open(*args)
file = File.new(*args)
begin
yield file
ensure
file.close
end
end
I know Python far better than Ruby, but it looks like the semantics are equivalent, it's just that Ruby's standard `open` implementation includes the cleanup rather than leaving it up to the caller as in Python.
Ruby does this at the cost of having non-local returns from those blocks, which complicates semantics quite a bit (since now you need to distinguish closures that capture "return" and other control transfer from those that don't, and you need to handle the scenario where a control transfer happens when the target is already gone).
On the other hand, it means any helper/builder has to handle this pattern on its own rather than be able to return a file object which can be "context managed" by a caller, which Python allows.
I agree that Python's `with` is nicer than Go's `defer` (and I use both languages almost daily). I'm not sure I agree that languages seem to need exceptions--stack traces in the presence of errors are certainly handy, and there's certainly a case for panicking, and it's arguably more ergonomic to have syntax or type-system support for eliminating error-handling boilerplate, but I still prefer the simplicity of returned errors. That said, I'm really interested to hear different view points.
That's really complicated. The great thing about the "with" clause is that it Just Works. The easiest way to handle opens and closes is thus exception-safe.
What's wrong with indenting? You're entering a scope; indenting is appropriate.
I wrote "if that's an issue". For most cases it's not an issue.
The primary use case ExitStack is not to reduce indentation levels; it's to support "a variable number of context managers and other cleanup operations in a single with statement".
You wrote "The easiest way to handle opens and closes is thus exception-safe".
Consider if you need to merge up to 50 files containing sorted lines, where the result must also be sorted, and where you should not load all of the data into memory first. (Sorted according to UTF-8.) How would you do it? Not so easy, is it?
With ExitStack you could write it as something like the following sketch:
filenames = [ list, of, file, names]
with ExitStack() as stack:
files = [stack.enter_context(open(fname, "rb")) for fname in filenames]
with open("merged_lines.txt", "wb") as output:
output.writelines(heapq.merge(*files))
As you say, the great thing about this with clause is that it Just Works.
A side-effect of making ExitStack work is that it might help if the indentation depth is a problem, which mappu apparently has experienced.
The variable isn't necessary, and contextlib.contextmanager is the standard wrapper for arbitrary functions. Here's the example code from the documentation:
from contextlib import contextmanager
@contextmanager
def tag(name):
print "<%s>" % name
yield
print "</%s>" % name
>>> with tag("h1"):
... print("foo")
...
<h1>
foo
</h1>
But how does that close over other locals? I'll admit I haven't looked too closely, but go defer makes it pretty easy to drop in a lambda. Getting stuff into your decorated functions by reference is going to be awkward, unless I'm missing something.
I'll need to see an example to understand the issue. The idioms between the two languages are different enough that idiomatic Go might be written a different way for idiomatic Python.
Remember, "with" isn't always required - you can also write the code using a try/finally.
I see a lot of people in this thread dumping on this idea, but in my opinion as someone who has built several large systems in C++, this pattern is not just good, it's essential. KJ (the C++ framework library built with Cap'n Proto that I use in all my projects now) has had KJ_DEFER() since just about the beginning, working basically exactly as described here, and I use it constantly.
If you write C++ with exceptions, you need this. If you simply place tear-down code at the end of the function instead, you are creating exception-unsafe code. Of course, ideally state is encapsulated in classes with destructors and tear-down happens there, but writing a whole class can often be overkill for a one-off use case. If you don't have a defer macro, you're going to get lazy and introduce bugs.
If you don't use exceptions, then you still need this, because your code almost certainly has lots of "if (error) return nullptr;" or whatever. Without `defer`, you need to stick the teardown code before every one of those returns, or you need to resort to "goto fail" error handling.
> If you simply place tear-down code at the end of the function
No one is suggesting this. They are saying that you use an RAII class such that it doesn't need tear down. In the example in the article, if std::ofstream calls close() in its destructor (which it does) then you don't need to DEFER a call to it. If you lock a mutex, use a RAII locker class so that you don't need to DEFER a call to unlock(). As someone once said of C++: "my favorite feature is the close brace", because it calls destructors for all objects on the stack, even during exception unwinding.
Of course, the article is just using RAII. But I disagree with the style; rather than re-writing a DEFER(myfile.close()) in every single function that uses a file object, you should make the file class have an appropriate destructor, or at least write a wrapper that does. That way you re-use that code rather than having to re-write it over and over.
> If you write C++ with exceptions, you need this.
You absolutely do not. You need to use the core language feature that it is using: deterministically-called destructors.
> They are saying that you use an RAII class such that it doesn't need tear down.
Yes, as I said in my original comment: "Of course, ideally state is encapsulated in classes with destructors and tear-down happens there, but writing a whole class can often be overkill for a one-off use case. If you don't have a defer macro, you're going to get lazy and introduce bugs."
In C++ the destructor handles this as part of the lifespan of the object. Making users of the object encapsulating the resource handle resource cleanup is an antipattern, which is why the standard library does this for you.
> You could totally just write the FdWrapper class
Yes, and that RAII class would be reusable all of the places you need to close a file descriptor. It's even a handy place to hang more methods that work on the descriptor... for instance if you wanted to abstract those ioctls.
That's why I personally never use ScopeGuards in C++. You are correct that writing a single-use RAII class requires a few lines of extra code. However I have found that after I write such a class I often find myself using it over and over.
There's plenty of one-time cases where you don't want to declare an entire class but still enjoy scope-based functions. However, the approach in the post is not very C++-esque, I like this one much more:
https://github.com/mawww/kakoune/blob/master/src/utils.hh#L5...
std::unique_ptr has a widely unknown second template argument called deleter. It is designed exactly for this purpose. You just do roughly like this(written from a phone):
The only time I can think of where this is useful is when you are trying to be super defensive with possible exceptions being thrown.
Otherwise it seems a lot like you can just make the call at the end of scope yourself for the same line count and _much_ more readable code.
I guess if you have multiple return paths it's more tenable to use scoped dtors for this but it all really seems questionable to me -- I can't think of a single use-case where it'd be preferable to use something like this rather than a dedicated class.
> The only time I can think of where this is useful is when you are trying to be super defensive with possible exceptions being thrown.
If your application uses exceptions at all, then the time to be super-defensive about exceptions is all the time. If you don't form the habit, then you'll forget in the cases where it matters.
Plus, most programmers are pretty bad at identifying where exceptions might be thrown, because it tends to reduce to the question: "Where will my code have bugs?" Obviously, if you knew, you'd have fixed them...
> _much_ more readable code.
I'd argue that it's more readable to put state build-up and tear-down next to each other, so that future readers can see and modify them together. If you put them far apart, it's easier for them to skew.
It allows you to keep the cleanup next to the allocation/initialization, which is a nice organization. It's easy to ensure you haven't forgotten to clean something up in any exit path.
(Obviously, classic RAII has pretty much the same benefit. But it requires a whole class per type, whereas this approach has less boilerplate.)
It's rare that there is no chance of exceptions being thrown anywhere in a function, especially in the futurer as the code changes.. But even if you're really sure, it's so idiomatic to do cleanup in a destructor in C++ that it looks wrong to leave some cleanup code at the end of a function. Plus, it means you can think about cleanup at the same time you're thinking about creation, then forget about it later; one less thing to think about during the lifetime of the function.
I think life is simpler if you just adopt a blanket rule: all cleanup is done in destructors/scoped guards. Occasionally code might be slightly longer than it needs to be, but your life is simpler overall. Plus you can forget the old C rule of only one return from a function that you mentioned. (If you use exceptions anywhere then you already stopped using that rule anyway.)
Sometimes you need to wrap C APIs. Sure you could put their guts in a class or struct, but you can also use a unique_ptr for some of the same lifetime stuff.
Also, defer in Go always uses the function exit, not the scope. Not sure you can do that in C++. Everything seems scoped.
You usually shouldn't use unique_ptr for C api stuff unless you really only need to free it.
Typically though there are custom delete/free API calls you need to make, and to use them with unique_ptr you have to pass that function pointer in as the second argument to the unique_ptr constructor.
At the very least you should have a C++ function that gives you the unique_ptr back that does the right thing; but at that point you almost might as well just make a class and give it member functions to map the C calls.
If you wanted to do things on function exit, you could have an “ENABLE_DEFER” macro that declares a scope for guards; the “DEFER” macro would register the deferred lambda with that scope, not define a new object, and the destructor of the scope would run all the registered guards.
Of course, in my C++ code I would just write a class/struct to wrap any RAII-like logic, not use a “defer”. A marginal amount of extra work that pays off in consistency and readability.
Otherwise commas will be problematic—it’ll be treated as too many arguments to the macro if they’re not wrapped in parentheses. The reason being that the preprocessor understands parentheses as grouping delimiters, but not other bracket characters. Also note that I changed the double underscores to a single underscore, since the former is reserved for use by the implementation & standard library.
It’s this type of constant minor language-lawyering and clerical work that makes me both very good with C++ and very averse to using it anymore. (Though to be fair this particular case is also a problem with C, which I like nevertheless.)
It seems that this might be for only limited cases. I'm not that familiar with Go, but I suspect that due to coroutines, the deferral may not execute for quite some time, while in C++ this could be limited to the single-threaded flow of the calling method (unless you manually move the scoped object out of the call). To match Go's behavior, you would need to attach the object lifetime to whatever future/asynch mechanism you were using. One very nice implementation (if heavyweight) is Seastar's then chaining: http://docs.seastar-project.org/master/md_doc_tutorial.html
is how it's commonly used. This makes sense because the cleanup lives next to the initialization of the actual thing it's cleaning up. In a way it becomes part of the initialization, itself.
Same with locks and release. Now you don't have to scan across the entire function body to make sure your inits and cleanups match up.
There is a substantial issue with this pattern for things like files -- the operating system imposes a max limit on the number of file descriptors you can have open.
Defer gives no guarantees about when that thing is going to run -- just that it is after the function returns. Because of this, you can get erratic behavior if you open and defer closing a lot; essentially openFile just stops working at some point.
This can happen with any limited resource and isn't theoretical; I had a friend who ran into this in long running process with a lot of IO.
Defer statements run exactly at the moment of a function return (in lifo order). They are equivalent to wrapping the rest of the function body in a try / finally. They have to, since you can change the return value in a defer (by using named return values).
A "defer" statement invokes a function whose execution is deferred to the moment the surrounding function returns, either because the surrounding function executed a return statement, reached the end of its function body, or because the corresponding goroutine is panicking.
There might have been something else going on with your friend's code. Maybe the function he was using didn't return before running out of file descriptors?
I somehow completely misunderstood that from the Go documentation when first reading it; thank you for clarifying that.
There must have been some other leaking going on. He was using a largish library when he saw the leak and eventual crash so maybe it was due to improper pooling or C api calls? I'll have to ask him more about it.
> con: "finally" puts the cleanup code at the end of the function. With "defer" you have it scattered at the beginning.
I would consider this a 'pro' as each action and defer are paired, executes in reverse order and a conditional action/defer doesn't need a second matching check.
The syntax of defer is a function call, but it isn't called like a normal function: the arguments are evaluated at the time the defer is encountered, and then the function is called later. That's one reason why defer is very often called with an anonymous function, so that variables can be captured.
It's a nit I guess but I feel like RAII is a pattern, not a feature of the language. Haven't done C++ in years but as I recall it doesn't have anything like python's with or finally blocks, which would both serve the purpose illustrated here. Using destructors to perform things when exiting a block is fine, I guess, but it does introduce hidden behaviors that will need to be well documented.
owns a mutex for duration of the scope. The constructor claims it, and the destructor releases it.
But yeah, it has downsides. If you would not name the variable, and do `std::lock_guard<std::mutex>(some_mutex);`, then its scope is limited to a single line, and the lock is immediately released.
And to toot its horn, Rust combines that with its ownership semantics to create a very neat pattern: many locks have to do with resources yet almost all languages split the resource and the lock and you've got to know that you need to acquire one lock before you use a resource without any intrinsic relation between the two.
In Rust however sync::Mutex::new takes a value, which the lock owns, and acquiring the lock yields a MutexGuard which acts as a reference to the wrapped value. You must acquire the lock to get access to the value (since it's owned by the lock itself), and once the lock is released you have no way to access the locked value (unless you created a copy).
> If you would not name the variable, and do `std::lock_guard<std::mutex>(some_mutex);`, then its scope is limited to a single line, and the lock is immediately released.
OTOH you can pretty trivially create a wrapper/extension which takes a lambda, acquires the lock, calls the lambda then drops the lock. It could even return the lambda's result while at it.
It's not a discovery per se, since its creation was deliberate, but it's a practice exercised throughout the STL (the C++ standard template library), and is a term coined by C++ creator Bjarne Stroustrup himself. It could be considered a "standard pattern" in lieu of a better term.
A bit of both, IMO. RAII is a design pattern, not a language feature. However, it’s a pattern made possible by the semantics of destructors and object lifetimes in C++. The creator of C++ (Stroustrup) developed RAII within the first few years of the language’s existence.
(It's "voilà", not "wollah". It's a borrowed French word meaning "there it is!" or "behold!". In English, the accent is optional: "voila" means the same thing.)
