There's a great story about a heckler at a Windows NT conference:
MS speaker: and we've got POSIX, and we've got POSIX ACLs, and we've got a Unix Subsystem and we've got Korn Shell, and it works just the same way as under Unix.
Heckler: NO IT DOESN'T
MS speaker: er, ok. It does work. Anyway...
Heckler: NO IT DOESN'T WORK
MS speaker: What doesn't work?
Heckler: (rattles off differences in behavior and brokenness of ksh on NT)
MS speaker: we'll we've had our top guys working on this ...
While I always enjoy a good rant, I don't see why it had to be so vicious and insulting. Yeah, the guy's clearly pretty confused (if he wanted to do these kind of tricks, I'd try to use setjmp, fiddle with the structure, then longjmp to it), but there's no reason to be mean about it.
I think it's because his indignant demand of the Cygwin team to fix this "bug" and make it work for him. Basically it's the attitude that I know my way is the right way, now fix the platform so that my program runs.
He questions whether it "should be fixed" in Cygwin, a perfectly reasonable request, and he fully admits that he isn't sure whether it's a problem in Cygwin or not.
"Should be fixed" is a passive aggressive way to demand it to be fixed.
His reference to Linus doing the same thing in kernel (not) sounds very much like a smack response.
Also he claims he knows what he's doing since he has done the same thing in DOS so when it's clear that he has no clue of what's going on just irrates people.
Instead of trying to walk "indignant demand" back to "passive aggressive way to demand" inch by inch, just concede that all the guy did was ask a question and make a case for why he thought Cygwin's behavior was broken. Nobody's saying Cygwin has to agree with him, but you've simply mischaracterized the start of the thread.
My interpretation is that Korn was trying to be funny (and he succeeded), but as usual the peanut gallery ruined the joke.
While setjmp/longjmp is a good trick (it's how Hanson did his thread library in CII), it's actually an even less good approach by the standards of this mailing list; there are even fewer constraints on how Cygwin needs to implement longjmp than there are on how it manages %%esp.
What are the problems with setjmp/longjmp? (a honest question; I've drafted a proof-of-concept implementation of C coroutines yesterday, just to see how long it will take me to get it working -- http://gist.github.com/491302).
> Do you actually realise that not all computer languages can
> be implemented with a model of one stack?
>
Do you actually realize that there are far better ways to implement stacks
for interpreters than dinking with esp? Ways that actually have a chance of
working? I suggest you investigate those alternatives instead of trying to
do something that cannot work, and then getting all pissy when people try to
dissuade you from wasting your time.
And don't make me break out my Turing Machine on you about that statement of
yours.
Strong disagree. Korn's response is at least funny. The rest of the responses are petulant, somewhat ill-informed, and orthodoxical. The argument over whether C does or doesn't have a runtime is particularly tedious. You get the impression that these people would all be very, very upset at the Detours paper, except that Detours works so well they'd know a priori they'd lose the argumet.
The original commenter --- despite his very fragile code --- is right: for what he wants to do, he needs to create temporary stacks. What he doesn't have is a mechanism to allocate and swap in and out of those stacks (he's hoping that switching stacks is a 2-liner that just relies on knowing the inline asm incantation for changing ESP).
Regardless, it is a uniquely and flagrantly bad way to educate (both the questioner and the world who's reading the thread) to start from the premise that "you're not ready to build the thing you want to build" --- at least when "the thing" we're talking about is systems code. 90% of C programmers are not "ready" to build production C code that will get deployed in hostile environments, and as a result, in every web project we ever do, we have to embark on a tedious inventory of all the 3rd party stuff our clients use to find the horrible C ZIP library they accidentally included. No C coder is blameless.
Unless I missed a whole leg of this thread (the part where they explained temp stacks and stack switching to this guy, who, in challenging the notion that C has a real runtime or the notion that the OS should [heh] be controlling ESP, is demonstrating more on-the-ground systems programming knowhow than several of the commenters), I think this is a remarkably douchey example of a programming help thread.
Because, as you've demonstrated amply downthread, the people making the most noise about how incompetent the original poster was for asking this question about Cygwin are the ones least qualified to have an opinion about this stuff, and are actually making Dave Korn (who was at least funny and who clearly does know what he's talking about) look worse.
But if you know how to use The Google, you can actually find code of mine that does this. I doubt very highly you can say the same thing.
I for one would pay good money to get a response like this.
He points out the errors,the incorrect assumptions and offers pointers of what to do. The general standard of discourse on most mailing lists tend to be quite a bit lower than this.
For one thing, he's using assembly of one type of machine and expects it to work on different type of machine. He confuses Linux as a machine abstraction and demands that Cygwin running on Windows to be the same.
For the low level mess, he sets the stack frame pointer to a uninitialized heap-allocated buffer which contains God-know-what garbage. When main() returns, it pops its return address from the stack frame pointer, which contains garbage and crashes. The printf() might sometime work depending on how the compiler emits code to set up the stack frame.
It just shows how dangerous some ignorant programmers can be.
First, he's writing a nonportable language runtime. For Cywgin. It's not a fair critique to say "he's making the mistake of writing x86-specific code".
Second, you don't have to initialize a stack before you use it. The stack isn't magically cleared before it gets used. In fact, that's rather the point of having the stack implemented with a single pointer, and is also the reason that in C you have to explicitly initialize (say) integers before using them.
Third, there's nothing "dangerous" about what he's doing, in that what he's doing can't ever work. He created a temp stack, write code that attempts to switch into it, and never wrote the code to switch out of it. That's not going to slip into production. But that's also not a fundamental critique of the approach.
What he's trying to do works fine, including under Cygwin, when completely implemented. Despite much peanut gallery whining that it can't possibly work, or that this guy should give up on doing inline/threaded/coroutine stacks and instead implement some CS101 interpreter instead.
He said his code works on Linux but not Cygwin. Linux is not a hardware architecture. The code generated for Linux Arm is very different from Linux i386. I merely pointed out he's confused about different machine abstractions and his claim of "working on Linux" is not a good basis to expect it would work on Cygwin. Does that make sense for you?
Second, re-read what I wrote. I talked about returning from main(). I wasn't talking about sub-function calls within main(). Returning from main() will take the garbage content from the uninitialized buffer as return address and crash. And if you initialize the buffer with the correct stack frame for the function you want to jump to, you can "return" to it. That's how co-routine work, or buffer overflow attack. Does that make sense for you?
Second (b) a stack is not necessarily implemented as a single pointer. On i386, it's a double-indirect pointer EBP:ESP. He only set up ESP and forgot about EBP. Depending on how the generated code sets up the runtime environment, EBP is not necessary (most likely not) the same as the heap.
Third, whatever, it's your pure speculation. He's jumping wildly to whatever location. It's lucky it crashes rather than silently executes whatever code and format his drive.
The fact remains that what he's doing doesn't work, despite what you insist. No one says he should give up doing thread/coroutine stacks. Just his approach is wrong and non-portable. There are better ways to do coroutine.
Every technical detail you try to provide makes it seem less likely you've ever implemented any of these things.
People targeting Cygwin aren't writing for ARM. They're by and large trying to port Unix code to Win32. Meanwhile: what on earth is your point? He wants to build a language runtime on top of the native stack. You propose that he should find an arch-neutral way to do that? That's silly. This post is from 2005, before you try to muddy the waters with LLVM.
Second, like I said upthread, it's clear that it his code doesn't work. Switching stacks isn't a 2-liner. He hasn't fully implemented the concept. Nobody is arguing that his code properly restores the system stack, because he didn't implement that. What does that have to do with how malloc() initializes buffers? You don't have to provide a valid return address at the end of a synthesized stack; you can just JMP directly back to where you want to rejoin the original code --- where you switch the stack back.
The x86 stack isn't "double-indirected" through EBP. EBP is where the compiler arranges to store the base of the stack frame. C code works just fine without it; you simply fix up ESP in the function epilogue. All you lose by doing so (besides negligible overhead) is the ability to trivially dump stack traces.
Why don't you tell us how you, in 2005, would have implemented portable native-stack execution of a high level language? (Note that we're using "threaded" in two very different ways, a fact made all the more confusing by some guy on that thread's insistence that he should spawn a thread every time he needs a new stack).
For what it's worth, I'm not super comfortable tit-for-tatting your comment, except that this whole thread is about a bunch of people pooping on someone who asked for help, and many of the principles being espoused on that thread are simply bogus. I think people felt like they could be puffed-up and self-righteous because they had cover from Korn's original post. And right now, I think that about you too.
He claimed his code work on Linux but didn't work when he ported to Cygwin. He's writing a Scheme interpreter which would aim for portability one assumes. Where did he say he's doing a non-portable exclusively Cygwin implementation? Stop putting your words into his mouth to make your argument.
I specifically wrote the return address of main() was messed up in his code and that was what not working. Please point out what is wrong with that assessment. You are bringing up far fetched ideas and arguments that I didn't mention. Stop putting words in my mouth.
Ok, I made a mistake with the register name. It's ESS:ESP instead of EBP:ESP. I meant he forgot to set up the stack segment register. The stack segment is most likely different from the heap segment. Switching the ESP but not the ESS is asking for trouble.
I will NOT tell you how to implement a portable native-stack. That's not my intention. What I have done is pointing out exactly what were not working in his code. You are the one failing on reading comprehension and making wild assumption and arguing in different directions.
There is no "ESS" register on x86. You've confused the stack segment selector, which is an addressing format quirk (and which virtually never changes), with the register file. Please, stop digging.
The stack segment descriptor virtually never changes in userland protected mode code. You're not "asking for trouble" by not fucking with it. It's also not simply a pointer, unlike EBP with frame pointers. And it's called SS, not "ESS", and it's implied in stack operations.
Have you ever changed a segment register in userland code? Tell me about the code that did it and why.
And while you're at it: have you ever written code that switched stacks? You might have had to if you wrote a userland threading library, or if you wrote an exception handling system, or a coroutine library. This stuff you're critiquing this guy on the Cygwin list about: are you an expert?
I saw your Reddit comments; I believe you're a competent WinAPI C programmer. What I want to know is, do you actually know assembly? Or just have a lot of opinions about it?
Yes, I had, in both user mode or kernel mode codes. Where do you think I got the idea about stack segment register? I don't need to prove how awesome an expert I am, unlike someone. My comments remain as they stand.
You can insist until your face turns blue that you've never seen a stack segment register changed. The fact remains the Intel standard reference states that the stack segment register can be changed and is used in stack operations. The difference between you and me is that I code defensively to handle the published standard contract so as not to worry about some corner cases coming up in some compiler implementation while you ignore the published standard and base decision solely on what you have seen. That's fine, to each their own. I just find the cavalier attitude surprising coming from a supposed security expert.
It's obvious that you don't have anything constructive to say and arguing purely for argument sake. That's the end of this discussion.
I asked if you'd ever changed a segment selector in userland code and why. You said "yes", didn't say why, blew smoke about how "Intel says its possible to change them!" (no shit?), said changing the stack segment selector is "defensive coding" (again: userland code practically disables segment addressing, except for conveniences like the TEB at GS), made reference to a "published standard", and then called me a "supposed" security expert. And, let's be clear, you're the expert that can't remember the eight (8) GPRs on x86.
I'm convinced you can code, but I'm equally convinced that you had no business berating someone else for asking questions about how Cygwin handles its stack, because you don't know either. Don't pile on to people.
Uh, actually, his original program (http://www.cygwin.com/ml/cygwin/2005-08/msg00542.html) did restore the stack pointer before returning from main. He "wanted to cut [the example's size] down for the sake of the mailing list", so he left that bit out.
Still a terrible thing to do. I think it's possible just calling printf() itself could do wacky things if printf is depending on the stack being set up a certain way by the C runtime startup code prior to main.
But the real issue is that what he's doing is completely unnecessary, because you could manage multiple stacks yourself using ordinary C data structures and ordinary C code. Evidently he thinks he's gaining some kind of speed advantage by doing things this way, but the risk is very high versus the gain. This kind of optimization may have worked OK on a DOS machine, but it's neither safe nor obviously faster to do things like this on a modern CPU and OS.
Well, doing terrible things is sometimes necessary to perform magic or to get a better understanding about how things work. From the less flamey parts of the discussion, I gather his ideas weren't all that unsound, just extremely hard to pull off.
Generally speaking if you have to do terrible things then you need to a: know exactly what you are doing before hand and b: have the fortitude to be able to withstand harsh criticism until you can prove your code.
In this case I think the harsh criticism was fully justified. People should be warned away from doing this sort of thing, precisely because it's hard to get right. As the thread makes abundantly clear the original programmer was fairly clueless, otherwise he wouldn't have gone off half-cocked thinking that the fault was with cygwin or somesuch. He was making his own dynamite and was too clueless to realize what he was doing wrong.
I don't agree: we have no reason to doubt his claim that he wrote a task switcher for DOS. That means he's reasonably clueful. It's easy to dismiss someone as a complete loony, because he's wrong/confused about some things and is getting burned to the ground for that. Remove the flaming and there is criticism and the pointing out of mistakes, but nothing to indicate the guy has no clue whatsoever. If that were the case, it would be hard to even being criticising the idea. The fact that people could succinctly point out the problems means the ideas were stated in reasonably clear language.
The code was critiqued as it was posted, not any other version.
Also depending on how the compiler sets up the runtime environment, setting ESP is not sufficient. Intel also has the segment registers. EBP is the stack segment pointer. I'm pretty sure the heap is not in the EBP segment.
C's function call mechanism requires internal maintenance of a call stack, which the programmer ordinarily never has to touch. The broken code author claims that he needs to manipulate said stack to write his Scheme interpreter. He does it wrong in a way that might pass the trivial "hello" test case under Linux, and blames Cygwin when it doesn't even pass that.
If you actually read further along in the thread, it looks like this guy is trying to implement coroutines. However, he cut some essential part of a sample code that he was initially looking at and then assumed that the problem is with cygwin. And then came some silly flame war.
I personally haven't done anything with coroutines but looks like from a low level perspective, ultimately what he is trying to do is not insane… aka do some form of task switching. Arguably, he shouldn't be doing this by himself and he should've resorted to something that is already available. Just a quick look at (http://en.wikipedia.org/wiki/Coroutine#Coroutine_alternative...) should've been sufficient. Yet.., maybe he had some reason to try and do it himself. Looks like at the end he managed to figure out what he wanted to do.
The short answer is that he's doing low-level stack-twiddly stuff using inline assembler in C. It's one of the standard perils of mixing languages: the C compiler assumes (and attempts to ensure) that it's the only thing manipulating the stack pointer, and the inline assembler violates that assumption.
The asm statement puts the pointer (the malloc(5000)+5000's result) into the stack pointer. When the arguments are passed into fprintf(), they're pushed on top of that frame (presumably back into the malloc'd block).
In my book, a worthy attempt to grow your stack on the heap, but you've really got to be ready to hit the asm debugger when things go awry.
Edit: as mentioned in other comments, the mistake was (at least) in not restoring the stack pointer before main returned.
It may be undefined, but gcc accepts it without warning under -Wall and does exactly what you'd expect. Note that (a) casting malloc's return is evil, (b) casting malloc to void* is silly since it's declared returning void* already, and (c) casting malloc to void* so it can be received as a char* is also pretty goofy.
There's nothing wrong with returning void. There is something wrong with casting void.
First, as someone else already pointed out, it can lead to an LP64 bug if you miss the declaration for malloc(), since int and caddr_t are not necessarily the same thing.
But the bigger problem is that casting to void* basically tells the compiler "stop all further type checking". That's not a big deal if the token "malloc" on that line never changes, but, in real codebases, allocators get changed all the time. If you change from foo_alloc() to bar_alloc(), the compiler will rightly freak out when you try to assign a foo* to a bar. But that (void) construction explicitly prevents that check from occurring, thus turning what should be a compile-time error into a runtime error.
C allows implict casting of void* to any other kind of pointer. Hence, you never need to cast the return from a malloc if you're storing it in a pointer.
C++, however, does not allow this. So, if for some reason you're calling malloc rather than new in C++, you need to cast.
You are not the poster who said "casting malloc's return is evil", so perhaps you don't agree with that statement, but I don't see how either of the points you make lead to the conclusion that it is "evil". You say that since void* is implicitly cast to all pointer types, you never need to explicitly cast it, but why would that make an explicit cast evil?
Actually, since st1 is a char* and the sizeof char is 1, that is a valid declaration. It just probably isn't doing what the writer expects. It allocates 5000 bytes and then moves the pointer to point at the very last byte.
edit: Though free()ing it would not work so well until you move the pointer back to the beginning of the 5000 bytes.
MS speaker: and we've got POSIX, and we've got POSIX ACLs, and we've got a Unix Subsystem and we've got Korn Shell, and it works just the same way as under Unix.
Heckler: NO IT DOESN'T
MS speaker: er, ok. It does work. Anyway...
Heckler: NO IT DOESN'T WORK
MS speaker: What doesn't work?
Heckler: (rattles off differences in behavior and brokenness of ksh on NT)
MS speaker: we'll we've had our top guys working on this ...
Heckler: I'M DAVID KORN.
Room erupts in laughter.
Update: source: http://slashdot.org/articles/01/02/06/2030205.shtml. See question 5.