“I wish I knew what to tell you that would lead you to write good Forth. I can demonstrate. I have demonstrated in the past, ad nauseam, applications where I can reduce the amount of code by 90% percent and in some cases 99%. It can be done, but in a case by case basis. The general principle still eludes me.”
I'm not surprised that the general principle that helps Moore reduce his code size eludes him. Authoring something blinds you to your own style—both to your mistakes, and to your brilliances. It's why prose writers can't be their own editors: you need someone who wasn't in your head during the writing process to actually be able to see what's ended up on the page, rather than just seeing reminders that immerse you back into your own implicit context.
I would expect that, if you took five of these rewrite projects of Moore's, and compare-and-contrasted their source with the source of the originals, principles would leap out at you.
(Why don't we programmers, as a culture, read code this way? It really seems like the only way to deduce novel facts about programming for yourself. Is it just that a rewrite where both the old and new codebases are shared-source is rare?)
To clarify a little Chuck's projects were not rewrites in the usual sense, but they can still be read that way, as derefr suggests. His famous BASIC compiler for example was an implementation of BASIC, something that's been done a thousand times before but Chuck's was not a rewrite of any particular program.
Much of Chuck's code reduction comes from simply not implementing parts of the problem that most people would not think of eliminating. Other parts he will eliminate that most people would think inconceivable to eliminate.
Other "secrets" of Chuck's ways are that he uses the CLI as a front end, or, spends a great deal of time working on an algorithm and finds a simpler way that is good enough in the application he's working on today. So instead of spending time optimizing for speed or modularity he's willing to specialize his subroutines per application to optimize for concision or simplicity. Somewhat like Woz's optimization to save chips.
I bet most people who investigate Chuck's methods will feel disappointed that the methods don't seem as clever as the end product. What I think most people can take away from Chuck is skepticism about the utility of most programming technologies, which do automate tedium but tend to substitute their own architecture and complexity, for a net loss.
I'm a big Chuck fan, but I get the feeling that he isn't on a short deadline to build some standard business app like 90% of coders where you almost have a template...sure it has lots of code you don't need and to paraphrase Rich Hickey, it isn't simple, but it's really easy (at least in the short run). He's doing things where he is allotted the time to really understand the problem and deliver a minimal solution. Listening to his interviews make me sad as they call up images for how programming could be if we'd gone about it differently. Small teams writing small and efficient software where you can grok the entire thing at once.
Chuck Moore was amazing before I even knew how to code (and was a big inspiration for me), so I'm not going to even pretend that I know how to do what he does. However my view of this is that we, as programmers, can't just look at our jobs from a technical perspective. Nobody is given the time to make great solutions. The best programmers know how to work so that they can get the time they need. This is not trivial.
You can't just wander up to management and say, "I need X to make an optimal solution". They will negotiate you down. That's their job -- to negotiate into a position of advantage. When you need protection from the chaos that surrounds the rest of the organisation, you can count your lucky stars when you have an expert negotiator on your team. But when you go up to that expert negotiator and say, "I want X", the first thing that's going to go through their minds is, "I wonder if I can get that cheaper".
So programming has to be a balancing act of constantly delivering functionality while at the same time ruthlessly refining your vision of how it's going to work. If you are delivering quickly, regularly and constantly nobody will touch you for fear of screwing it all up. But knowing how to do that while still marching inevitably closer to simpler and simpler solutions is hard. A normal person needs to practice (and get it wrong) a long time before they get good at it.
Chuck describes a lot of his techniques in his books, but it's fluency that you need -- and it's fluency while things are hitting the proverbial fan that makes all the difference. You need great judgement to look at what you need and to prioritise it appropriately. If I deliver X quickly, I'll show progress. Do I need to show progress now? How fast? Can I refactor Y while I'm here? Is all of this necessary? And so on and so on.
It's tempting to think, "If only I was given lots of time.", "If only I didn't have incompetent coworkers", "If only people would listen to my advice", "If only management could look past tomorrow"... But that's not useful. Anybody can be a great programmer if we remove every problem before they start. The measure of your worth is how many of these complicating factors you can deal with while still producing great code. That's what's rare.
“I wish I knew what to tell you that would lead you to write good Forth. I can demonstrate. I have demonstrated in the past, ad nauseam, applications where I can reduce the amount of code by 90% percent and in some cases 99%. It can be done, but in a case by case basis. The general principle still eludes me.”