The web page rubs me the wrong way. How can you claim to be the fastest anything without a single benchmark? How can you claim to be a "scripting language" when you're statically-typed and compile to C++? What does "scripting language" even mean then? How can you say things like "it will be a bit slow the first time but subsequent runs will load much faster than any VM." Any VM? Are you really "much faster" than:
$ time lua empty.lua
real 0m0.005s
user 0m0.002s
sys 0m0.002s
$ time ./luajit empty.lua
real 0m0.005s
user 0m0.001s
sys 0m0.002s
Maybe there's cool stuff going on here but I can't get past being annoyed at these over-hyped claims.
Indeed. In my experience, LuaJIT is insanely fast and is widely considered to be one of the fastest dynamic languages around. The term "scripting language" is, in my opinion, meaningless these days, but either way, claiming to be as faster than LuaJIT and aiming to be faster than C (while compiling to C++? [1]) and not demonstrating this to be true just seems pretentious.
[1] I guess it should be faster than hand-written C, which is, of course, possible. ATS supposedly generates C code which is very competitive.
What are the LuaJIT timings on the same machine also is this the proof that it is "The fastest scripting language on Earth."? Please provide your profiling results with the languages tested, the files ran and the timings.
My "good enough" definition: a scripting language is a language that does not require you to write a main() function or define a class / object in order to produce output and features automatic memory management.
I'm not familiar with APL, but after seeing the examples from Wikipedia [1], I think I know what you mean:
# generates a typical set of Pick 6 lottery numbers
x[⍋x←6?40]
# finds all prime numbers from 1 to R
(~R∊R∘.×R)/R←1↓⍳R
# game of life
life←{↑1 ⍵∨.∧3 4=+/,¯1 0 1∘.⊖¯1 0 1∘.⌽⊂⍵}
are you referring to the "The fastest scripting language on Earth" tagline? i saw that as a way to say "look, this is a compiled language that you can use as a scripting language" with a touch of whimsical humour.
Python and Lua are both compiled languages, almost all languages are compiled these days. They compile to bytecode rather than native machine code, then run an interpreter which may well do on-the-fly compilation to machine code (JIT). Bytecode target makes the compiler platform independent. Felix does the same, except the "bytecode" is ISO Standard C++.
Python and Lua are both compiled languages, almost all languages are compiled these days.
For some strange reason, certain misconceptions in CS/programming have half-lives measured in several decades, such as this annoying distinction between interpreters/VMs/compiled languages. We're at least 20 years out from interpreters being meaningfully distinct from VMs and compiled languages.
> Compiled languages generally don't support eval.
Python is compiled to bytecode. It supports eval. So does Smalltalk. Lua would fall into this category as well, and a whole bunch of others. I don't think "compiled" means what you think it means anymore, which is my whole point.
> I don't think "compiled" means what you think it means anymore, which is my whole point.
See my other reply (http://news.ycombinator.com/item?id=5012218). "Compiled language" is an informal term that usually means "to machine code." If you take it to mean "compiles to any kind of IR at all," then basically all languages are compiled and the term is meaningless. But that's not how the term is generally used -- for example, see the Wikipedia article (http://en.wikipedia.org/wiki/Compiled_language).
There is a real difference between languages that can meaningfully be compiled directly to machine code and those that can only JIT-compile type-specialized traces/functions (with guards that fall back to the interpreter if the assumptions do not hold).
> If you take it to mean "compiles to any kind of IR at all," then basically all languages are compiled and the term is meaningless.
That's my point. It's a misnomer. It's the same thing as calling technology with the ability to parse context free grammars "regexes." It's a common usage that pollutes the precise meaning of technical terms. (And at the same time, generates misconceptions based on those technical terms.)
> There is a real difference between languages that can meaningfully be compiled directly to machine code and those that can only JIT-compile type-specialized traces/functions
Well, not so much as you'd think. In theory, the ability to do things like eval cuts off a lot of direct compilation to machine code, but in practice, we know this isn't necessarily true.
The question is: can you execute this byte-code without implementing an entire Python interpreter? The answer is no, because the BINARY_ADD opcode has to handle the case where "x" is an object that implements an overloaded operator __add__(). In this case, the user's __add__() method can be arbitrary Python code, and therefore requires an entire Python interpreter to fully and generally execute.
I expect you will want to talk about the limited circumstances where you can specialize this function ahead-of-time, thereby avoiding the fully-general implementation. This would again be missing the point. Python and C are different beasts, and this difference has far-reaching consequences on their implementations. Trying to draw an equivalence between all "compiled languages" does a disservice to people who are trying to understand the differences between them.
I'm not sure what any of this has to do with the verbiage about "scripting languages," but I'd be much more interested in reading about Felix if it didn't involve so many attempts to redefine colloquially-understood terms in non-standard ways.
(Yes, most languages compile to bytecode, but people generally use the colloquial term "compiled language" to refer to a language whose implementations usually compile to machine code. JIT compilers for dynamic languages are quite a different thing because they generally only compile type-specialized fragments of code; such machine code has guards that fall back to the general-purpose interpreter if the expected preconditions do not hold.)
@nmcfarl thanks for resubmitting with a different url. I find the home page a bit baity but the tutorial page http://felix-lang.org/web/tutorial.fdoc is great.
Depending on how you look, it is C++ speed OCaML, or perhaps more correctly C++ in a fully type-inferred (unfortunately, nowadays almost everything gets called type-inferred. Hence the added qualification "fully"), ML like language.
It does whole program optimization. It uses a mix of lazy and eager evaluation strategies for speed.
I believe it can generate Python modules too, thanks to how well it interacts with C. The details have to be gleaned from the mailing list though.
While I enjoy reading about new and interesting languages (note: Felix has been around for over ten years), I really wish that language designers would put example code for something like FizzBuzz right on the front page to give people a flavor of the syntax.
I can't remember how many times in my life I've done a link expedition through a website or docs just to see a simple programming example.
There is an alternative: a quick start GUI mini-IDE which allows you to type in and edit the code and press a button to run it. Such tools are mandatory on Windows anyhow: Ocaml and Python both have this for example. Most Unix editors like Vim or Emacs could be programmed to do this fairly easily I guess. I even wrote such a tool once using Tcl/Tk.
I agree this is not the same as a REPL with line by line interactive execution/editing with an environment that saves well defined symbols. Ultimately this would be tough to see through because Felix binds functions statically and lookup is setwise (like function scope in C) not linear, so recursive definitions cannot be introduced one at a time.
i've been keeping a vaguely interested eye on it for years, because it does look interesting. it has failed to make it to the front of my "learn this language next" priority queue due to the fact that i can't find anyone using it for anything, not even a hobby project on github. also, the documentation is a bit sparse.
one killer feature that would make me start learning it immediately is good, well-documented qt bindings (i'm sure it is possible, since felix compiles down to c++, but with the sparse documentation i have no idea how i would go about getting it up and running, and insufficient motivation to first learn the language and then figure out how to do it).
See fridgescript (http://code.google.com/p/fridgescript/) - because you are going through C/C++ it is faster in its domain (x87 floating point work) if given good clean code. Its not big or clever - just that C/C++ kills its performance a little with restrictions like struct layout rules and standard library math functions being rubbish... (note the compiler uses almost no optimisation strategies and the language is very specific and almost useless though)
That being said, I like the idea very much, I just object to the claim of fastest and the value the claim implies. Beating C is very easy if you know some very basic things about the restrictions placed on it by standards... :)
If end of line always means end of statement, by all means get rid of semi colons. If some non-trivial algorithms is used to infer end-of-statement boundaries that is more complicated than detecting end-of-line, then...there will be subtle problems.
I went through this transition before in Scala, the error messages got a lot worse after semi-colon inference was added. I wound up telling people to add semi-colons to their code when they were scratching their head at some sort of parse/type error message.
Semicolons are distracting, both reading and writing, and mostly useless. If it's not "noise" in a technical sense, is it not still a bad thing? I don't think this should be a complicated question.
Two choices dictate a terminator: (a) free form 1D language like C vs 2D language like Python or Haskell, and, (b) support for assignment statements x = y and the like as in C vs all statements starting with a keyword as in say ATS. I personally do not like the semi-colons either.
And simpler to parse has another benefit Given an effort X spent writing the parser, users of the language get better error messages
Also, I do not think of those semicolons as noise They make text easier to parse for humans, too That's why you see most people end sentences, paragraphs and HN posts with a 'superfluous' period
EDIT: Some people will find the above quite readable I am somewhat in that camp Problem is, however, that in English and other languages capitals like I and E are not guaranteed to be sentence starters That muddles the waters considerably Many programming languages have worse problems.
Looks very interesting, well done to whoever did this. After an admittedly quick and superficial glance at the tutorial, I'm not fan of chapter 6 though. Why so many ways to write calls? With no idiomatic syntax, it means I have to know all these variations if I hope to understand Felix code (since it is ok for anyone to choose whichever style).
Functions aren't supposed to have side effects and procs are. It makes sense since functions without side effects would allow for some nice compiler optimizations. The part that scares me is that the compiler doesn't enforce no side effects. This means you need to be extra careful when using someone else's code.
Basically this is to support "expression" like syntax such as people are used to in C, for example i++ and x=y are expressions with side effects in C and a lot of C idioms depend on such things. This kind of thing was originally not allowed but I personally found it clumsy to manually split out the imperative parts from the functional parts, so I taught the compiler how to do it for me :)
But those expressions also return things. Why can't they be gen? Why would I use a proc instead? It seems like I must be missing something, but I can't see it.
I'll stick with Lua. I wouldn't give up the dynamic aspects of a language like Lua just for a bit more speed. LuaJIT is more than enough, and if you are doing the hard number crunching that makes speed an issue there's a good chance it's not trivially harder just to write it in C++ to begin with.
For raw number crunching, LuaJIT is plenty fast anyway (when you use the FFI to instantiate native types), so unless you need something real special (SSE perhaps), it may not even buy you that much to use C++ over LuaJIT.
Just a small clarificatation: the 2.x version of LuaJIT only compiles floating point operations to SSE2 (http://lua-users.org/lists/lua-l/2010-03/msg00142.html). I'm not sure, however, whether it does vectorization or just operates on one float at a time.
I don't come from much of a type theory background. I certainly have no formal education in the field. I just know how a Cartesian product works, and that's enough to make me get it.
Aren't product types just tuples, whereas arrays correspond to list types? The '^' operator can only construct arrays whose elements are of a single type, right?
That's correct. However, somewhat problematically, Felix considers the tuple int * int to actually be an array int ^ 2. It's not entirely clear this "automatically applied isomorphism" is a good idea though. However it means arrays "drop out" of the notion of a tuple as a special case.
Internally such arrays have a special representation which allows arrays of 100,000 values, something which could never be represented by a tuple type (and still get reasonable compile times :)
Yes, provided you collect all the required library headers and either sources or binaries: the generated C++ still requires run time libraries and the top level driver if it's a program.
There is actually an option of flx, --bundle=dirname, which puts the generated C++ in a single directory, to make it simpler to ship the generated C++ to another platform.
This does not do a full bundle, i.e. it doesn't package all the run time support code as well. That's on the TODO list, so you could literally copy the target directory to another machine and run "make" or something and only need a C++ compiler to build it.
There's another aspect to your question: if by "use" you also mean "modify" then the current state is that Felix generated C++ is a bit hard to read. The code is "good enough" to add debugging prints but not much more. It would be good to improve this so the code is more readable.
At first glance, it seems fairly complex. For instance, 4 different kinds of variables, 3 different kinds of function. But I guess that's understandable if it compiles to C++ and is looking to be highly efficient.
As a whole program analyser, Felix does a lot of optimisations. The most important one is inlining. Felix pretty much inlines everything :)
When a function is inlined, there are two things you can do with the arguments: assign them to variables representing the parameters (eager evaluation) or just replace the parameters in the code with the arguments (lazy evaluation).
Substitution doesn't just apply to functions: a sequence of straight line code with assignments can be converted into an expression by replacing occurrences of the variables with the initialising expressions.
For a small number of uses, substitution is the usually the most efficient. For many uses, lifting the common expressions to a variable is more efficient. If we're dealing with a function (in C++ the model is a class) for which a closure is formed (in C++ the model is an object of the class) lazy evaluation is very expensive because the argument itself must be wrapped in an object to delay evaluation.
By default, Felix val's and function arguments use indeterminate evaluation semantics, meaning the compiler gets to choose the strategy. This leads to high performance, but it also means we need a way to enforce a particular strategy: for example vars and var parameters always trigger eager evaluation. This leads to some complication in the language.
Felix also does other optimisations, for example it does the usual self-tail call optimisation. This one works best if you do inlining at the right point to convert a non-self tail call (which cannot be represented for functions in C) into a self-tail call (which is replaced by a goto).
Felix also does parallel assignment optimisation.
It ensures type-classes have zero cost (unlike Haskell which, by supporting separate compilation, may have to pass dictionaries around).
There is quite a lot more: eliminating useless variables, functions, unused arguments, etc. There are even user specified optimisations based on semantics, such as
which says reversing a list twice leaves the original list, so just get rid of these two calls.
Actually one important aspect to the optimisation process: by default a function is a C++ class with an apply() method. This allows forming a closure (object). The object is usually allocated on the heap. However Felix "knows" when it can get away with allocating such an object on the machine stack instead (saving a malloc and garbage collection). Furthermore, Felix "knows" when it can get away with a plain old C function, and generates one of those instead if it can. And all of that occurs only if the function wasn't entirely eliminated by inlining all the calls.
So although you should think of Felix functions and procedures as objects of C++ classes allocated on the heap and garbage collected, any significant program implemented with this model without optimisations would just drop dead.
These are very good optimisations indeed. Inlining adds a lot more speed than people think.
I don't understand why people want closures in their languages. You basically want a function that cheats her own scope... I don't get where the big deal is.
One needs closures, that is, functional values bound to some context, for higher order functions (HOFs). For example in C++ much of STL was pretty useless until C++11 added lambdas. Many systems provide for closures in C, by requiring you register callbacks as event handlers, and these callbacks invariably have an associated client data pointer. A C callback function together with a pointer to arbitrary client data object is a hand constructed closure.
The utility of closures derives from being able to split a data context into two pieces and program the two halves separately and independently. For example for many data structures you can write a visitor function which accepts a closure which is called with each visited value. Lets say we have N data structures.
Independently you can write different calculations on those values formed incrementally one value at a time, such as addition, or, multiplication. Lets say we have M operations.
With now you can perform N * M distinct calculations whilst only writing N + M functions. You have achieved this because both halves of the computation are functions: lambda abstraction reduces a quadratic problem to a linear one.
The downside of this is that the abstraction gets in the way of the compiler re-combining the visitor HOF and the calculation function to generate more efficient code.
There's another more serious structural problem though. The client of a HOF is a callback. In C, this is very lame because functions have no state. In more advanced languages they can have state. That's an improvement but it isn't good enough because it's still a callback, and callbacks are unusable for anything complex.
A callback is a slave. The HOF that calls it is a master. Even if the context of the slave is a finite state machine, where there is theory which tells you how to maintain the state, doing so is very hard. What you really want is for the calculation part of the problem to be a master, just like the HOF itself is. You want your calculation to read its data, and maintain state in the first instance on the stack.
Many people do not believe this and answer that they have no problems with callbacks, but these people are very ignorant. Just you try to write a simple program that is called with data from a file instead of reading the file. An operating system is just one big HOF that calls back into your program with stream data, but there's an important difference: both your program and the operating system are masters: your program reads the data, it isn't a function that accepts the data as an argument.
Another common example of master/master programming is client/server paradigm. This is usually implemented with two threads or processes and a communication link.
Felix special ability is high performance control inversion. It allows you to write threads which read data, and translates them into callbacks mechanically.
Some programming languages can do this in a limited context. For example Python has iterators and you can write functions that yield without losing their context, but this only works in the special case of a sequential visitor.
Felix can do this in general. It was actually designed to support monitoring half a million phone calls with threads that could perform complex calculations such as minimal cost routing. At the time no OS could come close to launching that number of pre-emptive threads yet Felix could do the job on a 1990's desktop PC (with a transaction rate around 500K/sec where a transaction was either a thread creation or sending a null message).
The webserver, which is written in Felix, translates a particular format called "fdoc" to create the tutorial and slides, including embedded colourised hyperlinked Felix and C++ code. The translation is to a mix of HTML5 and Javascript. This link shows the source of the first slideshow:
Yttrill, I see that Felix is 12+ years old. I remember reading about Felix on Lambda the Ultimate a few years ago. :)
Over those 12 years, what are some of the biggest changes (in the language or its implementation) you have made? Where the changes mostly evolutionary?
This is a hard question to answer. In general the emphasis changed from getting the compiler to work, and to generate efficient code, to using the technology to implement a rich set of libraries (feedback into compiler), and then writing some simple tools (feedback into libraries and compiler).
Probably the single biggest feature was the introduction of Haskell style type classes as a way to systematically provide "generic" features like comparisons and conversions to string which are de rigueur in dynamically typed scripting languages.
Less obvious but quite important was switching the parser from Ocamlyacc to Dypgen combined with OCScheme, which together put the grammar in user space, allowing almost the entire grammar to be put in the library. A lot of new features were added with very little or no change to the compiler by just adding some EBNF grammar and suitable Scheme action code to the parser.
This can be very effective. For example, with only one extra term in the compiler, I added a dynamic object system that looks a lot like Java with objects and interfaces including "implements" and "extends" stuff. It just uses records of closures (the compiler mod added record extension), but the syntax is neat and almost immediately I used it to factor the webserver into separately compiled plugins. I originally implemented this as a kind of joke, to show off the expressive power in respect of Domain Specific Sub-Languages, but not intended for use. The joke was on me :)
All the documentation and layout needs a lot more work.
Yes, clicking links to get a small page of content to only find another link to click .. gets old really fast. (Old as in it feels like the Java documentation circa 2001)
I find it definitely a nice language proposal, but there are a few issues, I guess. A first one is that many of the distinctions introduced (such as proc,fun,gen) are all optimization issues. I would appreciate if it were possible to do that long after you're done with just making the program work. The language may force the developer way too early in the process to think of things that do not matter at that point. In the best case, it will matter later on, when the program truly works. Another issue I have is that the "simplifications" introduced initially lead to introducing lots and lots of additional symbols such $ and #, just because the simplification was apparently not leading to something simple. So, the language suffers from enforcing a premature optimization mindframe as well as from overly complex administrative simplification.
The proc/gen/fun distinction isn't entirely an optimisation issue, its a semantic one, heavily related to the implementation model. In principle fun and gen use the machine stack for return addresses whilst proc uses a heap allocated list which allows cooperative multi-tasking using channels for communication: such so-called spaghetti stacks are easy to swap. Machine stacks can only be swapped in a conforming way in C/C++ by using pre-emptive threads which is precisely what we're trying to avoid.
I said "entirely" above because semantics and optimisation are heavily intertwined in any system.
The comment about "syntactic sugar" is valid: there are at least four operators with different precedences all meaning "application of function to arguments". However note that in most languages this is true anyhow: x + y is really just add (x,y). Finding a good set of squiggles and marks that's acceptable to many people isn't easy.
However the syntax is defined in the library, in user space, so you can add your own grammar or design your own domain specific sub-language, and add your favorite squiggles that way. Any contributions to making the standard syntax simpler would be welcome: I'm constantly struggling with this issue because I'm well aware syntax matters, especially early in learning a language.
I am sorry about this problem. I have had a look and I understand what is happening but not how to fix it. Python 3.2 does not provide a timeout in the subprocess module. fbuild is using a module by Peter Astrand which derived a new class and adds a timeout. Unfortunately Python 3.3 adds the exact same argument to the API, but defaults it no None, which is clobbering the value used in the derived class somehow, leading to timeout of None being added to a floating point value.
