Thanks! Out of curiosity — if I built an application with truffleruby, could I distribute it as a JAR (provided that it was run under JDK 9)? Are there ways to call out to Java like JRuby? Having some difficulty finding documentation around this.
Be warned, the jruby compatible interop isn’t complete, and I don’t have the jruby test suite running for it because that requires a much more complete implementation to run at all.
It doesn't do that - we compile the Ruby interpreter to native, not the Ruby application.
It's a little bit more complex than that in that we can run a Ruby application ahead of time up to a certain point where we compile it, and then when you run the application that state when it was compiled is restored. So we load the core library during compilation for example.
I'm not on the Graal team but I can take a crack at an answer - it's because as languages get more dynamic, static ahead of time compilation becomes less and less effective and can even be counterproductive. To compile a Ruby program it really needs to be just-in-time compiled, or at least a partial ahead of time compile using injected profiling runs.
The problem is that the semantics of Ruby are such that you can't compile it efficiently out of the box. TruffleRuby can but only by making tons of assumptions at runtime that might not be valid. So it compiles code on the assumption your program is normal and not weird, and if those assumptions are violated, it goes back to interpreting the source code.
Therefore even if you could pre-compile Ruby in some pedantic sense, you'd still need the source code to fall back to when the JITC has to bail out.
That's partly true, but depends on the particular program. There are Ruby programs that could be compiled ahead-of-time quite efficiently, despite their dynamic features.
The slow-path version of the code that violates runtime assumptions can still be compiled as an alternative variant - so instead of switching from fast compiled code to the interpreter, the code violating the assumptions can switch to slow-path compiled code.
While that's true, and switching to a slower path solves many problems, consider this common Ruby pattern:
Dir.glob("some dir/*").each do |filename|
require filename
end
A lot of Ruby code messes with how code is loaded (heck, "require" in almost every Ruby install is not the built in "require" but one that has been dynamically replaced). Sometimes that is intended to implement static behavior: The expectation is that the code remains the same. It's just a way of avoiding having to maintain a list manually.
But other times it is used as a plugin mechanism, where the expectation is for the code to be provided by a user or other developer, and processing this statically at compile time would remove capabilities that the user would expect.
You can't automatically determine things like that, because it is down to developer intent and the code does not document the intent.
So the problem goes deeper than being about being able to revert to a slow path: To compile Ruby in a meaningful way you need to make some level of trade-off in what aspects of Ruby you decide to treat as "ahead of time" vs runtime, because there is no clear rule that says "now the program has been loaded; everything after this is runtime".
> Genuinely interested in knowing this - if you can compile ruby , why cant you compile a program?
I would think that the fact that Ruby constructs that look like what would be compiler directives in a more static language (e.g., Ruby's “require” vs C’s “#include”) have their semantics defined in a way which both depends on and affects runtime state means that AOT compiling Ruby in the strict sense gets you very little, whereas running it to a certain point AOT and saving the runtime state gets you closer to what you'd expect from compiling based on usually-compiled languages.
Select your start and your target language and it will show you an example. Its a very basic example, but it basically works with all Java types. We will provide better docs soon.
How would I go about creating a native image for a gem I've installed? I can't seem to find the documentation for this.
Or have I misunderstood this totally? I thought I could do something like ruby script -> graal ruby --> jvm bytecode --> native image
Update:: Doesn't look like I read it wrong - from this reddit comment[1] from 3months ago
Ahh, I think there's some confusion here. What we do with the SVM is ahead of time compile the TruffleRuby interpreter, not an application. This is akin to what you'd do with MRI. Instead of treating Java as a language you compile to a .class file and load with the JVM, the SVM's native image generator compiles it to a native binary.
There's likely no reason we couldn't compile Ruby applications into a static binary.
I would also love to be able to distribute a ruby binary as a single statically compiled executable. Any pointers/directions?
> There's likely no reason we couldn't compile Ruby applications into a static binary.
I think this was a theoretical observation.
What is more likely in the near future is to be able to initialise your Ruby gem at compile time, so when you start your application it is already required and you can start execution as if all your require statements had already run.
This is actually the first time I heard of UPL, which from what i read in the FAQ [1] is basically a GPL v2 compatible Apache 2 license. Which was one of the problem Apple had with LLVM i think. I wonder why Apple decide not to use UPL?
There are research papers that discuss this. It depends a lot on the bytecode patterns. For Java it's usually as good as the current JITC unless you're relying heavily on auto vectorisation. For Scala it's about a 10% win on open source or a 20% win for enterprise Graal, according to Twitter at least.
For Clojure I don't know. But from what I understand Clojure produces quite inefficient bytecode, so it may benefit a lot.
I have been paying some scant attention to GraalVM for some time, having been excited by "universal VMs" for decades now (including setups like LLVM, which most people forget was designed as a JIT-compiled VM, despite that being in the name), and I develop Cycript (which is somewhat the "opposite" technology, binding together existing virtual machines all into the same process space and doing crazy interop; though I haven't done a public release now in well over a year).
I am excited to see such a beast coming from a major company with a strong track record in VMs, and particularly given that it is based on their existing JDK, and am particularly excited that it now seems to be available!
I have some questions that I did not managed to easily get answered from looking over the website.
Can I AOT with this (via SubstrateVM) and deploy to "mobile" targets, and specifically iOS?
Are there any examples of embedding GraalVM in an existing native application (such as the noted idea of having GraalVM accessible inside of MySQL)?
Is there an existing embedding for node.js? (I don't mean "can I run node.js apps on GraalVM?"; I mean: from node.js, can I instantiate a GraalVM and give it code to execute in a new context, or will I need to build my own bindings to the GraalVM embedding API for that first?)
I did find some documentation for using an embedded GraalVM in a Java application, but while I work on many projects in many languages, none of them have been written in Java now for over a decade. I guess I was expecting a C/C++ API to be documented somewhere, not a Java API (due to the listed prototypical embeddings, and based on my long experience with embedding other virtual machines, including the Oracle/Sun JVM).
> Can I AOT with this (via SubstrateVM) and deploy to "mobile" targets, and specifically iOS?
Technically there is no reason to not support it. But support is not implemented yet.
> Are there any examples of embedding GraalVM in an existing native application (such as the noted idea of having GraalVM accessible inside of MySQL)?
Yes. Oracle MLE[1] and MySQL already work. Oracle MLE you can try already.
We have a native embedding API (its very similar to the Java API). We could not yet quite finish the documentation of it for website. But its already included look for a file polyglot_api.h in the binary.
> Are there any examples of embedding GraalVM in an existing native application (such as the noted idea of having GraalVM accessible inside of MySQL)?
The GraalVM in MySQL will be released soon and that will be the best example of embedding GraalVM in native projects.
The API for embedding in native applications is still experimental: it might change slightly in the few following versions. You can find the API with documentation in the polyglot_api.h file in the GraalVM distribution.
That's what their "node" program actually is. It's regular node but using GraalVM instead of V8. The semantic difference you're getting at doesn't seem to exist.
I guess I was expecting a C/C++ API to be documented somewhere, not a Java API
SubstrateVM has a C API that can be used to start it up that's not the same as JNI, however, bear in mind Graal can be used as a plugin to regular HotSpot too. So you can just embed the JVM the usual way and ship the Graal compiler with it, enabled via a command line flag. So yes you can do this and it doesn't require any new documentation.
> you can just embed the JVM the usual way and ship the Graal compiler with it
How feasible would it then be to use Graal NFI for FFI from vanilla Java? That seems useful given the performance limits of JNA and the complexity of JNI.
> That's what their "node" program actually is. It's regular node but using GraalVM instead of V8. The semantic difference you're getting at doesn't seem to exist.
No, because what I'm trying to accomplish is "I'm inside of a node.js program and I want to instantiate a GraalVM virtual machine instance context, load a random Java class (or whatever), and run a function on that class". Even if I'm running my node.js code on top of GraalVM, the semantic difference fundamentally exists. Like, you are trying to tell me that somehow running my node.js program on top of GraalVM is the same thing as being able to run GraalVM from inside of my program (which just so happens to be written in node.js, but that's arbitrary and meaningless in a way: I just want good bindings for the language I happened to write it in) on arbitrary code, but that clearly doesn't make any sense. I mean, it certainly isn't true for their Java embedding API, why/how would it somehow be true for their node.js API?
> SubstrateVM has a C API that can be used to start it up that's not the same as JNI... So you can just embed the JVM the usual way and ship the Graal compiler with it...
But SubstrateVM is the offline VM for AOT use, not the runtime VM for JIT use, so I really am not that interested in the SubstrateVM API... it isn't at all relevant for the use case I'm describing for embedding GraalVM; and the Graal compiler in this context is also for AOT use, which again isn't relevant for the use case of embedding the GraalVM.
What makes GraalVM interesting here is that it is a JIT. Just take a step back for a second and imagine you have a C program, and you want to boot that JIT and use it, as a library, loaded into your process. That's the use case here. You seem to be confusing that with either running the C program inside of the VM (which doesn't even change the problem) and then later with pre-compiling the thing we want to load and bringing the compiled code into the process (which bypasses the wonderful goal of working with the JIT).
The way you do this using their Java embedding API is clearly documented, and it is absolutely not the same thing as running a Java application in the GraalVM: it is a set of APIs that let you instantiate Graal Context objects, eval() code in various languages, and then work with the proxy Value objects. I am asking if that API has been exposed to any languages other than Java yet. It seems (from the answer from someone who actually works on Graal from 8 hours before your comment) that the answer is "yes" for C/C++, but I'm still guessing the answer is "no" for node.js.
My point is that you can load GraalVM into an already running process using JNI and access objects that way. JNI isn't a great API by any means but it works and there are tools to bind to it. It isn't new and I don't think GraalVM without Substrate or Sulong innovates in this area.
So if you wanted to run JavaScript using GraalJS, from C, you would link against libjvm, use JNI to initialise the VM with Graal as the JIT compiler (there's a special flag for this), and then use the Polyglot API via JNI.
To bind from JavaScript to Polyglot without having Graal be the host would presumably require you to write a V8 extension that then loads the JVM using JNI and proxies back and forth. I think there are projects that do this already. But you wouldn't get the performance or convenience improvements than Graal/Truffle are pioneering.
In the end though, embedding the JVM this way - at least for node apps - would not be better or more efficient than using their node implementation, but ignoring things like performance and debuggability the results are presumably identical unless your code depends on an implementation detail of V8. So I don't get why you'd do that.
I wouldn't touch this with a ten-foot-pole without an explicit patent grant. Oracle has misbehaved too much in the past (and present) to ever be trusted again by the open source community in my opinion.
If I have a choice between running our stack 50 times slower (and thereby 50 times the cost) or running anything Oracle, I know which choice is cheaper and doesn't come with an existential threat to my company.
Only governments and big corp are rich and reckless enough to take on this kind of risk willingly.
There is like a bajillion people out there running Oracle Java, MySql, Virtualbox and so on. All of them Oracle products, and with permissive licenses. Where is this "existential treat" thing coming from?
>There is like a bajillion people out there running Oracle Java, MySql, Virtualbox and so on. All of them Oracle products, and with permissive licenses. Where is this "existential treat" thing coming from?
Oracle. (see, I can be sarcastic too)
Ignoring the fact that most of those users were likely acquired back when those products were still known as Sun Java, Sun MySql, and Sun Virtualbox, those "bajillion" people are not lucrative enough to warrant the effort. The moment one of those people becomes large enough, Oracle's lawyers will come knocking. If you're large enough, they will even go as far as to change how copyright law itself is interpreted to get their fill.
With a company like that, entering any form of contract or agreement is a major liability and an existential threat.
"The moment one of those people becomes large enough, Oracle's lawyers will come knocking."
Can we see an example of big Java user being called by Oracle's lawyers? Excluding Google, they created their own Java with no regards to license, so yeah they got sued. So... did anyone that just downloaded Java and used it, got sued? I do not see it. Nor do I see what they could possibly be sued for.
> Google, they created their own Java with no regards to license, so yeah they got sued.
You don't need a license to build a clone.. well, you didn't need one until recently thanks to Oracle.
While not exactly the same thing, I'm still glad that IBM didn't pursue IBM-compatibles using oracle's absurd logic (or ford and other car manufactures for that matter). I like competition.
> did anyone that just downloaded Java and used it, got sued?
Perhaps not sued, just threatened into paying. See sibling posts for your anecdotes.
My boss was careless and entered his contact details when he downloaded MySQL.
Fast forward a bit and Oracle sales people where calling him and trying to convince him that he needed a paid license - to run default MySql on AWS(!).
> There is like a bajillion people out there running Oracle Java, MySql, Virtualbox and so on. All of them Oracle products, and with permissive licenses.
All of those examples have dual commercial/copyleft licenses, none of them have permissive licenses. Moreover, the Free licenses are without explicit patent grants which may, therefore, be at risk of patent claims.
Where I work now, we went through 2 or 3 rounds of funding. Each round involved lawyers doing due dilligence, checking the licencing of all the software we use, and all the libraries we use, and so on.
No issues were raised regarding Java or MySql. Bottom line, I don't know the difference between this and that license, what I do know is that Oracle licensing for Java and MySql gave us zero grief. For the record, we just use Java and MySql as provided. I can see how Google making their own "Java" for Android can get messy. But that is kind of unusual, who makes their own Java? I know of several vendors, and they usually try and get certification/TCK and so on. Google did not, well, that is Google's problem.
Is there public documentation for the differences between the CE and EE editions? The website mentions that EE "provides improved performance and security for production deployments". What types of performance and security optimizations are locked behind the EE edition? What is your criteria for determining whether a new optimization belongs in CE or EE?
I also want to echo others' concerns about the license, lock-in, and lack of explicit patent grant. It's a shame that Oracle's (well-deserved) reputation is tainting so many excellent engineering efforts. I hope that these comments serve as some form of feedback about the very real cost the Oracle brand has suffered as a result of its legal activities.
By default it uses an arbitrary precision adder. But node rewriting on the AST (predicated by the "guard") provides specializations for long and String types. The long-type execution is a "fast path" that executes machine instructions directly; the String-type execution is a string concatenation.
Someone I know wrote a Java implementation of the small VM for an open source project that had a simple C interpreter. My first thought was "wait, why?", until they mentioned that it was more than 10x faster and much, much easier to write and extend. You basically get a very good JIT engine for free. Not gonna underestimate Truffle after that.
Hi, author of jaque here. I've been working on some other things for a little while, but I'll be getting back to jaque soon. It's quite out of date and needs to be brought up to 1.0 and cleaned up a bit before I start grousing about the stack issues I was running into on the mailing list, but when I last was working on it I was running into issues with Graal when trying to increase the java stack size. Deep but bounded head recursion is quite common in nock, and I seem to recall I needed about 8 megs of stack space (I just passed -Xss 8m or something on the command line) and Graal slowed waaaay down. That's where I left it.
Truffle tech lead here. Happy to hear you are not giving up :-) Do you mind if I include your language on the experiments list?
BTW.: I'd recommend upgrading one Truffle release at a time. So it should always continue to compile. Just fix the deprecation warnings and continue with the next version until you reached the last version. We always keep deprecations for at least one version before we remove/break APIs.
Definitely not dead -- it got stalled for a while because Urbit tends to use deep stacks and both Graal and the JVM seem to favor shallow ones. The author (not me) came up with a "meta-stack" design that may mitigate this issue and will go back and try it soon.
But the performance seen on a trivial Ackerman function was very good -- Nock via Graal got close to C performance. That's impressive for what's basically a Lisp.
> Are you working on a Java application? Do you want a faster runtime? Enhance your code with JavaScript!
First, the JVM already has the Nashorn JS engine.
Second, if I'm "working on a Java app", that means that it's already running in a JVM. Where does the faster runtime come from? Is executing JS from Java using Graal faster than running actual Java code? Is it just faster than Nashorn? Something else?
The Graal compiler is compiling Java code to faster machine code on top of the JVM. Also GraalVM's JavaScript implementation uses an entirely different approach by using Truffle for its implementation making it significantly faster. Also GraalVM ships with Node.js support, tooling and other languages.
"Are you working on a Java application? Do you want a faster runtime?"
Graal provides an ahead of time compiler for Java, which can speed things up by removing JVM warm up time. I'm unsure how much faster Graal is compared to steady-state JVM.
So Graal can do more than Hotspot's C2 compiler so may be able to run your code faster than a stick JVM, Twitter have had some very good experiences with it. The JS implementation is very different to Nashorn's so it may be faster for your application.
Note that Nashorn is very slow - Graal.JS is orders of magnitude faster, and its peak performance is very close to V8 (although with a somewhat slower startup, currently).
Might this (today, tomorrow) become another way to get access to the wonderful Lucene from non-Java environments? A long, long time ago there was a GCJ-based solution, but it was never loved much (either the port or GCJ itself). SubstrateVM looks potentially delicious here
edit: lol: talk about a low blow!
> The community version does not support DWARF information. The enterprise version supports all native tools that rely on DWARF information, like debuggers (gdb) and profilers (VTune).
This project is remarkable! Really excited to see where it goes. The biggest problem I see so far is sparse docs on interop. I see docs with a trivial example of importing simple types from java and using them in node, which is amazing! However, this is pretty far from a real world example. In the real world there are deep type graphs and dependencies and functions that look like IWidgetInterface createWidget(IAbstractWidgetFactory<CoolWidget> factory) etc... seeing examples of dealing with complex types, anonymous Callables, generics, etc... would be great.
> GraalVM CE is available for free for development and production use. It is built from the GraalVM sources available on GitHub
2. Enterprise Edition (EE)
> GraalVM EE provides additional performance, security, and scalability relevant for running critical applications in production. It is free for evaluation uses and available for download from the Oracle Technology Network.
"performance, security, and scalability" words don't present in p1. Is it just me or...?
This is wonderful. I'm looking forward to playing around with JavaFX. Being able to compile that to a native image is huge! It would effectively invalidate the Electron approach of building "native" applications.
However bear in mind you can already bundle JavaFX and the JRE together into native installers for each platform that don't require a JRE or JVM, and don't mention Java anywhere, using the javapackager tool. I've used it several times, it works well enough.
That was the earliest post I remember from HN. But I record there some other information scattered around a little before that.
So this is nearly 5 years coming. I am amazed what at first seems like fantasy, impossible work is now actually thing. 2012 / 2013 was the year when VM and JIT has lots of innovation. Rubinus, LuaJIT, SpiderMonkey and all those Javascript VM.
If we can successfully JIT it we normally do very well. The caveat is that we do have warm up time and if we can’t compile stuff then you won’t see the same level of performance.
I know lots of people are worry about Oracle. But the license is extremely clear, it is the same as Java.
So like the JVM or JDK, there is nothing stopping you from for forking the code, as long as you don't call it Java, and "your" JVM would not get the test suit.
GPL 2 with Classpath exception has been with us for so long. I know you hate Oracle, but I don't see any downside to it. I mean half of the Internet literally runs on JVM and MySQL.
I had to look hard and it doesn't seems to have any strings attached.
> With the SubstrateVM it is possible to produce a single, statically linked native binary executable version of Graal.js, which doesn't need any JVM to run.
How large is the resulting executable? There was some mention of shares libraries — is it possible to build a runtime that is callable from C?
Relatedly, does the GraalVM java executable support JNI?
I’m also curious about the runtime footprint in general; any idea what is the smallest possible VM for a real but relatively-small language like Lua?
> There was some mention of shares libraries — is it possible to build a runtime that is callable from C?
Yes you can compile a JAR to a shared library with C function entry points.
> Relatedly, does the GraalVM java executable support JNI?
Yes.
> I’m also curious about the runtime footprint in general; any idea what is the smallest possible VM for a real but relatively-small language like Lua?
Real language VMs are quite a bit bigger - a hundred MB or so.
>> I’m also curious about the runtime footprint in general; any idea what is the smallest possible VM for a real but relatively-small language like Lua?
> Real language VMs are quite a bit bigger - a hundred MB or so.
Don't know if you're trying to say Lua is not a real language, or just skipped the part about Lua.
No, hello-world executables built using SVM are 5 MB or so. Real world language VMs built using SVM are a hundred MB or so. If you had a language VM for the real world language that is Lua built using SVM it would probably be a hundred MB or so.
I'm not passing any opinions about any languages or VMs here.
> If you had a language VM for the real world language that is Lua built using SVM it would probably be a hundred MB or so.
Is this mostly JRE? Would Jigsaw help, or is SubstrateVM already doing something like that? (To be clear, this tech is really impressive! Just trying to get a sense of embeddability constraints)
The SubstrateVM already does whole-world analysis to only compile methods that are actually used, so reducing the methods in the JAR before hand (which is what Jigsaw) would do, would not change anything.
Language VMs compiled using SubstrateVM are larger than you may think because as well as the native code they also need to contain a representation of the code that the compiler can read (IR) so it can dynamically compile the methods that are combined to form the compiled version of your user's code, and we also need to compile a sort of runtime debug information that allows us to jump from optimised code back to the native code when an optimisation proves to be wrong. This is a novel thing - most language VMs jump from optimised code to a bytecode interpreter - we have to jump back to the native code as there is no bytecode interpreter. Finally, a language VM also needs to include all of the Graal compiler itself for dynamic compilation, which is quite a bit of code.
Keep in mind that while this is nice technology, it's Oracle-based, split in CE/EE (which in itself is not a problem, but CE doesn't run on macOS), and they will find a way to screw you over at some point in the future regardless of what you do, use or choose.
The reason why we don't build CE on Mac OS is purely technical. Its because there was no OpenJDK 8 build for Mac that we could use. We hope we can change that soon. OpenJDK builds got a lot more regular with Java version >= 10.
Currently we have the same APIs for CE and EE. So applications are easily portable between those two. We have no plans to change this.
But. You don't have to take our word for it. CE is open source and you may build, extend and maintain it yourself if you want to.
Is it Oracle Oracle, or Oracle SUN product? BTW, it sounds pretty cool, I am wondering how do you convert between complex datatypes while passing parameters between different languages and not losing much speed with it?
We don't convert. We have defined a contract that all languages can work with (quite a challenge and still evolving).
From the website:
> In order to provide foreign polyglot values meaning in languages we have developed the so-called polyglot interoperability protocol. This interoperability protocol consists of a set of standardized messages that every Graal language implements and uses for foreign polyglot values. The protocol allows GraalVM to support interoperability between any combination of languages without requiring them to know of each other.
Coincidentally I met the creators last week and it's actually developed at the Johannes Kepler University in Linz/Austria. They have a oracle sponsored lab working on compilers etc.
Me too. I’m thinking there must be some library that knows how to marshal between object representations. Surely Graal doesn’t have universal objects that suit every language.
That's the thing - there is no marshaling. The objects are passed between the languages as pointers under-the-hood. There is a standard way of invoking methods/functions on those pointers, that works across all languages.
To give you an idea of how deep this goes - you can pass a Ruby object to a C function and the C function can read and write it as if it was a pointer to a structure.
It has a form of universal object used for scripting languages (but not more static languages like Java or C++). But using it is not required, it's just a convenience to make implementing languages easier.
What it actually provides is a form of "universal abstract syntax tree". So if a C program wants to access a JavaScript object by writing "a->b = c" then the dereference is turned into a snippet of JavaScript code (in effect) and inlined right into the C function.
Like Google, Microsoft, Apple dropping tech, or replacing it with something else, just because.
Oracle has done lots of nice things for Java, some of them with uncertain future or even tabu at Sun, like Graal's percusor MaximeVM or AOT compilation.
It might not be a company loved by FOSS, but other than IBM there wasn't anyone else caring that much for Java's fate, Google specially did not even bothered.
So I appreciate every improvement Oracle does to Java.
Comparing Oracle to Google is like comparing China gov to US gov. They both do bad things, but they are not remotely the same, the difference is well over 10x.
Remember when Oracle sued Google for re-implementing the Java standard library.
Remember the OpenOffice -> LibreOffice thing.
Remember the Hudson -> Jenkins thing.
There's the enterprise sales/licensing tactics which I guess is not very familiar or visceral for most of us, me included. But I'm sure there's more I'm forgetting ...
> Comparing Oracle to Google is like comparing China gov to US gov. They both do bad things, but they are not remotely the same, the difference is well over 10x
Ask that to the civil victims of drone attacks and Apache pilots having fun with civil convoys, as proven by many available video footage.
It was a decision done by Apple, Oracle did not move a finger.
"As you are likely aware, Oracle owns US Trademark Registration No. 2416017 for JAVASCRIPT. The seller of this iTunes app prominently displays JAVASCRIPT without authorization from our client. The unauthorized display of our client's intellectual property is likely to cause consumers encountering this app to mistakenly believe that it emanates from, or is provided under a license from, Oracle. Use of our client's trademark in such a manner constitutes trademark infringement in violation of the Lanham Act. 15 U.S.C. § 1125(a)(1)(A). In order to prevent further consumer confusion and infringement of our client's intellectual property rights, we request that you immediately disable access to this app. We look forward to your confirmation that you have complied with this request."
> Remember when Oracle sued Google for re-implementing the Java standard library.
> It was a decision done by Apple, Oracle did not move a finger.
That's not true. If you read the text, the request comes from Oracle's lawyers. "our client's intellectual property":
> "As you are likely aware, Oracle owns US Trademark Registration No. 2416017 for JAVASCRIPT. The seller of this iTunes app prominently displays JAVASCRIPT without authorization from our client. The unauthorized display of our client's intellectual property is likely to cause consumers encountering this app to mistakenly believe that it emanates from, or is provided under a license from, Oracle. Use of our client's trademark in such a manner constitutes trademark infringement in violation of the Lanham Act. 15 U.S.C. § 1125(a)(1)(A). In order to prevent further consumer confusion and infringement of our client's intellectual property rights, we request that you immediately disable access to this app. We look forward to your confirmation that you have complied with this request."
The statement you quoted regarding the JavaScript trademark infringement reads like something a lawyer commissioned to represent Oracle would write, rather than Apple being proactive in enforcing Oracle's trademark.
I base this on the author's repeated use of the phrase "our client's IP".
And yet, we use techs from those companies on a daily basis. JVM especially. GraalVM is the next step in JVM's evolution. It's not hard to imagine that this will motivate other JVM vendors to add serious support for other languages.
CE edition with Mac and Windows builds is a must for me to consider it. I realize I am not a prospective Oracle customer, but I would have been a part of the common people making it more popular overall.
The idea of interoperating languages is great though. From the other polar end of the corporate world, we have .NET and its open source versions, and I hear IronPython, Javascript etc work pretty well in the .NET ecosystem. Not my cup of tea though.
Well, you may have a different experience, but for me and my companies (current/prev) it wasn't a lot of fun working with Oracle.
It's not that all the stuff is bad or that all the 'free' things turn sour, most of the things are technologically sound, but there is just so much lawyering, licensing and IP fighting that takes all the joy out of life. This isn't unique to Oracle, but not even IBM is as passive-agressive as their non-tech people and documents are.
I don't understand the point of the clip. He says that the code was similar to Sun's, but clearly admits that, not being involved with the trial, he didn't get to see Google's code.
Oracle has an extensive history of screwing developers and companies with IP laws, being suspicious of projects coming from Oracle is only a valid reaction on the history of the company.
Because WebAssembly is extremely limited? Unless you need to ship on the web, using WebAssembly is almost certainly a bad decision as it makes highly compromising tradeoffs that you don't need.
WebAssembly is a tool in work, supported by Google, Microsoft, Apple, etc... Currently it lacks a garbage collector, but this is planned. I would bet on wasm to be the holly graal in the future, more than anything else.
Many optimizations need to be performed at a higher level, using language details. By compiling something to WebAssembly you lose all the ability to do that.
For dynamic languages like JS you can’t actually make these optimizations AOT, which is why we JIT JS in the first place.
WebAssembly is a low level portable bytecode. You still need to write the code in C++ or Rust to get performance benefits.
GraalVM improves the performance of Ruby, Python, etc and also allows you to run C and C++ code in the same VM as a bonus with good performance. Unless you're using an intermediary bytecode like WebAssembly or JVM Bytecode you will also need to ship full sources with your C or C++ application.
The idea there:
"Graal is a research compiler. ... Truffle is a framework for implementing languages using nothing more than a simple abstract syntax tree interpreter."
As a bonus, you get high speed, good instrumentation for debugging and profiling, with great garbage collection, and hotswap code as you develop.
"The OCA gives Oracle and the contributor joint copyright interests in thre code."
What does 'thre' mean? ;P
They want it physically printed and signed.. sigh, I haven't printed anything in years.
Is this normal to require a scanned signature for CLAs? I noticed Apache CLA is the same (they want scanned paper signature), but all others I've seen are an online form.
Am I correct in reading the license in Github that this is all GPL? Albeit GPL2 but still not some Oracle proprietary license.
"One VM to rule them all" reminds me of Parrot VM (for Perl 6). Looks like development slowed down a year or two ago. Back then, it got me excited as it looked like the "open source community" alternative to "Big Corporate Java".
Is anyone else so leery of Oracle's lawyers you want nothing to do with this? Even if it's awesome work, as I expect it is, I don't trust Oracle not to someday find a new way to screw me, whatever the text of the license.
(I'm having doubts if this comment belongs here. But it's my genuine reaction, for what it's worth.)
0% chance I'll ever use this or let one of my team members use it.
I don't know a single person that's said a single good thing about Oracle, including people that work there. I know a ton of people that have been burned by Oracle and their lawyers.
I was very interested in Parrot VM for a while too for the same reasons. I always liked the idea of language interoperability and being able to use the best libraries of each language. This brings me to a question I have had for a while, why do we even have so many language specific libraries for common tasks instead of more "universal" libraries with bindings for each. Of course these exist, but I feel like this should be almost the default, and language libraries would just be wrappers to help it integrate better with the language (like making it more pythonic for python etc). That way you would have at most a few competing libraries for each task with higher quality. Also it would be easier to bootstrap new programming languages since that is a big initial barrier (having few libraries).
There are fundamental semantic differences between the languages. An immutable language will always have a serious impedance mismatch with a mutable library. A library that uses GC won't work in a language without them, a specific example of the general principle that memory management strategies vary widely. A library that uses dynamic types will have a huge conversion boundary with a statically-typed language, and vice versa. A library that uses threads, or expects threads, will have a hard time working in an event-based environment that depends on code voluntarily yielding to maintain its responsiveness. A language that wants to provide guarantees about correctness or whathaveyou is forced to give all those up to talk to a lowest-common-denominator library.
And that's just the top-level differences. It gets worse as you get into the details of API style and the costs of the indirection layers to convert into the local style, assuming conversion is even technically possible.
Even if you write with the lowest possible common denominator of C or Rust with no runtime, you'll still encounter a significant subset of those issues. And of course we still will use such things; all serious languages can communicate in some manner with a C implementation of a library. (Not for any theoretical reasons, really, C has more opinions than people often realize, but because it's still the baseline current systems are built on.) But it can't be as good as a native implementation.
> This brings me to a question I have had for a while, why do we even have so many language specific libraries for common tasks instead of more "universal" libraries with bindings for each.
A lot of language specific libraries incorporate foreign language (often C) libraries and add additional functionality thst fits the ergonomics of the target language, rather than being a thin wrapper around a language neutral library. This is more than aesthetic when the target language has, for example, differences in type system or fundamental guarantees from the underlying library's language.
OTOH, sometimes there aren't C (etc.) libraries, and it's more straightforward to implement a library in (say) Python. But once you have a Python library, calling into it from Rust, C, Erlang, or even a broadly similar language like Ruby is, to put it mildly, non-trivial.
Languages usually have a foreign function interface (FFI for short) to interface with external libraries. That's not as easy as one could expect, even for languages that are "closer to the metal": I remember Pascal and C different parameters passing conventions, left to right vs right to left and the cleaning of stack... you need to use compiler directives and careful thinking. Also which memory manager you use and the fact that DLLs might not "see" the same address space.
You mention Python that they say has an easy FFI with C. That's nice but it's usually used for the performance more than for reusing code.
Well, we do have some programming tasks that are almost exclusively performed through bindings on different platforms (cryptography, graphics/GUIs, scientific computing to some extent), but the hassle of maintaining bindings is enough that the “need barrier” is high: a given library has to be doing really important stuff that is also really hard to reimplement to end up in the set of tools that are almost always used via interop/bindings. This indicates a deeper problem hidden in your question: for a bunch of reasons (technical, personal, OSS-community-political), maintaining multi-platform bindings is a hassle. Things like Truffle/Graal aim to solve that by providing a common binding “middle” layer in various platforms’ runtimes. Things like libffi offer ways to make ad hoc binding easier. It’s a complicated problem.
This is exactly the case with JVM languages today: Every language has its own idiomatic abstractions, but everything is ultimately based on the underlying Java libraries.
> "One VM to rule them all" reminds me of Parrot VM (for Perl 6). Looks like development slowed down a year or two ago.
The concept lives on in the form of two "new" Perl 6 projects, namely MoarVM (started in 2012; a multi language VM) and NQP (started around 2008; a virtual VM and multi language compiler toolkit).
I think MoarVM/NQP are strikingly similar in their approach based on what I've learned so far about Graal/Truffle (which isn't much; perhaps a few hours of exploration; eg reading/watching the One VM to Rule Them All, One VM to Bind Them paper/video). [1]
Given the large number of Oracle employees working on Graal/Truffle (40 in 2016 according to the video) it doesn't surprise me that it's currently a lot faster.
Did you use Graal/Truffle friendly benchmarks (just as was done in the video)? Have you tried MoarVM friendly benchmarks to see where MoarVM shines? In particular, have you accounted for the new JIT that recently landed after 3 years of non-master branch development? [2]
Would you please elaborate a little on how MoarVM is insufficiently flexible?
> I think MoarVM/NQP are strikingly similar in their approach
Does MoarVM use a meta-compilation technique? Pypy for example uses meta-tracing and Truffle uses Partial Evaluation to get from an interpreter specification to dynamically compiled code without duplicating the language logic. If yes, could you please point me to further information?
Meta-compilation is the primary innovation in Truffle. It allows to compose languages together and to compile them as one unit.
If you want to know more about the Truffle approach to meta-compilation see this PLDI paper from 2017[1].
Its NQP that does metacompilation, not MoarVM (afaik).
I linked to NQP's (spartan) readme above.
Some other points/links that might be helpful:
* nqp is written in nqp.
* nqp has a first class grammar construct. See wikipedia's Perl 6 Rules page.[1] (nqp's grammar is defined using nqp's grammar construct. So is Perl 6's.)
* nqp and MoarVM implement a protocol for abstracting behavior (eg methods) and representation (memory layout and use) called 6model.[2]
To try set your expectations appropriately, let me emphasize this note from the NQP readme: "NOTE: there's no end-user support for NQP and the behaviour can change without notice. It's a tool for writing Perl 6 compilers, not a low-level module for Perl 6 programmers."
NQP is a tool for writing compilers, not just Perl 6 compilers, and is in fact much more than that, but the Perl 6 project is currently very focused on the Perl 6 language and definitely doesn't want ordinary Perl 6 programmers thinking NQP is a tool for them.
Doc is sparse, again reflecting the focus on the Perl 6 language and the Rakudo compiler that builds atop NQP, analogous to your team focusing on just one of your guest languages.
The best doc for getting into NQP is not even in the NQP repo. It's the support materials for a 2-day workshop held in 2013. The abstract is "This intensive 2-day workshop takes a deep dive into many areas of the Rakudo Perl 6 and NQP internals, mostly focusing on the backend-agnostic parts but with some coverage of the JVM and future MoarVM backends also. During the course, participants will build their own small compiler, complete with a simple class-based object system, to help them understand how the toolchain works.".[3]
Beyond that, there is a doc directory in the nqp repo.[4]
(nqp rules are Perl 6 Rules grammars but the general purpose DSL (!) for writing procedural code embedded in an nqp grammar is nqp's general purpose DSL rather than Perl 6's general purpose DSL.)
(MoarVM directly implements 6model -- MoarVM is named for "Metamodel On A Runtime". 6model is how one nqp based language is able to do magic like subclassing a class written in another language (whether nqp based or via an existing compiler) despite them having distinct OO semantics and implementation strategies.)
There is a community edition (CE) and an enterprise edition (EE) of GraalVM. The community edition is distributed under an open source license. It is free to use in production and comes with no strings attached, but also no guarantees or support. The enterprise edition is available from the Oracle Technology Network under an evaluation license. It provides improved performance and security for production deployments. If you are interested in using the enterprise edition in production, please contact graalvm-enterprise_grp_ww@oracle.com.
None, but that is not the point. The point is: can we use the code without hindrance in whatever plans we have now or in the future? Or is building upon the code like going into a long one-way street?
The fact that some people will use the code without giving back is secondary to that.
That is not just my opinion, because if a more liberal library comes into existence (inevitable), it will prevail.
I tested it out with some fairly dependency-heavy node JS projects and it worked great (albeit with a human-noticeable perf difference vs node). R projects suffer the usual R-on-JVM issues, see eg https://github.com/oracle/fastr#status-and-limitations . I could, however, use ggplot2 and dplyr, which I don't recall working the last time I tried R-on-JVM. So that's awesome.
Being able to reasonably use R stuff from clojure (and other languages better for general purpose programming) is my data science dream. Though I can't say I'm sad with my current unix-y solutions.
We have considered .NET support but we are not actively working on it right now. But GraalVM is an open platform. Everyone can develop a language for it:
http://www.graalvm.org/docs/graalvm-as-a-platform/
I could run node -v, but could not run npm and I was getting this error:
jvm library could not be loaded: /home/raj/graalvm/jre/lib/polyglot/libpolyglot.so: cannot enable executable stack as shared object requires: Invalid argument
why would you want that... it should run directly on Windows too?!? Not sure what you gain from using the linux binary from LXSS.
If you really insist, invoke instead as .exe so you can still use bash and other tools, but run the actual compile or runtime using the system-native implementation.
Can anyone comment on how mature the R and python support is? I'm coming at this from an ML/data science background, so anything that speeds up the above while maintaining broad library support would be a great help.
- FastR is around for several years. Language support is pretty much done. Its mostly about getting to run CRAN packages that use native extensions.
- Graal.python is pretty new and its really early days. So even basic language things might break right now. But we are investing a lot of resources in Python support.
Thanks for that, link seems down unfortunately. I use data.table a lot but would happily switch if the dplyr family of packages work.
Would love to use this in Rstudio for exploratory analysis which would also need ggplot to work. Quite often have to cancel long running tasks when exploring a data set, so even if its a bit slower for interactive tasks but much faster for longer running tasks would be huge. E.g. typical interactive finished in 1 second instead of 50ms, but large loops that would take an hour finish in 10 minutes.
On the "why graal" page, it says that one benefit is you can use a larger heap size for nodejs applications. I would imagine one other big benefit is that you can have true multithreading since the app now runs on the jvm. Is this indeed possible? Is there any guidance on how to do it? Rhino had the concepts of Scope's and Context's to somewhat safely parallelize code, so curious if there are similar primitives in Graal?
For Clojure, I think it means native compilation, with fast startup times, and the ability to interop with Python, Ruby, or any of the other Graal languages. Not sure there would be an advantage to re-implementing a Clojure on Graal. Clojure is not an interpreted language like Ruby, JavaScript and Python is, so I don't think Graal adds as much benefit here.
For ClojureScript, it would mean full circle. You could run your ClojureScript back on the JVM, and have Java interop from it.
We are getting there, lots of things work right out of the box just fine. C extensions are still being actively worked on as they often need some amount of patching because of assumptions they make about types and so forth. I think we can reduce that, but I doubt it can will disappear entirely, but in the end I hope extension authors will test on TruffleRuby themselves and patch the code themselves.
I know that you are using Rails as a reference point so is there any ETA when will be most of the features usable ? (openssl,activerecord,nokogiri..etc) ?
OpenSSL and nokogiri already work pretty well. ActiveRecord works providing you have a database driver and I’ve got a rough set of patches for pg that I’m working on right now.
I’m working on Discourse at the moment and can run the dB create and migrate tasks and I’m working on the asset compilation pipeline, but it’s all pretty rough and won’t be our final solution.
That sounds promising. I think it's important to keep community in loop and make sure people are aware of limitations and what's already possible and supported.
Not at the moment. I'm mostly doing some ugly hacks to get stuff working and generating a list of bugs for things we need to fix properly. Looking at my branch I think there's one small change that we will want to do PR for in the end, and some dependencies that we may want to make platform dependent (though they recently did change a couple for JRuby, which also helps us). Pretty much everything else I hope to back out when we've fixed a few of the root causes.
It depends on their C API. Ruby doesn't really have one, it just has the exposed internals of the vm, and assumptions that can be made about those which are normally true. That includes type assumptions like VALUE being an integer that you can switch on, and _some_ values such as Qfalse and Qnil being constants that can be used in such switch statements, but it also includes things like being able to have a custom GC mark function for a C data structure.
It's very hard to produce another Ruby implementation without breaking some of those assumptions, but we can hopefully get it down to a small set so the changes needed are minimal.
There are other things like throwing Ruby exceptions across C library boundaries which may work in MRI, but whose behaviour may not be intentional and could already have subtle problems, and I don't think we should be aiming for compatibility in those areas at all. Finding and fixing things like that is normally a good thing to do anyway.
This is an epic announcement and it further extends the JVM as the best VM to develop new languages for and port existing ones to. I'm curious though, is there a Truffle version of the Java language that the Graal JIT uses?
Graal: https://github.com/oracle/graal (GPLv2 with CPE)
JavaScript with Node Integration: https://github.com/graalvm/graaljs (UPL - BSD license)
Ruby: https://github.com/oracle/truffleruby (EPL, GPLv2, LGPL)
R: https://github.com/oracle/fastr (GPLv2)
Python: https://github.com/graalvm/graalpython (UPL - BSD license)
LLVM/Sulong: https://github.com/graalvm/sulong