It's unfortunate that just because Tensor Flow was released, you felt the need to replace references to deep learning (which is a somewhat well-defined application) by "machine intelligence" [1], which could mean so many things.
Otherwise, it's great to see such initiatives in Rust. I'm unsure if there's an equivalent that implements more traditional ML algorithms (logistic regression, decision tree, random forest, SVM, etc.)
We wanted to make clear that we are not "just" doing deep learning with Leaf. A lot of other ML algorithms that don't fall under the deep learning label can benefit of the tools provided (e.g. solvers) and can be implemented as layers. While not too popular, there are papers showing this being done with e.g. a SVM[0].
Great work -- very exciting to see a Rust ML framework!
Dist-Belief (TensorFlow's predecessor) was oriented around layers (as Leaf is) but TensorFlow (like Theano) is graph/flow-based rather than layer-based. I think in particular the layer-based approach makes it awkward to represent more complicated models since not every model can be decomposed into backward/forward passes.
Do you worry that by adopting a layer-based paradigm you are boxing yourself in? A graph-based approach does seem more flexible (and distributable across machines).
I am very optimistic concerning our plans to make Leaf work in a distributed fashion. It's not quite finalized yet, but the rough outline consists of abstracting other parts of the Network away as a special type of layer and using capnproto + zeromq for transportation. Sadly I haven't come across a good implementation of model parallelism to learn from, so that might become a bit tricky.
I think there is still a lot of room for improvement for layer-based networks, but even if the graph-based approach will become more prelevant I think the Rust trait system gives us great tools to migrate and reuse a lot from our layer based approach.
I am very excited to finally see this type of library in a modern (strongly typed) language, let alone my current passion language. In my opinion your pitch (ok, Readme), should put emphasis on the productivity and usability gain of types and a generally "smarter" language moreso than the not-offloaded-to-C performance (do people care what they code gets compiled to?)
I was pretty excited about the tensorflow announcement but I'm actually saddened that it might cast a shadow over this library, which in my opinion brings more to the arena (e.g. improvements on the current scripting approach of theano, torch et. al)
I can give a quick comparison on Leaf vs. TensorFlow - although this might seem somewhat biased.
We like what the Google engineers did on TensorFlow and share a lot of similar ideas on how future Machine Learning should be structured and implemented. Especially that in the end it is just a performant pipeline for numeric information processing.
We feel that the biggest difference (besides the different stages of the project) is the language and the ecosystem that it embraces. We are strong believers in Rust and think that it might have good chances to succeed in the long run - which we outline in more detail in the Q&A[1].
Due to your question I would recommend picking up on the basic concepts of machine intelligence and then going with whatever ecosystem fits the task/stack. Might become similar to choices of web frameworks in the next couple of years. I think Neon[2] - as a Python framework - seems to be easy and performant as well.
If you're just getting started in this area (machine learning), I'd advise you to look at simpler and more interpretable models first (things like logistic regression, decision tree, perceptron) using simple tools (scikit-learn for instance).
Something like Tensor Flow has a lot of mechanics you have to understand before getting started, and it's "lower-level" than other ML libraries. And I definitely wouldn't advise you to use "deep learning" as your first ML model. It's hard to understand, and the result of training it is hard to interpret.
I highly recommend Weka, http://www.cs.waikato.ac.nz/~ml/weka/. I used it for river flow forecasting (published a paper) and it was very good. We ran about 100 different models, various neural nets, SVM, logistic regression, multilinear regression, etc.
I spent about a month trying different tools and liked the understandability of Weka. Scikit-learn calls out to C for the heavy lifting, which is great but it's not exactly easy to read and understand the code.
Exciting to see this! I haven't taken a look at TensorFlow code yet but it looks very similar (method names and all) to Caffe. I've wanted a CNN library for Rust for a while.
It sounds like performance is important to you. Are there any plans to add GPU paths soon? If it's CUDA, are you going to use cuDNN?
Yes, GPU support has high priority but we are still investigating what library/tools we want to go with, since the current options in Rust are not that great. Maybe we'll use Arrayfire, maybe we'll use CUDA/cuDNN (would require writing Rust binding) or maybe something else.
If you're going for GPU support then please do not use CUDA. It's non-portable (NVIDIA only), proprietary, clunky and it requires a huge SDK which is very awkward to install. I highly recommend OpenCL which is portable, requires no SDK, is open and for properly optimized code is just as fast as CUDA.
While this sounds attractive, OpenCL performance just doesn't seem to be there at the moment.
In CNN benchmarks[1] the CL based implementations consistently finish at the bottom. I think this is mostly because of the weak nVidia OpenCL implementations, but that doesn't help, since most people use nVidia (eg, Amazon GPU instances are nVidia based)
I like your idea very much, especially implement in Rust. As we need more powerful computation, the native low-level language is a very good choose, we can control the memory, and call the native library to run on CPU/GPU.
Even the development of Rust may not very high enough compared with Java/Go(I do not sure), but we can keep the core as simple and steady, and use high-level language such as Python/R to implement the API.
Hope to hear more wonderful news about this Project.
Otherwise, it's great to see such initiatives in Rust. I'm unsure if there's an equivalent that implements more traditional ML algorithms (logistic regression, decision tree, random forest, SVM, etc.)
[1] https://github.com/autumnai/leaf/commit/58bfa92abd5721eae116...