Author here. IBM is currently focusing on the combination of machine learning and reasoning, also called neural-symbolic AI [1], to combine the best of both worlds (noise robustness and explainability, etc.). I am from Tokyo and will join MIT-IBM lab soon to perform this line of research.
I have been working on Latplan [AAAI-2018, 2,3,4,5,6], a neural-symbolic classical planning system that learns symbolic representations of noisy images and perform efficient symbolic planning in the latent space. Later, several other groups have started working in this field, like [7].
The problem with python is the lack of sequential execution speed, which is necessary for solving symbolic tasks, e.g. SAT solver, classical planning, theorem prover. At the same time, traditional languages like C++, typically used for writing solvers, are too inflexible to accommodate the workflow in machine learning. For a neural-symbolic task, personally, Lisp is best in this space (I don't try to repeat why, because there are plenty of articles on it.), with Julia being a contender.
I am also interested in adding the compile time type/shape checking on the deep learning code, since python debugging is painful (just a personal opinion -- not intending to start a war here).
By the way, I made the first prototype of this lib in 3 days, and am working on improving it for 3 month (for other ML projects).
Hi. Are there some openings over there? My entire career has been low-level development in C and C++ (embedded Linux in the last 15 years), with a passion for Lisp on the side (20 years of experience now). Since 2009 I've been developing the TXR language (http://www.nongnu.org/txr), which contains a Lisp dialect called TXR Lisp. My Japanese is okay-ish. I'm not a numerical researcher and have zero background in machine learning. I live in Vancouver, Canada.
I hope you enjoy your new job! I am also interested in hybrid AI systems. I have been doing symbolic AI in Lisp (and sometimes Prolog) and also neural networks since the 1980s but I have never professionally combined disciplines (but some side of desk hacking at home). Glad to see the MIT-IBM lab peruse this with resources.
Is there a specific reason you choose LGPL for the library? Are you aware of the issues to use that license with Common Lisp libraries? Common Lisp doesn't use shared library linking as C does, so some terms of the plain LGPL are not directly applicable to Common Lisp, Franz tries to compensate for this with the LLGPL, but it still more difficult to use correctly compared to a more permissive license.
Loading a Lisp .fasl is very much like loading a shared library. (Shared library loading can be regarded as a Greenspunned version of what goes on in Lisp.) There is code in there, referenced by symbols, which resolve. dlsym is like a hacky, low-level version of symbol-function or symbol-value.
C header files also have a ready analogy in Lisp: they are like Lisp files that provide macros and constants.
Where the LGPL says "The object code form of an Application may incorporate material from a header file that is part of the Library." we can understand this "header file" to refer to a Lisp file that provides macros that generate code within the Application's file.
The LGPL doesn't define what is a "header file"; it doesn't say that it's a C thing with typedefs and #defines. In that regard it has a flaw whether we apply it to a C shared library to a collection of Lisp object files.
I know how Lisp loads files (disclaimer, I am professionally using Common Lisp) and consequently are aware of the LGPL issues. First of all, does LGPL only allow using fasl files or also building applications? Can you create a fasl file of a modified library and load it instead? Especially, if the user does not have the source code to the rest of the application?
The answers depend on the Lisp implementation used. There is a good reason that Franz decided to augment the LGPL (at their time version 2) with a license preamble to clear up the ambiguities in the context of Common Lisp.
Building an application executable may or may not violate the LGPL. It likely depends on whether the application allows new versions of the LGPL-covered modules to be loaded into it. The simple question is: can users exercise their LGPL-granted rights to that library: to modify the library code, and still use it with the proprietary application?
(FWIW, I've written a Lisp implementation that has compile-file and load, similar to Common Lisp.)
A Lisp-ified LGPL might seem perfect, but on the other hand there are some good reasons not to introduce derivative versions of de facto standard licenses.
You are a lawyer? You are a professional Lisp programmer? How does the LGPL deal with Common Lisp macros?
"Franz did LLGPL" is not by itself a good enough reason to suspect there are or to use LLGPL.
As Franz is the most prominent Lisp vendor, employing some of the greatest Lisp programmers and undoubtfully having a good legal department, I see no reason why I shouldn't take their legal opinion seriously. Especially as it is not part of any commercial offerings of theirs, but just provided as a clearer license for Lisp libraries.
It also coincides with everything I heard from my lawyers at the company I work for.
You don't have to be a lawyer to read legal documents and use licenses. There is no reasonable interpretation of LGPL in Lisp context which would lead to a need of additional clarifications.
TL;DR if someone wanted to have Lisp code to be licensed under GPL terms they would license it that way. I'm a professional Lisp programmer and I've spend a fair share of hours on reading licenses and analyzing them.
This might work for ECL which is build as a shared library. But what if e.g. an SBCL user wants to use a LGPL Lisp library? How can the developer provide the ability to "link" against a modified version of the LGPL library? What about macros exported by the library?
The LGPL talks about the szenario of linking shared libraries. This is a clearly defined process in the C world, which has no exact counterpart in the Lisp world.
You claim that if the author of a library had other intentions, the author would have chosen the GPL. That might be true, but certainly isn't sufficient for legal certainty. Perhaps the author wanted allow only shared libraries where it is possible. The LLGPL is an easy amendment to make those intentions clear.
Even easier, and better suited for Lisp environments (even following the LLGPL to the letter can be technically difficult) are licenses like BSD, which in this case would be the obvious one in my eyes, as Numpy itself is under the BSD license.
> This might work for ECL which is build as a shared library. But what if e.g. an SBCL user wants to use a LGPL Lisp library?
If you had read the post you'd have my answer to that question.
There is no need to spread fear and uncertainty about LGPL and other copyleft licenses, I've seen too much of that over last few years (and I have a strong intuition that it is not an accident).
P.S. "are you a lawyer?" argument is very cheap (and often happening on threads about copyleft licenses) given it is HackerNews not LawyerNews -- I know of lawyers working for corporations who say that they wouldn't touch anything with GPL in the license name with a five-meter pole (not knowing a difference between LGPL, GPL and AGPL) -- there goes your "in-depth legal analysis". Same lawyers doesn't bat an eye when they give a green light for eula-crippled code from their "technical partners". Cargo cult is present everywhere, not only in software development.
If you had read the post you'd have my answer to that question.
I have read your post, and just reread it, and I don't see where you answer that question. How does an SBCL user deploy a program with an LGPL library so that the reciever of the program can relink it as required by the license.
P.S. "are you a lawyer?" argument is very cheap
I used this question on a post where the poster just makes a claim, not supporting it by any reasoning or linking to supporting documents. The question must be allowed how he justifies his statement. So for legal questions, this would be: are you a lawyer? as I would ask a random poster making definitely medical statements: are you a doctor?
And yes, I know, beeing a lawyer per se doesn't guarantee a correct evaluation, I have worked with far too many corporate lawyers, which are usually not experts on open source licenses. But I also had no doubts on the competence of those, who looked into the details of the licenses in question. I have been able to get their OK on LLGPL software for example, though GPL is of course limited to very isolated usages.
> This is a clearly defined process in the C world
Rather, shared-library mechanisms are platform-specific. ELF shared libraries are different from Windows DLL's, for instance. The LGPL covers both, because it's mum on implementation details.
The abstraction is basically the same. The application has some symbolic references satisfied dynamically by the library, and possibly vice versa.
> The LLGPL is an easy amendment to make those intentions clear.
I wouldn't use it because it's a rare license, and I'm not a lawyer. It's best to use one of the top five or so popular licenses. Licenses aren't programming languages; it's okay to use the popular thing.
Furthermore, bosses and customers don't want to hear that Lisp is so weird that it needs a modified version of a license that everyone else uses in stock form.
The LLGPL contains dubious, superfluous clauses like: "It is permitted to add proprietary source code to the Library, but it must be done in a way such that the Library will still run without that proprietary code present."
w...hy? The LGPL doesn't allow that, and there is no reason to do that in a Lisp library.
Code is not added to libraries in Lisp; it is added to the image. We can add the proprietary application into the image and we can load the library into the image. That's all the mixture of proprietary and LGPL that is needed; the proprietary sources don't have to appear to be mixed into the library in the project tree. Keep it in a separate directory, which contains a copy of the LGPL which applies to every source file in it. No brainer.
> This is a clearly defined process in the C world
Rather, shared-library mechanisms are platform-specific. ELF shared libraries are different from Windows DLL's, for instance. The LGPL covers both, because it's mum on implementation details.
The abstraction is basically the same. The application has some symbolic references satisfied dynamically by the library, and possibly vice versa.
Shared libraries are not limited to C, but in the context it should have been clear what I was talking about. Most importantly, most Common Lisp systems are not using shared libraries in the way C and languages using C style bindings do.
> The LLGPL is an easy amendment to make those intentions clear.
I wouldn't use it because it's a rare license, and I'm not a lawyer. It's best to use one of the top five or so popular licenses. Licenses aren't programming languages; it's okay to use the popular thing.
I also am strongly advocating not using rare licenses. But they are still preferrable to licenses which do not work as intended for technical reasons. LLGPL is one thing, I would rather recommend a BSD style license, as it is very well known, has no technical problems involved and is actually the license of the original Numpy code. Why choose a different license for a reimplementation of Numpy?
Because it's not a derived work (not based on numpy code) thus the library author is free to choose a license that he feels best protects his code and aligns with his interests.
LGPL offers significant protections that can not be found in BSD.
Because it's not a derived work (not based on numpy code) thus the library author is free to choose a license that he feels best protects his code and aligns with his interests.
In this case, the author could choose LGPL even if it were derived on the numpy code. So yes, technically the author is free to choose about the license. However, unless specific and significant reasons require this, I would find it just appropriate to choose the same or a similar license.
LGPL offers significant protections that can not be found in BSD.
Well, I asked the author - without ever getting an answer - whether there would be specific reasons, why the LGPL would be needed. For example, because of requirements by his employer or other specific reasons.
Other than that, yes, people always claim that the GPL style license offers especial "protections". But one has to look at the environment. We are talking about a port (derived or not) of the BSD licensed Numpy, intended to run e.g. on the BSD licensed SBCL. So if the library would require special "protections", one could name them.
All legal text, and the interpretation of it is ambiguous to some extent. Sometimes by design, but always through the very nature of not being an unambigous formal language, and hence not trivially machine interpretable.
Thank you, Google found the pdf for me :). I am going to read the pdf in detail, but skipping to the conclusion the author only states:
Even so,this article has shown that the clarifications made by the LLGPL to the original GNU license are largely unnecessary, and that the LGPL would probably be interpreted in a similar fashion withoutthe clarifications proposed by the LLGPL.
The useage of the words "largely" and "probably" would make me less confident of the verdict, that the LGPL is enough.
On a different angle: I checked Numpy and it is using a BSD-license. What was your motivation to license your library differently? Is your library derived from Numpy or just implementing the same api? A BSD-style license would make any technical issues - both legal and actual distribution of the product - just go away.
The "largely" and "probably" are the mentions to the fact that there aren't yet legal cases concerning LLGPL. I am not an expert in law, but given that the journal is peer-reviewed, I will give some credit. https://www.ifosslr.org/index.php/ifosslr/about/editorialPol...
A BSD-style license does not protect contributions/changes to the library so it follows that it is not a drop-in replacement for LGPL.
Multiple people have told you that there are no technical issues with LGPL applied to Lisp programs but of course you may persist in thinking so. Do not expect others to share your anxieties though, particularly when it comes to deciding which license to use for their code.
A BSD-style license does not protect contributions/changes to the library so it follows that it is not a drop-in replacement for LGPL.
I suggested a BSD-style license, as it would resolve any possible legal problems and is actually the license of the library that was cloned. I asked the author, whether he could name a concrete reason why he switched his work to a different license than the original library, which I would find an appropriate default.
Multiple people have told you that there are no technical issues with LGPL applied to Lisp programs but of course you may persist in thinking so. Do not expect others to share your anxieties though, particularly when it comes to deciding which license to use for their code.
How many of those people are lawyers? Law isn't decided by a majority vote. I am a professional programmer working with far too many lawyers to do my work. Calling my concerns about the legal technicalities of a licence "anxieties" is quite inappropriate.
I have been working on Latplan [AAAI-2018, 2,3,4,5,6], a neural-symbolic classical planning system that learns symbolic representations of noisy images and perform efficient symbolic planning in the latent space. Later, several other groups have started working in this field, like [7].
The problem with python is the lack of sequential execution speed, which is necessary for solving symbolic tasks, e.g. SAT solver, classical planning, theorem prover. At the same time, traditional languages like C++, typically used for writing solvers, are too inflexible to accommodate the workflow in machine learning. For a neural-symbolic task, personally, Lisp is best in this space (I don't try to repeat why, because there are plenty of articles on it.), with Julia being a contender.
I am also interested in adding the compile time type/shape checking on the deep learning code, since python debugging is painful (just a personal opinion -- not intending to start a war here).
By the way, I made the first prototype of this lib in 3 days, and am working on improving it for 3 month (for other ML projects).
[1] https://medium.com/@MITIBMLab/the-shared-history-and-vision-... [2] https://github.com/guicho271828/latplan [3] AAAI18 presentation at : https://asai-research-presentation.github.io/2018-2-5-aaai/ [4] AAAI18 paper : Classical Planning in Deep Latent Space: Bridging the Subsymbolic-Symbolic Boundary, https://arxiv.org/abs/1705.00154 [5] Follow-up papers (ICAPS2019) : https://arxiv.org/abs/1903.11277 [6] Follow-up papers (ICAPS2019), extension to first order logic : https://arxiv.org/abs/1902.08093 [7] Learning Plannable Representations with Causal InfoGAN, https://arxiv.org/abs/1807.09341v1