4. This is an overview of Meta-Interpretive Learning (MIL), a new approach to ILP that overcomes many difficulties of earlier approaches (Full disclosure: my own work is on MIL, though not the article linked):
Meta-Interpretive Learning: achievements and challenges (Stephen Muggleton, 2017)
That should be enough to get you started. I recommend reading in the order I linked to the various articles. I tried to give links to documents that I know can be read for free.
Unfortunately most of the material on ILP is either in scholarly articles, or, where there are textbooks, they tend to be older. That sounds bad, but there has been much new work recently with several new approaches.
Let me know if you're looking for more specific information. See my signature for contact details- I'm happy to answer emails about ILP :)
It should be emphasised that inductive programming is not tied to logic programming, and works for every other programming paradigm as well, e.g.
functional programming [1, 2]. We could also do IP for imperative programming, although, as far as I am aware, nobody has done this.
That's absolutely true! But the OP asked about ILP in particular.
To be fair, logic and functional programming languages do have some advantages as target languages for Inductive Programming compared to imperative languages in that they have very simple syntax. For example, Prolog doesn't even have variable declarations. That's very convenient because the learning system only needs to learn the logic of the program, not the syntax of the language also. It's also much simpler to define language bias or program schemata etc constraints on the form of hypotheses in such languages, or even order programs by generality. For instance, Prolog has unification built-in and unification is used in ILP to order programs by generality (by testing for subsumption). All this machinery would have to be implemented from scratch in an imperative language.
Although the reason that logic and functional programming languages are given more weight in IP is probably for historical reasons, because Lisp and Prolog were, for a long time, "the languages of AI".
I'm trying to remember... I think there's been some IP work on imperative languages, maybe even Python. I'll need to check my notes.
Not naive at all! One common categorisation of ILP approaches is by whether they search for programs from the most to the least general (least general is more specific), or from the least to the most general. Some approaches do a little bit of both. Approaches that search from general to specific are known as "top-down" and approaches that search from specific to general are known as "bottom-up".
The "top" and "bottom" terms refer to a lattice of generality between programs, where generality is typically measured by subsumption or entailment etc. Subsumption in particular is a syntactic relation (that implies a semantic one, entailment) so "searching" a space of logic programs ordered by subsumption means in practice that the space of programs is constructed by generalising or specialising some starting program by means of syntactic transformation according to subsumption (e.g. a first order clause can be specialised by adding literals to it: P(x):- Q(x) subsumes P(x):- Q(x), R(x). The simplest intuition is to remember that by adding more conditions to a rule we make it harder to satisfy).
A more general program entails more logical atoms and ILP algorithms are typically trained on both positive and negative example atoms of a target program, so top-down approaches begin with an over-general program that entails all the positive examples and some or all of the negative examples and specialise that program until it entails only the positive examples. Bottom-up approaches start with an over-specialised program that entails none of the positive examples and generalise it until it entails all the positive examples.
The mathematics of generalisation are at the core of ILP theory and practice. It's what sets ILP apart from statistical machine learning which is based on the mathematics of optimisation.
- First, some more recent work, mostly overviews.
1. The following is the most recent overview of the field I'm aware of:
Inductive logic programming at 30 (Cropper et al, 2020)
https://www.doc.ic.ac.uk/~shm/Papers/ilp30.pdf
2. And a slightly shorter version of the same paper that summarises new trends:
Turning 30: New Ideas in Inductive Logic Programming (Cropper et al, 2020)
https://www.ijcai.org/Proceedings/2020/0673.pdf
3. Here's a short introduction to the relatively new ILP direction of learning Answer Set Programming:
Inductive Logic Programming in Answer Set Programming (Corapi et al, 2011)
https://link.springer.com/chapter/10.1007/978-3-642-31951-8_...
4. This is an overview of Meta-Interpretive Learning (MIL), a new approach to ILP that overcomes many difficulties of earlier approaches (Full disclosure: my own work is on MIL, though not the article linked):
Meta-Interpretive Learning: achievements and challenges (Stephen Muggleton, 2017)
https://www.doc.ic.ac.uk/~shm/Papers/rulemlabs.pdf
5. And this is a (short vesion) of a paper on δILP, a neural-net based ILP system:
Learning Explanatory Rules from Noisy Data (Evans and Grefenstette, 2018)
https://www.ijcai.org/Proceedings/2018/0792.pdf
- Next, some earlier work that is still relevant:
6. This is the inaugural paper of the field, that first named it (a little heavy reading though):
Inductive Logic Programming (Stephen Muggleton, 1990)
https://www.doc.ic.ac.uk/~shm/Papers/ilp.pdf
7. Here's an early paper on predicate invention, an important technique in ILP (only recently fully realised via MIL):
Predicate Invention in ILP - an Overview (Irene Stahl, 1993)
https://link.springer.com/chapter/10.1007%2F3-540-56602-3_14...
8. And an early overview of learning recursion (and performing predicate invention) that also lists several early ILP systems:
Inductive synthesis of recursive logic programs:achievements and prospects (Flener and Yilmaz, 1999)
https://core.ac.uk/download/pdf/82810434.pdf
That should be enough to get you started. I recommend reading in the order I linked to the various articles. I tried to give links to documents that I know can be read for free.
Unfortunately most of the material on ILP is either in scholarly articles, or, where there are textbooks, they tend to be older. That sounds bad, but there has been much new work recently with several new approaches.
Let me know if you're looking for more specific information. See my signature for contact details- I'm happy to answer emails about ILP :)