Author of Symbolica here: Symbolica is used in physics calculations to do arithmetic on rational polynomials that are hundreds of megabytes long.
For my physics research I have worked with expressions that was just shy of a terabyte long and had > 100M terms. The way that works is that you stream terms from disk, perform manipulations on them and write them to disk again. Using a mergesort, terms that add up can be identified by sorting them to be adjacent.
This is fascinating. What is the real-world application of these polynomials? I mean, what technical-scientific problems can only be solved with such large objects? I love the thought of "if I don't solve this enormous polynomial, this solar panel won't have very good efficiency" or something like that.
These polynomials appear when computing Feynman diagrams, which are used to make predictions for the Large Hadron Collider. The collider can measure collisions with such astonishing precision (<1% error) that predictions of the same order are also needed. The more precise you want to be, the more terms in a series approximation of the mathematical description of the collision you need to compute. For example, I computed the fifth-order approximation of the QCD beta function, which governs how intense the strong force affects matter. This takes 5 days of symbolic manipulations (pattern matching, substitutions, rational polynomial arithmetic, etc) on 32 cores.
The large polynomials appear in the middle of the computation, often referred to as intermediate expression swell. This also happens when you do a Gaussian elimination or compute greatest common divisors: the final result will be small, but intermediately, the expressions can get large.
Is there a document/book you can recommend that includes a simplified example/tutorial of this computation process from beginning to end? (Or, what are the right keywords to search for such a thing?)
I'm looking for something like: here's the particle interaction we will work on, this is a very simple Feynman diagram, and here's the simplified data the LHC gave us about it, here's the resulting equation from which we'll derive a series, etc.
Not looking for how to program it, but actually for seeing the problem structure, and the solution design from beginning to end. (Familiar with high level physics concepts, and comfortable with any math).
You can even make a toy-problem with large polynomials yourself.
Just make a Taylor approximation of some e.g. transcendental function like sin(x) around some point, and use more and more terms to get a higher precision.
I used polynomials and specifically special symmetric polynomials extensively during my Phd research (mathematical physics).
For instance symmetric polynomials (x_1^2 + x_2^2 + ...) can describe the state of a system of particles where exchanging any 2 particles does not change the system at all (exchange x_1 and x_2 in the previous expression, same polynomial = same system & state).
If you have a system of equation that you can solve exactly with special polynomials, you can approximate real world system governed by similar equations by using your special polynomials as a starting point and adding a correction to your solution.
There's so much to say about polynomials, but I'll leave you with a basic example that shows how multiplying infinite polynomials allow you to count the number of ways there are to hand you back your change at the till:
The basic units of change are 0.01$, 0.05$, 0.10$, 0.25$, 1$, 5$, 10$, ...
For example, you can always give exact change back with only 0.01$.
So the set of all the different amount of change you can produce with 0.01$ is given by the exponents in the following
There are thus 4 ways of handing back exactly 10 cents.
So for any amount, you take the following:
product(c in (0.01, 0.05, 0.10, 0.25,...) (sum_n (q^c)^n)
= sum_(a >= 0.01) [Number Of Way To Give Back Change for amount `a`] * q^(a)
So that would be the "generating series" of the number of ways to hand back change.
In this context, polynomials bridge the gaps between combinatorics and analytical computation.
I forgot. You can do SICP over the web without installing an Scheme interpreter.
The web has a complete enough Scheme interpreter to do and eval the whole book exercises.
While I agree with this attitude towards obscure acronyms, SICP is widely known among those who voluntary enter a site called Hacker News, and it's easily searchable too.
I can remember the days! I suppose as problems grow in size, it's not too surprising that older methods of coping with what seemed like lots of data are still applicable.
Indeed, it's hard to me to imagine such an expression without any of the following qualities:
- immediately converges to zero
- immediately heads to infinity
- is dominated by only a few terms (thus obviating the needs for the other X million terms)
It sounds like the problem is that it’s dominated by a few terms, but finding them is tricky. If you have 1.0 x + 2.0 x - 3.0 x - 5/4 x + x - x + x - x … (etc for a few GB) and then quadratic terms, it’ll take work to figure out whether the dominant term is linear or quadratic. Cancelling out the potentially important intermediate terms (especially in higher dimensions) sounds like a mess.
Could you share the equation/code you worked on?
I can't imagine a mathematical object being 84KB long, that's insanely huge