Summer of Code summary

Summer of Code deadline has passed, now is the evaluation period, however I think I can already say, that it was a success, since all projects were finished.

Here is some statistics

ondra@fuji:~/sympy$ svn log |grep ondrej | wc -l
64
ondra@fuji:~/sympy$ svn log |grep inferno | wc -l
48
ondra@fuji:~/sympy$ svn log |grep fredrik | wc -l
39
ondra@fuji:~/sympy$ svn log |grep mattpap | wc -l
39
ondra@fuji:~/sympy$ svn log |grep lethargo | wc -l
29
ondra@fuji:~/sympy$ svn log |grep brian | wc -l
44
ondra@fuji:~/sympy$ svn log |grep pearu | wc -l
14
ondra@fuji:~/sympy$ svn log |grep Chris.Wu | wc -l
3

But one shouldn't draw too many conclusions from that, because people like me prefer to commit very often and people like Mateusz prefer to work offline and then commit when they have a bigger debugged chunk of code ready. Also the logs start with the sympy-merge branch, that was created on July 19. So everything before that (Pearu's contribution and everything that happened before the merging) is not present in the statistics. Another statistics are the Issues changes mailinglist, I did a search for all the names:

Ondrej 974
Fabian 335
Jason 227
Fredrik 151
Robert 104
Mateusz 88
Pearu 76
Brian 65
Chris 28


The above table are all issues from the beginning of the project. Fabian was the most active and then Jason is catching up recently. :)

One should have in mind the old joke though: There are three kinds of lies: lies, damned lies, and statistics.

I'd like to thank all of you for bringing SymPy this far. Basically all major things are working, now comes the phase of polishing things and reducing the number of open Issues and improving doctests, so that SymPy is very easy to use for newcomers.

Progress

The midterm evaluation is over and noone has failed, which is great. We started to work more closely on the #sympy at freenode IRC channel and SymPy has made a tremendous progress in past couple of weeks.

Our team was joined by Fredrik and Pearu, so now there are 8 regular contributors.

Fredrik wrote fast floating point arithmetic (in pure Python), faster than the python Decimal module. For example 10 000 decimal digits of Pi can be computed in 0.3s (and one million in less than 20 minutes) with SymPy - just run the pidigits.py in the examples directory. He among other things wrote a fast algorithm for calculating the gamma function, that in some cases is faster than in Mathematica.

We moved to the new core written by Pearu, that has made everything 10x faster or more. Basically, Pearu very cleverly designed everything so that no operation is made in vain. For example my little hobby relativity.py for calculating the Schwarzschild solution now calculates it in 1s (compared to more than 10s before)!


ondra@pc232:~/sympy/examples$ time python relativity.py
----------------------------------------
Christoffel symbols:
(1/2)*(FD(1,)(nu))(r)
(1/2)*(FD(1,)(nu))(r)

(1/2)*exp(-lam(r) + nu(r))*(FD(1,)(nu))(r)
(1/2)*(FD(1,)(lam))(r)
-r/exp(lam(r))
-r*sin(\theta)**2/exp(lam(r))

1/r
1/r
-sin(\theta)*cos(\theta)

1/sin(\theta)*cos(\theta)
1/sin(\theta)*cos(\theta)
1/r
1/r
----------------------------------------
Ricci tensor:
(1/2)*exp(-lam(r) + nu(r))*(FD(1, 1)(nu))(r) - 1/4*exp(-lam(r) + nu(r))*(FD(1,)(nu))(r)**2 + 1/r*exp(-lam(r) + nu(r))*(FD(1,)(nu))(r) + (1/2)*exp(-lam(r) + nu(r))*(-(FD(1,)(lam))(r) + (FD(1,)(nu))(r))*(FD(1,)(nu))(r) + (1/4)*exp(-lam(r) + nu(r))*(FD(1,)(lam))(r)*(FD(1,)(nu))(r)
-1/2*(FD(1, 1)(nu))(r) - 1/4*(FD(1,)(nu))(r)**2 + 1/r*(FD(1,)(lam))(r) + (1/4)*(FD(1,)(nu))(r)*(FD(1,)(lam))(r)
1 - 1/exp(lam(r)) + (1/2)*r/exp(lam(r))*(FD(1,)(lam))(r) - 1/2*r/exp(lam(r))*(FD(1,)(nu))(r)
sin(\theta)**2 - sin(\theta)**2/exp(lam(r)) + (1/2)*r*sin(\theta)**2/exp(lam(r))*(FD(1,)(lam))(r) - 1/2*r*sin(\theta)**2/exp(lam(r))*(FD(1,)(nu))(r)
----------------------------------------
solve the Einstein's equations:
lam(r) = -log(C1 + C2/r)
metric:
-C1 - C2/r 0 0 0
0 1/(C1 + C2/r) 0 0
0 0 r**2 0
0 0 0 r**2*sin(\theta)**2

real 0m1.092s
user 0m1.024s
sys 0m0.068s


Now we are in the phase of polishing things so that everything that used to work before works in the new core the same way (or better). Things that remain are printing and some limits are failing, everything else was already fixed.

One conclusion that can be made is, that Python on today computers is already fast enough to do something useful. Concerning the Pi, it's amusing to read some articles about people trying to calculate many digits, for example in one article, it took them a day to calculate one million digits on Pentium 90 (SymPy can do that in 20 minutes).

Also I found some page, that I cannot rediscover at the moment, that stated it took like 15 minutes to calculate the Schwarzschild with Maple on some older computer, so I am glad SymPy can do that in 1s (and actually the code is very slow, I am doing some calculations again and again, but I didn't find time yet to optimize it).

Anyway, the overall progress on SymPy looks very promising, I have no doubts that it will become a useful symbolic python library.

USA plans

Last 8 days I spent in Porto at a workshop and my plans for August are:
from August 13 to August 19 I am at Caltech at the SciPy2007 conference and from August 20 till August 25 at Berkeley at a workshop. Let me know if anyone is going to be around, we can meet and do some coding together.


Next direction in SymPy is to try to merge the sympy-research with the trunk, this should make SymPy a lot faster.

RISC and coding

I just returned from the workshop at RISC where I was with Mateusz. We met a lot of interesting people there and did some coding together, so I really enjoyed that. And we decided we should try to come together again and do some coding. I checked with Robert and he also likes the idea.

So the preliminary proposition is to meet in Prague at the end of August or the beginning of September, currently me, Mateusz and Robert should be able to make it. It's a lot more enjoyable to work together, because you can discuss things more quickly and more work is done. We will join it with sightseeing and pubs of course. Let us know, if anyone else would like to attend. We will discuss the details later. Most of you are in the US, but those in Europe could take it as a trip to Prague. :)

The topic will be anyting that interests us, from fixing bugs, to discussing features, implementing new things, creating some nice examples or nontrivial new usages of SymPy.

As to less distant future - most of the most urgent bugs are fixed, so I will release 0.4.0 tomorrow.

relativity.py is working

I finally got it working. :)

If you want to try it, go to the examples directory and:

ondra@syslik:~/sympy/examples$ python relativity.py
----------------------------------------
Christoffel symbols:
1/2*\nu'(r)
1/2*\nu'(r)

1/2*exp(\lambda(r))**(-1)*exp(\nu(r))*\nu'(r)
1/2*\lambda'(r)
-exp(\lambda(r))**(-1)*r
-exp(\lambda(r))**(-1)*sin(\theta)**2*r

r**(-1)
r**(-1)
-cos(\theta)*sin(\theta)

sin(\theta)**(-1)*cos(\theta)
sin(\theta)**(-1)*cos(\theta)
r**(-1)
r**(-1)
----------------------------------------
Ricci tensor:
-1/4*exp(\lambda(r))**(-1)*exp(\nu(r))*\lambda'(r)*\nu'(r)+1/2*exp(\lambda(r))**(-1)*exp(\nu(r))*(\nu'(r))'+exp(\lambda(r))**(-1)*exp(\nu(r))*r**(-1)*\nu'(r)+1/4*exp(\lambda(r))**(-1)*exp(\nu(r))*\nu'(r)**2
-1/4*\nu'(r)**2-1/2*(\nu'(r))'+r**(-1)*\lambda'(r)+1/4*\nu'(r)*\lambda'(r)
-exp(\lambda(r))**(-1)-1/2*exp(\lambda(r))**(-1)*r*\nu'(r)-(-1-sin(\theta)**(-2)*cos(\theta)**2)-sin(\theta)**(-2)*cos(\theta)**2+1/2*exp(\lambda(r))**(-1)*r*\lambda'(r)
1/2*exp(\lambda(r))**(-1)*sin(\theta)**2*r*\lambda'(r)-exp(\lambda(r))**(-1)*sin(\theta)**2+sin(\theta)**2-1/2*exp(\lambda(r))**(-1)*sin(\theta)**2*r*\nu'(r)
----------------------------------------
solve the Einstein's equations:
\lambda(r) = -log(C1+C2*r**(-1))
metric:
-(C1+C2*r**(-1)) 0 0 0
0 (C1+C2*r**(-1))**(-1) 0 0
0 0 r**2 0
0 0 0 r**2*sin(\theta)**2


It will start with a general, spherically symmetric metric:

gdd=Matrix((
(-exp(nu(r)),0,0,0),
(0, exp(lam(r)), 0, 0),
(0, 0, r**2, 0),
(0, 0, 0, r**2*sin(theta)**2)
))

notice the two unknown functions nu(r) and lam(r) and it calculates the Christoffel symbols, Riemann and Ricci tensors, then solves the differential (Einstein) equations for nu and lam and substitute it back to the metric to get the Schwarzschild metric (last matrix in the output from relativity.py).

If you look into the relativity.py, you will see, that when solving the equations, I am a little cheating in that I am assuming

nu(r) == -lam(r)

that can be shown by playing with the equations. That is so that I get just one differential equation for just one unknown function and then I plug it into the dsolve() and I am done.

Some more ideas what I want SymPy to do

Today I found several Mathematica packages for doing things in general relativity and quantum field theory, I updated the links here. I found xTensor, SusyMath and cadabra (this one is written in C++). Many more can be found in the Computer Physics Communications journal (CPC). Those packages are opensource, but depend on a proprietary product. And I cannot help myself, but when I look at the sources, I just find the Mathematica language completely messy.

So now I am concentrating to get the relativity.py working and then calculate something that they did. And if it turns out that it is reasonably fast (i.e. I will get the correct result in minutes, or hours) and if the source code is going to be short and nice (I hope so), I'll try to write an article about it to CPC. As a side effect, it will be a real world test for SymPy. Also I would like then to do the same calculation in SAGE, to see how it compares to SymPy. What I don't like in SAGE is that I prefer to import the library as a normal python module and I don't want to be trapped in another environment (SAGE is actually preproccessing the input a little, like converting ^ to ** etc, which I don't like at all). I think the symbolic package should be a regular standard python library. And who knows python, should easily understand how to work with that.

Right now I am struggling with the mattern matching (issue 82) in order for the dsolve() to be able to solve the differential equations that arise in relativity.py.
I would like to make it into 0.4. I think we can release a little after sympy gets into debian unstable (hopefuly in a week or two), so that we can check how it behaves etc. and fix everything for 0.4.

I know that there are still some issues, that are quite embarrassing they still don't work in SymPy, but I think more important is to "release early, release often".

some future plans

If everything goes well, SymPy should be quite well tested when the summer is over and GSoC projects will improve it in the areas: geometry, graphing, concrete mathematics, linear algebra and polynomial equations (and stuff).

For me SymPy starts to be usable enough so that I can actually calculate something interesting. I started with the Schwarzschild solution to the Einstein's equations of the general relativity. So I start with a diagonal metric tensor, and calculate the Ricci tensor using SymPy and this works remarkably well and I get the correct differential equations. Now I am trying to make the SymPy's dsolve() to solve them. And then I would like to make pictures like this one using the graphing module. The code is in examples/relativity.py

My other application is the Quantum Field Theory, where I am trying to calculate some simple cross sections, however, SymPy is pretty slow on that, but when we optimize the polynomials stuff, it should work just fine. The code is in examples/qft.py

Being able to use symbolic manipulation in the Python language allows me to be really productive. And also it's fun, because it's a pleasure, like any other programming in Python. Programming in Maple is not a pleasure for me at all.

So over the summer, I'll try to make SymPy calculate the above things because that will show, if SymPy is usable, or not. Besides that, I want to polish the web page, and documentation and installation especially. And get it to Debian and Ubuntu.

Is any of you using something else than Debian/Ubuntu/Windows? If so, would you be willing to create a package for your distribution? :)