Google Highly Open Participation Contest 2007-8

SymPy participates in the Google Highly Open Participation Contest 2007-8, if you are a high school student, go for it. Last chance to claim a task is January 22!

SymPy 0.5.9 released

Changes:

* Differential solvers were polished
* isympy now predefines f as a function
* Matrix printing improved
* Printing internals were documented

See Changes for more info.

Blogs

We'll use this blog for official announcements only from now on. Personal opinions will be on people's blogs, all of which are synchronised at planet.sympy.org.

We have implemented some new cool features in the Mercurial repository, so a new release will soon come out.

If you are curious, you can always find what code has been committed here:

http://hg.sympy.org/sympy

SAGE Days 6 wrap-up

SD6 is over, here are my final notes about the conference:

http://ondrejcertik.blogspot.com/2007/11/sage-days-6.html

SAGE is an exciting project. On SD6 I managed with the help of other SAGE developers to make SymPy play more nicely (after my patches are accepted, you will be able to freely mix SymPy and SAGE expressions and it will work).

SAGE Days 6

I am at SAGE Days 6, you can watch the progress on my blog.

Unicode printing

One very cool feature of SymPy, that Kirill recently committed, is a unicode printing. Start isympy and see for yourself:

$ bin/isympy 
Python 2.4.4 console for SymPy 0.5.6-hg. These commands were executed:
>>> from __future__ import division
>>> from sympy import *
>>> x, y, z = symbols('xyz')
>>> k, m, n = symbols('kmn', integer=True)
>>> Basic.set_repr_level(2)     # 2D output
>>> pprint_try_use_unicode()    # try to setup unicode pprint


In [1]: sqrt((sqrt(x+1))+1)
Out[1]: 
   ⎽⎽⎽⎽⎽⎽⎽⎽⎽⎽⎽⎽⎽⎽⎽
  ╱       ⎽⎽⎽⎽⎽⎽⎽ 
╲╱  1 + ╲╱ 1 + x  

In [2]: th, ph = Symbol("theta"), Symbol("phi")

In [3]: Integral(sin(th)/cos(ph), (th,0,pi), (ph, 0, 2*pi))
Out[3]: 
2*π π             
 ⌠  ⌠             
 ⎮  ⎮ sin(θ)      
 ⎮  ⎮ ────── dθ dφ
 ⎮  ⎮ cos(φ)      
 ⌡  ⌡             
 0  0             

More examples can be found on the wiki page PrettyPrinting.

Fredrik also played with the idea of ascii plotting. I am a huge fan of a terminal, because this is very versatile - I can use it over ssh, I can copy & paste, save the results to a file etc. I can easily script it. And the most important - I just prefer to write my programs in vim and execute them as a simple script. Any GUI, even the best one (or in the browser), is not giving me such a power. So I like the idea of making the terminal experience as pleasant as possible.

Google stopped showing sympy's webpage

This was written on October 18:

I was used to finding the sympy webpage by googling "sympy". When I did that today, the http://code.google.com/p/sympy/ isn't shown in the results (instead the sympy's freshmeat page, or sympy's Debian page is shown). I tried yahoo and msn, and sympy webpage always shows as the first or second link in there (as it used to in google).

It's interesting and it reminds me how important it is to have several search engines and do not depend on only one, so that one can switch to a different one if the first one is apparently not working correctly.

Update (October 21): Google fixed the problem and sympy's webpage is the first link again.

New SymPy release and beyond

Things have been a little quiet on planet.sympy.org, but the less we talk, the more we work. :) With 468 svn commits in September and 89 in October (till October 5) [1], we are working hard. We made a new release yesterday, where we simplified the internals of SymPy a lot and I also fixed the limits finally.

What remains is to fix some problematic series expansion (that also makes some limits fail, but the limit algorithm itself seems to be bugfree so far) and then I'll concentrate on making SymPy more SAGE friendly. SymPy will remain as a small Python library, doing one job and doing it well. But when used inside SAGE, it will know how to convert expressions to and from SAGE, so that people can use SymPy from SAGE ideally the same way as they can now use Maxima.

I think there are already things that SymPy is better than Maxima in terms of pure functionality, like pattern matching, limits (after I'll fix the series code, hopefully in the next release). There are however things, where Maxima is better, it still has a lot more features (more integrals, more differential equations, tensors) and it's faster. Except for the speed, all the other things are really easy to implement in SymPy. The main advantage of SymPy over Maxima is that it is written in Python and it is really easy to implement new features in it.

A huge deficiency of Maxima is that it uses lisp and it's own language for doing the calculations. SAGE fixes this problem (but it doesn't support all features of Maxima yet, for example substitutions of subexpressions, that is needed in the Gruntz limit algorithm), so SAGE is in fact like a platform, a common ground for all these mathematical libraries, allowing them to be used using the same interface and thus one can compare the real features (and speed), instead of some possibly good libraries being disqualified just because they use an unfortunate language/interface (for whatever historical reasons), which is great and I want SymPy to compete in the same ground as well, because only this will tell, if it is good or not and which things to improve. Competition is very healthy, it's always good to have a choice.

[1] http://groups.google.com/group/sympy-commits/about

Summer of Code summary

Summer of Code deadline has passed, now is the evaluation period, however I think I can already say, that it was a success, since all projects were finished.

Here is some statistics

ondra@fuji:~/sympy$ svn log |grep ondrej | wc -l
64
ondra@fuji:~/sympy$ svn log |grep inferno | wc -l
48
ondra@fuji:~/sympy$ svn log |grep fredrik | wc -l
39
ondra@fuji:~/sympy$ svn log |grep mattpap | wc -l
39
ondra@fuji:~/sympy$ svn log |grep lethargo | wc -l
29
ondra@fuji:~/sympy$ svn log |grep brian | wc -l
44
ondra@fuji:~/sympy$ svn log |grep pearu | wc -l
14
ondra@fuji:~/sympy$ svn log |grep Chris.Wu | wc -l
3

But one shouldn't draw too many conclusions from that, because people like me prefer to commit very often and people like Mateusz prefer to work offline and then commit when they have a bigger debugged chunk of code ready. Also the logs start with the sympy-merge branch, that was created on July 19. So everything before that (Pearu's contribution and everything that happened before the merging) is not present in the statistics. Another statistics are the Issues changes mailinglist, I did a search for all the names:

Ondrej 974
Fabian 335
Jason 227
Fredrik 151
Robert 104
Mateusz 88
Pearu 76
Brian 65
Chris 28


The above table are all issues from the beginning of the project. Fabian was the most active and then Jason is catching up recently. :)

One should have in mind the old joke though: There are three kinds of lies: lies, damned lies, and statistics.

I'd like to thank all of you for bringing SymPy this far. Basically all major things are working, now comes the phase of polishing things and reducing the number of open Issues and improving doctests, so that SymPy is very easy to use for newcomers.

Progress

The midterm evaluation is over and noone has failed, which is great. We started to work more closely on the #sympy at freenode IRC channel and SymPy has made a tremendous progress in past couple of weeks.

Our team was joined by Fredrik and Pearu, so now there are 8 regular contributors.

Fredrik wrote fast floating point arithmetic (in pure Python), faster than the python Decimal module. For example 10 000 decimal digits of Pi can be computed in 0.3s (and one million in less than 20 minutes) with SymPy - just run the pidigits.py in the examples directory. He among other things wrote a fast algorithm for calculating the gamma function, that in some cases is faster than in Mathematica.

We moved to the new core written by Pearu, that has made everything 10x faster or more. Basically, Pearu very cleverly designed everything so that no operation is made in vain. For example my little hobby relativity.py for calculating the Schwarzschild solution now calculates it in 1s (compared to more than 10s before)!


ondra@pc232:~/sympy/examples$ time python relativity.py
----------------------------------------
Christoffel symbols:
(1/2)*(FD(1,)(nu))(r)
(1/2)*(FD(1,)(nu))(r)

(1/2)*exp(-lam(r) + nu(r))*(FD(1,)(nu))(r)
(1/2)*(FD(1,)(lam))(r)
-r/exp(lam(r))
-r*sin(\theta)**2/exp(lam(r))

1/r
1/r
-sin(\theta)*cos(\theta)

1/sin(\theta)*cos(\theta)
1/sin(\theta)*cos(\theta)
1/r
1/r
----------------------------------------
Ricci tensor:
(1/2)*exp(-lam(r) + nu(r))*(FD(1, 1)(nu))(r) - 1/4*exp(-lam(r) + nu(r))*(FD(1,)(nu))(r)**2 + 1/r*exp(-lam(r) + nu(r))*(FD(1,)(nu))(r) + (1/2)*exp(-lam(r) + nu(r))*(-(FD(1,)(lam))(r) + (FD(1,)(nu))(r))*(FD(1,)(nu))(r) + (1/4)*exp(-lam(r) + nu(r))*(FD(1,)(lam))(r)*(FD(1,)(nu))(r)
-1/2*(FD(1, 1)(nu))(r) - 1/4*(FD(1,)(nu))(r)**2 + 1/r*(FD(1,)(lam))(r) + (1/4)*(FD(1,)(nu))(r)*(FD(1,)(lam))(r)
1 - 1/exp(lam(r)) + (1/2)*r/exp(lam(r))*(FD(1,)(lam))(r) - 1/2*r/exp(lam(r))*(FD(1,)(nu))(r)
sin(\theta)**2 - sin(\theta)**2/exp(lam(r)) + (1/2)*r*sin(\theta)**2/exp(lam(r))*(FD(1,)(lam))(r) - 1/2*r*sin(\theta)**2/exp(lam(r))*(FD(1,)(nu))(r)
----------------------------------------
solve the Einstein's equations:
lam(r) = -log(C1 + C2/r)
metric:
-C1 - C2/r 0 0 0
0 1/(C1 + C2/r) 0 0
0 0 r**2 0
0 0 0 r**2*sin(\theta)**2

real 0m1.092s
user 0m1.024s
sys 0m0.068s


Now we are in the phase of polishing things so that everything that used to work before works in the new core the same way (or better). Things that remain are printing and some limits are failing, everything else was already fixed.

One conclusion that can be made is, that Python on today computers is already fast enough to do something useful. Concerning the Pi, it's amusing to read some articles about people trying to calculate many digits, for example in one article, it took them a day to calculate one million digits on Pentium 90 (SymPy can do that in 20 minutes).

Also I found some page, that I cannot rediscover at the moment, that stated it took like 15 minutes to calculate the Schwarzschild with Maple on some older computer, so I am glad SymPy can do that in 1s (and actually the code is very slow, I am doing some calculations again and again, but I didn't find time yet to optimize it).

Anyway, the overall progress on SymPy looks very promising, I have no doubts that it will become a useful symbolic python library.

USA plans

Last 8 days I spent in Porto at a workshop and my plans for August are:
from August 13 to August 19 I am at Caltech at the SciPy2007 conference and from August 20 till August 25 at Berkeley at a workshop. Let me know if anyone is going to be around, we can meet and do some coding together.


Next direction in SymPy is to try to merge the sympy-research with the trunk, this should make SymPy a lot faster.

RISC and coding

I just returned from the workshop at RISC where I was with Mateusz. We met a lot of interesting people there and did some coding together, so I really enjoyed that. And we decided we should try to come together again and do some coding. I checked with Robert and he also likes the idea.

So the preliminary proposition is to meet in Prague at the end of August or the beginning of September, currently me, Mateusz and Robert should be able to make it. It's a lot more enjoyable to work together, because you can discuss things more quickly and more work is done. We will join it with sightseeing and pubs of course. Let us know, if anyone else would like to attend. We will discuss the details later. Most of you are in the US, but those in Europe could take it as a trip to Prague. :)

The topic will be anyting that interests us, from fixing bugs, to discussing features, implementing new things, creating some nice examples or nontrivial new usages of SymPy.

As to less distant future - most of the most urgent bugs are fixed, so I will release 0.4.0 tomorrow.

relativity.py is working

I finally got it working. :)

If you want to try it, go to the examples directory and:

ondra@syslik:~/sympy/examples$ python relativity.py
----------------------------------------
Christoffel symbols:
1/2*\nu'(r)
1/2*\nu'(r)

1/2*exp(\lambda(r))**(-1)*exp(\nu(r))*\nu'(r)
1/2*\lambda'(r)
-exp(\lambda(r))**(-1)*r
-exp(\lambda(r))**(-1)*sin(\theta)**2*r

r**(-1)
r**(-1)
-cos(\theta)*sin(\theta)

sin(\theta)**(-1)*cos(\theta)
sin(\theta)**(-1)*cos(\theta)
r**(-1)
r**(-1)
----------------------------------------
Ricci tensor:
-1/4*exp(\lambda(r))**(-1)*exp(\nu(r))*\lambda'(r)*\nu'(r)+1/2*exp(\lambda(r))**(-1)*exp(\nu(r))*(\nu'(r))'+exp(\lambda(r))**(-1)*exp(\nu(r))*r**(-1)*\nu'(r)+1/4*exp(\lambda(r))**(-1)*exp(\nu(r))*\nu'(r)**2
-1/4*\nu'(r)**2-1/2*(\nu'(r))'+r**(-1)*\lambda'(r)+1/4*\nu'(r)*\lambda'(r)
-exp(\lambda(r))**(-1)-1/2*exp(\lambda(r))**(-1)*r*\nu'(r)-(-1-sin(\theta)**(-2)*cos(\theta)**2)-sin(\theta)**(-2)*cos(\theta)**2+1/2*exp(\lambda(r))**(-1)*r*\lambda'(r)
1/2*exp(\lambda(r))**(-1)*sin(\theta)**2*r*\lambda'(r)-exp(\lambda(r))**(-1)*sin(\theta)**2+sin(\theta)**2-1/2*exp(\lambda(r))**(-1)*sin(\theta)**2*r*\nu'(r)
----------------------------------------
solve the Einstein's equations:
\lambda(r) = -log(C1+C2*r**(-1))
metric:
-(C1+C2*r**(-1)) 0 0 0
0 (C1+C2*r**(-1))**(-1) 0 0
0 0 r**2 0
0 0 0 r**2*sin(\theta)**2


It will start with a general, spherically symmetric metric:

gdd=Matrix((
(-exp(nu(r)),0,0,0),
(0, exp(lam(r)), 0, 0),
(0, 0, r**2, 0),
(0, 0, 0, r**2*sin(theta)**2)
))

notice the two unknown functions nu(r) and lam(r) and it calculates the Christoffel symbols, Riemann and Ricci tensors, then solves the differential (Einstein) equations for nu and lam and substitute it back to the metric to get the Schwarzschild metric (last matrix in the output from relativity.py).

If you look into the relativity.py, you will see, that when solving the equations, I am a little cheating in that I am assuming

nu(r) == -lam(r)

that can be shown by playing with the equations. That is so that I get just one differential equation for just one unknown function and then I plug it into the dsolve() and I am done.

Some more ideas what I want SymPy to do

Today I found several Mathematica packages for doing things in general relativity and quantum field theory, I updated the links here. I found xTensor, SusyMath and cadabra (this one is written in C++). Many more can be found in the Computer Physics Communications journal (CPC). Those packages are opensource, but depend on a proprietary product. And I cannot help myself, but when I look at the sources, I just find the Mathematica language completely messy.

So now I am concentrating to get the relativity.py working and then calculate something that they did. And if it turns out that it is reasonably fast (i.e. I will get the correct result in minutes, or hours) and if the source code is going to be short and nice (I hope so), I'll try to write an article about it to CPC. As a side effect, it will be a real world test for SymPy. Also I would like then to do the same calculation in SAGE, to see how it compares to SymPy. What I don't like in SAGE is that I prefer to import the library as a normal python module and I don't want to be trapped in another environment (SAGE is actually preproccessing the input a little, like converting ^ to ** etc, which I don't like at all). I think the symbolic package should be a regular standard python library. And who knows python, should easily understand how to work with that.

Right now I am struggling with the mattern matching (issue 82) in order for the dsolve() to be able to solve the differential equations that arise in relativity.py.
I would like to make it into 0.4. I think we can release a little after sympy gets into debian unstable (hopefuly in a week or two), so that we can check how it behaves etc. and fix everything for 0.4.

I know that there are still some issues, that are quite embarrassing they still don't work in SymPy, but I think more important is to "release early, release often".

some future plans

If everything goes well, SymPy should be quite well tested when the summer is over and GSoC projects will improve it in the areas: geometry, graphing, concrete mathematics, linear algebra and polynomial equations (and stuff).

For me SymPy starts to be usable enough so that I can actually calculate something interesting. I started with the Schwarzschild solution to the Einstein's equations of the general relativity. So I start with a diagonal metric tensor, and calculate the Ricci tensor using SymPy and this works remarkably well and I get the correct differential equations. Now I am trying to make the SymPy's dsolve() to solve them. And then I would like to make pictures like this one using the graphing module. The code is in examples/relativity.py

My other application is the Quantum Field Theory, where I am trying to calculate some simple cross sections, however, SymPy is pretty slow on that, but when we optimize the polynomials stuff, it should work just fine. The code is in examples/qft.py

Being able to use symbolic manipulation in the Python language allows me to be really productive. And also it's fun, because it's a pleasure, like any other programming in Python. Programming in Maple is not a pleasure for me at all.

So over the summer, I'll try to make SymPy calculate the above things because that will show, if SymPy is usable, or not. Besides that, I want to polish the web page, and documentation and installation especially. And get it to Debian and Ubuntu.

Is any of you using something else than Debian/Ubuntu/Windows? If so, would you be willing to create a package for your distribution? :)

SymPy

This blog will be used to coordinate the Google Summer of Code for the SymPy project.

I'm Ondrej Certik, the author of SymPy, and together with Fabian Seoane (who joined SymPy in February 2007), Jim Jewett and Mike Haertel we'll be mentoring 5 students: Robert, Mateusz, Brian, Chris and Jason.

Here is a picture of me in Prague:

and in Washington: