Archive for the ‘free software’ Category

PyPy 2.5 features – news

Tuesday, May 13th, 2008

Changes! Now I’m free to work “full-time” on my GSoC project, last Friday was my last day on my old job (Ikwa) :-)

Today I’ve read lots of docs and code, specially the pyparser code. I’ll try to maintain a list of features I’m currently working on here.

Last week (ok, Friday before last Friday) we went to Werneck’s cave to code. One thing I’ve made there was implement the defaultdict object in the collections module. This module is written in C originally, so I needed to port it.

The code is relatively simple and I’ve just “copied” the C implementation (of course it’s clearer and more beautiful in Python :P ). Werneck also helped me with the pyparser but we couldn’t find a solution for supporting the full Python 2.5 Grammar .

Today I’ve worked on a small Python 2.5 change, the empty base classes list syntax.

Until Python 2.5 the list of base classes could not be empty. So you couldn’t define a class like:

class A():
    pass

The right way for doing this was

class A:
    pass

Now, in Python 2.5, both ways are legal.

Last week I’ve committed a slight changed Grammar for Python 2.5 syntax support to PyPy’s 2.5-features branch (the only change I’ve made was removing the support for the new import syntax, as it crashes the parser, I need to work on that).

With part of the 2.5 Grammar already supported, to support this new class definition syntax the only change needed was in pyparser (more specifically to the AST builder).

While the build_classdef method expected 4 atoms (< class keyword >, < white space >, < class name > and < comma >) or 7 (the same 4 plus < ( >, < base classes > and < ) > before the < comma >) in Python 2.4, now in Python 2.5 it can receive 6 atoms too (< class keyword >, < white space >, < class name >, < ( >, < ) > and < comma >). The change was really simple, but it was a good exercise for me because I could apply what I’m learning in my compilers classes at school (while reading the sources) :-)

Some code (pypy/interpreter/pyparser/astbuilder.py:634):

def build_classdef(builder, nb):
    ...
    if l == 4: # class NAME:
        basenames = []
        body = atoms[3]
    elif l == 6: # class NAME():  # 2.5
        basenames = []
        body = atoms[5]
    else:
        assert l == 7
        basenames = []
        body = atoms[6]
        base = atoms[3]
    ...

I don’t think I need to explain this code, it’s very simple (of course, it’s Python! :-) ).

I’ve made some tests and it seemed to work great, but I’ve decided to compare the bytecode generated by PyPy and by CPython (both 2.4 and 2.5). While in CPython 2.4 the statement “class A: pass” produces the following bytecode:

>>> from dis import dis; c = compile("class A: pass", "/dev/null", "exec"); dis(c)
  1           0 LOAD_CONST               0 ('A')
              3 BUILD_TUPLE              0
              6 LOAD_CONST               1 ()
              9 MAKE_FUNCTION            0
             12 CALL_FUNCTION            0
             15 BUILD_CLASS
             16 STORE_NAME               0 (A)
             19 LOAD_CONST               2 (None)
             22 RETURN_VALUE

CPython 2.5 produces for both statements (the old syntax and the new one) something slight different:

>>> from dis import dis; c = compile("class A: pass", "/dev/null", "exec"); dis(c)
  1           0 LOAD_CONST               0 ('A')
              3 LOAD_CONST               3 (())
              6 LOAD_CONST               1 ()
              9 MAKE_FUNCTION            0
             12 CALL_FUNCTION            0
             15 BUILD_CLASS
             16 STORE_NAME               0 (A)
             19 LOAD_CONST               2 (None)
             22 RETURN_VALUE

The only difference between 2.4 and 2.5 is the instruction BUILD_TUPLE in 2.4 against the LOAD_CONST in 2.5. I'm not a bytecode expert, but it seems that this is an optimization, of course I may be completely wrong :P

Well, as I expected the bytecode produced by PyPy is identical the one produced by CPython 2.4, so I think I have more things to change to complete this task. ;)

But not tonight, after 14 hours of PyPy code reading and some debugging (plus Linear Programming and Computer Theory classes) I think I should sleep.

:-)

Global Python Sprint Weekend in São Paulo

Thursday, May 8th, 2008

This weekend (May 10th and 11th) will happen the Global Python Sprint Weekend. Like the last PythonBugDay our Python User Group (GruPy-SP) will meet to work together.

This time our meeting will be held at Universidade de São Paulo IME (Instituto de Matemática e Estatística) thanks to RBP and the CCSL (Free Software Competence Centre) =)

Other Python User Groups from Brazil are also meeting to work together, check the wikipage for the PythonBugDay to know if there will be a meeting near you :-) (or you can work from your home sweet home).

Python friends meeting

Monday, May 5th, 2008

Last Friday, me, Luciano Pacheco and Marcelo Honório went to Pedro Werneck‘s house (or werneck’s cave) to talk, eat and, of course, code python :-)

We wrote a small guide (portugues only, sorry… I promise i’Il translate it to English) about preparing the environment for those who want to participate of the PythonBugDay (or even for the curious who want to play with the cpython code).

Of course I’ve worked on my GSoC project implementing the new (2.5) “defaultdict” object (in the collections module), but this is subject of another post :-)

Werneck and Pacheco also worked on some issues/bugs listed in the CPython bugtracker and I’ve introduced them to PyPy (and vice-versa hehe).

After all this coding we ate Japanese food and our friend Andrews suggested a new kind of development method, Sushi-Driven Development… nice! :)

FLISOL 2008 – Python Talk

Saturday, May 3rd, 2008

Last Saturday I gave a Python talk at FLISOL 2008 in Sào Paulo. The public was students who were learning programming (C and Visual Basic), so it was a very basic talk. The event was great and so was the talk, the only problem was that I thought that the public would be programmers with some experience, but not really. Anyway, the FATEC students seemed to like Python and some of then came to talk to me after the event to know more about the language and to ask where they could find more information!

I’ll upload my presentation somewhere.

I’ve helped to install GNU/Linux (Ubuntu) on a FATEC lab, people there were very nice!

Well, I felt like my “mission” was accomplished :-)

FLISOL 2008 in São Paulo – Python Talk

Friday, April 25th, 2008

I’ll give a talk about Python in FLISOL 2008 São Paulo. It’s scheduled to start at 15h00. It’s a basic talk for people who code in other languages and don’t know much about Python.

Take a look here for more information about FLISOL.

FLISOL 2008 in São Paulo

Thursday, April 24th, 2008

Time to install free software :-)

FLISOL, the Latin-America Free Software Installfest (Festival Latinoamericano de Instalação de SOftware Livre) will happend this Saturday, 04/26, simultaneously in more than 20 countries in Latin America.

In São Paulo it will be held at FATEC Zona Sul, R. Frederico Grotte, 353 – Jd São Luís.

Our schedule:

Installfest

09:00 – 18:00

Talks:

10:00 – 12:00
Free Software and Security
Rodrigo Montoro

Workshops

09:00 – 11:00
Desenvolvendo em C com GTK
Alex Marcio Ribeiro Nunes

13:00 – 16:00
Shell Script
Edegar C. Berlezzi

More information at http://www.flisol.org.br (portuguese) and http://installfest.info/ (spanish and portuguese).

If you live in São Paulo, I hope to see you there :-)

PyPy 2.5 features: __hash__ behavior

Thursday, April 24th, 2008

One small change in Python 2.5 is the __hash__ method behavior. In earlier versions the __hash__ method was supposed to return a normal integer always, but now due to changes in the id() builtin function the __hash__ method behavior was changed and it can return a long or an int.

This was easy to change in PyPy, the hash() builtin function implementation is part of the ObjectSpace as almost all the builtin functions and types.

It’s in pypy/objspace/descroperation.py:

def hash(space, w_obj):
        w_hash = space.lookup(w_obj, '__hash__')
        if w_hash is None:
            if space.lookup(w_obj, '__eq__') is not None or \
               space.lookup(w_obj, '__cmp__') is not None:
                raise OperationError(space.w_TypeError,
                                     space.wrap("unhashable type"))
            return default_identity_hash(space, w_obj)
        w_result = space.get_and_call_function(w_hash, w_obj)
        if space.is_true(space.isinstance(w_result, space.w_int)):
            return w_result
        else:
            raise OperationError(space.w_TypeError,
                     space.wrap("__hash__() should return an int or long"))

One don’t even need to know PyPy well to understand that the lines

if space.is_true(space.isinstance(w_result, space.w_int)):
    return w_result
else:
    raise OperationError(space.w_TypeError,
             space.wrap("__hash__() should return an int or long"))

are responsible for raising a TypeError when the returning value from __hash__ is not an integer.

So just changing the condition

space.is_true(space.isinstance(w_result, space.w_int)):

to

(space.is_true(space.isinstance(w_result, space.w_int)) or
 space.is_true(space.isinstance(w_result, space.w_long))):

will solve this, I hope!

But that’s not all the code needed, I promised Carl to write tests for everything, but as I like to use TDD (test-driven development) it was already done.

Oh, and should I mention that I love PyPy’s documentation? Thanks guys!

Supporting Python 2.5 features in PyPy – GSoC 2008

Tuesday, April 22nd, 2008

As you probably know my GSoC proposal to work on PyPy interpreter, supporting the Python 2.5 features, has been accepted. This post is a short explanation of my Google Summer of Code project.

PyPy

The PyPy project aims at producing a flexible and fast Python implementation. The guiding idea is to translate a Python-level description of the Python language itself to lower level languages. Rumors have it that the secret goal is being faster-than-C which is nonsense, isn’t it?

PyPy is divided into the python interpreter and the translator framework. The interpreter is written in RPython (a subset of Python) and the main goal is to provide an easy-to-hack (for python programmers ;) ) Python interpreter.

PyPy’s interpreter currently supports only a few features introduced in Python 2.5, like __index__ (for slicing) and some part of the ‘with’ statement. Also, the 2.5 changes to the language and to the standard library have not been ported yet (although some have been, like the ctypes).

Goal

This proposal idea is to bring Python 2.5 features and changes to PyPy interpreter, this includes porting the modules and writing code for the features it self.

Here are some of the features I’ll work on:

  • PEP 308: Conditional Expressions[*]
  • 328: Absolute and Relative Imports
  • PEP 342: New Generator Features
  • PEP 343: The with statement[*]
  • PEP 352: Exceptions as New-Style Classes

[*] part of this features are already supported by PyPy

Benefits

With support to 2.5 features and changes more the PyPy interpreter will be up-to-date so people who wants to use this features can be happy using PyPy. This probably means more people using PyPy (i hope!) :-)

Why this project?

The first reason I’ve chosen the PyPy project is that I really like their ideas and goals. I was looking for collaborating to it in some way for a long time, but it took me some time to read the docs and to really understand the project.

As I like PyPy ideas and I’m starting to plan my graduation project/bachelor’s thesis, my work on PyPy interpreter will be a big part of it. Maybe I can explain my ideas in another post :-)

I would like to thank Google (and the SoC folks), Python Software Foundation, PyPy and specially to Carl Friedrich Bolz* who will be my GSoC mentor and Leonardo Santagada who helped me to understand PyPy and to write my application.

* morepypy is a blog where PyPy developers talk to the community, it’s not Carl’s personal blog.

Thanks to my friends from Python Brasil community too.

Approved in GSoC

Monday, April 21st, 2008

As mentioned before I’ve applied to GSoC to work on PyPy for the Python Software Foundation.

Today I’m very happy to say that my project got approved and I will work on PyPy interpreter the next three months. It means that a lot of PyPy posts are coming, and the first one will be a introduction to the project, what is PyPy, the interpreter and the translation framework :)

Stay tuned if you are interested in PyPy! Congratulations to everybody that have applied to GSoC this year =)

FISL 9.0 – 1. São Paulo -> Porto Alegre -> FISL!

Sunday, April 20th, 2008

There isn’t words to describe how I’m happy after this 3 days in Porto Alegre, attending to FISL. I’ll try to make a report of what happened, but it’s so much things that I’ll post it in parts.

So this is part one, and it starts at the São Paulo international airport (at Guarulhos-SP) on April 17th. Destination: Porto Alegre, Rio Grande do Sul!

Airplane

With me were my two friends Marcelo Honório and David Kwast. Our flight was scheduled to leave at 4h30am. Well, it left exactly 4h25am, after one hour flying they announced that the Porto Alegre airport was closed due to bad weather conditions, and worse, all near airports were closed so went back to São Paulo international airport. We had to wait until 11am, when the Porto Alegre aiport opened again and all flights that were late started to leave.

After that we finally did it and went to PUC (the university where FISL happened) but what happened there is history for the next post, the first day report :-)