Archive for the 'python' Category

S3 demo using Django

I have been amazed at how reliable storage is these days with services like . The S3 doesn’t mention using python and django so I thought I’d implement the demo in django. It really is very simple to take advantage of S3 in your django apps, as Adrian shows .

Grab the S3 libary from .

My view looks like this:

BUCKET_NAME = 's3demo'

def upload_and_view(request):
    conn = S3.AWSAuthConnection('your access key', 'your secret key')

    if request.method == 'POST':
        post_data = request.POST.copy()
        post_data.update(request.FILES)
        filedata = post_data['uploaded_file']['content']
        filename = post_data['uploaded_file']['filename']
        content_type = mimetypes.guess_type(filename)[0]
        if not content_type:
            content_type = 'text/plain'

        conn.put(BUCKET_NAME, filename, S3.S3Object(filedata),
                    {'x-amz-acl': 'public-read', 'Content-Type': content_type})

    bucket = conn.list_bucket(BUCKET_NAME)
    bucket_entries = bucket.entries

    return render_to_response('s3demo/upload_and_view.html', {'entries': bucket_entries})

The template is a simple web form. bucket_entries contains a list of files in the S3 bucket. Since I’ve set the ACL to be public readable I can print a list of urls like so:

{% for entry in entries %}
<a href="http://s3demo.s3.amazonaws.com/{{ entry.key }}">{{ entry.key }}</a><br />
{% endfor %}

Great stuff!

Django version issues on upgrade to 0.96

Seeing as that there will be coming in future releases of Django I installed 0.96 in my environment today. I simply ran setup.py and the libraries were put in the appropriate site-packages directory. The new version seemed to get picked up:

>>> import django
>>> django.VERSION
(0, 96, None)

The mention an additional view for adding users in admin. It didn’t seem to exist for me. It seemed like the old 0.95.1 library I was using was being picked up in this instance, so I removed the egg package for it and all seemed well.

It turns out that the problem was that the older 0.95.x packages used where the new one doesn’t. The new install didn’t create an egg package and the reference to the old one still existed in the easy-install.pth file in my site-packages:

$ cat easy-install.pth
import sys; sys.__plen = len(sys.path)
./setuptools-0.6c5-py2.5.egg
./Django-0.95.1-py2.5.egg
./vobject-0.4.8-py2.5.egg
./python_dateutil-1.1-py2.5.egg
./flup-0.5-py2.5.egg
./MySQL_python-1.2.2-py2.5-macosx-10.3-i386.egg
import sys; new=sys.path[sys.__plen:]; del sys.path[sys.__plen:]; p=getattr(sys,’__egginsert’,0); sys.path[p:p]=new; sys.__egginsert = p+len(new)

Simply removing the reference to Django-0.95.1-py2.5.egg will also do the trick.

Django and database neutrality

I use the framework on a day to day basis in my job and absolutely love it, especially the database model. The idea is that you generate a “model” for your database and use it using a common API, no matter the backend you’re using. Key supported databases are MySQL, PostgreSQL and sqlite. This works well for the most part.

One key area where it doesn’t is due to the fact that sqlite stores data no matter how it looks. Problems can arise, for example, if you expect a datetime field to be a python datetime object when you pull it out. Believe it or not, this may not always be the case with sqlite.

Another relates to the schema evolution branch that was a google summer of code project. I had an sqlite database that I wanted to update to my latest model, so I thought I’d check out the . With my model updated, I pointed my settings to the old sqlite file and ran manage.py sqlevolve and immediately got the following exception:

File "/path/to/django/core/management.py", line 583, in get_sql_evolution_check_for_changed_field_flags
if column_flags['allow_null']!=f.null or \
KeyError: 'allow_null'

from the function I see the :

column_flags = introspection.get_known_column_flags(cursor, db_table, cf)

Looking at the introspection library for sqlite I see:

def get_known_column_flags( cursor, table_name, column_name ):
    cursor.execute("PRAGMA table_info(%s)" % quote_name(table_name))
    dict = {}
    for row in cursor.fetchall():
        if row[1] == column_name:

            # maxlength check goes here
            if row[2][0:7]=='varchar':
                dict['maxlength'] = row[2][8:len(row[2])-1]

            # default flag check goes here
            #if row[2]=='YES': dict['allow_null'] = True
            #else: dict['allow_null'] = False

            # primary/foreign/unique key flag check goes here
            #if row[3]=='PRI': dict['primary_key'] = True
            #else: dict['primary_key'] = False
            #if row[3]=='FOR': dict['foreign_key'] = True
            #else: dict['foreign_key'] = False
            #if row[3]=='UNI': dict['unique'] = True
            #else: dict['unique'] = False

            # default value check goes here
            # if row[4]=='NULL': dict['default'] = None
            # else: dict['default'] = row[4]
            #dict['default'] = row[4]

    print table_name, column_name, dict
    return dict

Note the commented out code! This, along with the introspection code:

    for row in cursor.fetchall():
        if row[0] == column_name:

            # maxlength check goes here
            if row[1][0:17]=='character varying':
                dict['maxlength'] = row[1][18:len(row[1])-1]

            # null flag check goes here
            dict['allow_null'] = not row[3]

where the dictionary value in question may or may not get assigned.

Looks like django isn’t quite as database neutral as I’d hoped.