Heroku django new database settings

Heroku just announced that injection of database settings, meaning:

heroku run cat your_project/settings.py

Outputs something like this
import os
import sys
import urlparse

# Register database schemes in URLs.
urlparse.uses_netloc.append('postgres')
urlparse.uses_netloc.append('mysql')

try:

    # Check to make sure DATABASES is set in settings.py file.
    # If not default to {}

    if 'DATABASES' not in locals():
        DATABASES = {}

    if 'DATABASE_URL' in os.environ:
        url = urlparse.urlparse(os.environ['DATABASE_URL'])

        # Ensure default database exists.
        DATABASES['default'] = DATABASES.get('default', {})

        # Update with environment configuration.
        DATABASES['default'].update({
            'NAME': url.path[1:],
            'USER': url.username,
            'PASSWORD': url.password,
            'HOST': url.hostname,
            'PORT': url.port,
        })
        if url.scheme == 'postgres':
            DATABASES['default']['ENGINE'] = 'django.db.backends.postgresql_psycopg2'

        if url.scheme == 'mysql':
            DATABASES['default']['ENGINE'] = 'django.db.backends.mysql'
except Exception:
    print 'Unexpected error:', sys.exc_info()

Will cease to happen, heroku won't be adding the database settings, luckly they provided a simple solution. here are the steps i followed for my django app.
Note: i recommend you try this first in a copy of your app, you are using a staging copy right?
Assumptions/Requirements

• heroku gem past 2.18.1. version(this is important, upgrade your gem now)
  gem install heroku
  otherwise you will get a
  No superclass method 'app'
  error
•  Django 1.3 or higher
• You will need a custom buildpack thats updated to account the user_env_compile addon, add it to your app like this
  heroku config:add BUILDPACK_URL=git@github.com:heroku/heroku-buildpack-python.git --app your_app_name
  Or if you need M2Crypto support you will need my custom buildpack that is just a mix of the standard python pack with a guybowdens pack that enabled the support for SWIG needed by M2Crypto.
  heroku config:add BUILDPACK_URL=git://github.com/grillermo/heroku-buildpack-python.git --app your_app_name

Ok now you will have to add a this line to the bottom of your requirements.txt
dj-database-url==0.2.1
Then add the heroku plugin user_env_compile to your app

heroku labs:enable user_env_compile --app your_app_name

And then add the ENV variable to disable injection

heroku config:add DISABLE_INJECTION=1

Now add these lines to your settings.py

import dj_database_url
DATABASES = {'default': dj_database_url.config(default='postgres://localhost')} 

Now make a commit and push to heroku

git commit -m 'disable heroku injection of database settings, manually add them'

git push origin master

You can check is not injecting the settings anymore with:

heroku run cat your_project/settings.py

And at the end of your settings there should not be anything you did not put.
  
References:

• The heroku news article 
• My custom heroku buildpack 

recursive s3 uploading from python

I recently had the need to upload a directory with many files to s3 so i wrote a small function to to this for me, this is not optimized at all, as it will only send one file at the time, and it doesn't take into account the 5gb limitation on s3 uploads, but is the simplest that could get the job done.
This function traverse through a folder and uploads the files with the correct key so they show up inside their respective 'folders' on the s3 management panel, i say 'folders' because there is no such concept on s3 only, keys that can be optionally prepended by slashes so '/home/static/files/lol.py' is the key of the 'lol.py' file, and thats how you request it.
This function checks if the key already exists only uploading missing keys.
So here it is, in all its slow glory.
Also, sometimes amazon fails to find a key its supposed to be there, so i had to add a little try, except to handle these cases, i had to upload ~4000 files and only had problems with 10, so i guess this is the 99.9% availability amazon claims to have with s3 files.

import os
import boto
from boto.s3.key import Key

failed = open('failers','w')
def uploadResultToS3(awsid,awskey,bucket,source_folder):
    c = boto.connect_s3(awsid,awskey)
    b = c.get_bucket(bucket)
    k = Key(b)
    for path,dir,files in os.walk(source_folder):
        for file in files:
            relpath = os.path.relpath(os.path.join(path,file))
            if not b.get_key(relpath):
                print 'sending...',relpath
                k.key = relpath
                k.set_contents_from_filename(relpath)
                try:
                    k.set_acl('public-read')
                except:
                    failed.write(relpath+', ') 
    failed.close()

You obviously need the boto library, get it first

pip install boto

fixing @font-face problem in firefox free

So you open your brand new webpage on your browser, you are using @font-face on your css you host your own fonts its all good, its works awesome.

Suddenly you start to grow and you need a cdn such as amazon's S3 or google storage, to host your static files for you, so you naturally expect to host your fonts there, WRONG! Firefox will not load @font-face if they are not hosted on the same domain and your-domain.com is not the same as https://s3.amazonaws.com/bandtastic_heroku/

I found a couple of solutions, one involved getting your own google-fonts serivce with a sinatra app, it bugged for me i couldnt get it to work on heroku.
Don't waste more time.
Go to http://typefront.com/fonts register and get your free font hosted there, they will maintain a list of available domains and firefox, and older ie browsers will happily load your font with a simple css @font-face rule.

You depend on them, but such is life.

Testing PIL for jpeg support

If you are unsure that your PIL instalation found libjpeg and thus you have jpeg support in django or any app that uses PIL, you can enter the django shell
./manage.py shell

>>>from PIL import Image
>>>im = Image.open("bride.jpg")
>>>im.rotate(45).show()

<PIL.Image.Image image mode=RGB size=960x293 at 0x2D2D560>

The last line is the confirmation that it worked, you have jpeg support. 

heroku django celery redistogo

If you want to use redistogo as a backend and broker for django celery

These are the relevant lines on settings.py

import os
if [(i,k) for i,k in os.environ.items() if 'heroku' in k]: #detect heroku somehow reliably
    REDIS_DB = 0
    REDIS_CONNECT_RETRY = True
    CELERY_RESULT_BACKEND = 'redis'
    CELERY_REDIS_PORT = 9104
    CELERY_REDIS_PASSWORD = '93a4b2c92dc0e07f0f9c82e806108fc1'
    CELERY_REDIS_HOST = 'gar.redistogo.com'
    BROKER_URL = os.getenv('REDISTOGO_URL', 'redis://localhost')
    BROKER_TRANSPORT = 'redis'
    BROKER_BACKEND = 'redis'
    BROKER_PORT = CELERY_REDIS_PORT
    BROKER_HOST = CELERY_REDIS_HOST
    BROKER_VHOST = '/'
    BROKER_PASSWORD = CELERY_REDIS_PASSWORD
    CELERYD_CONCURRENCY = 1
    CELERY_RESULT_PORT = 9104
    CELERY_TASK_RESULT_EXPIRES = 60 * 10

 

import djcelery
djcelery.setup_loader() #not sure if needed

Dont forget to add your celery worker on the Procfile

web: bin/gunicorn_django --workers=4 --bind=0.0.0.0:$PORT name_of_your_project/settings.py;
worker: bin/python name_of_your_project/manage.py celeryd -E -B --loglevel=INFO

And turn in it on
heroku ps:scale worker=1
And then you can test it if it works

heroku run name_of_your_project/manage.py shell

from your_app.tasks import task_name

result = task_name.apply_async(args=[arg1,arg2])

result.wait()

Please leave a comment if you have problems with this, i'm more than glad to help because i suffered this. Doesnt matter if its a year or many months later leave a comment i'll be here.

Install m2crypt in heroku

If you ever had the need to install m2crypt in a heroku app, but you keep getting

       unable to execute swig: No such file or directory
      
       error: command 'swig' failed with exit status 1

when doing a normal push with your requirements.txt on place.
I got a solution for you, you just need to use a thing heroku calls buildpacks, these are bash scripts that heroku uses to setup your application, dependencies and all.
So you need a custom buildpack that preinstalls m2crypto for you.

For an existing heroku app
heroku config:add BUILDPACK_URL=https://github.com/guybowden/heroku-buildpack-python-paybox.git

If you want to create an app with m2crypto builtin just do
 heroku create myapp --buildpack https://github.com/guybowden/heroku-buildpack-python-paybox.git

And thats it, heroku will use this custom buildpack with the necessary code to install m2crypt.
Cheers!