domingo, 4 de noviembre de 2012

Connection pooling in heroku with Django

You should use this:
So Postgresql is your thing on heroku with django, and you think to your self, how can i shave miliseconds of my response time, while doing practically nothing.
Your answer, database connectiong pooling.
In your settings.py

import urlparse
url = urlparse.urlparse(os.environ['HEROKU_POSTGRESQL_GOLD_URL'])
path = url.path[1:]
path = path.split('?', 2)[0]

DATABASES = {
    'default': {
        'ENGINE': 'dbpool.db.backends.postgresql_psycopg2',         
        'OPTIONS': {'max_conns': 1},
        'HOST': url.hostname,
        'NAME': path,
        'OPTIONS': {},
        'PASSWORD': url.password,
        'PORT': url.port,
        'TEST_CHARSET': None,
        'TEST_COLLATION': None,
        'TEST_MIRROR': None,
        'TEST_NAME': None,
        'TIME_ZONE': 'America/Mexico_City',
        'USER': url.username
        }
    }
#If you use south migrations
SOUTH_DATABASE_ADAPTERS = {
    'default': 'south.db.postgresql_psycopg2',
}


And in your requirements.txt

django-db-pool==0.0.7
So, numbers, i'll just leave my new relic graph here.


This is a graph of 7 days with response time on the x axis and time on the y axis.
That bump in response time, that orangeish growth is the database taking 50% more time answering because django has to setup the connection to the database on each request.
It is pretty clear with connection pooling > 200 ms response time, after > 300 ms. thats a 50% increase in your app performance for basically changing a setting.

Notice:
Heroku limits the 9 dls database plan to 20 connections, and because the connections are persistent thanks to the pool, youll be permanently using more(the exact number i don't know) but be careful when running out connections, if you have a lot of workers writing to the database and you restart the web app it might run out of connections and you'll site will go down(experience talking here).

You'll have to migrate to crane the 50 dls plan to avoid the connection limit. So be careful out there, it happened to me with a cronjob using the connections leaving my main app connection thirsthy and really down.

Update:
I was totally wrong, as it turns out,  i changed the pooling setting at the same time heroku migrated to the new databases, and they are the guilty ones for this increase in response time. tzk tzk tzk heroku.



No hay comentarios:

Publicar un comentario