Install pygraphviz on mountain lion

So you are trying to install pygraphviz on OSX and you get this:
$pip install pygraphviz
Traceback (most recent call last):

  File "<string>", line 16, in <module>

  File "/Users/grillermo/.virtualenvs/bandtastic/build/pygraphviz/setup.py", line 89, in <module>

    raise OSError,"Error locating graphviz."

OSError: Error locating graphviz.

Don´t worry you can install graphviz with Brew(you are already using brew arent you?)
like this:
brew install graphviz
But you now have to add an extra enviromental variable for the pip installer, all you have to do after brew is done is:
export PKG_CONFIG_PATH=/usr/local/Cellar/graphviz/2.28.0/lib/pkgconfig

And voilá, pip installer will run without problems.
You might be wondering why we exported that dir and not other, because thats where the file libcgraph.pc is located.

Johnny-cache on Heroku

Hi, today i got the awesome caching library for Django johnny cache working on heroku.

What you will need
1. My fork of johnny cache that adds a new cacheing backend with support for
django-pylibmc-sasl==0.2.4
pylibmc==1.2.3
2. A heroku memcachier addon
3. The settings.

STEP 1
For development install my fork locally

pip install git+git://github.com/grillermo/johnny-cache.git

Now add these lines to your requirements.txt, at the end of it

git+git://github.com/grillermo/johnny-cache.git

django-pylibmc-sasl==0.2.4
pylibmc==1.2.3


STEP 2
Install the addon on your app

heroku addons:add memcachier:dev --app yourHerokuApp

STEP 3

Add these to your settings file

    os.environ['MEMCACHE_SERVERS'] = os.environ.get('MEMCACHIER_SERVERS', '')
    os.environ['MEMCACHE_USERNAME'] = os.environ.get('MEMCACHIER_USERNAME', '')
    os.environ['MEMCACHE_PASSWORD'] = os.environ.get('MEMCACHIER_PASSWORD', '')

    CACHES = {}
    CACHES['default'] = {
            'BACKEND': 'johnny.backends.memcached.PyLibMCCacheSasl',
            'BINARY': True,
            'JOHNNY_CACHE': True,
            'LOCATION': 'localhost:11211',
            'OPTIONS': {
                'ketama': True,
                'tcp_nodelay': True,
            },
            'TIMEOUT': 500,
        }
    JOHNNY_MIDDLEWARE_KEY_PREFIX='a_nice_string_of_your_choosing'

    # The first middleware on the list

    MIDDLEWARE_CLASSES = (
        'johnny.middleware.LocalStoreClearMiddleware',
        'johnny.middleware.QueryCacheMiddleware',
        ...
    ) 

By all means if you encounter problems, contact me @grillermo or leave a comment here, i suffered this installation and my experience could help somebody in need.

On a related issue i couldnt uninstall already installed packages by pip so i had to fix my buildpack so heroku respects these command
heroku labs:enable user_env_compile --app bandtastic
heroku config:add CLEAN_VIRTUALENV=true
What these will do is make heroku reinstall all the packages from your requirements.txt everytime you push, so make sure you only do this once you update your johnny cache installation with my repo and then you should do

heroku config:remove CLEAN_VIRTUALENV

To go back to normality
To use my build pack run
heroku config:add BUILDPACK_URL=git://github.com/grillermo/heroku-buildpack-python.git

 UPDATE

DONT DO IT, do not install johnny cache on heroku, at least using Memcachier this is my new relic performance log

 Look at that! wtf im using the C client pylibmc,  response time almost trippled!

Heroku django new database settings

Heroku just announced that injection of database settings, meaning:

heroku run cat your_project/settings.py

Outputs something like this
import os
import sys
import urlparse

# Register database schemes in URLs.
urlparse.uses_netloc.append('postgres')
urlparse.uses_netloc.append('mysql')

try:

    # Check to make sure DATABASES is set in settings.py file.
    # If not default to {}

    if 'DATABASES' not in locals():
        DATABASES = {}

    if 'DATABASE_URL' in os.environ:
        url = urlparse.urlparse(os.environ['DATABASE_URL'])

        # Ensure default database exists.
        DATABASES['default'] = DATABASES.get('default', {})

        # Update with environment configuration.
        DATABASES['default'].update({
            'NAME': url.path[1:],
            'USER': url.username,
            'PASSWORD': url.password,
            'HOST': url.hostname,
            'PORT': url.port,
        })
        if url.scheme == 'postgres':
            DATABASES['default']['ENGINE'] = 'django.db.backends.postgresql_psycopg2'

        if url.scheme == 'mysql':
            DATABASES['default']['ENGINE'] = 'django.db.backends.mysql'
except Exception:
    print 'Unexpected error:', sys.exc_info()

Will cease to happen, heroku won't be adding the database settings, luckly they provided a simple solution. here are the steps i followed for my django app.
Note: i recommend you try this first in a copy of your app, you are using a staging copy right?
Assumptions/Requirements

• heroku gem past 2.18.1. version(this is important, upgrade your gem now)
  gem install heroku
  otherwise you will get a
  No superclass method 'app'
  error
•  Django 1.3 or higher
• You will need a custom buildpack thats updated to account the user_env_compile addon, add it to your app like this
  heroku config:add BUILDPACK_URL=git@github.com:heroku/heroku-buildpack-python.git --app your_app_name
  Or if you need M2Crypto support you will need my custom buildpack that is just a mix of the standard python pack with a guybowdens pack that enabled the support for SWIG needed by M2Crypto.
  heroku config:add BUILDPACK_URL=git://github.com/grillermo/heroku-buildpack-python.git --app your_app_name

Ok now you will have to add a this line to the bottom of your requirements.txt
dj-database-url==0.2.1
Then add the heroku plugin user_env_compile to your app

heroku labs:enable user_env_compile --app your_app_name

And then add the ENV variable to disable injection

heroku config:add DISABLE_INJECTION=1

Now add these lines to your settings.py

import dj_database_url
DATABASES = {'default': dj_database_url.config(default='postgres://localhost')} 

Now make a commit and push to heroku

git commit -m 'disable heroku injection of database settings, manually add them'

git push origin master

You can check is not injecting the settings anymore with:

heroku run cat your_project/settings.py

And at the end of your settings there should not be anything you did not put.
  
References:

• The heroku news article 
• My custom heroku buildpack 

recursive s3 uploading from python

I recently had the need to upload a directory with many files to s3 so i wrote a small function to to this for me, this is not optimized at all, as it will only send one file at the time, and it doesn't take into account the 5gb limitation on s3 uploads, but is the simplest that could get the job done.
This function traverse through a folder and uploads the files with the correct key so they show up inside their respective 'folders' on the s3 management panel, i say 'folders' because there is no such concept on s3 only, keys that can be optionally prepended by slashes so '/home/static/files/lol.py' is the key of the 'lol.py' file, and thats how you request it.
This function checks if the key already exists only uploading missing keys.
So here it is, in all its slow glory.
Also, sometimes amazon fails to find a key its supposed to be there, so i had to add a little try, except to handle these cases, i had to upload ~4000 files and only had problems with 10, so i guess this is the 99.9% availability amazon claims to have with s3 files.

import os
import boto
from boto.s3.key import Key

failed = open('failers','w')
def uploadResultToS3(awsid,awskey,bucket,source_folder):
    c = boto.connect_s3(awsid,awskey)
    b = c.get_bucket(bucket)
    k = Key(b)
    for path,dir,files in os.walk(source_folder):
        for file in files:
            relpath = os.path.relpath(os.path.join(path,file))
            if not b.get_key(relpath):
                print 'sending...',relpath
                k.key = relpath
                k.set_contents_from_filename(relpath)
                try:
                    k.set_acl('public-read')
                except:
                    failed.write(relpath+', ') 
    failed.close()

You obviously need the boto library, get it first

pip install boto

fixing @font-face problem in firefox free

So you open your brand new webpage on your browser, you are using @font-face on your css you host your own fonts its all good, its works awesome.

Suddenly you start to grow and you need a cdn such as amazon's S3 or google storage, to host your static files for you, so you naturally expect to host your fonts there, WRONG! Firefox will not load @font-face if they are not hosted on the same domain and your-domain.com is not the same as https://s3.amazonaws.com/bandtastic_heroku/

I found a couple of solutions, one involved getting your own google-fonts serivce with a sinatra app, it bugged for me i couldnt get it to work on heroku.
Don't waste more time.
Go to http://typefront.com/fonts register and get your free font hosted there, they will maintain a list of available domains and firefox, and older ie browsers will happily load your font with a simple css @font-face rule.

You depend on them, but such is life.

Testing PIL for jpeg support

If you are unsure that your PIL instalation found libjpeg and thus you have jpeg support in django or any app that uses PIL, you can enter the django shell
./manage.py shell

>>>from PIL import Image
>>>im = Image.open("bride.jpg")
>>>im.rotate(45).show()

<PIL.Image.Image image mode=RGB size=960x293 at 0x2D2D560>

The last line is the confirmation that it worked, you have jpeg support. 

heroku django celery redistogo

If you want to use redistogo as a backend and broker for django celery

These are the relevant lines on settings.py

import os
if [(i,k) for i,k in os.environ.items() if 'heroku' in k]: #detect heroku somehow reliably
    REDIS_DB = 0
    REDIS_CONNECT_RETRY = True
    CELERY_RESULT_BACKEND = 'redis'
    CELERY_REDIS_PORT = 9104
    CELERY_REDIS_PASSWORD = '93a4b2c92dc0e07f0f9c82e806108fc1'
    CELERY_REDIS_HOST = 'gar.redistogo.com'
    BROKER_URL = os.getenv('REDISTOGO_URL', 'redis://localhost')
    BROKER_TRANSPORT = 'redis'
    BROKER_BACKEND = 'redis'
    BROKER_PORT = CELERY_REDIS_PORT
    BROKER_HOST = CELERY_REDIS_HOST
    BROKER_VHOST = '/'
    BROKER_PASSWORD = CELERY_REDIS_PASSWORD
    CELERYD_CONCURRENCY = 1
    CELERY_RESULT_PORT = 9104
    CELERY_TASK_RESULT_EXPIRES = 60 * 10

 

import djcelery
djcelery.setup_loader() #not sure if needed

Dont forget to add your celery worker on the Procfile

web: bin/gunicorn_django --workers=4 --bind=0.0.0.0:$PORT name_of_your_project/settings.py;
worker: bin/python name_of_your_project/manage.py celeryd -E -B --loglevel=INFO

And turn in it on
heroku ps:scale worker=1
And then you can test it if it works

heroku run name_of_your_project/manage.py shell

from your_app.tasks import task_name

result = task_name.apply_async(args=[arg1,arg2])

result.wait()

Please leave a comment if you have problems with this, i'm more than glad to help because i suffered this. Doesnt matter if its a year or many months later leave a comment i'll be here.