I wanted a function that 'crawled' all the users in a django queryset so the whole problem was the old, how to work on a whole list but in small chunks. I needed this for performance reasons because it was expensive to try to generate all the user's thumbnails at once and expecting that nothing would go wrong was insane. Apparently facebook servers are not as stable as you think.
So less bla bla y more code:
def fanfunders_subset(self,start=None,end=None):
return UserProfile.objects.fanfunders()[start:end]
So i would call this function like this in a loop
fanfunders_subset(0,10)
# and in the next iteration
fanfunders_subset(10,20)
At this moment i was thankful python lists end in -1, so slicing [0,10] will get you the first number the decond and up to the ninth, if it didnt ended in -1 i would have to do something akward like [0,10] [11,20] [21,30] and that looks ugly.
So what was the trick? oh yeah, you can slice with None! in this context
>>> list[0,None] == list[0,-1]
True
And then
>>> list[None,-1] == list[0,-1]
True
So you can only give on the two arguments to the function and it will return the correct slice, yeah this trick is more for that, to get this flexible interface, sometimes you want a chunk from the end of the list and sometimes you want it from the begining and somethings you want in the middle, lots of cases are covered safely here it just looks a little weird if you see this at first, thats why i wrote this post :P So more people know it and you don't judge me for using weird tricks in Python.
sábado, 10 de noviembre de 2012
domingo, 4 de noviembre de 2012
Connection pooling in heroku with Django
You should use this:
So Postgresql is your thing on heroku with django, and you think to your self, how can i shave miliseconds of my response time, while doing practically nothing.
Your answer, database connectiong pooling.
In your settings.py
import urlparse
url = urlparse.urlparse(os.environ['HEROKU_POSTGRESQL_GOLD_URL'])
path = url.path[1:]
path = path.split('?', 2)[0]
DATABASES = {
'default': {
'ENGINE': 'dbpool.db.backends.postgresql_psycopg2',
'OPTIONS': {'max_conns': 1},
'HOST': url.hostname,
'NAME': path,
'OPTIONS': {},
'PASSWORD': url.password,
'PORT': url.port,
'TEST_CHARSET': None,
'TEST_COLLATION': None,
'TEST_MIRROR': None,
'TEST_NAME': None,
'TIME_ZONE': 'America/Mexico_City',
'USER': url.username
}
}
#If you use south migrations
SOUTH_DATABASE_ADAPTERS = {
'default': 'south.db.postgresql_psycopg2',
}
And in your requirements.txt
django-db-pool==0.0.7
So, numbers, i'll just leave my new relic graph here.
This is a graph of 7 days with response time on the x axis and time on the y axis.
That bump in response time, that orangeish growth is the database taking 50% more time answering because django has to setup the connection to the database on each request.
It is pretty clear with connection pooling > 200 ms response time, after > 300 ms. thats a 50% increase in your app performance for basically changing a setting.
Notice:
Heroku limits the 9 dls database plan to 20 connections, and because the connections are persistent thanks to the pool, youll be permanently using more(the exact number i don't know) but be careful when running out connections, if you have a lot of workers writing to the database and you restart the web app it might run out of connections and you'll site will go down(experience talking here).
You'll have to migrate to crane the 50 dls plan to avoid the connection limit. So be careful out there, it happened to me with a cronjob using the connections leaving my main app connection thirsthy and really down.
Update:
I was totally wrong, as it turns out, i changed the pooling setting at the same time heroku migrated to the new databases, and they are the guilty ones for this increase in response time. tzk tzk tzk heroku.
So Postgresql is your thing on heroku with django, and you think to your self, how can i shave miliseconds of my response time, while doing practically nothing.
Your answer, database connectiong pooling.
In your settings.py
import urlparse
url = urlparse.urlparse(os.environ['HEROKU_POSTGRESQL_GOLD_URL'])
path = url.path[1:]
path = path.split('?', 2)[0]
DATABASES = {
'default': {
'ENGINE': 'dbpool.db.backends.postgresql_psycopg2',
'OPTIONS': {'max_conns': 1},
'HOST': url.hostname,
'NAME': path,
'OPTIONS': {},
'PASSWORD': url.password,
'PORT': url.port,
'TEST_CHARSET': None,
'TEST_COLLATION': None,
'TEST_MIRROR': None,
'TEST_NAME': None,
'TIME_ZONE': 'America/Mexico_City',
'USER': url.username
}
}
#If you use south migrations
SOUTH_DATABASE_ADAPTERS = {
'default': 'south.db.postgresql_psycopg2',
}
And in your requirements.txt
django-db-pool==0.0.7
So, numbers, i'll just leave my new relic graph here.
This is a graph of 7 days with response time on the x axis and time on the y axis.
That bump in response time, that orangeish growth is the database taking 50% more time answering because django has to setup the connection to the database on each request.
It is pretty clear with connection pooling > 200 ms response time, after > 300 ms. thats a 50% increase in your app performance for basically changing a setting.
Notice:
Heroku limits the 9 dls database plan to 20 connections, and because the connections are persistent thanks to the pool, youll be permanently using more(the exact number i don't know) but be careful when running out connections, if you have a lot of workers writing to the database and you restart the web app it might run out of connections and you'll site will go down(experience talking here).
You'll have to migrate to crane the 50 dls plan to avoid the connection limit. So be careful out there, it happened to me with a cronjob using the connections leaving my main app connection thirsthy and really down.
Update:
I was totally wrong, as it turns out, i changed the pooling setting at the same time heroku migrated to the new databases, and they are the guilty ones for this increase in response time. tzk tzk tzk heroku.
domingo, 23 de septiembre de 2012
Install pygraphviz on mountain lion
So you are trying to install pygraphviz on OSX and you get this:
$pip install pygraphviz
Traceback (most recent call last):
File "<string>", line 16, in <module>
File "/Users/grillermo/.virtualenvs/bandtastic/build/pygraphviz/setup.py", line 89, in <module>
raise OSError,"Error locating graphviz."
OSError: Error locating graphviz.
Don´t worry you can install graphviz with Brew(you are already using brew arent you?)
like this:
brew install graphviz
But you now have to add an extra enviromental variable for the pip installer, all you have to do after brew is done is:
export PKG_CONFIG_PATH=/usr/local/Cellar/graphviz/2.28.0/lib/pkgconfig
And voilá, pip installer will run without problems.
You might be wondering why we exported that dir and not other, because thats where the file libcgraph.pc is located.
$pip install pygraphviz
Traceback (most recent call last):
File "<string>", line 16, in <module>
File "/Users/grillermo/.virtualenvs/bandtastic/build/pygraphviz/setup.py", line 89, in <module>
raise OSError,"Error locating graphviz."
OSError: Error locating graphviz.
Don´t worry you can install graphviz with Brew(you are already using brew arent you?)
like this:
brew install graphviz
But you now have to add an extra enviromental variable for the pip installer, all you have to do after brew is done is:
export PKG_CONFIG_PATH=/usr/local/Cellar/graphviz/2.28.0/lib/pkgconfig
And voilá, pip installer will run without problems.
You might be wondering why we exported that dir and not other, because thats where the file libcgraph.pc is located.
miércoles, 15 de agosto de 2012
Johnny-cache on Heroku
Hi, today i got the awesome caching library for Django johnny cache working on heroku.
What you will need
1. My fork of johnny cache that adds a new cacheing backend with support for
django-pylibmc-sasl==0.2.4
pylibmc==1.2.3
2. A heroku memcachier addon
3. The settings.
STEP 1
For development install my fork locally
Now add these lines to your requirements.txt, at the end of it
pylibmc==1.2.3
STEP 2
Install the addon on your app
By all means if you encounter problems, contact me @grillermo or leave a comment here, i suffered this installation and my experience could help somebody in need.
On a related issue i couldnt uninstall already installed packages by pip so i had to fix my buildpack so heroku respects these command
heroku labs:enable user_env_compile --app bandtastic
heroku config:add CLEAN_VIRTUALENV=true
What these will do is make heroku reinstall all the packages from your requirements.txt everytime you push, so make sure you only do this once you update your johnny cache installation with my repo and then you should do
To use my build pack run
heroku config:add BUILDPACK_URL=git://github.com/grillermo/heroku-buildpack-python.git
UPDATE
DONT DO IT, do not install johnny cache on heroku, at least using Memcachier this is my new relic performance log
Look at that! wtf im using the C client pylibmc, response time almost trippled!
What you will need
1. My fork of johnny cache that adds a new cacheing backend with support for
django-pylibmc-sasl==0.2.4
pylibmc==1.2.3
2. A heroku memcachier addon
3. The settings.
STEP 1
For development install my fork locally
pip install git+git://github.com/grillermo/johnny-cache.git
Now add these lines to your requirements.txt, at the end of it
git+git://github.com/grillermo/johnny-cache.git
django-pylibmc-sasl==0.2.4pylibmc==1.2.3
STEP 2
Install the addon on your app
heroku addons:add memcachier:dev --app yourHerokuApp
STEP 3
Add these to your settings file
os.environ['MEMCACHE_SERVERS'] = os.environ.get('MEMCACHIER_SERVERS', '')
os.environ['MEMCACHE_USERNAME'] = os.environ.get('MEMCACHIER_USERNAME', '')
os.environ['MEMCACHE_PASSWORD'] = os.environ.get('MEMCACHIER_PASSWORD', '')
CACHES = {}
CACHES['default'] = {
'BACKEND': 'johnny.backends.memcached.PyLibMCCacheSasl',
'BINARY': True,
'JOHNNY_CACHE': True,
'LOCATION': 'localhost:11211',
'OPTIONS': {
'ketama': True,
'tcp_nodelay': True,
},
'TIMEOUT': 500,
}
JOHNNY_MIDDLEWARE_KEY_PREFIX='a_nice_string_of_your_choosing'
# The first middleware on the list
MIDDLEWARE_CLASSES = (
'johnny.middleware.LocalStoreClearMiddleware',
'johnny.middleware.QueryCacheMiddleware',
...
)
By all means if you encounter problems, contact me @grillermo or leave a comment here, i suffered this installation and my experience could help somebody in need.
On a related issue i couldnt uninstall already installed packages by pip so i had to fix my buildpack so heroku respects these command
heroku labs:enable user_env_compile --app bandtastic
heroku config:add CLEAN_VIRTUALENV=true
What these will do is make heroku reinstall all the packages from your requirements.txt everytime you push, so make sure you only do this once you update your johnny cache installation with my repo and then you should do
heroku config:remove CLEAN_VIRTUALENV
To go back to normalityTo use my build pack run
heroku config:add BUILDPACK_URL=git://github.com/grillermo/heroku-buildpack-python.git
DONT DO IT, do not install johnny cache on heroku, at least using Memcachier this is my new relic performance log
Look at that! wtf im using the C client pylibmc, response time almost trippled!
lunes, 25 de junio de 2012
Heroku django new database settings
Heroku just announced that injection of database settings, meaning:
import os
import sys
import urlparse
# Register database schemes in URLs.
urlparse.uses_netloc.append('postgres')
urlparse.uses_netloc.append('mysql')
try:
# Check to make sure DATABASES is set in settings.py file.
# If not default to {}
if 'DATABASES' not in locals():
DATABASES = {}
if 'DATABASE_URL' in os.environ:
url = urlparse.urlparse(os.environ['DATABASE_URL'])
# Ensure default database exists.
DATABASES['default'] = DATABASES.get('default', {})
# Update with environment configuration.
DATABASES['default'].update({
'NAME': url.path[1:],
'USER': url.username,
'PASSWORD': url.password,
'HOST': url.hostname,
'PORT': url.port,
})
if url.scheme == 'postgres':
DATABASES['default']['ENGINE'] = 'django.db.backends.postgresql_psycopg2'
if url.scheme == 'mysql':
DATABASES['default']['ENGINE'] = 'django.db.backends.mysql'
except Exception:
print 'Unexpected error:', sys.exc_info()
Will cease to happen, heroku won't be adding the database settings, luckly they provided a simple solution. here are the steps i followed for my django app.
Note: i recommend you try this first in a copy of your app, you are using a staging copy right?
Assumptions/Requirements
dj-database-url==0.2.1
Then add the heroku plugin user_env_compile to your app
heroku run cat your_project/settings.py
Outputs something like thisimport os
import sys
import urlparse
# Register database schemes in URLs.
urlparse.uses_netloc.append('postgres')
urlparse.uses_netloc.append('mysql')
try:
# Check to make sure DATABASES is set in settings.py file.
# If not default to {}
if 'DATABASES' not in locals():
DATABASES = {}
if 'DATABASE_URL' in os.environ:
url = urlparse.urlparse(os.environ['DATABASE_URL'])
# Ensure default database exists.
DATABASES['default'] = DATABASES.get('default', {})
# Update with environment configuration.
DATABASES['default'].update({
'NAME': url.path[1:],
'USER': url.username,
'PASSWORD': url.password,
'HOST': url.hostname,
'PORT': url.port,
})
if url.scheme == 'postgres':
DATABASES['default']['ENGINE'] = 'django.db.backends.postgresql_psycopg2'
if url.scheme == 'mysql':
DATABASES['default']['ENGINE'] = 'django.db.backends.mysql'
except Exception:
print 'Unexpected error:', sys.exc_info()
Will cease to happen, heroku won't be adding the database settings, luckly they provided a simple solution. here are the steps i followed for my django app.
Note: i recommend you try this first in a copy of your app, you are using a staging copy right?
Assumptions/Requirements
- heroku gem past 2.18.1. version(this is important, upgrade your gem now)
gem install heroku
otherwise you will get a
No superclass method 'app'
error - Django 1.3 or higher
- You will need a custom buildpack thats updated to account the user_env_compile addon, add it to your app like this
heroku config:add BUILDPACK_URL=git@github.com:heroku/heroku-buildpack-python.git --app your_app_name
Or if you need M2Crypto support you will need my custom buildpack that is just a mix of the standard python pack with a guybowdens pack that enabled the support for SWIG needed by M2Crypto.
heroku config:add BUILDPACK_URL=git://github.com/grillermo/heroku-buildpack-python.git --app your_app_name
dj-database-url==0.2.1
Then add the heroku plugin user_env_compile to your app
heroku labs:enable user_env_compile --app your_app_name
And then add the ENV variable to disable injectionheroku config:add DISABLE_INJECTION=1
Now add these lines to your settings.py
import dj_database_url
DATABASES = {'default': dj_database_url.config(default='postgres://localhost')}
Now make a commit and push to heroku
git commit -m 'disable heroku injection of database settings, manually add them'
git push origin master
You can check is not injecting the settings anymore with:
heroku run cat your_project/settings.py
And at the end of your settings there should not be anything you did not put.
References:
References:
miércoles, 13 de junio de 2012
recursive s3 uploading from python
I recently had the need to upload a directory with many files to s3
so i wrote a small function to to this for me, this is not optimized at
all, as it will only send one file at the time, and it doesn't take into
account the 5gb limitation on s3 uploads, but is the simplest that
could get the job done.
This function traverse through a folder and uploads the files with the correct key so they show up inside their respective 'folders' on the s3 management panel, i say 'folders' because there is no such concept on s3 only, keys that can be optionally prepended by slashes so '/home/static/files/lol.py' is the key of the 'lol.py' file, and thats how you request it.
This function checks if the key already exists only uploading missing keys.
So here it is, in all its slow glory.
Also, sometimes amazon fails to find a key its supposed to be there, so i had to add a little try, except to handle these cases, i had to upload ~4000 files and only had problems with 10, so i guess this is the 99.9% availability amazon claims to have with s3 files.
def uploadResultToS3(awsid,awskey,bucket,source_folder):
c = boto.connect_s3(awsid,awskey)
b = c.get_bucket(bucket)
k = Key(b)
for path,dir,files in os.walk(source_folder):
for file in files:
relpath = os.path.relpath(os.path.join(path,file))
if not b.get_key(relpath):
print 'sending...',relpath
k.key = relpath
k.set_contents_from_filename(relpath)
try:
k.set_acl('public-read')
except:
failed.write(relpath+', ')
failed.close()
This function traverse through a folder and uploads the files with the correct key so they show up inside their respective 'folders' on the s3 management panel, i say 'folders' because there is no such concept on s3 only, keys that can be optionally prepended by slashes so '/home/static/files/lol.py' is the key of the 'lol.py' file, and thats how you request it.
This function checks if the key already exists only uploading missing keys.
So here it is, in all its slow glory.
Also, sometimes amazon fails to find a key its supposed to be there, so i had to add a little try, except to handle these cases, i had to upload ~4000 files and only had problems with 10, so i guess this is the 99.9% availability amazon claims to have with s3 files.
import os
import boto
from boto.s3.key import Key
failed = open('failers','w') import boto
from boto.s3.key import Key
def uploadResultToS3(awsid,awskey,bucket,source_folder):
c = boto.connect_s3(awsid,awskey)
b = c.get_bucket(bucket)
k = Key(b)
for path,dir,files in os.walk(source_folder):
for file in files:
relpath = os.path.relpath(os.path.join(path,file))
if not b.get_key(relpath):
print 'sending...',relpath
k.key = relpath
k.set_contents_from_filename(relpath)
try:
k.set_acl('public-read')
except:
failed.write(relpath+', ')
failed.close()
You obviously need the boto library, get it first
pip install boto
martes, 12 de junio de 2012
fixing @font-face problem in firefox free
So you open your brand new webpage on your browser, you are using @font-face on your css you host your own fonts its all good, its works awesome.
Suddenly you start to grow and you need a cdn such as amazon's S3 or google storage, to host your static files for you, so you naturally expect to host your fonts there, WRONG! Firefox will not load @font-face if they are not hosted on the same domain and your-domain.com is not the same as https://s3.amazonaws.com/bandtastic_heroku/
I found a couple of solutions, one involved getting your own google-fonts serivce with a sinatra app, it bugged for me i couldnt get it to work on heroku.
Don't waste more time.
Go to http://typefront.com/fonts register and get your free font hosted there, they will maintain a list of available domains and firefox, and older ie browsers will happily load your font with a simple css @font-face rule.
You depend on them, but such is life.
Suddenly you start to grow and you need a cdn such as amazon's S3 or google storage, to host your static files for you, so you naturally expect to host your fonts there, WRONG! Firefox will not load @font-face if they are not hosted on the same domain and your-domain.com is not the same as https://s3.amazonaws.com/bandtastic_heroku/
I found a couple of solutions, one involved getting your own google-fonts serivce with a sinatra app, it bugged for me i couldnt get it to work on heroku.
Don't waste more time.
Go to http://typefront.com/fonts register and get your free font hosted there, they will maintain a list of available domains and firefox, and older ie browsers will happily load your font with a simple css @font-face rule.
You depend on them, but such is life.
lunes, 11 de junio de 2012
Testing PIL for jpeg support
If you are unsure that your PIL instalation found libjpeg and thus you have jpeg support in django or any app that uses PIL, you can enter the django shell
./manage.py shell
./manage.py shell
>>>from PIL import Image >>>im = Image.open("bride.jpg") >>>im.rotate(45).show()
<PIL.Image.Image image mode=RGB size=960x293 at 0x2D2D560>
The last line is the confirmation that it worked, you have jpeg support.
domingo, 10 de junio de 2012
heroku django celery redistogo
If you want to use redistogo as a backend and broker for django celery
These are the relevant lines on settings.py
djcelery.setup_loader() #not sure if needed
Dont forget to add your celery worker on the Procfile
And turn in it on
heroku ps:scale worker=1
And then you can test it if it works
Please leave a comment if you have problems with this, i'm more than glad to help because i suffered this. Doesnt matter if its a year or many months later leave a comment i'll be here.
These are the relevant lines on settings.py
import os
if [(i,k) for i,k in os.environ.items() if 'heroku' in k]: #detect heroku somehow reliably
REDIS_DB = 0
REDIS_CONNECT_RETRY = True
CELERY_RESULT_BACKEND = 'redis'
CELERY_REDIS_PORT = 9104
CELERY_REDIS_PASSWORD = '93a4b2c92dc0e07f0f9c82e806108fc1'
CELERY_REDIS_HOST = 'gar.redistogo.com'
BROKER_URL = os.getenv('REDISTOGO_URL', 'redis://localhost')
BROKER_TRANSPORT = 'redis'
BROKER_BACKEND = 'redis'
BROKER_PORT = CELERY_REDIS_PORT
BROKER_HOST = CELERY_REDIS_HOST
BROKER_VHOST = '/'
BROKER_PASSWORD = CELERY_REDIS_PASSWORD
CELERYD_CONCURRENCY = 1
CELERY_RESULT_PORT = 9104
CELERY_TASK_RESULT_EXPIRES = 60 * 10
import djceleryif [(i,k) for i,k in os.environ.items() if 'heroku' in k]: #detect heroku somehow reliably
REDIS_DB = 0
REDIS_CONNECT_RETRY = True
CELERY_RESULT_BACKEND = 'redis'
CELERY_REDIS_PORT = 9104
CELERY_REDIS_PASSWORD = '93a4b2c92dc0e07f0f9c82e806108fc1'
CELERY_REDIS_HOST = 'gar.redistogo.com'
BROKER_URL = os.getenv('REDISTOGO_URL', 'redis://localhost')
BROKER_TRANSPORT = 'redis'
BROKER_BACKEND = 'redis'
BROKER_PORT = CELERY_REDIS_PORT
BROKER_HOST = CELERY_REDIS_HOST
BROKER_VHOST = '/'
BROKER_PASSWORD = CELERY_REDIS_PASSWORD
CELERYD_CONCURRENCY = 1
CELERY_RESULT_PORT = 9104
CELERY_TASK_RESULT_EXPIRES = 60 * 10
djcelery.setup_loader() #not sure if needed
Dont forget to add your celery worker on the Procfile
web: bin/gunicorn_django --workers=4 --bind=0.0.0.0:$PORT name_of_your_project/settings.py;
worker: bin/python name_of_your_project/manage.py celeryd -E -B --loglevel=INFO
worker: bin/python name_of_your_project/manage.py celeryd -E -B --loglevel=INFO
And turn in it on
heroku ps:scale worker=1
And then you can test it if it works
heroku run name_of_your_project/manage.py shell
from your_app.tasks import task_name
result = task_name.apply_async(args=[arg1,arg2])
result.wait()
Please leave a comment if you have problems with this, i'm more than glad to help because i suffered this. Doesnt matter if its a year or many months later leave a comment i'll be here.
sábado, 9 de junio de 2012
Install m2crypt in heroku
If you ever had the need to install m2crypt in a heroku app, but you keep getting
I got a solution for you, you just need to use a thing heroku calls buildpacks, these are bash scripts that heroku uses to setup your application, dependencies and all.
So you need a custom buildpack that preinstalls m2crypto for you.
For an existing heroku app
heroku config:add BUILDPACK_URL=https://github.com/guybowden/heroku-buildpack-python-paybox.git
If you want to create an app with m2crypto builtin just do
heroku create myapp --buildpack https://github.com/guybowden/heroku-buildpack-python-paybox.git
And thats it, heroku will use this custom buildpack with the necessary code to install m2crypt.
Cheers!
unable to execute swig: No such file or directorywhen doing a normal push with your requirements.txt on place.
error: command 'swig' failed with exit status 1
I got a solution for you, you just need to use a thing heroku calls buildpacks, these are bash scripts that heroku uses to setup your application, dependencies and all.
So you need a custom buildpack that preinstalls m2crypto for you.
For an existing heroku app
heroku config:add BUILDPACK_URL=https://github.com/guybowden/heroku-buildpack-python-paybox.git
If you want to create an app with m2crypto builtin just do
heroku create myapp --buildpack https://github.com/guybowden/heroku-buildpack-python-paybox.git
And thats it, heroku will use this custom buildpack with the necessary code to install m2crypt.
Cheers!
miércoles, 30 de mayo de 2012
Django, Sentry, Redis, Raven
Its been a pain installing the logging and async tasks setup, i sort of did it in parallel.
The hardest part wasnt actually installing the setup, it was getting my views to actually log to sentry.
The documentation is pretty bad, so i'm putting my settings here so hopefully somebody suffers less.
In my settings i added this monstruosity.
import logging
from raven.handlers.logging import SentryHandler
logging.getLogger().setLevel(logging.INFO)
logger = logging.getLogger()# ensure we havent already registered the handler
handler = SentryHandler('http://the magic url you get from sentry')
logger.addHandler(handler)
# Add StreamHandler to sentry's default so you can catch missed exceptions
logger = logging.getLogger('sentry.errors')
logger.propagate = False
logger.addHandler(logging.StreamHandler())
from raven.conf import setup_logging
setup_logging(handler)
And in my views
import logging
logger = logging.getLogger(__name__)def home(request,context={},template_name=None):
logger.info(str(request), exc_info=True)
return render_to_response(template_name,context, context_instance=RequestContext(request))
And thats it.
lunes, 21 de mayo de 2012
javascript parseInt
parseInt is a javascript function that can't be relied to return NaN in some strings that are not numbers
parseInt('12lol')
returns 12
parseInt('lol12')
returns Nan
A more reliable function to do this is Number()
Number('12lol')
returns Nan
Number('293.2')
returns 293.2
parseFloat has the same defect, don't rely on those.
parseInt('12lol')
returns 12
parseInt('lol12')
returns Nan
A more reliable function to do this is Number()
Number('12lol')
returns Nan
Number('293.2')
returns 293.2
parseFloat has the same defect, don't rely on those.
miércoles, 7 de marzo de 2012
Forma de dineromail comentada
Forma de dineromail comentada:
Me topé con el problema de implementar una forma de dineromail y la escasa documentación no ayudaba asi que espero esto le sirva a mas personas.
Si necesitas enviar a dineromail a un usuario y quieres saber que campos necesitas pasar en el POST esta forma te será útil, no es una referencia completa, para eso necesitas ir a por el manual que dineromail ofrece
jueves, 1 de marzo de 2012
My django snippets
In this post i'd like to share info on snippets i've shared on the very nice django-snippets.org.
My list
My list
-
Encrypted paypal buttons :
Use the code here if you want to encrypt your paypal buttons to pass sensitive information, like the receipt_id if you use it to validate a payment in your end. -
Redirect to no slash:
In django there is this setting APPEND_SLASH that does the oposite to my snippet, adds a slash and redirects there, but i wanted the oposite, i think urls look better like that.
So far the only problem i've had with this approach is Facebook likes code. When you insert a facebook like iframe it checks the url you send it and follows redirections, so for urls with these kinds of codes you should disable the redirection. - Save image in field :I ended up not using this code, and instead i went with the more generic get_img_obj because i wanted a one function fits all to get images from external source inside my imagefields.
-
Send templated email with text | html | optional files
Use this function to send email with html that automatically degrades to text if the client does not support html.
- Generate a dineromail form in python
This code is pretty generic and its just to show some idiosyncrasies of the dineromail API.
jueves, 23 de febrero de 2012
Why the rotate does not work on chrome
I think i need to rebrand this blog as a general web development blog.
The case today is about the css property to rotate an element, for good measure this is the code for all browsers to rotate elements
If chrome refuses to work using this line
And thats it. have fun!
UPDATE:
The sass mixing to rotate an element
The case today is about the css property to rotate an element, for good measure this is the code for all browsers to rotate elements
#yourelement { -moz-transform: rotate(180deg) -o-transform: rotate(180deg) -webkit-transform: rotate(180deg) filter: progid:DXImageTransform.Microsoft.Matrix(sizingMethod='auto expand', M11=#{cos(180deg)}, M12=-#{sin(180deg)}, M21=#{sin(180deg)}, M22=#{cos(180deg)}) -ms-filter: "progid:DXImageTransform.Microsoft.Matrix(sizingMethod='auto expand', M11=#{cos(180deg)}, M12=-#{sin(180deg)}, M21=#{sin(180deg)}, M22=#{cos(180deg)})" zoom: 1 }So thats the code to rotate an element 180 degrees, i used it to rotate a bang ! sing because in spanish you need the rotated version to start a sentence.
If chrome refuses to work using this line
-webkit-transform: rotate(10deg);You are probably tring to target an inline element, this could be a inline element by default so i solved my problem enveloping what i wanted to rotate in a div, and setting its display property to inline.
And thats it. have fun!
UPDATE:
The sass mixing to rotate an element
@mixin rotate($degrees) -moz-transform: rotate($degrees) -o-transform: rotate($degrees) -webkit-transform: rotate($degrees) filter: progid:DXImageTransform.Microsoft.Matrix(sizingMethod='auto expand', M11=#{cos($degrees)}, M12=-#{sin($degrees)}, M21=#{sin($degrees)}, M22=#{cos($degrees)}) -ms-filter: "progid:DXImageTransform.Microsoft.Matrix(sizingMethod='auto expand', M11=#{cos($degrees)}, M12=-#{sin($degrees)}, M21=#{sin($degrees)}, M22=#{cos($degrees)})" zoom: 1And you would use it like this
@include rotate(90deg)Dont forget to import them if you have the mixins in another file!
Etiquetas:
browsers rotate,
css rotate,
rotate element
miércoles, 22 de febrero de 2012
How to submit a form with a click on an arbitrary element.
This is real live code for Bandtastic.me
What i wanted to accomplish was a clickable div with a text box in it that submits a form, this is to bypass the limitations of a regular form that needs a submit button.
I'm posting this because it was a real pain to get this functionallity and i hope somebody can save itself the trouble.
So you have a div with a textbox in it, roughly this

And you want to be able to submit the contents of the textbox as a regular html form.
You can style your form as a regular div, like i did in the screenshot.
Now the magic would rely on jquery so remember to include it in your html and then would be this
First we target all the elements on our form using the *, so when any or subelement can be used to submit the form
We submit the form with another handy jquery method.
Thats it, if you have any questions or remarks leave a comment.
What i wanted to accomplish was a clickable div with a text box in it that submits a form, this is to bypass the limitations of a regular form that needs a submit button.
I'm posting this because it was a real pain to get this functionallity and i hope somebody can save itself the trouble.
So you have a div with a textbox in it, roughly this

And you want to be able to submit the contents of the textbox as a regular html form.
<form method='GET' action='/post' id='myform'> <input type='text' id='mytextbox' value='100' /> </form>Something like that, so no button anywhere to be found.
You can style your form as a regular div, like i did in the screenshot.
Now the magic would rely on jquery so remember to include it in your html and then would be this
$(function(){
$('.custom_contribution *').click(function(event){ event.stopImmediatePropagation() if(! $(event.currentTarget).is('input#custom_ammount')){ $('.custom_contribution').submit(); } }); })Lets analyze this
First we target all the elements on our form using the *, so when any or subelement can be used to submit the form
$('.custom_contribution *').click(function(event)(Then we stop the propagation of the event so if there are elements underneath that jquery also considers clicked we only get one
event.stopImmediatePropagation()Then we check with if the clicked element is not our textbox
if(! $(e.currentTarget).is('input#custom_ammount')){We do this converting the dom element that e.currenTarget returns to a jquery object surrounding it with
$()we do this so we can the use
is()method that will return true if the element is our textbox, and because we have a boolean negation ! the next line of code only will run if we clicked anything but the textbox.
We submit the form with another handy jquery method.
$('.custom_contribution').submit();This allow us to have a huge clickable button while mantaining the form functionality, because if you type something in the textbox and then hit enter, it will also submit it.
Thats it, if you have any questions or remarks leave a comment.
Etiquetas:
capture click,
click multiple elements.,
form,
jquery,
submit without button
jueves, 9 de febrero de 2012
Facebook chat is not loading
Troubleshotting Facebook code
Any Facebook code got in https://developers.facebook.com/docs/plugins/ such as:
Any Facebook code got in https://developers.facebook.com/docs/plugins/ such as:
- Like Button
- Send Button
- Subscribe Button
- Comments
- Activity Feed
- Recommendations
- Like Box
- Login Button
- Registration
- Facepile
- Live Stream
- First go to the facebook debugger and input the problematic url, sometimes just doing it, fixes problems such as wrong meta data or wrong likes numbers.
- Is the required Javascript sdk in the header of your site? it doesnt work if you move it to the bottom
- Is the code copy pasted right, some template systems can mess your code, so make sure yours get the appropriate VERBATIM, :plain or whatever tag your templating system uses to pass textual content.
- Are your running the code on the same url the code is configured in.
- Is the code inside a div or an element with an id, this can be tricky to spot, but some Facebook code requires some css selecting that can be messed up if you put it inside an html element with an id.
- Any other common mistakes, please post them in the comments.
Fixing facebook on firefox
If you use Firefox and the new Facebook photo display mode the photos are out of place, and by out of place i mean they are wrongly placed.
Go get the stylish addon and add this rule for facebook:
Go get the stylish addon and add this rule for facebook:
@namespace url(http://www.w3.org/1999/xhtml); @-moz-document domain("facebook.com") { .stage img{ position:relative; top: -100%; } }
domingo, 15 de enero de 2012
Confusing python path2
Lets clear things up:
I want to add a path to PYTHONPATH so Python doesn't complain about not finding a module(a library)/cannot import a module.
From my terminal emulator(console, terminal)
I want python to always find this or that module
From the first file.py you run
This other path is used to find binaries/executables in your console/terminal emulator. You modify this other path using:
$user: binary
And if you want to modify this path from inside python you do:
import os.environ['PATH'] += ':/new/path'
This only affects the current session and its practically useless unless you are doing some weird bash inside python stuff. Remember that you use : to separate paths.
I want to add a path to PYTHONPATH so Python doesn't complain about not finding a module(a library)/cannot import a module.
From my terminal emulator(console, terminal)
$user: export PYTHONPATH=$PYTHONPATH:/path/to/my/module
From inside a python module
import sys
sys.path.append('path/to/my/module')
I want python to always find this or that module
$user: gedit ~/.bashrc
export PYTHONPATH=$PYTHONPATH:/path/to/my/module
OrFrom the first file.py you run
import sys
sys.path.append('path/to/my/module')
This is different that then system PATHThis other path is used to find binaries/executables in your console/terminal emulator. You modify this other path using:
export PATH=$PATH:/path/to/a/binary
So you can run your shiny new binary/executable like this$user: binary
And if you want to modify this path from inside python you do:
import os.environ['PATH'] += ':/new/path'
This only affects the current session and its practically useless unless you are doing some weird bash inside python stuff. Remember that you use : to separate paths.
jueves, 12 de enero de 2012
Automatically modifing data from models in django
If you want Django to automatically add new data, update existing data or delete from a model you got various choices with various pros and cons.
This post will attempt to give you the necesary information to decide whats the best strategy, you can see how they are implemented in each of the provided links.
You basically have 3 options:
Signals
The good
Signals is the most flexible way to work with your models, you can do pretty much anything at anytime with signals.
The bad
Because they get sent all the time and with many arguments, you might have to do many checks to know if you need them in that particular time and that migh easily lead to unnecessary code, and if you forget some check, your database will suffer.
Conclusion
Use them as your last resort,when the other options do not cut it, go for signals.
Overriding the save and/or the delete method in model classes
The good
This will run EVERY time a model is deleted or saved, you will never miss a chance to change your model data.
The bad
It can lead to hard to track bugs, if the operations you are doing inside the method assume too many things. This can be dangerous, and you might not now the time you are writing them how are you going to use your models.
Conclusion
Go safe with this one, only use things you are sure will always be available at the point of the save method. And try not to change other models data.
Overriding the save_model/delete_model/save_formset methods inside your model_admin class
The good
These methods are only called when you do things in your admin, so this are pretty safe to use, and allow you provide additional features and automation to your admin interface.
The bad
Limited use, won't do anything outside the admin.
This post will attempt to give you the necesary information to decide whats the best strategy, you can see how they are implemented in each of the provided links.
You basically have 3 options:
- Using signals
- Overriding the save method inside your model class
- Overriding the save_model/delete_model/save_formset methods inside your model_admin class
Signals
The good
Signals is the most flexible way to work with your models, you can do pretty much anything at anytime with signals.
The bad
Because they get sent all the time and with many arguments, you might have to do many checks to know if you need them in that particular time and that migh easily lead to unnecessary code, and if you forget some check, your database will suffer.
Conclusion
Use them as your last resort,when the other options do not cut it, go for signals.
Overriding the save and/or the delete method in model classes
The good
This will run EVERY time a model is deleted or saved, you will never miss a chance to change your model data.
The bad
It can lead to hard to track bugs, if the operations you are doing inside the method assume too many things. This can be dangerous, and you might not now the time you are writing them how are you going to use your models.
Conclusion
Go safe with this one, only use things you are sure will always be available at the point of the save method. And try not to change other models data.
Overriding the save_model/delete_model/save_formset methods inside your model_admin class
The good
These methods are only called when you do things in your admin, so this are pretty safe to use, and allow you provide additional features and automation to your admin interface.
The bad
Limited use, won't do anything outside the admin.
martes, 10 de enero de 2012
Pushing to a working webfaction repo from your dev machine
I was able to succesfully push to a git repository in webfaction using this general strategy. Save your self(and my future self) some googling and follow this links.
Set up the ssh connection
Follow the webfaction guide to set up a git repo
Follow Abhijit Menon-Sen guide to setup the push
Set up the ssh connection
Follow the webfaction guide to set up a git repo
Follow Abhijit Menon-Sen guide to setup the push
lunes, 9 de enero de 2012
On yakuake in gnome
Note: I know this blog is called Notes on Using Python, but i'll also do random posts on using linux.
Yakuake its hands down the best drop down terminal emulator on Linux, but it can be tricky to get it to play nice with gnome.
THE PROBLEM
The behavior should be this:
My solution involves
Dont set a hotkey in the yakuake shortcut settings(delete any shortcuts)
Set a hotkey using the keyboard shortcuts gnome system application: adding a new shortcut that runs the command yakuake, to my desired hotkey.
To fix the show desktop im using wmctrl, which you can get using a simple:
Boom, now all works as expected.
Yakuake its hands down the best drop down terminal emulator on Linux, but it can be tricky to get it to play nice with gnome.
THE PROBLEM
The behavior should be this:
- I hit my yakuake hotkeys(defined in the yakuake shortcut settings) and yakuake toggles between hidden or shown.
- I hit my show desktop hotkey and it toggles all the windows off and on.
- I hit my yakuake hotkeys(defined in the yakuake shortcut settings) and yakuake does nothing.
- If yakuake is being show if i hit my showdesktop hotkey, yakuake ignores it and remains.
My solution involves
Dont set a hotkey in the yakuake shortcut settings(delete any shortcuts)
Set a hotkey using the keyboard shortcuts gnome system application: adding a new shortcut that runs the command yakuake, to my desired hotkey.
To fix the show desktop im using wmctrl, which you can get using a simple:
sudo apt-get install wmctrl
Then create the file showdesktop
sudo gedit /usr/local/bin/showdesktop
Paste this content into the file and save it
#!/bin/sh
if wmctrl -m | grep "mode: ON"; then
wmctrl -k off
if ! wmctrl -l | grep Yakuake; then
yakuake
fi
else
wmctrl -k on
if wmctrl -l | grep Yakuake; then
yakuake
fi
fi
Give it permission to runif wmctrl -m | grep "mode: ON"; then
wmctrl -k off
if ! wmctrl -l | grep Yakuake; then
yakuake
fi
else
wmctrl -k on
if wmctrl -l | grep Yakuake; then
yakuake
fi
fi
sudo chmod +x /usr/local/bin/showdesktop
And then go again to the keyboard shortcuts gnome system application and add a new shortcut that simply runs showdesktop.Boom, now all works as expected.
domingo, 8 de enero de 2012
Different ways to get urls in a django view
During production i've had the need to various urls, and i've found a few ways to do this, i leave this here so hopefully somebody can use it, and as a self reminder.
1. Using the Sites app to get the current domain url
Good side:
It does'nt need the request to work, only the Sites app with your list of sites.
Bad side:
The problem with this approach is that if it depends on the sites app, this is good enough if you are diciplined enough to keep that app updated with your development enviroment, your stating server and your production server, it can be annoying.
Via Fragments of code
2. Using the HTTP_REFERER to get the current url
Good side:
I'm not sure where this function could be used, maybe as a helper to prevent sites from outside yours to load some urls. Because of the:
Bad side:
It only works if the current view was called from another url. If the user types directly the url of the current view, it won't work because there is not http referer. Use it only if you are sure your view will always be called from a hyperlink.
Plus the client can modify the http header to send a different referer so it cannot be relied.
3. Using the request HTTP_HOST get the current url
Good side:
It's the simplest form to get your current url. All it takes is a request.
Bad side:
Same problem as the HTTP_REFERER the client can modify the http header so it can't be relied upon.
Credit goes to limodou, the author of this django snippet
4. Using the built in build_absolute_uri to get the current url
Good side:
It's pretty solid, it only takes the current view, or an url name, basically everything reverse() accepts this functions does.
Bad side:
This function needs a relative url you want to get the full url, so its uses is limited. Also, the build_absolute_uri was introduce in Django 1.2+ so that could be a problem for older apps. And i'm not really sure about its short comings i haven't .
1. Using the Sites app to get the current domain url
def current_domain_url(): """Returns fully qualified URL (no trailing slash) for the
current site.""" from django.contrib.sites.models import Site current_site = Site.objects.get_current() protocol = getattr(settings, 'MY_SITE_PROTOCOL', 'http') port = getattr(settings, 'MY_SITE_PORT', '') url = '%s://%s' % (protocol, current_site.domain) if port: url += ':%s' % port return url
Good side:
It does'nt need the request to work, only the Sites app with your list of sites.
Bad side:
The problem with this approach is that if it depends on the sites app, this is good enough if you are diciplined enough to keep that app updated with your development enviroment, your stating server and your production server, it can be annoying.
Via Fragments of code
2. Using the HTTP_REFERER to get the current url
def current_site_url(request): """Returns fully qualified URL (no trailing slash) for the
current site.""" return '/'.join(request.meta['HTTP_REFERER'].split('/')[:3])
Good side:
I'm not sure where this function could be used, maybe as a helper to prevent sites from outside yours to load some urls. Because of the:
Bad side:
It only works if the current view was called from another url. If the user types directly the url of the current view, it won't work because there is not http referer. Use it only if you are sure your view will always be called from a hyperlink.
Plus the client can modify the http header to send a different referer so it cannot be relied.
3. Using the request HTTP_HOST get the current url
def current_site_url(request):
full_path = ('http', ('', 's')[request.is_secure()],
'://', request.META['HTTP_HOST'], request.path) return ''.join(full_path)
Good side:
It's the simplest form to get your current url. All it takes is a request.
Bad side:
Same problem as the HTTP_REFERER the client can modify the http header so it can't be relied upon.
Credit goes to limodou, the author of this django snippet
4. Using the built in build_absolute_uri to get the current url
from django.core.urlresolvers import reverse def current_site_url(request,*args,**kwargs): """Returns fully qualified URL (no trailing slash) for the current site.""" relative_url = reverse(*args,**kwargs) return request.build_absolute_uri(relative_url)
Good side:
It's pretty solid, it only takes the current view, or an url name, basically everything reverse() accepts this functions does.
Bad side:
This function needs a relative url you want to get the full url, so its uses is limited. Also, the build_absolute_uri was introduce in Django 1.2+ so that could be a problem for older apps. And i'm not really sure about its short comings i haven't .
Etiquetas:
current url,
django,
domain url,
function,
how to
Suscribirse a:
Entradas (Atom)