Setting up Django(Python) with Nginx, Gunicorn, virtualenv, supervisor and PostgreSQL

nginx_django_uwsgi-520x245

 

Django is an efficient, versatile and dynamically evolving web application development framework. When Django initially gained popularity, the recommended setup for running Django applications was based around Apache with mod_wsgi. The art of running Django advanced and these days the recommended configuration is more efficient and resilient, but also more complex and includes such tools as: Nginx, Gunicorn, virtualenv, supervisord and PostgreSQL.

1. Update Your System

$sudo apt-get update

$sudo apt-get upgrade

2. Install and create application database

$sudo apt-get install postgresql postgresql-contrib

$sudo su – postgres

postgres@django:~$createuser –interactive -P
Enter name of role to add: hello_django
Enter password for new role:
Enter it again:
Shall the new role be a superuser? (y/n) n
Shall the new role be allowed to create databases? (y/n) n
Shall the new role be allowed to create more new roles? (y/n) n

postgres@django:~$createdb –owner hello_django hello

postgres@django:~$logout

3. Create Application User

$sudo groupadd –system webapps

$ sudo useradd –system –gid webapps –shell /bin/bash –home /webapps/hello_django hello

4. Install Virtualenv and create environment for your application

$ sudo apt-get install python-virtualenv

$ sudo mkdir -p /webapps/hello_django/

$ sudo chown hello /webapps/hello_django/

As the application user create a virtual Python environment in the application directory:

$ sudo su – hello

hello@django:~$ cd /webapps/hello_django/

hello@django:~$ virtualenv .
New python executable in hello_django/bin/python
Installing distribute…………..done.
Installing pip…………………done.

hello@django:~$ source bin/activate

(hello_django)hello@django:~$

Your environment is now activated and you can proceed to install Django inside it.

(hello_django)hello@django:~$ pip install django
Downloading/unpacking django
(…)
Installing collected packages: django
(…)
Successfully installed django
Cleaning up…

Your environment with Django should be ready to use. Go ahead and create an empty Django project.

(hello_django)hello@django:~$ django-admin.py startproject hello

You can test it by running the development server:

(hello_django)hello@django:~$ cd hello

(hello_django)hello@django:~$ python manage.py runserver example.com:8000
Validating models…
0 errors found
June 09, 2013 – 06:12:00
Django version 1.5.1, using settings ‘hello.settings’
Development server is running at http://example.com:8000/
Quit the server with CONTROL-C.

You should now be able to access your development server from http://example.com:8000

5. Allowing other users write access to the application directory

Your application will run as the user hello, who owns the entire application directory. If you want regular user to be able to change application files, you can set the group owner of the directory to users and give the group write permissions.

$ sudo chown -R hello:users /webapps/hello_django

$ sudo chmod -R g+w /webapps/hello_django

If you’re not a member of users, you can add yourself to the group with this command:

$ sudo usermod -a -G users `whoami`

6. Configure PostgreSQL to work with Django

$ sudo apt-get install libpq-dev python-dev

Install psycopg2 database adapter:

(hello_django)hello@django:~$ pip install psycopg2

You can now configure the databases section in your settings.py. Change it to:

DATABASES = {
‘default’: {
‘ENGINE’: ‘django.db.backends.postgresql_psycopg2’,
‘NAME’: ‘hello’,
‘USER’: ‘hello_django’,
‘PASSWORD’: ‘password’,
‘HOST’: ‘localhost’,
‘PORT’: ”,                      # Set to empty string for default.
}
}

And finally build the initial database for Django:

(hello_django)hello@django:~$ python manage.py syncdb

7. Install and configure Gunicorn

(hello_django)hello@django:~$ pip install gunicorn

Now that you have gunicorn, you can test whether it can serve your Django application by running the following command:

(hello_django)hello@django:~$ gunicorn hello.wsgi:application –bind example.com:8001

You should now be able to access the Gunicorn server from http://example.com:8001 . I intentionally changed port 8000 to 8001 to force your browser to establish a new connection.

Gunicorn is installed and ready to serve your app. Let’s set some configuration options to make it more useful. I like to set a number of parameters, so let’s put them all into a small BASH script, which I save as bin/gunicorn_start

#!/bin/bash
NAME=”hello_app” # Name of the application
DJANGODIR=/webapps/hello_django/hello # Django project directory
SOCKFILE=/webapps/hello_django/run/gunicorn.sock # we will communicte using this unix socket
USER=hello # the user to run as
GROUP=webapps # the group to run as
NUM_WORKERS=3 # how many worker processes should Gunicorn spawn
DJANGO_SETTINGS_MODULE=hello.settings # which settings file should Django use
DJANGO_WSGI_MODULE=hello.wsgi # WSGI module name
echo “Starting $NAME as `whoami`”
# Activate the virtual environment
cd $DJANGODIR
source ../bin/activate
export DJANGO_SETTINGS_MODULE=$DJANGO_SETTINGS_MODULE
export PYTHONPATH=$DJANGODIR:$PYTHONPATH
# Create the run directory if it doesn’t exist
RUNDIR=$(dirname $SOCKFILE)
test -d $RUNDIR || mkdir -p $RUNDIR
# Start your Django Unicorn
# Programs meant to be run under supervisor should not daemonize themselves (do not use –daemon)
exec ../bin/gunicorn ${DJANGO_WSGI_MODULE}:application \
–name $NAME \
–workers $NUM_WORKERS \
–user=$USER –group=$GROUP \
–bind=unix:$SOCKFILE \
–log-level=debug \
–log-file=-

Set the executable bit on the gunicorn_start script:

$ sudo chmod u+x bin/gunicorn_start

You can test your gunicorn_start script by running it as the user hello.

$ sudo su – hello
hello@django:~$ bin/gunicorn_start

In order for the –name argument to have an effect you need to install a Python module called setproctitle. To build this native extension pip needs to have access to C header files for Python. You can add them to your system with the python-dev package and then install setproctitle.

$ sudo aptitude install python-dev
(hello_django)hello@django:~$ pip install setproctitle

Now when you list processes, you should see which gunicorn belongs to which application.

$ ps aux
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
(…)
hello    11588  0.7  0.2  58400 11568 ?        S    14:52   0:00 gunicorn: master [hello_app]
hello    11602  0.5  0.3  66584 16040 ?        S    14:52   0:00 gunicorn: worker [hello_app]
hello    11603  0.5  0.3  66592 16044 ?        S    14:52   0:00 gunicorn: worker [hello_app]
hello    11604  0.5  0.3  66604 16052 ?        S    14:52   0:00 gunicorn: worker [hello_app]

8. Configure Monitoring using Supervisor

$ sudo apt-get install supervisor

When Supervisor is installed you can give it programs to start and watch by creating configuration files in the /etc/supervisor/conf.d directory. For our hello application we’ll create a file named /etc/supervisor/conf.d/hello.conf with this content:

[program:hello]
command = /webapps/hello_django/bin/gunicorn_start ; Command to start app
user = hello ; User to run as
stdout_logfile = /webapps/hello_django/logs/gunicorn_supervisor.log ; Where to write log messages
redirect_stderr = true ; Save stderr in the same log
environment=LANG=en_US.UTF-8,LC_ALL=en_US.UTF-8

Create the file to store your application’s log messages:

hello@django:~$ mkdir -p /webapps/hello_django/logs/

hello@django:~$ touch /webapps/hello_django/logs/gunicorn_supervisor.log

After you save the configuration file for your program you can ask supervisor to reread configuration files and update (which will start your the newly registered app).

$ sudo supervisorctl reread
hello: available

$ sudo supervisorctl update
hello: added process group

You can also check the status of your app or start, stop or restart it using supervisor.

$ sudo supervisorctl status hello
hello                            RUNNING    pid 18020, uptime 0:00:50

$ sudo supervisorctl stop hello
hello: stopped

$ sudo supervisorctl start hello
hello: started

$ sudo supervisorctl restart hello
hello: stopped
hello: started

9. Install and configure Nginx for web hosting

$ sudo apt-get install nginx

$ sudo service nginx start

You can navigate to your server (http://example.com) with your browser and Nginx should greet you with the words “Welcome to nginx!”.

Create a new nginx server configuration file for your Django application running on example.com in /etc/nginx/sites-available/hello.

upstream hello_app_server {
# fail_timeout=0 means we always retry an upstream even if it failed
# to return a good HTTP response (in case the Unicorn master nukes a
# single worker for timing out).
server unix:/webapps/hello_django/run/gunicorn.sock fail_timeout=0;
}
server {
listen 80;
server_name example.com;
client_max_body_size 4G;
access_log /webapps/hello_django/logs/nginx-access.log;
error_log /webapps/hello_django/logs/nginx-error.log;
location /static/ {
alias /webapps/hello_django/static/;
}
location /media/ {
alias /webapps/hello_django/media/;
}
location / {
# an HTTP header important enough to have its own Wikipedia entry:
# http://en.wikipedia.org/wiki/X-Forwarded-For
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# enable this if and only if you use HTTPS, this helps Rack
# set the proper protocol for doing redirects:
# proxy_set_header X-Forwarded-Proto https;
# pass the Host: header from the client right along so redirects
# can be set properly within the Rack application
proxy_set_header Host $http_host;
# we don’t want nginx trying to do something clever with
# redirects, we set the Host: header above already.
proxy_redirect off;
# set “proxy_buffering off” *only* for Rainbows! when doing
# Comet/long-poll stuff. It’s also safe to set if you’re
# using only serving fast clients with Unicorn + nginx.
# Otherwise you _want_ nginx to buffer responses to slow
# clients, really.
# proxy_buffering off;
# Try to serve static files from nginx, no point in making an
# *application* server like Unicorn/Rainbows! serve static files.
if (!-f $request_filename) {
proxy_pass http://hello_app_server;
break;
}
}
# Error pages
error_page 500 502 503 504 /500.html;
location = /500.html {
root /webapps/hello_django/static/;
}
}

Create a symbolic link in the sites-enabled folder:

$ sudo ln -s /etc/nginx/sites-available/hello /etc/nginx/sites-enabled/hello

Restart Nginx:

$ sudo service nginx restart

check http://example.com on your browser

21 Comments