Setup a Django VM with Vagrant, VirtualBox, and Chef

If you’ve decided that you want to learn the Django framework, but you don’t know where to begin, you’ve come to the right place. In this guide we’ll be walking step-by-step through the setup of a Django development VM. By the time we’re done we’ll have used a variety of amazing open-source tools to create a virtual machine designed just for Django development.

We’re going to use the following tools in the process:

  • VirtualBox (download link)
    • This is the virtualization software we will use to run our Django virtual machine. Simply download and install.
  • Vagrant (download link)
    • Created in 2010 by Mitchell Hashimoto (Twitter) and John Bender (Twitter), Vagrant allows for the programmatic configuration of virtual machines. No longer must we endure the tedious process of clicking around in VirtualBox dialog boxes to configure our VM. We will create a Vagrantfile, a text file which specifies how we want our virtual machine configured, and Vagrant will handle creating and configuring the VM to our specifications. In this tutorial, we’re assuming version 1.02, but as newer versions are released, they should be fine to use as well.
  • Chef Solo (we’ll download our Chef cookbooks later in this guide)
    • Once Vagrant finishes booting our VM, we need a way to install the various supporting packages (dependencies) that are required to run Django on our virtual machine. We could manually go through this process, but this is the way of the past. Instead, we will download and apply “cookbooks” — collections or abstract definitions as source code that describe how we want our virtual machine to be built — and we’ll apply these cookbooks to our VM immediately after Vagrant finishes booting it up. Don’t worry, it’s not as complicated as it sounds. It actually makes life a lot easier!

Workstation Configuration

The beauty of using a development VM is that very few configuration changes are required on the host workstation. To begin, we’ll want to set up a directory on our machine for use with Vagrant. In this guide we’ll create a django_guide directory in our home directory:

# Create django_guide directory in our home directory
mkdir ~/django_guide

# Change into the newly created ~/django_guide directory
cd ~/django_guide

Once we’re in this directory, we’re going to instruct vagrant to “initialize” this directory. This drops a file called a Vagrantfile in the directory:

# Generate a Vagrantfile in the ~/django_guide directory.
vagrant init

If everything worked as planned, the following message should appear:

A `Vagrantfile` has been placed in this directory. You are now
ready to `vagrant up` your first virtual environment! Please read
the comments in the Vagrantfile as well as documentation on
`vagrantup.com` for more information on using Vagrant.

Then, we’ll use git (for more information on git, check out Using Git and GitHub to Manage Your Dotfiles) to clone the Chef cookbooks repository from GitHub. We will not need all of the cookbooks downloaded from this repository, but cookbooks don’t take up much space — it’s okay to keep them in case they’re needed in the future.

# Create a place for the Chef cookbooks
mkdir ~/django_guide/cookbooks

# Change into the newly created ~/django_guide/cookbooks directory
cd ~/django_guide/cookbooks

# Clone the Chef cookbooks repositories as needed (we will use the following cookbooks in this guide)
git clone git://github.com/opscode-cookbooks/apache2.git
git clone git://github.com/opscode-cookbooks/apt.git
git clone git://github.com/opscode-cookbooks/build-essential.git
git clone git://github.com/opscode-cookbooks/git.git
git clone git://github.com/opscode-cookbooks/vim.git

At this point the django_guide directory should look like this:

django_guide/
--------- Vagrantfile
--------- cookbooks/
    --------- apache2/
    --------- apt/
    --------- build-essential
    --------- git
    --------- python
    --------- vim

What is a Vagrantfile?

vagrant is controlled completely from the command-line. This is what makes it so powerful. We can manage our VirtualBox VMs without ever leaving our terminal window. The Vagrantfile is a special file that is read by vagrant, and provides instructions about how to provision a virtual machine from scratch. Open the newly created Vagrantfileand edit it to look exactly like this:

Vagrant::Config.run do |config|
  config.vm.define :djangovm do |django_config|
    # Every Vagrant virtual environment requires a box to build off of.
    django_config.vm.box = "lucid64"

    # The url from where the 'config.vm.box' box will be fetched if it
    # doesn't already exist on the user's system.
    django_config.vm.box_url = "http://files.vagrantup.com/lucid64.box"

    # Forward a port from the guest to the host, which allows for outside
    # computers to access the VM, whereas host only networking does not.
    django_config.vm.forward_port 80, 8080
    django_config.vm.forward_port 8000, 8001

    # Enable provisioning with chef solo, specifying a cookbooks path (relative
    # to this Vagrantfile), and adding some recipes and/or roles.
    #
    django_config.vm.provision :chef_solo do |chef|
      chef.cookbooks_path = "cookbooks"
      chef.add_recipe "apt"
      chef.add_recipe "apache2::mod_wsgi"
      chef.add_recipe "build-essential"
      chef.add_recipe "git"
      chef.add_recipe "vim"
    #
    #   # You may also specify custom JSON attributes:
    #   chef.json = { :mysql_password => "foo" }
    end
  end
end

Let’s take a moment to analyze the above Vagrantfile to see what’s actually going on in it.

  • django_config.vm.box = "lucid64" – Every virtual machine created with vagrant is based off of something called a base box. This is typically a very minimal image of an operating system without any special customizations applied to it, specifically designed to be used as a foundation. We’ll be using Ubuntu 10.04 “Lucid Lynx” 64-bit as the base box for our Django VM. Here, we’re telling vagrant just that.
  • django_config.vm.box_url = "http://files.vagrantup.com/lucid64.box" – If the base box we want to use hasn’t already been downloaded to our computer, vagrant is smart enough to download it from the Internet, provided we tell it where to look. The address here is the web address where vagrant can find the base box we want to use.
  • django_config.vm.forward_port 80, 8080 and config.vm.forward_port 8000, 8001 – These lines set up port forwarding. Port forwarding allows network access to virtual machine ports. In our virtual machine, the Django test server will run on port 8000 by default. Any ports defined here can be accessed from our workstation. Traffic to these ports will get forwarded to our virtual machine. This makes for much quicker and more convenient testing. For example, to access our website, we can open a web browser on our workstation and go to http://localhost:8001. This request will get forwarded into the virtual machine, where Django will render our website to be sent back and displayed by our web browser.
  • django_config.vm.provision :chef_solo do |chef| – This section loops through the cookbooks and recipes contained within in our cookbooks directory on our workstation. It uses these cookbooks to automatically install and configure all of the listed recipes. For example, chef.add_recipe "vim" installs the vim text editor on our virtual machine.

Starting the Django VM

To start our VM, simply type:

vagrant up

A bunch of interesting stuff will start to scroll by in our terminal. Here’s an excerpt of what we might see after running vagrant up:

[djangovm] Importing base box 'lucid64'...
[djangovm] Matching MAC address for NAT networking...
[djangovm] Clearing any previously set forwarded ports...
[djangovm] Forwarding ports...
[djangovm] -- 22 => 2222 (adapter 1)
[djangovm] -- 80 => 8080 (adapter 1)
[djangovm] -- 8000 => 8001 (adapter 1)
[djangovm] Creating shared folders metadata...
[djangovm] Clearing any previously set network interfaces...
[djangovm] Booting VM...
[djangovm] Waiting for VM to boot. This can take a few minutes.
[djangovm] VM booted and ready for use!
[djangovm] Mounting shared folders...
[djangovm] -- v-root: /vagrant
[djangovm] -- v-csc-1: /tmp/vagrant-chef-1/chef-solo-1/cookbooks
[djangovm] Running provisioner: Vagrant::Provisioners::ChefSolo...
[djangovm] Generating chef JSON and uploading...
[djangovm] Running chef-solo...

SSH Into the Virtual Machine

To SSH into our djangovm VirtualBox VM, type:

vagrant ssh djangovm

This will drop us a terminal where we can begin to work on the VM. There isn’t much more to do before we have a working Django VM.

Final Steps

Install pip, a Python package installer:

sudo apt-get install python-pip

Install Django using pip:

sudo pip install django

Testing the Django Install

To test our Django install we will start a web server on djangovm, then we’ll try to access it from our host workstation in a web browser. To create and initialize a directory ~/django_project for use with Django and start a web server:

# Change to our home directory
cd

# Create and initialize a new directory called django_project
django-admin.py startproject django_project

# Change to the django_project directory
cd ~/django_project

# Start a web server that is accessible from anywhere
python manage.py runserver [::]:8000

This should produce the following output:

Validating models...

0 errors found
Django version 1.4, using settings 'django_project.settings'
Development server is running at http://[::]:8000/
Quit the server with CONTROL-C.

The Django web server is now running. On our host workstation we should be able to access our Django-powered website by going to http://localhost:8001. Recall that this works because in our Vagrantfile we set up a port forwarding rule that forwards requests to localhost:8001 to port 8000 on djangovm.

That’s it! Now we’re ready to start developing with Django.

Note: The Django Book is a great resource for getting started!

Comments

32 responses to “Setup a Django VM with Vagrant, VirtualBox, and Chef”

  1. Juanje Avatar

    Hi, great post 🙂

    I got some minor advises to avoid some common issues if you don’t mind.

    When Vagrant try to run the Chef-solo, it load all the cookbooks at the indicated path. This could give you some troubles trying to load some cookbooks that actually you don’t need. For example, the Windows’ cookbook:
    http://stackoverflow.com/questions/8027310/chef-error-loaderror-no-such-file-to-load-win32-open3

    I rather recomend you to download just the cookbooks you need. And now the official cookbooks from Opscode have been moved to separated repos. I link you a gist with alternate steps:
    https://gist.github.com/2210054

    Another issue is the recipes order. I’d put first the ‘apt’ recipe and then the rest. This recipe will update the ‘apt’ cache at the VM. In my fresh VM with your Vagrantfile, the recipe ‘apache2::mod_wsgi’ gave me an error because the cache hadn’t been updated.
    If you move up the recipe ‘apt’ the problem will solve. The order in Chef is very important.

    Thanks for this post and I hope this help you.

    1. Michael Smalley Avatar

      I appreciate you taking the time to comment and put that Gist together. I’ve slightly tweaked the guide to account for your suggestions. I think this is a perfect opportunity to mention that my primary motive for maintaining this blog is helping to teach others about things they may have otherwise deemed too difficult or confusing to grasp. I don’t like the elitist notion of privileged knowledge. I deeply believe that technology is something that belongs to and should be accessible to everyone. When I can do something better, I always want to know about it. Comments like yours help in this effort, so thank you for contributing and helping to make this a better resource for everyone.

      Also, while I’m at it: I want to send out a big indirect thanks to Joshua Timberman (Twitter) for his work on writing most of the awesome Chef cookbooks that are available today, and for originally pointing out that only required cookbooks should be downloaded.

      1. Juanje Avatar

        Thanks Michael for your kinds words. Actually I believe the same about sharing knowlege.
        I found your post a really easy and simple point of entry for many people to these awesome tools. Sometimes documentation and tutorials about them get overcomplicated.

        Maybe I’m abusing of your kindness, but I think it could be good for the tutorial to see the whole setup automated with Chef. You could replace the manual installation of Django with a recipe and all the setup would be ready to use from the begining and every time.

        With this few lines you have it done and could be useful to see the Chef potencial:
        https://gist.github.com/2212077

        Well, there is the idea (that I’ve already tested with your setup), you will explain it better if you like.

        Thanks for your time and the sharing.

  2. Gilson Filho Avatar

    I suggest my gist for installing last version of the Django using chef solo: https://gist.github.com/2141645

    For that, need checkout django cookbook of this repository: https://github.com/tarsis/vagrant-ci

    It’s a small suggestion. We can get this cookbook django and make a repository only for him. What do you think?

    1. Michael Smalley Avatar

      Gilson,

      Thank you for your suggestion. While writing this guide I did strongly consider including a Django cookbook so that no further configuration would be required. This would further illustrate how much of a time-saver this entire process can be. The reason I ended up omitting the Django cookbook is because currently Opscode doesn’t offer an official Django cookbook in their repository. Opscode is very strict (for good reason) about the quality of the cookbooks in their repository. I simply cannot guarantee this level of quality from third party cookbooks. If I suggest a cookbook, I need to make sure it will continue to work properly for as long as people read and follow this post.

      That said, depending on demand, I may expand on this guide by writing one on custom cookbooks and alternative sources for cookbooks (like the repo you suggested), so stay tuned!

      In the meantime, anyone who wants to try the cookbooks you posted are more than welcome to do so by putting them in the ~/django_guide/cookbooks/ directory and making the appropriate changes to the ~/django_guide/Vagrantfile.

      Thank you again for commenting!

  3. Tharshan Avatar
    Tharshan

    Hi,
    Thanks for the tutorial but I have been having alot of trouble booting up the actual VM using vagrant. When it gets to this step : [djangovm] Waiting for VM to boot. This can take a few minutes.

    It just hangs, I looked at the VirtualBox preview window and saw a login prompt. I am not really sure whats happening. It loaded once, when I destroyed the vm and reloaded it. However it gave an error like so:

    [Wed, 28 Mar 2012 08:30:44 -0700] ERROR: bash[install-pip] (python::pip line 33) has had an error
    [Wed, 28 Mar 2012 08:30:44 -0700] ERROR: Running exception handlers
    [Wed, 28 Mar 2012 08:30:44 -0700] ERROR: Exception handlers complete
    [Wed, 28 Mar 2012 08:30:44 -0700] FATAL: Stacktrace dumped to /tmp/vagrant-chef-1/chef-stacktrace.out
    [Wed, 28 Mar 2012 08:30:44 -0700] FATAL: Chef::Exceptions::ShellCommandFailed: bash[install-pip] (python::pip line 33) had an error: Expected process to exit 0, but it exited with 127
    ---- Begin output of "bash"  "/tmp/chef-script20120328-966-qvx7pa-0" ----
    STDOUT: 
    STDERR: /tmp/chef-script20120328-966-qvx7pa-0: line 1: /usr/local/bin/python: No such file or directory
    /tmp/chef-script20120328-966-qvx7pa-0: line 2: /usr/local/bin/easy_install: No such file or directory
    ---- End output of "bash"  "/tmp/chef-script20120328-966-qvx7pa-0" ----
    Ran "bash"  "/tmp/chef-script20120328-966-qvx7pa-0" returned 127
    Chef never successfully completed! Any errors should be visible in the. output above. Please fix your recipes so that they properly complete.
    
    1. Michael Smalley Avatar

      Tharshan,

      This appears to be an issue with how paths are handled in the Python cookbook provided by OpsCode. I did some homework for you and found that there have been some bug reports filed already regarding this. I assume this Gist was created by you and that you’re viperfx over at GitHub. I suggest you try again, but exclude the Python cookbook. For now, just install pip manually after logging in the VM. A default Ubuntu install includes Python anyway, so the only thing needed is pip. I know it’s a pain in the butt, and it negates the whole point of having a cookbook, but until a fix for this gets merged into the OpsCode repository it’s my cleanest suggestion. You’ll be pleased to know that I’ve updated my post to account for this change (hopefully temporary).

      Thank you for letting me know about this!

      – Michael

  4. anon Avatar
    anon

    I have no affiliation with this project, but a simpler way to setup a django server on virtualbox is to use http://www.turnkeylinux.org/django. Although, if you want the educational experience of setting up the server yourself – you’re probably better off following the tutorial.

    1. Michael Smalley Avatar

      I can see this being useful. If configuration isn’t your thing, you may be interested in BitNami, who makes AMI’s available for Amazon EC2 use. They have a Django AMI available and from what I understand it’s pretty solid in terms of stability and security. As for my post, it wasn’t so much intended to be a tutorial on how to install Django as it was a demonstrative tutorial on the power of rolling your own virtual machines with Vagrant + VirtualBox + Chef Solo.

      Thank you for your post!

  5. David Talbot Avatar

    Thanks a lot for this guide – I use a Mac and this really helped me get continue with my Django development (off Ubuntu).

    Very much appreciated!

    1. Michael Smalley Avatar

      You’re quite welcome. Thanks for taking the time to comment.

  6. […] Setup a Django VM with Vagrant, VirtualBox, and Chef […]

  7. […] Setup a Django VM with Vagrant, VirtualBox, and Chef […]

  8. kyle Avatar
    kyle

    this is a great tutorial. helps me get started with django – but also enhances my familiarity with the Vagrant, vbox, chef toolset.

    thanks again.

  9. Aaron Avatar

    I followed this on windows and I had a strange error
    [Sun, 15 Jul 2012 13:19:43 -0400] FATAL: Errno::ENOENT: execute[generate-module-list] (apache2::default li
    ne 83) had an error: No such file or directory – /usr/local/bin/apache2_module_conf_generate.pl /usr/lib64
    /httpd /etc/httpd/mods-available

    turns out that the perl script picked up windows line endings and tried to execute
    !/usr/bin/perl^M

    re-saving the cookbooks with unix line endings solved the problem.

    thanks for the post!

    1. Michael Smalley Avatar

      Good tip! Thanks for reading and commenting!

  10. Kimball Avatar
    Kimball

    Now…if someone could just tell me how to connect to the VM’s harddrive so I can edit files in my text editor of choice on the host machine. Then I would be golden.

    1. Michael Smalley Avatar

      What is your text editor of choice? I might be able to help. If you’re like me, and it’s vim, then you can use a vim plugin called netrw. I recommend you use a tool like pathogen to manage your vim plugins as well. It’ll keep things organized as you get more comfortable with using various plugins simultaneously.

  11. todbot Avatar
    todbot

    Great! Thanks.

    1. Michael Smalley Avatar

      No problem. Thank you for commenting.

    2. Michael Smalley Avatar

      No problem. Pay it forward!

  12. Joe Avatar
    Joe

    This tutorial is suburb. 4 things I wanted to learn about: Django, virtual boxes, Chef, and vim. Boom. I just got out of school where stuff was mostly theoretical or highly structured in an environment setup by the lab manager, so it’s great to learn some practical stuff. Thanks for it!

  13. PeterL Avatar
    PeterL

    With an exception of cookies all worked perfectly – thanks for this excellent post!

    Naive question: I would like to shut down django VM – is it enough to do so using vagrant hal and no need for root ? In other words, no need to shut down ubuntu using a terminal but exclusively vagrant interface?

    Thanks
    Peter

  14. PeterL Avatar
    PeterL

    Additional questions:
    I believe we need root password to install additional packages e.g. django-noose

  15. PeterL Avatar
    PeterL

    PLEASE DISREGARD my two previous emails – vagrant does the setup: Ubuntu has the
    root password= sudo su –

    thanks

  16. Cocobuster Avatar
    Cocobuster

    Thank you, very useful, good explanations! Makes it easy to understand chef with vagrant!

  17. Ian Avatar
    Ian

    Thank you for this helpful post, Michael.
    I followed the steps in the post on a windows 7 laptop as host.
    Everything went well and was able to vagrant up the djangovm,
    and inside windows, via browser, saw the apache default page at
    http://localhost:8080/. So the “forward_port 80, 8080” works.
    However, after I fired out django in the vm, and hop over to windows
    browser to hit http://127.0.0.1:8001/, I got “The connection was reset” problem
    loading page errror. Wonder anyone here have run into similar scenario?

    1. Ian Avatar
      Ian

      Answer my own question: using “python manage.py runserver [::]:8000” as you have described in the post, everything works beautifully. I was using the generic “python manage.py runserver” to fire up the server. I enjoyed this post very much.

      1. Tom Avatar
        Tom

        Thank you for the above suggestion for the port forwarding option! I was getting the connection reset message in Firefox and luckily stumbled upon this. I’m not using Chef but just using Vagrant and manually set up Django on the default Ubuntu ‘precise32’ box, which has been a bit of. I was about to go all in and set up a cookbook with Chef, which I hope to do in the future. This will definitely cut down on tweaking all the different setting across different OS settings. Consider this a lesson learned :). Thanks for this insight Ian and great article Mike!

  18. Laurence Avatar
    Laurence

    A noobish question perhaps but – where is the python directory supposed to have come from? It isn’t clear from the instructions:

    django_guide/
    └── Vagrantfile
    └── cookbooks/
    └── apache2/
    └── apt/
    └── build-essential
    └── git
    └── python <—- How/where do I get this?
    └── vim

  19. Florian Tatzel Avatar
    Florian Tatzel

    “# Start a web server that is accessible from anywhere
    python manage.py runserver [::]:8000”

    This saved my first steps, many thanks to that. 🙂
    I will have to look up whats with that “[::]” notation (which was my only problem). Never saw that before. :\

Leave a Reply

Your email address will not be published. Required fields are marked *