Serving Multiple Sites with Django

Today we’re releasing our dynamicsites code, which we’ve been developing as a pure django solution to serve multiple sites from a single django project.

The code is here: https://bitbucket.org/uysrc/django-dynamicsites/src

Background

Django includes the Django Sites Framework as part of its contrib package, django.contrib.sites (docs).  The documentation for this framework primarily describes example usages:

  • Associating content with multiple sites, or a single site
  • Hooking into the current site from views
  • Getting the current domain for display
  • Getting the current domain for constructing full URLs.

The examples presented in the documentation assume a common database, set of applications, templates, views, urls, public files, and project settings to be shared across multiple sites.

Shortcomings

There are cases where developers would like to share a single Django project installation across multiple sites, with some sites using different settings, different urls, different templates, different public files, and different data. Examples of this include domain portfolio owners, and development shops with numerous clients who sites generally share 80% functionality, but completely look ‘n feel and data.

The documentation for the Django Sites Framework does not support these cases.  Furthermore, the documentation really only describes using a small number of sites (a couple), where we would like to explore a larger number of sites (hundreds, if not thousands)

Our Requirements

  • Host multiple, potentially hundreds, of sites from a single Django project efficiently
  • A site may have its own settings,py, urls.py, apps, templates, views, public files and data
  • A single site may accept requests from multiple hostnames
  • A site must be served from a single canonical hostname.  Any other hostnames supported by the site must permanently redirect to the canonical hostname, preserving the request path, query args, method, protocol, port, and subdomain as best as possible.
  • Multiple sites may be hosted under a single hostname by using subdomains
  • A site’s hostname may use international and vanity “gTld”‘s (for when ICANN fully commits to them, that is)
  • We will need to access sites in our development and test environments using different hostnames than production
  • The solution should be implemented entirely in Django
  • The should must function with django-nonrel on Google App Engine

A note on redirects

A typical domain portfolio will need to account for intellectual property protection, and redirecting typos or alternate keywords.  These are very common use cases, for example try navigating to http://finance.yhoo.com/ and notice you’re redirected to finance.yahoo.com.  (Note, as I write this, http://news.googel.com/ only redirects you to www.google.com — where’s my news??)   While a very logically oriented programmer type might define these as user errors, the business would like to try our best to get the user where we intended them to go.   Additionally while these types of redirects can be handled in DNS or Apache/web server configs, we’d like to use an all-inclusive Django based solution, if feasable.

Example Sites Structure

The sites presented here are all fictitious.  If somebody registers these domains and fills them with spam, malware, pr0n or any other offensive material, it’s not our fault.  We didn’t do it.

We’re going to mix a lot of requirements together in one big bundle, as if a marketing manager was giving them to us.  We’re not looking for a one-tool-for-everything solution here, although we would like the solution to be contained in django as much as possible.

Example Configuration:

Site 1 (default site)

  • Name: Corporate Umbrella Site
  • Site hostname: corp-umbrella-site.com
  • Which hostnames redirect here: *Every request that comes to the django project for which it cannot find a site will permanently redirect to this default site.
  • Subdomains this site supports:  www
  • Subdomain redirects: Any non-www subdomains should permanently redirect to www.
  • Hostname in dev environment: cus.dev
  • Hostname in test environment: cus.test

Site 2

  • Name: Site About Food
  • Site hostname: about-food.com
  • Hostnames redirects: aboutfood.com, about-food.net
  • Subdomains this site supports: www, fruit, meat, vegetables, dairy
  • Subdomain redirects: meats.about-food.com should permanently redirect to meat.about-food.com, fruits to fruit, vegetable to vegetables, and diary to dairy.
  • Hostname in dev environment: af.dev
  • Hostname in test environment: af.test

Site 3

  • Name: About Food Subdomain Site
  • Site hostname: restaurants.about-food.com
  • Subdomains this site supports: None (it already has one: restaurants)
  • Subdomain redirects: restaurant.about-food.com should redirect to restaurants.about-food.com, as well as dining
  • Hostname in dev environment: res.af.dev
  • Hostname in test environment: res.af.test

Site 4

  • Name: About Food Brazil Site
  • Site hostname: sobre-comida.com.br
  • Subdomains this site supports: www, fruta, carne, legumes, leite
  • Subdomain redirects: carnes.sobre-comida.com.br should permanently redirect to carne.sobre-comida.com.br, frutas to fruta, legume to legumes, and leites to leite.
  • Hostname in dev environment: sc.dev
  • Hostname in test environment: sc.test

Site 5

  • Name: About Games Site
  • Site hostname: about.gam.es
  • Hostnames redirects: about-games.com redirects to about.gam.es
  • Subdomains this site supports: None (it already has one: about)
  • Subdomain redirects: Any non-about subdomains should permanently redirect to about
  • Hostname in dev environment: ag.dev
  • Hostname in test environment: ag.test

Step 1: Set Up A Test Project & Environment

It is assumed you can set up a test project.  We’re using the django-testapp from allbuttonspressed in a Google App Engine environment, modified to include the admin interface.  I’m also going to run the same tests on a mysql environment, just to be sure.  If you can’t get that far on your own, at least with another datastore and environment, you probably shouldn’t be reading this article but rather a more basic intro-to-django article instead.

Hosts file

We’ll be running this test-setup on an OSX development machine, and using /etc/hosts file for hostname management.  Here’s what the /etc/hosts file looks like for (most of) the above configuration:

127.0.0.1    corp-umbrella-site.com
127.0.0.1    www.corp-umbrella-site.com
127.0.0.1    garbage.www.corp-umbrella-site.com
127.0.0.1    garbage.corp-umbrella-site.com
127.0.0.1    cus.dev
127.0.0.1    www.cus.dev
127.0.0.1    garbage.cus.dev
127.0.0.1    cus.test
127.0.0.1    www.cus.test
127.0.0.1    garbage.cus.test

127.0.0.1    about-food.com
127.0.0.1    www.about-food.com
127.0.0.1    fruit.about-food.com
127.0.0.1    meat.about-food.com
127.0.0.1    vegetables.about-food.com
127.0.0.1    dairy.about-food.com
127.0.0.1    meats.about-food.com
127.0.0.1    fruits.about-food.com
127.0.0.1    vegetables.about-food.com
127.0.0.1    diary.about-food.com
127.0.0.1    garbage.about-food.com

127.0.0.1    aboutfood.com
127.0.0.1    www.aboutfood.com
127.0.0.1    fruit.aboutfood.com
127.0.0.1    meat.aboutfood.com
127.0.0.1    vegetables.aboutfood.com
127.0.0.1    dairy.aboutfood.com
127.0.0.1    meats.aboutfood.com
127.0.0.1    fruits.aboutfood.com
127.0.0.1    vegetables.aboutfood.com
127.0.0.1    diary.aboutfood.com
127.0.0.1    garbage.aboutfood.com

127.0.0.1    about-food.net
127.0.0.1    www.about-food.net
127.0.0.1    fruit.about-food.net
127.0.0.1    meat.about-food.net
127.0.0.1    vegetables.about-food.net
127.0.0.1    dairy.about-food.net
127.0.0.1    meats.about-food.net
127.0.0.1    fruits.about-food.net
127.0.0.1    vegetables.about-food.net
127.0.0.1    diary.about-food.net
127.0.0.1    garbage.about-food.net

127.0.0.1    af.dev
127.0.0.1    www.af.dev
127.0.0.1    fruit.af.dev
127.0.0.1    meat.af.dev
127.0.0.1    vegetables.af.dev
127.0.0.1    dairy.af.dev
127.0.0.1    meats.af.dev
127.0.0.1    fruits.af.dev
127.0.0.1    vegetables.af.dev
127.0.0.1    diary.af.dev
127.0.0.1    garbage.af.dev

127.0.0.1    restaurants.about-food.com
127.0.0.1    restaurant.about-food.com
127.0.0.1    dining.about-food.com
127.0.0.1    garbage.dining.about-food.com
127.0.0.1    garbage.restaurant.about-food.com
127.0.0.1    garbage.restaurants.about-food.com

127.0.0.1    res.af.dev
127.0.0.1    restaurants.af.dev
127.0.0.1    restaurant.af.dev
127.0.0.1    dining.af.dev
127.0.0.1    garbage.dining.af.dev
127.0.0.1    garbage.restaurant.af.dev
127.0.0.1    garbage.restaurants.af.dev

127.0.0.1    sobre-comida.com.br
127.0.0.1    www.sobre-comida.com.br
127.0.0.1    fruta.sobre-comida.com.br
127.0.0.1    carne.sobre-comida.com.br
127.0.0.1    legumes.sobre-comida.com.br
127.0.0.1    leite.sobre-comida.com.br
127.0.0.1    carnes.sobre-comida.com.br
127.0.0.1    frutas.sobre-comida.com.br
127.0.0.1    legume.sobre-comida.com.br
127.0.0.1    leites.sobre-comida.com.br
127.0.0.1    garbage.sobre-comida.com.br
127.0.0.1    garbage.www.sobre-comida.com.br
127.0.0.1    garbage.fruta.sobre-comida.com.br

127.0.0.1    sc.dev
127.0.0.1    www.sc.dev
127.0.0.1    fruta.sc.dev
127.0.0.1    carne.sc.dev
127.0.0.1    legumes.sc.dev
127.0.0.1    leite.sc.dev
127.0.0.1    carnes.sc.dev
127.0.0.1    frutas.sc.dev
127.0.0.1    legume.sc.dev
127.0.0.1    leites.sc.dev
127.0.0.1    garbage.sc.dev
127.0.0.1    garbage.www.sc.dev
127.0.0.1    garbage.fruta.sc.dev

127.0.0.1    about.gam.es
127.0.0.1    garbage.gam.es
127.0.0.1    garbage.about.gam.es

127.0.0.1    about.ag.dev
127.0.0.1    garbage.ag.dev
127.0.0.1    garbage.about.ag.dev

Step 2: Try To Use What Is Given To Us

Sites Framework

While it appears the Django Sites Framework does not support our needs, let us begin by seeing how far we can get with what is provided to us via django.contrib.sites. A lot of django plugins already are designed to work with the sites framework, so it would be best to integrate with it than have to rewrite/extend so many plugins.

First, I created all the sites in the admin interface:

Configuring the Django Sites Framework

Out of the box, the Django Sites Framework wants a SITE_ID defined in the global settings.py.  This won’t work for us, as we want the SITE_ID to be dynamic.

Djangotoolbox.sites.dynamicsite

The folks at allbuttonspressed has a djangotoolbox project available, with a dynamic site middleware component (code).  Let’s try using this with our configuration.

The first thing I notice is that the middleware is trying to find the site using the request.get_host() with the port number, unless the port number is 80 or 443.  The problem I’m encountering is my development environment is running by default on port 8000, yet none of the sites I defined are defined with port 8000.

  1. I don’t want to define the site by port number.  I’m using 8000 as my dev environment.  I may use some other port later on for my dev environment and I don’t want to go back and change all my site definitions.  We’ll probably run the test environment on a different port, also.
  2. To host multiple sites over SSL you need to have a unique IP address or port number for each site.  Large hosting farms will usually host HTTPS over nonstandard ports for large numbers of virtual hosts.  Again, I don’t want to have to manage these port numbers in the Site definition.

For these reasons I’m going to remove the port number all together from the domain being passed to Site.objects.get(domain=domain).  Let the forking begin!

The second thing I notice is lack of subdomain support, which we’ll need.  With the dynamic site module, it is stripping off “www” subdomain from request.get_host() before doing the site lookup.  However, for our about-food site, which supports a number of subdomains, we need to create an individual site for each subdomain.  What I need is a way to find the site, and then determine if the subdomain that came with the request is supported or not.  If it’s not supported, redirect to the default subdomain for that site.

Redirect App

Before jumping in and writing our own middleware, let’s take a quick check at any possible solutions for our hostname redirecting needs.

Django provides a redirects app, django.contrib.redirects (docs). however it is not designed to work with the hostname, with specific instructions on the admin form not to use the hostname.  So, out of the box, this app will not support the various hostname redirect requirements we have.  This makes sense, as we should have the site serving from the correct hostname before we try to make any path adjustments.

Note the text: This should be an absolute path, excluding the domain name. Example: '/events/search/'.

Enforce Hostname Middleware

From http://code.djangoproject.com/ticket/12662 – Enforce Hostname Middleware we are led to some middleware to enforce a hostname: https://gist.github.com/283860.  This will support preserving the protocol and query args, and will support allowing multiple hostnames, however in the case an unexpected hostname is found, it will redirect only to the first hostname in the ENFORCE_HOSTNAME settings.  It doesn’t appear to check for port, or anything but HTTP GET requests, neither.  Ultimately, it’s a good base and on-the-right-track, so let’s take what we can from it and combine it with what we’ll be taking from allbuttonspressed’s Djangotoolbox.sites.dynamicsite code.

Requirements for hostname redirect middleware

  • Must maintain protocol, port, and path information when redirecting
  • Must redirect to a canonical hostname
  • Must redirect to a canonical subdomain, noting that a site may be configured with multiple subdomains
  • If a hostname cannot be found, must redirect to the default site (preserving protocol, port, and path should the default site care to throw a 404 error, or redirect of its own)
  • Should provide an admin configuration form along the lines of the django redirects app

Step 3: Let the coding begin!

I’m going to borrow parts of the dynamicsite code from allbuttonspressed to strip the port number and not create any new sites dynamically. Secondly, let’s introduce an algorithm for handling subdomains.   Lastly, I’ll pull in parts from the Enforce Hostname Middleware and modify them for our test cases above.  Once it’s all coded up, and tested, I’ll put the first-pass up on a github account for anyone to play with, study, and submit feedback/code patches.

Extend the Django.contrib.sites model

If you ask around for how to extend an existing model in Django, you’ll find varying opinions on implementation preference.  I decided to go for one of the more discouraged routes, to dynamically extend the Site class, adding a “subdomains” field to store a comma separated list of strings, rather than creating a subclass.  Why?  Because I didn’t want a separate table just to store subdomains.  I just really want the subdomains as a Python list to see whether the subdomain requested is supported by the site or not.  I have no need for a many to many relationship and the overhead involved, both in coding time and system performance.

This is actually pretty easy to do on the model side.  Just one line of code really takes care of it:

SubdomainListField(blank=True).contribute_to_class(Site,'subdomains')

Of course, the custom model field, form field, and widget, are a little more involved, basically converting the comma separated string into a Python list internally. I could have opted to simply persist a serialized form of the Python list, however I thought that would be too non-portable a solution.  In the end, I’ve got a textarea added to the django sites admin form which accepts a comma separated list of subdomains, cleaning them up and validating them against Django’s URLValidator (which I realize has bugs, however the team seems to be working to fix)

I also added a “folder_name” field, as the site specific configuration and templates will go in site specific folders under a common SITES_DIR.  I thought about trying to a) scan a folder at runtime and extract configuration dynamically (cool, but more work, so I skipped it) and then, secondly, trying to auto-deduce a folder name from the hostname, however in the end that left for some ugly looking folder names.  So I added this extra field to the Site class so the developer/admin can define from which folder the site config will be read, along with some jQuery to auto populate the field.

An screen shot of the modified Site admin screen is presented below:

Note, for my initial implementation, I’m not looking to support dynamic subdomains, like 37signals products generally do, for example, using account names as subdomains.  When I need to support {account_name}.{site_domain} I’ll come back and extend this code again to tie in with an external accounts system, likely implementing the external table I decided not to use for now.  So, if you want {account_name}.{site_domain} functionality, it’s not here (yet).

The Heart and Meat Of It All: The Middleware

I put the hostname redirects in a dictionary inside of settings.py.

DEFAULT_HOST = 'www.corp-umbrella-site.com'
HOSTNAME_REDIRECTS = {
'aboutfood.com':              'www.about-food.com',
'about-food.net':             'www.about-food.net',
'meats.about-food.com':       'meat.about-food.com',
'fruits.about-food.com':      'fruit.about-food.com',
'vegetable.about-food.com':   'vegetables.about-food.com',
'diary.about-food.com':       'dairy.about-food.com',
'restaurant.about-food.com':  'restaurants.about-food.com',
'dining.about-food.com':      'restaurants.about-food.com',
'carnes.sobre-comida.com.br': 'carne.sobre-comida.com.br',
'frutas.sobre-comida.com.br': 'fruta.sobre-comida.com.br',
'legume.sobre-comida.com.br': 'legumes.sobre-comida.com.br',
'leites.sobre-comida.com.br': 'leite.sobre-comida.com.br',
'about-games.com':            'about.gam.es'
}

An admin panel for the redirects was defined as a “should” requirement, and so we’ll punt for the next revision to make one.  Besides, since this redirect data will be accessed on every request, I felt it would make sense to have it defined in python code.  Later we can look at caching the entire table in memcache if the settings file proves to be a headache.

Next, a function to clean up django’s request.get_host() to separate the host and the port number, and make sure the host is indeed lowercase.

    def get_domain_and_port(self):
        """
        Django's request.get_host() returns the requested host and possibly the
        port number.  Return a tuple of domain, port number.
        Domain is lowercased
        """
        if ':' in self.request.get_host():
            domain, port = self.request.get_host().split(':')
            return (domain.lower(), port)
        else:
            return (self.request.get_host().lower(),
                self.request.META.get(SERVER_PORT))

Then, we get into the main loop, which will split apart the requested domain name, subdomain, by subdomain, until a matching site is found with the lookup function. You can check out the source code for this implementation.

The lookup function does a quick check in the cache to see if the site_id is already in the cache (ala allbuttonspressed code) and if not, checks the database for any sites matching the currently requested domain.  If no site is found, the main loop proceeds to the next subdomain.

This way, we can handle international domain names and sites with numerous levels of subdomains with more capability.  I couldn’t rely on any code assuming that a domain will always be structured as:  {subdomain}.{hostname}.{tld} because we want to develop sites for international audiences, like here in Uruguay, where all of our tlds have our country code at the end.  (.uy)

In the first release of dynamic sites, I have a redirect function that only supports HTTP GET, for now. The requirements said to retain the HTTP method as best as possible, however I suspect that GET is only really necessary.

    def redirect(self, new_host, subdomain=None):
        """
        Tries its best to preserve request protocol, port, path,
        and query args.  Only works with HTTP GET
        """
        return HttpResponsePermanentRedirect('%s://%s%s%s%s%s' % (
            self.request.is_secure() and 'https' or 'http',
            (subdomain) and '%s.' % subdomain or '',
            new_host,
            (int(self.port) not in (80, 443)) and ':%s' % self.port or '',
            urlquote(self.request.path),
            (self.request.method == 'GET'
                and len(self.request.GET) > 0)
                    and '?%s' % self.request.GET.urlencode() or ''
        ))

Lastly, thanks to http://effbot.org/zone/django-multihost.htm, we add a process_response handler to call patch_vary_headers and instruct the Django caching system that the cache keys must depend on the HTTP_HOST, instead of just the request path (default). [docs]

Step 4: Dynamically Loading urls.py, and modifying settings. TEMPLATE_DIRS by site

Here’s where the real power comes in.  Now that we have our site identified, we need to add some logic to modify from where urls.py and templates are loaded.

To do this, I ripped out a function from djangotoolbox called make_tls_property which, from the pydoc, “Creates a class-wide instance property with a thread-specific value.”

In short, I’m able to dynamically modify values from settings.py on a per request basis without worrying about upsetting other threads.

You can see an example of this in dynamicsites/middleware.py

SITE_ID = settings.__class__.SITE_ID = make_tls_property()
TEMPLATES_DIR = settings.__class__.TEMPLATES_DIR = make_tls_property()
...
self.site = Site.objects.get(domain=domain)
SITE_ID.value = self.site.pk

Notice, that by dynamically loading a site-specific urls.py, you can specify exactly which views will be used for which URL’s, giving you dynamic views per site.

Note: You must define the sites in the database (using the admin panel or some other means) *before* enabling the middleware.  If you enable the middleware, and no sites are defined in the database, your django site will throw 404′s for all requests.

Step 5: Dynamically map development environment hostnames to sites

In my development environment, I use very short codenames for the sites I’m developing.  Mostly so I can access the site via my browser’s address bar as fast as possible, but also so I have a strong visual indicator which environment I’m looking at, to avoid making painful mistakes like accidentally editing data in the production environment.

Compare the following:

http://www.sobre-comida.com.br/some/path

http://www.sc.dev:8000/some/path

We just need to add an extra ENV_HOSTNAMES setting and a little extra logic to the middleware layer to support different hostnames depending on the environment. Note you’ll need to add the correct magic to your settings.py or deployment scripts to set ENV_HOSTNAMES correctly for your local dev, staging, test environments. ENV_HOSTNAMES should not be used in production!

    'cus.dev':    'corp-umbrella-site.com',
    'af.dev':     'about-food.com',
    'res.af.dev': 'restaurants.about-food.com',
    'sc.dev':     'sobre-comida.com.br',
    'ag.dev':     'about.gam.es'

Step 6: Context Processor

Well, one last point.  Let’s add the current site as a variable available to all templates via a context processor.

def current_site(request):
    return (settings.SITE_ID) and {'site': Site.objects.get_current()} or None

If you notice in the screenshot for the modified admin interface above, it prints the current site name in the header bar. This was easy to do using this new context processor and a simple template override:

Simple make a /templates/admin/base_site.html template as such:

{% extends "admin/base.html" %}
{% load i18n %}

{% block title %}{{ title }} | {{ site.name }} {{% trans 'Administration' %}{% endblock %}

{% block branding %}
<h1 id="site-name">{{ site.name }} {% trans 'Administration' %}</h1>
{% endblock %}

{% block nav-global %}{% endblock %}

Wrapping It All Up

I encourage you to grab the source and set up your own little test environment. I’d be very interested to hear back any results of your tests, if you find any bugs, or if you have any suggestions for improvements to the code. In the case of the latter, I prefer to receive code samples or a link to a forked project on Mercurial.

UYSRC

References

Share
  • White

    Thanks for the detailed tutorial. I’m trying to adapt your code for my needs and I’ve run in 2 issues;

    1) https://bitbucket.org/uysrc/django-dynamicsites/src/2f930feac3bb/dynamicsites/middleware.py#cl-105 here shouldn’t this be TEMPLATE_DIRS and not TEMPLATES_DIR?

    2) on my stuff when I’m trying to add the new folder to TEMPLATE_DIR I get an UnboundLocalError: local variable referenced before assignment error.

    • Anonymous

      Yes you are correct. I am fixing this and another related item I found. Thanks for pointing this out!

      • E Lavoine

        thnx for the app. It helps me a lot. But is it not unsecured to change dynamically the SITE_ID? and how many hostnanes can be supported here? I want to use it for more than 300 hostnames.

        • Anonymous

          Please see:  http://permalink.gmane.org/gmane.comp.python.django.devel/30806

          It is from this make_tls_property() function that we can modify the SITE_ID dynamically.

          If you can, I’d encourage you to set up a simulation of 300 sites using dynamicsites.  Dynamicsites comes with a “site_info” view which displays which site dynamicsites thinks it’s seeing.  You could set up a database and an environment with 300 test sites, and then write a multi-threaded test client (or just run multiple test client processes) which requests a random hostname and checks the output from site_info to ensure the site dynamicsites is reporting is correct.

          I’d encourage you to run this kind of test no matter which solution you use for hosting multiple sites in django.  If you run the test with dynamicsites, I’d love to hear any feedback.  

          Good luck!

          • Django_guest

            This is an app i was exactly looking for. Two questions:

            1. Has anybody used this on shared hosting like WebFaction as production code? How the memory consumption is compared to separate projects with defined site_ids in each settings file? In particular i am curious to know whether this has any advantage over virtualhosts with timeout for passive sites in regards to memorycache? Keeping the django instance up to serve all the sites with their cached pages make me wonder vs taking some of these sites down when timeout hits with virtualhost.

            2. I see some branches in Bitbucket. Is the upstream most up-to-date version now?

          • Anonymous

            yes the upstream is the most up-to-date version

            it is an interesting study to compare the memory utilizations of #1, virtual hosts w/ timeout vs. one vhost and all sites in django.  It will depend on your application type, of course.  If it’s a wide variety of sites with much different function/content, I’d suggest to go w/ multiple vhosts.  If most of the functionality is generally the same across all sites, it may be simpler to load it all at once.  

    • Anonymous

      This has been fixed with the latest release.  Many thanks to crass on bitbucket for their bugfixes!

  • White

    Thanks for the detailed tutorial. I’m trying to adapt your code for my needs and I’ve run in 2 issues;

    1) https://bitbucket.org/uysrc/django-dynamicsites/src/2f930feac3bb/dynamicsites/middleware.py#cl-105 here shouldn’t this be TEMPLATE_DIRS and not TEMPLATES_DIR?

    2) on my stuff when I’m trying to add the new folder to TEMPLATE_DIR I get an UnboundLocalError: local variable referenced before assignment error.

    • uysrc

      Yes you are correct. I am fixing this and another related item I found. Thanks for pointing this out!

      • E Lavoine

        thnx for the app. It helps me a lot. But is it not unsecured to change dynamically the SITE_ID? and how many hostnanes can be supported here? I want to use it for more than 300 hostnames.

        • uysrc

          Please see:  http://permalink.gmane.org/gmane.comp.python.django.devel/30806

          It is from this make_tls_property() function that we can modify the SITE_ID dynamically.

          If you can, I’d encourage you to set up a simulation of 300 sites using dynamicsites.  Dynamicsites comes with a “site_info” view which displays which site dynamicsites thinks it’s seeing.  You could set up a database and an environment with 300 test sites, and then write a multi-threaded test client (or just run multiple test client processes) which requests a random hostname and checks the output from site_info to ensure the site dynamicsites is reporting is correct.

          I’d encourage you to run this kind of test no matter which solution you use for hosting multiple sites in django.  If you run the test with dynamicsites, I’d love to hear any feedback.  

          Good luck!

          • Django_guest

            This is an app i was exactly looking for. Two questions:

            1. Has anybody used this on shared hosting like WebFaction as production code? How the memory consumption is compared to separate projects with defined site_ids in each settings file? In particular i am curious to know whether this has any advantage over virtualhosts with timeout for passive sites in regards to memorycache? Keeping the django instance up to serve all the sites with their cached pages make me wonder vs taking some of these sites down when timeout hits with virtualhost.

            2. I see some branches in Bitbucket. Is the upstream most up-to-date version now?

          • uysrc

            yes the upstream is the most up-to-date version

            it is an interesting study to compare the memory utilizations of #1, virtual hosts w/ timeout vs. one vhost and all sites in django.  It will depend on your application type, of course.  If it’s a wide variety of sites with much different function/content, I’d suggest to go w/ multiple vhosts.  If most of the functionality is generally the same across all sites, it may be simpler to load it all at once.  

    • uysrc

      This has been fixed with the latest release.  Many thanks to crass on bitbucket for their bugfixes!

  • Stephen Jones

    sorry being stupid but i get The folder sites/subsite_1/ does not exist or is missing the __init__.py file
    when editing the sites in admin.
    created project/sites/folder_in_question
    contains
     __init__.py
    url.py
    can’t see what is the problem?
     

    • Anonymous

      Hi Stephen, we pushed some recent updates to django-dynamicsites this morning.  Still working on bug testing, however let me know if this new build may work better for you

  • Stephen Jones

    sorry being stupid but i get The folder sites/subsite_1/ does not exist or is missing the __init__.py file
    when editing the sites in admin.
    created project/sites/folder_in_question
    contains
     __init__.py
    url.py
    can’t see what is the problem?
     

    • uysrc

      Hi Stephen, we pushed some recent updates to django-dynamicsites this morning.  Still working on bug testing, however let me know if this new build may work better for you

  • samuel goldszmidt

    Hello, I try to use your app, but, I have this “DatabaseError” : column django_site.folder_name does not exist
    LINE 1: …d”, “django_site”.”domain”, “django_site”.”name”, “django_si…Before adding dynamicsites, I set up a first django site in django admin.Have you any idea about what’s going wrong in my case ?

    • samuel goldszmidt

      2 things to solve my problem above:
      - you need to run syncdb after dynamicsites is installed (to be sure the fields folder_name and subdomains is added to the standard Site model)
      - in sites folder, each folder must have a __init__.py file.

      • Anonymous

        Thanks Sam I put your notes inside the README.rst file

        • samuel goldszmidt

          Also it could be great to have a setup.py file at root level to install the package with pip

    • Anonymous

      I updated the README w/ the sql needed to fix this

      alter table django_site add column folder_name varchar(255);
      alter table django_site add column subdomains varchar(255);

  • samuel goldszmidt

    Hello, I try to use your app, but, I have this “DatabaseError” : column django_site.folder_name does not exist
    LINE 1: …d”, “django_site”.”domain”, “django_site”.”name”, “django_si…Before adding dynamicsites, I set up a first django site in django admin.Have you any idea about what’s going wrong in my case ?

    • samuel goldszmidt

      2 things to solve my problem above:
      - you need to run syncdb after dynamicsites is installed (to be sure the fields folder_name and subdomains is added to the standard Site model)
      - in sites folder, each folder must have a __init__.py file.

      • uysrc

        Thanks Sam I put your notes inside the README.rst file

        • samuel goldszmidt

          Also it could be great to have a setup.py file at root level to install the package with pip

    • uysrc

      I updated the README w/ the sql needed to fix this

      alter table django_site add column folder_name varchar(255);
      alter table django_site add column subdomains varchar(255);

  • Palmitoto

    Hummm; how to instal this module ?? setup ??

    • Anonymous

      please see https://bitbucket.org/uysrc/django-dynamicsites

  • Palmitoto

    Hummm; how to instal this module ?? setup ??

    • uysrc

      please see https://bitbucket.org/uysrc/django-dynamicsites

  • Wes

    It seems installing dynamicsites breaks save() on any of my site objects.  I’m including an example from ./manage.py syncdb.


    Traceback (most recent call last):
      File "./manage.py", line 14, in
        execute_manager(settings)
      File "/opt/django-trunk/django/core/management/__init__.py", line 442, in execute_manager
        utility.execute()
      File "/opt/django-trunk/django/core/management/__init__.py", line 379, in execute
        self.fetch_command(subcommand).run_from_argv(self.argv)
      File "/opt/django-trunk/django/core/management/base.py", line 191, in run_from_argv
        self.execute(*args, **options.__dict__)
      File "/opt/django-trunk/django/core/management/base.py", line 220, in execute
        output = self.handle(*args, **options)
      File "/opt/django-trunk/django/core/management/base.py", line 351, in handle
        return self.handle_noargs(**options)
      File "/opt/django-trunk/django/core/management/commands/syncdb.py", line 109, in handle_noargs
        emit_post_sync_signal(created_models, verbosity, interactive, db)
      File "/opt/django-trunk/django/core/management/sql.py", line 190, in emit_post_sync_signal
        interactive=interactive, db=db)
      File "/opt/django-trunk/django/dispatch/dispatcher.py", line 172, in send
        response = receiver(signal=self, sender=sender, **named)
      File "/opt/django-trunk/django/contrib/sites/management.py", line 14, in create_default_site
        s.save(using=db)
      File "/opt/django-trunk/django/contrib/sites/models.py", line 51, in save
        super(Site, self).save(*args, **kwargs)
      File "/opt/django-trunk/django/db/models/base.py", line 464, in save
        self.save_base(using=using, force_insert=force_insert, force_update=force_update)
      File "/opt/django-trunk/django/db/models/base.py", line 547, in save_base
        for f in meta.local_fields if not isinstance(f, AutoField)]
      File "/opt/django-trunk/django/db/models/fields/__init__.py", line 276, in get_db_prep_save
        return self.get_db_prep_value(value, connection=connection, prepared=False)
    TypeError: get_db_prep_value() got an unexpected keyword argument 'connection'

  • Wes

    It seems installing dynamicsites breaks save() on any of my site objects.  I’m including an example from ./manage.py syncdb.


    Traceback (most recent call last):
      File "./manage.py", line 14, in
        execute_manager(settings)
      File "/opt/django-trunk/django/core/management/__init__.py", line 442, in execute_manager
        utility.execute()
      File "/opt/django-trunk/django/core/management/__init__.py", line 379, in execute
        self.fetch_command(subcommand).run_from_argv(self.argv)
      File "/opt/django-trunk/django/core/management/base.py", line 191, in run_from_argv
        self.execute(*args, **options.__dict__)
      File "/opt/django-trunk/django/core/management/base.py", line 220, in execute
        output = self.handle(*args, **options)
      File "/opt/django-trunk/django/core/management/base.py", line 351, in handle
        return self.handle_noargs(**options)
      File "/opt/django-trunk/django/core/management/commands/syncdb.py", line 109, in handle_noargs
        emit_post_sync_signal(created_models, verbosity, interactive, db)
      File "/opt/django-trunk/django/core/management/sql.py", line 190, in emit_post_sync_signal
        interactive=interactive, db=db)
      File "/opt/django-trunk/django/dispatch/dispatcher.py", line 172, in send
        response = receiver(signal=self, sender=sender, **named)
      File "/opt/django-trunk/django/contrib/sites/management.py", line 14, in create_default_site
        s.save(using=db)
      File "/opt/django-trunk/django/contrib/sites/models.py", line 51, in save
        super(Site, self).save(*args, **kwargs)
      File "/opt/django-trunk/django/db/models/base.py", line 464, in save
        self.save_base(using=using, force_insert=force_insert, force_update=force_update)
      File "/opt/django-trunk/django/db/models/base.py", line 547, in save_base
        for f in meta.local_fields if not isinstance(f, AutoField)]
      File "/opt/django-trunk/django/db/models/fields/__init__.py", line 276, in get_db_prep_save
        return self.get_db_prep_value(value, connection=connection, prepared=False)
    TypeError: get_db_prep_value() got an unexpected keyword argument 'connection'

  • http://pulse.yahoo.com/_PV6UMJBSWXB6KVSG54FZFHDRUY MPMike2000

    After I installed this app, I’m now getting the following error:

    no such column: django_site.folder_name

    I’ve run syncdb and everything else seems to be fine.

    Also, I’m getting a Deprecation Warning:

    DeprecationWarning: A Field class whose get_db_prep_value method hasn’t been updated to take `connection` and `prepared` arguments.  new_class = super(SubfieldBase, cls).__new__(cls, name, bases, attrs)Any ideas on how to fix?

    • Anonymous

      I updated the README w/ the sql needed to fix this

      alter table django_site add column folder_name varchar(255);
      alter table django_site add column subdomains varchar(255);

  • http://pulse.yahoo.com/_PV6UMJBSWXB6KVSG54FZFHDRUY MPMike2000

    After I installed this app, I’m now getting the following error:

    no such column: django_site.folder_name

    I’ve run syncdb and everything else seems to be fine.

    Any ideas how to get around this? I’m using Django 1.3.1

    • uysrc

      I updated the README w/ the sql needed to fix this

      alter table django_site add column folder_name varchar(255);
      alter table django_site add column subdomains varchar(255);

  • http://pulse.yahoo.com/_UKYLC5LD2RTTSN7W3B3SPZMUYA Johnie
  • Anonymous

    thank you for this great module!  

    question – i have several settings that are site specific.  do you have any method to allow for that?  it would be great if i could drop a settings.py into the site folder and it would pick up and override the global settings with the site settings.If not, I might add it to my fork.

    • Anonymous

      That’s a great idea.  Please let me know if you have any ideas how to enable this feature. 

    • Gathole

      Hi mynameisgabe,

      Can you please share how did make site specific settings to override global settings.   Thanks

    • http://twitter.com/russferriday Russ Ferriday

      Settings are also essential for my purposes. I’ve added the ability to override settings by dropping an optional settings.py into any site directory. Just the settings provided are overridden. https://bitbucket.org/topiaruss/django-dynamicsites

  • mynameisgabe

    thank you for this great module!  

    question – i have several settings that are site specific.  do you have any method to allow for that?  it would be great if i could drop a settings.py into the site folder and it would pick up and override the global settings with the site settings.If not, I might add it to my fork.

    • uysrc

      That’s a great idea.  Please let me know if you have any ideas how to enable this feature. 

    • Gathole

      Hi mynameisgabe,

      Can you please share how did make site specific settings to override global settings.   Thanks

    • http://twitter.com/russferriday Russ Ferriday

      Settings are also essential for my purposes. I’ve added the ability to override settings by dropping an optional settings.py into any site directory. Just the settings provided are overridden. https://bitbucket.org/topiaruss/django-dynamicsites

  • http://www.webhostings.in Website hosting india

    Really superb tutorial you have shared here and it’s very easy to understand through this post about this concept.

  • http://www.webhostings.in Website hosting india

    Really superb tutorial you have shared here and it’s very easy to understand through this post about this concept.

  • avb

    I have an interesting and (extremely) annoying error. I get the typical “no such column: django_site.subdomains” error, in which I go into dbshell and run the alter table command you have already posted many times. Expectedly, I get “Error: duplicate column name: subdomain” in dbshell, yet when I refresh the page it works just fine. It seems to break more often when I am in the django-admin shell.

    You have any ideas? Am I some how resetting the sql db? (I’m using SQLite) I must be fucking something up in which Django isn’t reading the sql file correctly….

    • Anonymous

      I don’t quite understand where/when you’re seeing the  ”no such column: django_site.subdomains” error.  You wrote:

      > yet when I refresh the page it works just fine

      FYI – I’ve not tested w/ SQLite (only Postgres and Mysql) but generally SQLite is quite robust.

  • avb

    I have an interesting and (extremely) annoying error. I get the typical “no such column: django_site.subdomains” error, in which I go into dbshell and run the alter table command you have already posted many times. Expectedly, I get “Error: duplicate column name: subdomain” in dbshell, yet when I refresh the page it works just fine. It seems to break more often when I am in the django-admin shell.

    You have any ideas? Am I some how resetting the sql db? (I’m using SQLite) I must be fucking something up in which Django isn’t reading the sql file correctly….

    • uysrc

      I don’t quite understand where/when you’re seeing the  ”no such column: django_site.subdomains” error.  You wrote:

      > yet when I refresh the page it works just fine

      FYI – I’ve not tested w/ SQLite (only Postgres and Mysql) but generally SQLite is quite robust.

  • Michael

    would you please make an example project so that we better understand it

  • Michael

    would you please make an example project so that we better understand it

  • Alex

    I’m getting the following error:
    Site matching query does not exist.

  • Alex

    I’m getting the following error:
    Site matching query does not exist.

  • Alex

    I somehow solved the Site mathing query problem adding the middleware before the django-cms middlewares.

    Now I have a problem with dynamicaly adding templates to the settings:
    Exception Type:TypeErrorException Value:can only concatenate tuple (not “list”) to tupleException Location:/Users/loner/projects/miramare3/miramare/apps/dynamicsites/middleware.py in process_request, line 113

  • Alex

    I somehow solved the Site mathing query problem adding the middleware before the django-cms middlewares.

    Now I have a problem with dynamicaly adding templates to the settings:
    Exception Type:TypeErrorException Value:can only concatenate tuple (not “list”) to tupleException Location:/Users/loner/projects/miramare3/miramare/apps/dynamicsites/middleware.py in process_request, line 113

  • cheem

    Hi, I do actually have a real problem to understand how this app work. Can we have a real  example? 

  • cheem

    Hi, I do actually have a real problem to understand how this app work. Can we have a real  example? 

  • E Lavoine

    Hi, I think i’ve found an error in the middleware.py.
    self.request.urlconf remains unchanged after you import  the new urlconf. 
    So whatever domainname, it always lookup in the old urlconf.

  • E Lavoine

    Hi, I think i’ve found an error in the middleware.py.
    self.request.urlconf remains unchanged after you import  the new urlconf. 
    So whatever domainname, it always lookup in the old urlconf.

  • Gathole

    Hi,

    It’s great article which helps me a lot. Thanks…

    I have one problem, I am not able to override some other settings. I have two sites (site1 and site2) into sites folder. site1 and site2 have settings file and both has a variable called “TEST_VARIABLE”.

    Lets say main settings has value “main settings”, site1 has “site1 settings” and site2 has “site2 settings” for ”TEST_VARIABLE”, but when I hit either site1 or site2 it gives me the value of main settings “main settings”.

    can please tell me where should I make changes for “TEST_VARIABLE”?? so I can get correct site value. Thanks

  • Gathole

    Hi,

    It’s great article which helps me a lot. Thanks…

    I have one problem, I am not able to override some other settings. I have two sites (site1 and site2) into sites folder. site1 and site2 have settings file and both has a variable called “TEST_VARIABLE”.

    Lets say main settings has value “main settings”, site1 has “site1 settings” and site2 has “site2 settings” for ”TEST_VARIABLE”, but when I hit either site1 or site2 it gives me the value of main settings “main settings”.

    can please tell me where should I make changes for “TEST_VARIABLE”?? so I can get correct site value. Thanks

  • django_guest

    This is an app I was looking for.  I would like to use it in my django project that is SaaS offering model to host my clients sites along with the template customization on shared hosting for now.

    I have two questions. Please, help me figure out the answers:

    1. Has anybody used this on shared hosting like WebFaction in production? How the memory consumption is compared to separate
    projects with defined site_ids in each settings file? In particular i am
    curious to know whether this has any advantage over apache virtualhosts configured with timeouts for passive sites in regards to memorycache? Keeping the django
    instance up to serve all the sites with their cached pages make me
    wonder vs taking some of these sites down when timeout hits with
    virtualhost.

    2. I see some branches in Bitbucket. Is the upstream most up-to-date version now?

     

  • django_guest

    This is an app I was looking for.  I would like to use it in my django project that is SaaS offering model to host my clients sites along with the template customization on shared hosting for now.

    I have two questions. Please, help me figure out the answers:

    1. Has anybody used this on shared hosting like WebFaction in production? How the memory consumption is compared to separate
    projects with defined site_ids in each settings file? In particular i am
    curious to know whether this has any advantage over apache virtualhosts configured with timeouts for passive sites in regards to memorycache? Keeping the django
    instance up to serve all the sites with their cached pages make me
    wonder vs taking some of these sites down when timeout hits with
    virtualhost.

    2. I see some branches in Bitbucket. Is the upstream most up-to-date version now?

     

    • Jaimin

      Were you able to do it on webfaction? We are currently using webfaction and would like to implement subdomains with this approach. It would be helpful to get some insight from you.

  • Alex

    Doesn’t work unless I change row 101 in middleware.py to
    self.request.urlconf = ‘sites.’ + urlconf_pkg

    Without this it throws an exception

    ImportError at /

    No module named mysite_com.urls

    traceback:

    /usr/lib64/python2.7/site-packages/django/core/handlers/base.py in get_response

    request.path_info) …

    /usr/lib64/python2.7/site-packages/django/core/urlresolvers.py in resolve

    for pattern in self.url_patterns: …

    /usr/lib64/python2.7/site-packages/django/core/urlresolvers.py in _get_url_patterns

    patterns = getattr(self.urlconf_module, “urlpatterns”, self.urlconf_module) …

    /usr/lib64/python2.7/site-packages/django/core/urlresolvers.py in _get_urlconf_module

    self._urlconf_module = import_module(self.urlconf_name) …

    /usr/lib64/python2.7/site-packages/django/utils/importlib.py in import_module

    __import__(name)

    django 1.3.1

  • Alex

    Doesn’t work unless I change row 101 in middleware.py to
    self.request.urlconf = ‘sites.’ + urlconf_pkg

    Without this it throws an exception

    ImportError at /

    No module named mysite_com.urls

    traceback:

    /usr/lib64/python2.7/site-packages/django/core/handlers/base.py in get_response

    request.path_info) …

    /usr/lib64/python2.7/site-packages/django/core/urlresolvers.py in resolve

    for pattern in self.url_patterns: …

    /usr/lib64/python2.7/site-packages/django/core/urlresolvers.py in _get_url_patterns

    patterns = getattr(self.urlconf_module, “urlpatterns”, self.urlconf_module) …

    /usr/lib64/python2.7/site-packages/django/core/urlresolvers.py in _get_urlconf_module

    self._urlconf_module = import_module(self.urlconf_name) …

    /usr/lib64/python2.7/site-packages/django/utils/importlib.py in import_module

    __import__(name)

    django 1.3.1

  • Henri van de Geest

    oeps… sry

  • Henri van de Geest

    oeps… sry

  • Travistanderson

    This is a great article and solves the problem I have working on as well. What is your method for giving each site it’s own urls.py. I am currently hacking a method into your middleware that emptys and reloads the main urls.py with the one that is in the active site’s folder. Am I missing something obvious? Thanks for the help

  • Travistanderson

    This is a great article and solves the problem I have working on as well. What is your method for giving each site it’s own urls.py. I am currently hacking a method into your middleware that emptys and reloads the main urls.py with the one that is in the active site’s folder. Am I missing something obvious? Thanks for the help

  • krsarmiento

    Alex said it before, to make it work we have to change in middleware.py:

    self.request.urlconf = urlconf_pkg

    For

    self.request.urlconf = ‘sites.’ + urlconf_pkg

  • krsarmiento

    Alex said it before, to make it work we have to change in middleware.py:

    self.request.urlconf = urlconf_pkg

    For

    self.request.urlconf = ‘sites.’ + urlconf_pkg

  • http://www.dirtymonkey.co.uk/ Matt Stevens

    Great write-up, thanks it helped.

  • http://www.dirtymonkey.co.uk/ Matt Stevens

    Great write-up, thanks it helped.

  • Pavel

    Hello! Problem with TEMPLATE_DIRS. I have 20 dynamicsites, sometimes template set not correct (from other site, folder_name not correct). It is may be 1-2 times in day. In debug I can see, what SITE_ID set correct, but TEMPLATE_DIRS set not correct (folder_name from other SITE_ID). After reloading page – template set correct.