Tunnel Lights

When you're slogging away on a project like a migrations framework, it can seem a little unforgiving. You spend so many commits just laying down foundations and groundwork that it's such a relief when things finally start to work together - something I've always known as a black triangle.

You can imagine, then, how happy I am with the progress made this week. Not only did the autodetector get a lot better, but there are now commands. Not only that, but you can frigging migrate stuff with them.

I Should Calm Down A Bit

I know, that might seem like the entire purpose of a migrations framework, but it's nice to finally get all this code I've been planning for years (quite literally in some cases) working together and playing nicely.

Enough talk. Let's look at some examples. Here's a models.py file I found lying around:

from django.db import models

class Author(models.Model):
    name = models.CharField(max_length=255)
    featured = models.BooleanField()

Pretty simple, eh? Let's make a migration for it!

Of course, the command to do that has changed; in South, you had to run manage.py schemamigration appname for each app you wanted to migrate. Now, you just run manage.py makemigrations:

$ ./manage.py makemigrations
Do you want to enable migrations for app 'books'? yes
Migrations for 'books':
  0001_initial.py:
    - Create model Author

makemigrations will scan all apps and output all changes at once, possibly involving multiple migrations for each app. It will know to add dependencies between apps with ForeignKeys to each other, and to split up a migration into two if there's a circular ForeignKey reference.

There's no more --auto, no more --initial, and it'll even remind you to create migrations for new apps (don't worry, it won't prompt more than once).

Let's make some changes to our models.py file; in particular, I'm going to allow longer names, and add an integer rating column:

from django.db import models

class Author(models.Model):
    name = models.CharField(max_length=500)
    featured = models.BooleanField()
    rating = models.IntegerField(null=True)

Let's run makemigrations:

$ ./manage.py makemigrations
Migrations for 'books':
  0002_auto.py:
    - Add field rating to author
    - Alter field name on author

Notice how it's telling you nicely what each migration contains. There's also some colour on these commands to make them easier to read on the console; making migrations should be a pleasant experience, after all.

The Challenges

Of course, the result is lovely, but I'd like to look closer at one particular challenge - that of autodetection.

Autodetection is a very deep topic, and one I'll doubtless return to. However, the particular problem this week was having the autodetector intelligently discover new apps to add migrations to.

In South, you have to manually enable migrations for an app using the --initial switch to schemamigration, but I wanted to eliminate that distinction here. It's trivial enough to detect apps without migrations which need them, but that's not quite enough.

The problem is, you see, that there's plenty of apps that don't have migrations and don't need them. Third-party libraries, old internal packages with manual SQL migrations, and of course our own django.contrib (though I'm sure a few of those will inevitably grow migrations).

Thus, without any sort of extra code, makemigrations will prompt every time you run it about each of these unmigrated apps. That's going to get very annoying, and so I devoted quite a bit of thought to how to address this.

There's no way to write a marker into the apps themselves - third-party libraries may be shared and/or read-only - and so there are only two solutions:

• A setting, called UNMIGRATED_APPS
• An autogenerated file in the project directory

While we're trying to have less settings as part of Django - there's currently way too many - I feel that UNMIGRATED_APPS is a good fit to INSTALLED_APPS and fits the Django culture better.

It does mean having to update the settings file each time you add an app you don't want to migrate, rather than makemigrations updating an autogenerated file for you, but the command can remind you of this and even print you the new value ready to paste into your settings file.

Plus, it means migration refuseniks can just set UNMIGRATED_APPS = INSTALLED_APPS and get on with coding.

Opinions on this are of course always welcome, via Twitter or email.

One More Thing

There's one final feature I want to show off. If you ever renamed a field in South you'd know that it detected it as a removal and an addition and lost all your data. That's no more. Behold:

from django.db import models

class Author(models.Model):
    name = models.CharField(max_length=500)
    featured_top = models.BooleanField()
    rating = models.IntegerField(null=True)

$ ./manage.py makemigrations
Migrations for 'books':
  0003_auto.py:
    - Rename field featured on author to featured_top

Don't worry, there's more nice features like that in the works.