Django Diaries / 25th Jul 2013

A Thousand Papercuts

It's been a busy few weeks over here. A holiday directly followed by moving house is a great way to alternate relaxation and stress, especially when the flight back from said holiday is almost delayed by six hours.

Nevertheless, I'm back, and sorry for the hiatus - as much as I love databases and migrations, they're hard to handle when you've got your hands full of cardboard boxes. Still, things are coming along nicely, and we're almost at a merge point.

More Operations

The first set of changes over the last few weeks have been to do with improving the number of operations and the autodetector. In particular, I'm very proud of the fact that the autodetector now detects ForeignKeys and automatically adds dependencies to the migration so it runs in the right order - a feature that's been requested for South for almost as long as I can remember.

Even better, if you have two apps that have ForeignKeys to each other, you no longer have to worry. While in South this would have resulted in two migrations that both needed the other to run first (but which didn't declare this via dependencies), the new system recognises the problem and automatically splits one side up into two migrations.

It's this sort of more complex handling that made me have a single makemigrations command for the entire project; some changes have to be done all at once. The dependency detection should also work for ManyToManyField and anything else which somehow stems from RelatedField.

An operation for unique_together has also come into the fold, along with index_together - now you can change either of these and watch the migrations spill out.

All in all, there's now a very decent set of operations in place - addition and removal of models, addition, alteration and removal of fields, and changes to db_table, unique_together and index_together. This definitely seems enough to me to consider the code a working alpha.

The death of syncdb

The other great news is that syncdb is no longer a command. It is, of course, still there - we have deprecation rules in Django for this sort of thing - but it's now just an alias to a bigger, more powerful command called migrate.

migrate replaces syncdb entirely, performing both its old function of synchronizing apps without migrations, as well as its new function of running migrations. It looks rather like the South migrate command, though it hasn't grown all the same options yet and it's still a bit lacking in the multiple database department.

Still, this is what I and the other core developers agreed to be the best route; the naming of the old command no longer makes sense, and having two commands would just be confusing things too much. Things will just keep working if you call syncdb, though, there will just be prettier console output.

Documentation Troubles

Of course, the problem with deprecating something as old and core to Django as syncdb is that the Django documentation suddenly needs a lot of changes.

There's not only the syncdb references, though - the tutorial needs work, the sections talking about the ORM and tables need work, and a whole new section on migrations needs to appear. I've made a good start at this - most of the small references have been changed, and there's a start of a Migrations document - but there's still a lot more to come.

The SchemaEditor part of the database backends also needs a big reference document, both for potential end-users of that API (other schema changing apps, essentially) but also for third-party database backends who will need to implement it. That shouldn't be too hard, but there's quite a few functions in there to get through.

The Merge

I'm pleased, however, that things finally seem to be settling down to a slower, more polish-like level. I expect I'll go for a merge into master quite soon - once I've had some good code review - as the longer my branch sits unmerged the harder it's getting to work on other ORM parts of Django.

In particular, my branch has a few changes that aren't directly schema alteration - notably the new app cache, the new introspection functions and the new Field API. The last of those I merged across separately to help along my mentee's GSOC project, but it's getting to the point where having everything in core would help a great deal.

That means that this week and early next week I'll be trying to clean up the code and add enough documentation that it can be merged into master - after all, we don't let patches land without working code, tests, and documentation!

If you're interested, have a look through the pull request, and let me know about anything you see that's broken. This is a big patch, but not the biggest Django has ever had, so I'm confident we can get it merged soon.