I've been working on Django Channels over the last few weeks, mainly on settling things down into more of a final shape than they've been up to this point - and I'd like to discuss some of the progress that's happening, and the reasoning behind it.
The first major step was deciding on the plan of how to distribute the Channels code. We could have just put it all in 1.10, and left other Django versions out in the dark, but that would leave a massive gap, and a long time until many existing sites could start using it.
Of course, I would only have considered bringing it to earlier versions of Django if it wasn't prohibitively difficult to do; thankfully, the Channels design fits cleanly around the existing Django abstraction, which is part of why I'm so fond of it.
The plan, then, is to ship Channels as two main separate parts:
- django.channels, which runs Django consumers and view code when messages arrive on routed channels. Will ship as a pluggable app called channels for earlier Django versions.
- daphne, a HTTP/WebSocket protocol server that does the job of terminating client connections and encoding requests and packets as messages on channels.
Daphne represents some of the most complex code needed - a mixed-mode HTTP and WebSocket server that can serve thousands of clients at once - and so by making it a separate package, all versions of Channels can use it. Additionally, its release cycle can be decoupled from Django's if needed, though the intention is to keep it within the same contributor models.
Now, because these two different projects are going to have to communicate and function as different codebases and maybe on different release cycles, a specification for how they communicate is needed. In fact, if you look at the problem slightly more generally, Daphne could be just one of several protocol server options.
What we end up with, then, is something that defines how application code and servers communicate - and indeed, Python already has one of these, called WSGI. WSGI is a deceptively simple specification that allows all Python web applications to be written to a common interface - that of a callable that takes two arguments - allowing us to change servers and frameworks with ease.
Unfortunately, the callable pattern isn't quite enough for what we need for Channels, but the idea of a simple, defined interface is appealing, and so I've spent the last few weeks drawing up the spec for what I've initially called ASGI.
ASGI aims to define a common interface for protocol servers (like daphne) and application code (like Django Channels and the consumers it wraps) to communicate, with two key concepts:
- A channel layer API for sending and receiving messages between different system components
- Formats for those messages for defined protocols (e.g. HTTP, WebSocket)
The code API means that the actual mechanism of transporting messages between protocol servers and application code can be abstracted away, and users can choose between several options (or write their own) - much like database drivers now. In fact, channel layers, as they are called, can play to different strengths; Django Channels will likely recommend a Redis-based one for production use, but provide in-memory and database ones for development purposes.
Rather than provide a server with an application object, as in WSGI, with ASGI you provide servers and applications with channel layer objects. The API is pretty simple - the key mechanics are just two functions, send and receive_many.
The message formats fill in the rest of the specification, allowing different code to understand what they're sending. For example, a HTTP Request message format outlines how to represent the request method and headers; Django uses this to construct a Request object when it gets the message.
The specification is still not final - I'm working through it as the code develops and I see how things fit together and perform - but you can read through it if you want, and send along any comments. It's already being used to build the daphne and django.channels code, and seems to fit in quite well with the channels consumer/router design patterns that I had before.
Most Django users won't need to look under the hood to see what's going on - runserver will just launch daphne and a worker thread and tie them together with an in-memory backend, for example, while projects will grow an asgi module alongside the wsgi module if you want to deploy that way - you'd just pass projectname.asgi.channel_layer into a protocol server, and use manage.py runworker to run a worker process.
Django will still support WSGI of course - ASGI is there for deployments that want features that fall outside of the WSGI envelope, like long-polling, WebSockets or HTTP/2 - but you will also be able to run entirely on ASGI if you want. ASGI is also designed to be able to interoperate with WSGI, running as either a WSGI application or providing a WSGI server, so it should allow Django Channels to slip in to most stacks with relative ease.
There's potential to make ASGI a python-wide specification, and invite other application code to be able to run against it (and other protocol servers to be written against it, or existing webservers modified to talk it as well), but I'd rather have it developed, stable and working for Django before starting on that path.
With this phase complete, though, Channels is looking in good shape; the consumer pattern still seems to work well, there's now a server that natively services HTTP and WebSocket on all URL paths, and there's plans in place for handling large request and response bodies well. The next steps are to ensure the code runs well with large bodies, and start doing basic performance tests, and then, hopefully, start work on a Django patch!
You can see the Django Channels codebase at https://github.com/andrewgodwin/channels, and the Daphne codebase at https://github.com/andrewgodwin/daphne