Django Raspberry Pi 2
Last time, we started trying to deploy a Django app to a Raspberry Pi. I came up with the following steps, and did some of them in the last post:
- Get our python code onto the Pi.
- Get the dependencies installed on the Pi.
- Configure apache webserver to forward requests to django.
- Set up a database
- Probably other things I haven’t thought of yet.
1 and 2 took a little longer than I thought, and they are covered in the previous post. This post details my attempts at step 3. It turns out that I only thought I was done with step 2, but we’ll learn about that as we go.
I mostly followed this
tutorial, with the difference
that I used
rsync to copy over an existing Django app, rather than just
initiating one on the Pi. I also had more trouble getting Python working,
because I have some dependencies that need a newer Python version than ships on
the Raspberry Pi, and because I have some non-PyPi dependencies that need to
download over https. I also had an existing project running on the apache2
webserver on my Raspberry Pi, and needed to turn it down to free up port 80,
which is the usual port for HTTP. I got all that done, but now when I visit my
Raspberry Pi in the browser, I see the very boring 404 page:
Not Found The requested URL was not found on this server. Apache/2.4.38 (Raspbian) Server at 192.168.0.105 Port 80
So this leaves us squarely in “Step 3. Configure apache webserver to forward requests to django.” And we have a genuine mystery on our hands, since I have no idea what’s causing the 404.
First, I notice that this isn’t a Django error page, it’s an apache2 error page. That makes me think that something is configured wrong such that the apache2 webserver doesn’t know about the routes that Django is serving, or maybe just can’t find the Django app at all.
To try to fix this, I’m going to take some time trying to understand the basics of configuring the apache2 webserver and getting it to talk to Django at all, and then maybe I’ll be able to figure it out.
Here are some initial hypotheses:
- Apache is trying to start using the wrong python. We had to install a new python interpreter to get the Django app to install its dependencies, and so it seems possible that apache doesn’t know about this interpreter, or about it’s virtual environment, and so can’t really start the server process.
- It’s possible I just missed a step, and I need to tell the django app to receive requests.
- It’s possible I’m missing pieces of Apache’s config, and it doesn’t know I want it to forward requests to Django.
I’ve never tried to configure this setup before, so any of these could be happening. I feel like 2 and 3, or a combination, is likeliest. I thought there would be a setting that said something like “Dear Apache2 webserver, when you get a request on port 80, use this interpreter to invoke this python script.”
So let’s try to learn a bit about that. Here’s a super useful command I’ve been using to see what went wrong on the server:
$ tail -f /var/log/apache2/*.log
I like this command because the
*.log on the end keeps me from having to
figure out log rotation, or naming, or whether
has what I want.
With that command, I’m seeing errors in the log that look like:
[Sat Jun 17 19:39:25.378551 2023] [wsgi:error] [pid 21900] [client 192.168.0.34:58180] \ Target WSGI script not found or unable to stat: /home/pi/pidjango/pidjango
WSGIScriptAlias path in the apache2 config worked. Now I get a
500. Time for a celebration dance! I got to a better error message. Let’s see
what the logs say now!
Now the error looks like this:
Traceback (most recent call last): File "/home/pi/pidjango/library_django/wsgi.py", line 12, in <module> from django.core.wsgi import get_wsgi_application ModuleNotFoundError: No module named 'django'
That makes me think there’s a problem with the virtual environment. In other
words, when apache2 tries to invoke the django process, there’s no dependencies
loaded. An experiment I should have run at this point but didn’t think of,
would be to run
python manage.py runserver directly on the Pi from inside the
virtual environment. If that had worked, it would prove that the virtual
environment was correctly configured. As it was, I just tentatively reached
that conclusion by googling error messages.
Here are some of the things I found that made me suspect a different problem:
So I guess the other way to put this is that if you want libapache2-mod-wsgi-py3 to work in this image and use our Python instead of Debian’s, you’re going to have to compile mod-wsgi from source against our Python plus Debian’s httpd headers (something that’s likely to be somewhat involved, but to what degree I couldn’t really wager a guess since I’m unfortunately totally unfamiliar with mod-wsgi).
And an old StackOverflow answer, which suggested I check how apache2’s mod_wsgi was linked:
$ ldd /usr/lib/apache2/modules/mod_wsgi.so linux-vdso.so.1 (0x7eaec000) libpython3.7m.so.1.0 => /usr/lib/arm-linux-gnueabihf/libpython3.7m.so.1.0 (0x76a69000) ... snip ...
This is concerning. Last time, I fixed
dependencies that were too new for the system python by installing a newer
python alongside the existing one. But the
ldd command above makes me think
that apache2 knows that it’s supposed to work with python 3.7, and not a newer
python. I think maybe I need to upgrade apache2 to get it to link with newer
Maybe we’ll get lucky and I can upgrade this easily?
$ sudo apt-get install apache2 libapache2-mod-wsgi-py3 Reading package lists... Done Building dependency tree Reading state information... Done apache2 is already the newest version (2.4.38-3+deb10u10). libapache2-mod-wsgi-py3 is already the newest version (4.6.5-1+deb10u1). 0 upgraded, 0 newly installed, 0 to remove and 22 not upgraded.
What this command and output are telling me is: you are already on the newest version of apache2 that works with your version of Debian (the Linux distro running on the Pi). In other words, I have a limited number of choices to get this setup working:
- Downgrade django
libapache2-mod-wsgi-py3from source, linking against a new enough
- Try to upgrade Debian, in the hopes that a newer version will have a newer python and newer apache2 to match it.
I rate these as follows:
- Gross / disappointing
- Really hard
- My best hope.
Given those options, I think I’ll take a swing at “my best hope” first, but let’s just try to see (1) what the latest debian is and (2) what the system python3 is on that. I’ve followed this guide and succeeded in upgrading my Pi to a newer OS distro once before, but it was kind of a pain. I’d really like the answers to two questions first:
- What’s the oldest python that works with a maintained version of django?
- What’s the python3 that ships with debian 11 / “bullseye”, the newer version of the Linux distro running on the Pi.
For number 1, it looks like django 4.2 supports python 3.8 and 3.9 For number 2, it looks like 3.9 will be included in the upgrade. So I guess it’s time to try upgrading the Raspberry Pi again. My other concern is that the Pi is like 4 or 5 years old, and might just not work on the newer OS image. I need to know 2 more things:
- Will my pi handle the new OS?
- Will my django app run on python3.9?
The second one is probably easier to answer, and a no on either stops this project, so let’s answer that one on my laptop, where the django app at least works:
- Install Python 3.9 using
- Edit my
pyrpoject.tomlto permit python version 3.9
poetry env use 3.9
poetry run python manage.py runserverand make sure it starts.
So far so good! Now let’s see about upgrading the whole distro on the Pi!
It took me a while to get upgraded. I had to run
sudo apt install gcc-8-base
to get a new enough libc that I could upgrade other things, and I skipped the
sudo rpi-update because it’s own output says, “don’t do this as part
of normal upgrades,” and I sure hope what I’m doing is a normal upgrade.
I also had some issues where downloads would hang, and an
apt-get install or
sudo apt dist-upgrade would print the same
GET line over and over again
without making any progress. So I learned that if you send
CTRL+C to a
apt install, it might stop and then be able to restart where it left
off. I didn’t want to learn this, and I don’t recommend it, but it did happen
and get me unstuck! I thought I would have to switch
mirrors, but luckily just
retrying everything worked.
So now, after running a full upgrade of the OS, I refresh my browser, hoping I’ll see a django page!
Instead, I see the most helpful stacktrace I have seen in my life:
File "/home/pi/pidjango/library_django/wsgi.py", line 16, in <module> application = get_wsgi_application() File "/home/pi/pidjango/djenv/lib/python3.9/site-packages/django/core/wsgi.py", line 12, in get_wsgi_application django.setup(set_prefix=False) File "/home/pi/pidjango/djenv/lib/python3.9/site-packages/django/__init__.py", line 24, in setup apps.populate(settings.INSTALLED_APPS) File "/home/pi/pidjango/djenv/lib/python3.9/site-packages/django/apps/registry.py", line 83, in populat RuntimeError: populate() isn't reentrant
This error is apparently thown by django itself
In my humble opinion, this is a terrible error message, and when I have time, I want to write a post about good error messages and why they’re so hard to write. In the meantime, let’s try to get unblocked here.
One nice thing about getting a distinctive (but completely mad!) error message
on a super common framework is that I can just paste it into google, yielding
this SO answer, which tells me that I
need to modify some Django code to remove the raise statement and get a better
error message. So I edited some core django code in Vim on the server to remove
raise statement, so that I could get a better error message. This is
also not something I super recommend, but is something that worked so yay.
Now I get
ModuleNotFoundError: No module named 'catalog', which is actually
probably a problem with my app, since my app has a module named
This answer makes me suspect my python path is wrong.
Trying to understand this error better, I did some experiments.
- I activated the virtual env and ran
python manage.py runserver. It worked.
- I had python print out the path it’s looking for dependencies in:
>>> import sys >>> print(sys.path) ['', '/usr/lib/python39.zip', '/usr/lib/python3.9', '/usr/lib/python3.9/lib-dynload', '/home/pi/pidjango/djenv/lib/python3.9/site-packages']
I poked around for a little while. At this point I was pretty frustrated, because I’ve compiled python from source twice, done a full distro upgrade, and I am still getting generic 500 pages from the webserver. After taking a break to calm down, I realized that it was a stilly error:
I had the python path wrong in
000-default.conf define the
VirtualHosts that apache2 can route
traffic to. Here’s the offending snippet and the fix:
<VirtualHost *:80> # snip ... # THIS PATH IS WRONG: WSGIDaemonProcess django python-path=/home/pi/pidjango/library_django/:/home/pi/pidjango/djenv/lib/python3.9/site-packages/ python-home=/home/pi/pidjango/djenv WSGIProcessGroup django WSGIScriptAlias / /home/pi/pidjango/wsgi.py </VirtualHost>
I needed to change it to:
<VirtualHost *:80> # snip ... WSGIDaemonProcess django python-path=/home/pi/pidjango/:/home/pi/pidjango/djenv/lib/python3.9/site-packages/ python-home=/home/pi/pidjango/djenv WSGIProcessGroup django WSGIScriptAlias / /home/pi/pidjango/wsgi.py </VirtualHost>
In other words, the python path was one directory too specific, and pointed into one of my apps, instead of into my whole project. With that fixed, I can at least see django-rendered pages in the browser.
I still have no static assets and no database, but those will have to wait until next time.
Till then, happy learning!