In this post we're going to learn how to configure watchman to reload a django development server faster.
The enviroment we're going to create is quite simplistic, a hello world django app using watchman as reloader. If you want to add a postgres database, here you have more information if you want to add a database.
The image we are going to use is alpine, to take the advantage of a small image that use less resources.
In alpine, the watchman package is on the edge testing repo, so we need to add that repository to the list of repositories.
Here we install bash, in case you want to access to a shell in your container.
FROM alpine
ENV PYTHONUNBUFFERED 1
RUN echo "http://dl-cdn.alpinelinux.org/alpine/edge/testing" >> /etc/apk/repositories
RUN apk update --no-cache && \
apk upgrade && \
rm -fr /var/cache/apk/*
RUN apk add \
bash \
py3-pip \
watchman
RUN mkdir /var/run/watchman/
RUN mkdir /code
COPY requirements.txt /code/
WORKDIR /code
RUN pip3 install setuptools==41.6.0 wheel==0.33.6
RUN pip3 install --upgrade pip && pip install -r requirements.txt
COPY . /code/
Example requirements.txt file for the project:
Django
pywatchman
We need to install pywatchman. With pywatchman and the Watchman service, kernel signals will be used to autoreload the server, for large projects, that configuration improves the performance reloading the code.
We can add a docker-compose.yml file, that will be handy if you want to add more services into the environment.
version: '3.1'
services:
website:
container_name: website
build:
context: .
ports:
- 8000:8000
command: ["python3.8", "manage.py", "runserver", "0.0.0.0:8000"]
restart: always
environment:
DJANGO_WATCHMAN_TIMEOUT: 20
volumes:
- .:/code
Here the more relevant configuration is DJANGO_WATCHMAN_TIMEOUT.
In this example is configured to 20 (seconds). That's the time django project will wait to have a response from watchman on the startup. If django didn't receive an answer 20 seconds, will use StatReloader, printing something like that in the console:
Watching for file changes with StatReloader
That's the default django reloader. Watchman takes a bit to start, that amount of time can differ on your machine, try with a higher value.
Let's start our application, following the container logs:
docker-compose build
docker-compose up -d
docker-compose logs -f
Now, if we visit our test app in a browser (http://0.0.0.0:8000), we'll see the django default website after a successful installation:
The example project github repo: https://github.com/jesusenlanet/django-watchman-example
If you want to exclude some folders from the watchman scope, add a .watchmanconfig file like that:
{
"ignore_dirs": [".git", ".idea", "venv"]
}
And add that folder to the ignore_dirs list.
May 28 2020 *Last updated May 31 2020
#docker #development #docker-compose #python #django #watchman
We are going to see how to create a django development environment with docker-compose.
Why? because in that way we don't need to install dependencies, databases and other stuff in our own machine.
Well, that's not completely true, not all IDEs have support for docker environments, in that case, you'll need to use a local virtual environment where to install all the dependencies and avoid some disturbing errors in your IDE, but that' isn't a big deal since is really easy to create a virtual environment for that concrete problem (or you can pay a PyCharm professional licence and you will have an IDE with docker environment support).
Step 1 - Install Docker CE
To avoid to repeat contents, just follow the official manual:
https://docs.docker.com/install/linux/docker-ce/ubuntu/
Once installed, it's a good idea to do some extra steps from that other manual:
https://docs.docker.com/install/linux/linux-postinstall/
The steps I followed, and I recommend are:
-
Manage Docker as a non-root user
-
Configure remote access with systemd unit file
Managing docker as non-root user avoid us to preface all the docker commands with sudo
Configuring the remote access, will allow us to link our docker applications with the PyCharm Community Edition to enjoy some features.
Step 2 - Install docker-compose
As in the previous step, we'll follow the official documentation to install docker-compose:
https://docs.docker.com/compose/install/
Step 3 - Configuring our Django project
Once we have installed our the docker and docker-compose requirements, we are going to configure our Django project to be launched into a docker container.
requirements.txt
We need to add some requirements to our django project to create make it able to communicate with our MySql server.
Django==3.0
mysqlclient==1.4.6
Dockerfile
Create a Dockerfile in your's project root path:
FROM python:3.6
ENV PYTHONUNBUFFERED 1
RUN mkdir /code
WORKDIR /code
COPY requirements.txt /code/
RUN pip install --upgrade pip && pip install -r requirements.txt
COPY . /code/
Explanation:
FROM python:3.6
ENV PYTHONUNBUFFERED 1
With that lines we specify what python interpreter to use (python 3.6), and we create a environment variable to make python to be unbuffered (read that python documentation page for more information)
RUN mkdir /code
WORKDIR /code
With that lines we create the folder where our project code will be, and we move to that directory.
COPY requirements.txt /code/
RUN pip install --upgrade pip && pip install -r requirements.txt
COPY . /code/
We copy our project requirements file, upgrade pip and install our django requirements.
To finalize we copy our code into the container.
Here someone can ask: And why not to use ADD if it have the same functionality? Well, for that just read that docker best practices.
docker-compose file
version: '3'
services:
db:
container_name: db
image: mysql/mysql-server:5.7
ports:
- '3306:3306'
environment:
MYSQL_DATABASE: 'your_project_database'
MYSQL_USER: 'root'
MYSQL_PASSWORD: 'password'
MYSQL_ROOT_PASSWORD: 'password'
web:
container_name: web
build: .
command: python manage.py runserver 0.0.0.0:8000
volumes:
- .:/code
ports:
- "8000:8000"
depends_on:
- db
In that file you just need to modify your database.
The docker-compose.yml defines to services, a service called db, our database, and another service called web, our project service.
The web service have an options depends_on, that makes our service web to wait for the service db.
The first time you run your docker-compose you will receive an error from the web service (in logs or from the running app inside) saying that is unable to connect with the database server.
Don't worry, have an explanation but it's not needed to do anything more, second time you run the docker-compose services that error will disappear (or you can use some script like wait-for-it).
settings.py
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'project_database_name',
'USER': 'root',
'PASSWORD': 'password',
'HOST': 'db',
'PORT': 3306,
'OPTIONS': {
'init_command': "SET sql_mode='STRICT_TRANS_TABLES'"
}
}
}
Step 4 - Launching the environment
In bash, we need to execute that commands to build the docker images, and create and start the containers.
docker-compose build
docker-compose up -d
The -d option of the up command makes all to run in background.
To see the logs:
docker-compose logs [-f] [service]
The -f option makes to follow the logs in console. You can specify the containers to log (db, web in that case). If no services are specified show all the containers log.
Extra Step - PyCharm configuration
As extra step, we can connect PyCharm CE with docker in (limited features, more features with the professional edition).
To do that you need:
- Install the docker plugin
- Connect with the container
To connect with the container open File > Settings > Build, Execution, Deployment > Docker and add a configuration like that:
January 31 2020 *Last updated May 17 2020
#docker #development #docker-compose #python #django #mysql
Pylancer is proud to present the new blog, Be welcome!
That's blog will replace my old personal blog, hosted in blogspot, a site that's great to start writing your blog, but not enough when you want to generate contents as a professional.
This blog will be focused not only in a brand image, that blog will host technical information too, wishing the people to find it useful.
More entries soon ...
January 31 2020
