Django Workflow

Create project §

$ cd ~/projects
$ mkdir django-project-name
$ cd django-project-name
$ python3 -m venv .venv
$ source .venv/bin/activate
$ python3 -m pip install django
$ python3 -m pip install gunicorn
# Note the dot at the end of the line below:
$ django-admin startproject django_project .
$ python3 manage.py startapp app-name
$ touch README.md
$ touch .gitignore
# Do not initial migrate before you create Custom User Model!
$ python3 manage.py migrate
$ mkdir templates
$ python -m pip freeze > requirements.txt
$ python manage.py runserver

.venv/
.idea/
media/
__pycache__/
.DS_Store
.env
*.pyc
db.sqlite3

Configure git §

$ git status
$ git init

NB: do git add . and initial commit after removing secrets from settings.py!

Change basic project settings §

Use env vars for settings §

import secrets
print(secrets.token_urlsafe())

# python -m pip install 'environs[django]'
# environs==9.5.0
from environs import Env
env = Env()
env.read_env()
...
SECRET_KEY = env.str("SECRET_KEY")
DEBUG = env.bool("DEBUG", False)

DEBUG=True
SECRET_KEY=VsjFqreIGNU...........FWi7aziSvzRZ1M_cP-vKbI

Set template paths, etc. §

INSTALLED_APPS = [
		...,
    'posts.apps.PostsConfig',  # new
]
LANGUAGE_CODE = 'ru-RU'
TIME_ZONE = "Europe/Moscow"
AUTH_USER_MODEL = "accounts.CustomUser"  # new

ALLOWED_HOSTS = ['localhost', '127.0.0.1']  # + add prod host name
TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': [BASE_DIR / "templates"],
...
# Before deploy:
ALLOWED_HOSTS = ['85.193.89.177', 'localhost', '127.0.0.1']
CSRF_TRUSTED_ORIGINS = ['http://85.193.89.177','http://*.127.0.0.1']
ALLOWED_HOSTS = ['85.193.89.177', 'localhost', '127.0.0.1']
CSRF_TRUSTED_ORIGINS = ['http://85.193.89.177','http://*.127.0.0.1']

STATIC_ROOT = 'staticfiles/'
STATIC_URL = 'static/'
STATICFILES_DIRS = (
    os.path.join(BASE_DIR, 'static'),
		# BASE_DIR / 'static',
)
# This is for uploads (added by me):
MEDIA_URL = '/media/'
MEDIA_ROOT = os.path.join(BASE_DIR, 'media')
# or MEDIA_ROOT = BASE_DIR / 'media'

from django.contrib import admin
from django.urls import path, include
urlpatterns = [
    path('admin/', admin.site.urls),
    path('', include('posts.urls')),
]

Create CustomUser model §

See “Django for Professionals” page 55. NB: add GitHub link to my demo project, too.

python manage.py startapp accounts
# ... edit accounts/models.py
# ... edit accounts/forms.py
# ... edit accounts/admin.py

# accounts/models.py
from django.db import models
from django.contrib.auth.models import AbstractUser

class CustomUser(AbstractUser):
    address = models.TextField("Delivery address", null=True, blank=True)
    def __str__(self):
        return self.username

# settings.py
# add new setting:
AUTH_USER_MODEL = "accounts.CustomUser"

# accounts/forms.py
from django.contrib.auth import get_user_model
from django.contrib.auth.forms import UserCreationForm, UserChangeForm

class CustomUserCreationForm(UserCreationForm):
    class Meta:
        model = get_user_model()
        # The password field is implicitly included, no need to explicitly include it
        fields = (
            "email",
            "username",
        )

class CustomUserChangeForm(UserChangeForm):
    class Meta:
        model = get_user_model()
        # The password field is implicitly included, no need to explicitly include it
        fields = (
            "email",
            "username",
            "address",
        )

# accounts/admin.py
from django.contrib import admin
from django.contrib.auth import get_user_model
from django.contrib.auth.admin import UserAdmin
from .forms import CustomUserCreationForm, CustomUserChangeForm
custom_user = get_user_model()

class CustomUserAdmin(UserAdmin):
    """
    Configures admin panel views for CustomUsers.
    https://docs.djangoproject.com/en/4.1/ref/contrib/admin/\#modeladmin-options
    """
    add_form = CustomUserCreationForm
    form = CustomUserChangeForm
    model = custom_user
    # Which fields will be listed in admin section (on the list, not 'details' page):
    list_display = [
        "email",
        "username",
        "is_superuser",
        "address",
    ]
    # Which fields to show when editing user via admin panel:
    fieldsets = UserAdmin.fieldsets + ((None, {"fields": ("address",)}), )
    # Which fields to show when creating user via admin panel:
    add_fieldsets = UserAdmin.add_fieldsets + ((None, {"fields": ("address",)}), )

admin.site.register(custom_user, CustomUserAdmin)

After creating CustomUser, we can safely do the initial migration!

Initial Migrations §

python manage.py makemigrations
python manage.py migrate

Don’t forget to add all migrations to git!

Add superuser §

python manage.py createsuperuser

Add model, view, url §

from django.urls import path
from .views import HomePageView
urlpatterns = [
    path('', HomePageView.as_view(), name='home'),
]

from django.db import models

class Post(models.Model):
    text = models.TextField()
    def __str__(self):
        return self.text[:50]

# Add new model to Admin page
from django.contrib import admin
from .models import Post
admin.site.register(Post)

$ python manage.py makemigrations app-name
$ python manage.py migrate

from django.views.generic import ListView
from .models import Post

class HomePageView(ListView):
    model = Post
    template_name = "home.html"

# FBVs example
from django.shortcuts import render, get_object_or_404
from django.http import HttpRequest, HttpResponse
from .models import Category, Product

def product_list(request: HttpRequest, category_slug=None) -> HttpResponse:
    category = None
    categories = Category.objects.all()
    products = Product.objects.filter(available=True)
    if category_slug:
        category = get_object_or_404(Category, slug=category_slug)
        products = products.filter(category=category)
    return render(request, "shop/product_list.html", {
        "category": category,
        "categories": categories,
        "products": products
    })

<h1>Message Board Homepage</h1>
<ul>
    {% for post in post_list %}
        <li>{{  post.text }}</li>
    {% endfor %}
</ul>

Add static files folder §

# Project's settings.py (bottom):
STATIC_URL = '/static/'
STATICFILES_DIRS = [BASE_DIR / 'static']

{% load static %}
<html>
    <head>
        <title>Django Blog App</title>
    </head>
    <body>
        <header>
            <h1><a href="{% url 'home' %}">Django Blog - Home</a></h1>
            <link rel="stylesheet" href="{% static 'css/base.css' %}">
        </header>
...

File Uploads §

Upload and process files §

• Example from python_django/09_Files

  # forms.py
  from django import forms
  
  class UploadPostsForm(forms.Form):
      file = forms.FileField(label="Выберите файл в формате CSV на вашем устройстве")
  
  
  
  # views.py
  @login_required
  def upload_posts(request: HttpRequest) -> HttpResponse:
      context = {}
  
      if request.method == "POST":
          upload_form = UploadPostsForm(request.POST, request.FILES)
  
          if upload_form.is_valid():
              # Note that file itself is not saved in MEDIA dir!
              file = upload_form.cleaned_data["file"].read()
              # Don't forget to split rows for csv reader:
              file_str = file.decode("utf-8").split("\n")
              csv_reader = reader(file_str, delimiter="|", quotechar='"')
  
              csv_errors = []
              new_posts = []
              for num, row in enumerate(csv_reader):
                  # Check number of rows in line:
                  if not len(row) == 3:
                      csv_errors.append("Строка {line}: неверный формат строки.".format(
                          line=num+1
                      ))
                      continue
  
                  # Try to parse datetime:
                  try:
                      post_date = datetime.strptime(row[1], "%d.%m.%Y %H:%M")
                      print(num, post_date)
                  except:
                      csv_errors.append("Строка {line}: неверный формат даты.".format(
                          line=num+1
                      ))
                      continue
  
                  # Add blog post if data is OK:
                  new_post = Post.objects.create(
                      title=row[0],
                      body=row[2],
                      author=request.user
                  )
                  # Overwrite 'auto_now_add' datetime value on 'created' field and save again:
                  new_post.created = post_date
                  new_post.save()
                  new_posts.append(new_post)
  
  
              context["filename"] = upload_form.cleaned_data["file"].name
              context["csv_errors"] = csv_errors
              context["new_posts"] = new_posts
      else:
          upload_form = UploadPostsForm()
  
      context["form"] = upload_form
      return render(request, "post_upload.html", context=context)

Add tests §

from django.test import TestCase
from django.urls import reverse
from .models import Post

class PostTests(TestCase):
    @classmethod
    def setUpTestData(cls):
        """
        Put some data into DB for test.
        """
        cls.post = Post.objects.create(text="This is a test!")
    def test_model_content(self):
        """
        Check if the test data exists in DB.
        """
        self.assertEqual(self.post.text, "This is a test!")
    def test_url_exists_at_correct_location(self):
        response = self.client.get("/")
        self.assertEqual(response.status_code, 200)
		# Following three test can be combined in one:
    def test_url_available_by_name(self):
        response = self.client.get(reverse("home"))
        self.assertEqual(response.status_code, 200)
    def test_template_name_correct(self):
        response = self.client.get(reverse("home"))
        self.assertTemplateUsed(response, "home.html")
    def test_template_content(self):
        response = self.client.get(reverse("home"))
        self.assertContains(response, "<h1>Message Board Homepage</h1>")

# Run tests
$ python manage.py test

Internationalisation and Localisation §

i18n - Internationalisation L10n - Localisation

Poedit Translation Editor - Poedit

Poedit Full gettext support.
https://poedit.net/

LANGUAGE_CODE = "ru-RU"  # or "ru"
LANGUAGES = [
	("ru", "Russian"),
	("en", "English"),
]
LOCALE_PATHS = [os.path.join(BASE_DIR, "locale")]
USE_I18N = True
USE_L10N = True
# Check Middleware:
#...
#...Session
'django.middleware.locale.LocaleMiddleware',
# ...Common
# Template:
{% load i18n %}
...
{% trans "English text" %}

# create "locale/ru/LC_MESSAGES/django.po"
python -m manage makemessages -l ru
# ...translate po file ...
python -m manage compilemessages

Интернационализация административной панели §

Caching §

View caching §

https://docs.djangoproject.com/en/4.1/topics/cache/#specifying-per-view-cache-in-the-urlconf

from django.views.decorators.cache import cache_page
urlpatterns = [
    path('foo/<int:code>/', cache_page(60 * 15)(my_view)),
]

Template fragment caching §

https://docs.djangoproject.com/en/4.1/topics/cache/#template-fragment-caching

{% load cache %}
{% cache 500 sidebar request.user.username %}
    .. sidebar for logged in user ..
{% endcache %}

Low-level caching §

https://stackoverflow.com/questions/40772832/caching-results-of-a-django-function-call-with-cache-get-or-set

from django.core.cache import cache
# ...
class OfferListView(ListView):
    model = Offer
    template_name = "offer_list.html"
    context_object_name = "offer_list"
    def get_context_data(self, **kwargs):
        context = super().get_context_data(**kwargs)
				# Caching:
        offer_list = cache.get("offer_list")
        if not offer_list:
            language = self.request.LANGUAGE_CODE
            offer_list = Offer.objects.translated(language).all().order_by("-translations__title")
            cache.set("offer_list", offer_list, 60 * 10)
        context["offer_list"] = offer_list
        return context

---

New Deployment §

Migrations §

https://realpython.com/django-migrations-a-primer/ Next workflow tested for ‘clean’ DB setup:

• remove migrations directories with all files from apps;
• run makemigrations for each app:

$ python manage.py makemigrations accounts
$ python manage.py makemigrations articles
$ python manage.py makemigrations pages

• run migrate, createsuperuser:

$ python manage.py migrate
$ python manage.py createsuperuser

New DB will be created, along with migrations dirs in app dirs, and 0001_initial.py files there.

• use showmigrations to check which migrations were applied to DB:

$ manage.py showmigrations

Nginx §

Readme: GitHub - hazadus/django-website: Basic Django website. For learning purposes. Demo: http://85.193.89.177/

# sudo nano /etc/nginx/conf.d/virtual.conf
# sudo nginx -t
# sudo service nginx restart
# NOTE trailing '/'s in paths!!
server {
    listen 80;
    server_name 85.193.89.177;
    location / {
        proxy_pass http://127.0.0.1:8000/;
    }
    
    # STATIC_URL from settings.py
    location /static/ {
        autoindex on;
        alias /home/hazadus/django-static/; # STATIC_ROOT from settings.py
    }
    location /media/ {
        autoindex on;
        alias /home/hazadus/projects/django-website/media/;
    }
}

Nginx need + x to all dirs along the path to static, media dirs.

# chmod +x /root
# chmod +x /root/projects/
# chmod +x /root/projects/newspaper/
# chmod +x /root/projects/newspaper/media/
# chmod +x /root/projects/newspaper/media/images/

View nginx logs:

$ sudo tail -f /var/log/nginx/access.log
$ sudo tail -f /var/log/nginx/error.log

Collect static §

# Remove all previous stuff
rm -rf /home/hazadus/django-static/*
# Collect new static
source .venv/bin/activate
python3 -m manage collectstatic

Copy files §

scp ./db.sqlite3 hazadus@85.193.89.177:~/projects/django-c8-newspaper/
scp ./media/images/*.* hazadus@85.193.89.177:~/projects/django-c8-newspaper/media/images

Docker §

Docker compose with Django 4, Celery, Redis and Postgres

Deploying Django application that is using Celery and Redis might be challenging.
https://saasitive.com/tutorial/django-celery-redis-postgres-docker-compose/?utm_campaign=Django%2BNewsletter&utm_medium=email&utm_source=Django_Newsletter_157

# docker-compose
# SERVER version
# note 'network: host' must be UNDER build!
# https://forums.docker.com/t/option-network-mode-host-in-docker-compose-file-not-working-as-expected/51682
version: "3.9"
services:
  web:
    build:
      context: .
      network: host
    ports:
      - "8000:8000"
    command: python manage.py runserver 0.0.0.0:8000
# command: gunicorn django_project.wsgi -b 0.0.0.0:8000
    volumes:
      - .:/code

# By choosing network=host the container has access to the network of the host.
# https://github.com/docker/cli/issues/2707
sudo docker build . --network=host
# compose
sudo docker compose --file docker-compose.prod.yml build
sudo docker compose --file docker-compose.prod.yml up
sudo docker compose --file docker-compose.prod.yml up -d
# list running containers
sudo docker ps
# stop
sudo docker compose down

sudo docker compose build
sudo docker compose up -d --build
# New DB for Django:
sudo docker compose exec web python manage.py migrate
sudo docker compose exec web python manage.py createsuperuser

Back up and restore Postgres DB data:

# Creates 'backup.tar' with postgresql data in current directory
#
# Container id of Postgres (f0de8da471d2) got from 'sudo docker ps' when containers are up
# '/var/lib/postgresql/data/' is from 'docker-compose.yml', lines:
# volumes:
#      - postgres_data:/var/lib/postgresql/data/
cd /Users/hazadus/backup
sudo docker run --rm --volumes-from c0e3a99224e3 -v $(pwd):/backup ubuntu tar cvf /backup/backup.tar /var/lib/postgresql/data/

# Change web to running container name, e.g. newspaper-web-1
sudo docker compose exec web python manage.py dumpdata --indent=2 --output=mysite_data.json
# Or, connect to it then run command:
docker exec -it newspaper-web-1 bash
python manage.py dumpdata --indent=2 --output=mysite_data.json
# Restore data:
sudo docker compose exec web python manage.py loaddata mysite_data.json

# Restoring postgresql data from backup tar
#
#
# ls -al
sudo docker run --rm -v django-c8-newspaper_postgres_data:/var/lib/postgresql/data/ -v $PWD:/backup-dir bash -c "cd /var/lib/postgresql/data/ && ls -al"
#
# untar - RUN FROM DIR WITH 'backup.tar' in it!!!
#
cd /Users/hazadus/backup
sudo docker run --rm -v django-c8-newspaper_postgres_data:/var/lib/postgresql/data/ -v $PWD:/backup-dir bash -c "tar xvf /backup-dir/backup.tar"

# Copy DB contents from CONTAINER (not volume!) to a folder:
# 3501a6e46de1 = containerID from `docker ps`
docker cp 3501a6e46de1:/var/lib/postgresql/data ~/PycharmProjects/polstagramm/data/postgres/

# enter CLI:
docker exec -it django-c8-newspaper-db-1 bash
# then run psql:
psql -U postgres
# then in psql
\dt - list all tables

python -m manage dumpdata users --format json -o users.json

Email – setup and test §

docker compose exec web python manage.py shell

from django.core.mail import send_mail
send_mail('Django mail', 'This was sent with Django', 'hazadus7@gmail.com', ['hazadus7@gmail.com'], fail_silently=False)

Complete script – rebuild and run containers §

#!/bin/bash
cd /home/hazadus/projects/django-c8-newspaper
source .venv/bin/activate
echo '[0] Update virtual environment...'
python3 -m pip install -r requirements.txt
echo '[1] Removing static files...'
rm -rf /home/hazadus/django-static/*
echo '[2] Collecting static files...'
python3 -m manage collectstatic
deactivate
echo '[3] Shutting down containers...'
sudo docker compose down
echo '[4] Build containers...'
sudo docker compose build
echo '[5] Run containers...'
sudo docker compose up -d

After collectstatic copy files from project staticfiles dir to /home/hazadus/django-static/

Load testing §

GitHub - rakyll/hey: HTTP load generator, ApacheBench (ab) replacement

hey is a tiny program that sends some load to a web application.
https://github.com/rakyll/hey

brew install hey
hey -n 1000 https://www.polstagramm.ru

Getting started - Locust 2.13.1 documentation

A Locust test is essentially a Python program.
https://docs.locust.io/en/stable/quickstart.html

• use Locust

Backup via rsync §

# copy all the stuff directly to polsta-30.12/
sync -arv --exclude=.venv --exclude=.git --exclude=__pycache__ root@85.193.89.177:/root/projects/newspaper/ /Users/hazadus/webbackup/polsta-30.12/

---

📂 Django | Последнее изменение: 07.02.2024 15:04