diff --git a/src/markdown-pages/books/django-5-by-example.md b/src/markdown-pages/books/django-5-by-example.md new file mode 100644 index 000000000..31ab128c1 --- /dev/null +++ b/src/markdown-pages/books/django-5-by-example.md @@ -0,0 +1,2526 @@ +--- +title: Django 5 by Example +date: 2025-03-24 +tags: + - book + - django +--- + +## Install Django + +```bash +asdf set python 3.12.9 +asdf install +python -m venv .venv +. .venv/bin/activate +python -m pip install Django~=5.0.4 +``` + +## Manage Commands + +- `python manage.py runserver` start dev server +- `python manage.py shell` django shell +- `python manage.py createsuperuser` create superuser + + +## Create Project + +```bash +django-admin startproject mysite + +cd mysite +python manage.py migrate +``` + +`python manage.py runserver` - run server + +[Settings Reference](https://docs.djangoproject.com/en/5.0/ref/settings/) + + +## Create Application + +1. `python manage.py startapp blog` create app called blog +1. Edit settings.py and include the blog app `'blog.apps.BlogConfig',` + +| File | Purpose | +|---------------|---------------------------------------------------------| +| `__init__.py` | python module | +| `admin.py` | register models to include in the admin site (optional) | +| `apps.py` | main config of the blog application | +| `migrations` | db migrations | +| `models.py` | data models | +| `tests.py` | add tests | +| `views.py` | logic goes here, request/response | + + +## Models + +- [Field Reference](https://docs.djangoproject.com/en/5.0/ref/models/fields/) +- [Foreign Key On Delete](https://docs.djangoproject.com/en/5.0/ref/models/fields/#django.db.models.ForeignKey.on_delete) +- [Model API](https://docs.djangoproject.com/en/5.0/ref/models/) + +`publish = models.DateTimeField(default=timezone.now)` uses python now with timezone aware +`publish = models.DateTimeField(db_default=Now())` uses the database now + +`auto_now_add` - will use now on create +`auto_now` - will use now on update + +Meta can be added to model classes to set ordering and indexes. '-publish' will order by latest first. + +```python +class Meta: + ordering = ["-publish"] + indexes = [models.Index(fields=['-publish'])] +``` + +### Foreign Keys + +Define a one to many relationship with the default user model and posts. + +related_name allows us to reverse the look up with `user.blog_posts`. + +```python +author = models.ForeignKey( + settings.AUTH_USER_MODEL, + on_delete=models.CASCADE, + related_name='blog_posts', + ) +``` + +## Shell + +`python manage.py shell` runs interactive python shell with django loaded + +## Migrations + +1. Initial migration `python manage.py makemigrations blog` +1. Sync databse with new model `python manage.py migrate` + +`python manage.py sqlmigrate blog 0001` - inspect migration + +Any edit to `models.py` requires two steps. `makemigrations` and `migrate`. + +## Admin Site + +1. create super user: `python manage.py createsuperuser` + +Register models in `admin.py` + +```python +from django.contrib import admin +from .models import Post + +admin.site.register(Post) + +# or + +@admin.register(Post) +class PostAdmin(admin.ModelAdmin): + list_display = ['title', 'slug', 'author', 'publish', 'status'] + list_filter = ['status', 'created', 'publish', 'author'] + search_fields = ['title', 'body'] + prepopulated_fields = {'slug': ('title',)} + raw_id_fields = ['author'] + date_hierarchy = 'publish' + ordering = ['status', 'publish'] + show_facets = admin.ShowFacets.ALWAYS +``` + +## ORM + +- [ORM Reference](https://docs.djangoproject.com/en/5.0/topics/db/queries/) + +- Filters are SQL joined with AND +- + +`get` or `get_or_create` to retrieve +`save` to persist objects +`create` to define and create +`delete` to delete + +`Post.objects.all()` get all posts +`Post.objects.filter(title='Who was Django Reinhardt?')` filter by query +`Post.objects.filter(id__exact=1)` get by id 1 +`Post.objects.filter(id=1)` sae as `id__exact=1`, without lookup type is assumed exact +`Post.objects.filter(title__iexact='who was django reinhardt?')` case insensitive exact match +`Post.objects.filter(title__contains='Django')` title contains django +`Post.objects.filter(title__icontains='django')` case insensive contains +`Post.objects.filter(id__in=[1,3])` find posts with id 1 or 3 +`Post.objects.filter(id__gt=3)` find posts with id greater than 3 +`Post.objects.filter(id__gte=3)` find posts with id greater than equal to 3 +`Post.objects.filter(id__lt=3)` find posts with id less than 3 +`Post.objects.filter(id__lte=3)` find posts with id less than equal to 3 +`Post.objects.filter(title__istartswith='who')` find posts with titles that start with who case-insensitve +`Post.objects.filter(title__iendswith='reinhardt')` find posts with titles that ends with reinhardt case-insensitve +`Post.objects.filter(publish__date=date(2024, 1, 31))` find posts published on a date +`Post.objects.filter(publish__year=2024)` find posts from 2024 +`Post.objects.filter(publish__month=1)` find posts published in jan +`Post.objects.filter(publish__day=1)` find posts published on the first of the month +`Post.objects.filter(publish__date__gt=date(2024, 1, 1))` posts published after a date +`Post.objects.filter(author__username='admin')` posts published by a user +`Post.objects.filter(author__username__startswith='ad')` posts published by a user whose name starts with ad +`Post.objects.filter(publish__year=2024, author__username='admin')` posts published by a user in 2024 + + +```python +posts = Post.objects.filter(title='Who was Django Reinhardt?') +print(posts.query) # print sql +``` + +### Chaining + +```python +Post.objects.filter(publish__year=2024) + .filter(author__username='admin') +``` + +### Excluding + +```python +Post.objects.filter(publich_year=2024) + .exclude(title__startswith='Why') +``` + +### Ordering + +```python +Post.objects.order_by('title') # ascending +Post.objects.order_by('-title') # descending +Post.objects.order_by('author', 'title') # multiple fields +Post.objects.order_by('?') # random order +``` + +### Limiting + +Negative indexing is not supported. + +```python +Post.objects.all()[:5] # limit 5 +Post.objects.all()[3:6] # offset 3 limit 6 +Post.objects.order_by('?')[0] # get a single object randomly +``` + +### Counting + +```python +Post.objects.filter(id_lt=3).count() +``` + +### Exists + +```python +Post.objects.filter(title__startswith='Why').exists() +``` + +### Q objects + +- [Query Objects](https://docs.djangoproject.com/en/5.0/topics/db/queries/#complex-lookups-with-q-objects) + + +```python +from django.db.models import Q +starts_who = Q(title__istartswith='who') +starts_why = Q(title__istartswith='why') +Post.objects.filter(starts_who | starts_why) +``` + +### QuerySet Evaluation + +- [QuerySet API](https://docs.djangoproject.com/en/5.0/ref/models/querysets/) + +QuerySets are only evalutated in the following cases: + +- The first time you iterate over them. +- When you slice them +- When you pickle or cache them +- When you call `repr()` or `len()` on them +- When you explicity call `list()` on them +- When you test them in a statement, such as `bool()`, `or`, `and`, or `if` + +### Model Managers + +- Default manager is `objects` +- Two ways to create custom managers: add extra manager methods or create a new manager by modifying the initial QuerySet that a mangare returns + +```python +class PublishedManager(models.Manager): + def get_queryset(self): + return super().get_queryset() \ + .filter(status=Post.Status.PUBLISHED) + + +objects = models.Manager() +published = PublishedManager() +``` + +## Urls + +- [Path Converters](https://docs.djangoproject.com/en/5.0/topics/http/urls/#path-converters) +- [Regex Reference](https://docs.djangoproject.com/en/5.0/ref/urls/#django.urls.re_path) +- [URL Namespace Reference](https://docs.djangoproject.com/en/5.0/topics/http/urls/#url-namespaces) + +1. Define urls in a `urls.py` for each application +1. Add the include to the `urls.py` in mysite. + +`` is captured as a string + +Path converters such as `` will convert the string to an integer. `` would match a slug. + +## Templates + +- [Template Reference](https://docs.djangoproject.com/en/5.0/ref/templates/language/) +- [Template Builtins](https://docs.djangoproject.com/en/5.0/ref/templates/builtins/) + +- `{% tag %}` controls the rendering of the template +- `{{ variable }}` gets replaced with values +- `{{ variable|filter }}` allow you to modify variables for display +- `{% with comments.count as total_comments %}` will assign the value of `comments.count` to `total_comments` + +## Pagination + +- [Paginator](https://docs.djangoproject.com/en/5.0/ref/paginator/) + +```python +def post_list(request): + post_list = Post.published.all() + paginator = Paginator(post_list, 3) + page_number = request.GET.get('page', 1) + + try: + posts = paginator.page(page_number) + except PageNotAnInteger: + posts = paginator.page(1) + except EmptyPage: + posts = paginator.page(paginator.num_pages) + + return render( + request, + 'blog/post/list.html', + {'posts': posts}, + ) + +``` + +templates/pagination.html + +```html +
+ + {% if page.has_previous %} + Previous + {% endif %} + Page {{ page.number }} of {{ page.paginator.num_pages }}. + {% if page.has_next %} + Next + {% endif %} + +
+``` + +## Class Based Views + +- [Class-based Views Intro](https://docs.djangoproject.com/en/5.0/topics/class-based-views/intro/) + +### WHy? + +- Organize code related to HTTP methods in separate methods instead of conditional branching +- Multiple inheritence to create reusable view classes (mixins) + + +## Forms + +- [Fields](https://docs.djangoproject.com/en/5.0/ref/forms/fields/) + +```python +class EmailPostForm(forms.Form): + name = forms.CharField(max_length=25) + email = forms.EmailField() + to = forms.EmailField() + comments = forms.CharField( + required=False, + widget=forms.Textarea + ) +``` + +```python +def post_share(request, post_id): + post = get_object_or_404( + Post, + id=post_id, + status=Post.Status.PUBLISHED + ) + if request.method == 'POST': + form = EmailPostForm(request.POST) + if form.is_valid(): + cd = form.cleaned_data + else: + form = EmailPostForm() + + return render( + request, + 'blog/post/share.html', + { + 'post': post, + 'form': form + } + ) +``` + +## Tagging + +- [Taggit](https://django-taggit.readthedocs.io/en/latest/) + +```python +# django, third party, local +INSTALLED_APPS = [ + 'django.contrib.admin', + 'django.contrib.auth', + 'django.contrib.contenttypes', + 'django.contrib.sessions', + 'django.contrib.messages', + 'django.contrib.staticfiles', + 'taggit', + 'blog.apps.BlogConfig', +] +``` + +```python +path('', views.post_list, name='post_list'), +# path('', views.PostListView.as_view(), name='post_list'), +path('tag//', views.post_list, name='post_list_by_tag'), +``` + +```python +def post_list(request, tag_slug=None): + post_list = Post.published.all() + tag = None + if tag_slug: + tag = get_object_or_404(Tag, slug=tag_slug) + post_list = post_list.filter(tags__in=[tag]) + + paginator = Paginator(post_list, 3) + page_number = request.GET.get('page', 1) + + try: + posts = paginator.page(page_number) + except PageNotAnInteger: + posts = paginator.page(1) + except EmptyPage: + posts = paginator.page(paginator.num_pages) + + return render( + request, + 'blog/post/list.html', + {'posts': posts}, + ) +``` + +Find similar posts by tags + +```python +post_tags_ids = post.tags.values_list('id', flat=True) +similar_posts = Post.published.filter(tags__in=post_tags_ids).exclude(id=post.id) +similar_posts = similar_posts.annotate(same_tags=Count('tags')).order_by('-same_tags', '-publish')[:4] +``` + +## Custom Template Tags + +[Custom Template Tags](https://docs.djangoproject.com/en/5.0/howto/custom-template-tags/) + +Add `__init__.py` to `templatetags` + +`templatetags/blog_tags.py` + +```python +from django import template +from django.db.models import Count +from ..models import Post + + +register = template.Library() +@register.simple_tag +def total_posts(): + return Post.published.count() + +@register.inclusion_tag('blog/post/latest_posts.html') +def show_latest_posts(count=5): + latest_posts = Post.published.order_by('-publish')[:count] + return {'latest_posts': latest_posts} + +@register.simple_tag +def get_most_commented_posts(count=5): + return Post.published.annotate( + total_comments=Count('comments') + ).order_by('-total_comments')[:count] +``` + +```html +{% load blog_tags %} +{% load static %} + + + + {% block title %}{% endblock %} + + + +
+ {% block content %} + {% endblock %} +
+ + + +``` + +## Template Filters + +[Built-in Template Filters](https://docs.djangoproject.com/en/5.0/ref/templates/builtins/#built-in-filter-reference) + +- `{{ variable|filter }}` basic filter +- `{{ variable|filter:"foo" }}` filter with argument +- `{{ variable|filter|filter }}` filter chaining + +```python +import markdown +from django.utils.safestring import mark_safe + +@register.filter(name='markdown') +def markdown_format(text): + return mark_safe(markdown.markdown(text)) +``` + +```html + {{ post.body|markdown }} +``` + +## Sitemaps + +- [Sitemaps](https://docs.djangoproject.com/en/5.0/ref/contrib/sitemaps/) +- [Sites Framework](https://docs.djangoproject.com/en/5.0/ref/contrib/sites/) + +```python +# settings.py +SITE_ID = 1 + +# Application definition + +INSTALLED_APPS = [ + 'django.contrib.sites', + 'django.contrib.sitemaps', +] + +# sitemaps.py +from django.contrib.sitemaps import Sitemap +from .models import Post + +class PostSitemap(Sitemap): + changefreq = 'weekly' + priority = 0.9 + + def items(self): + return Post.published.all() + + def lastmod(self, obj): + return obj.updated + +# urls.py (mysite) +from django.urls import include, path +from django.contrib.sitemaps.views import sitemap +from blog.sitemaps import PostSitemap + + +sitemaps = { + 'posts': PostSitemap, +} + +urlpatterns = [ + path('admin/', admin.site.urls), + path('blog/', include('blog.urls', namespace='blog')), + path('sitemap.xml', sitemap, {'sitemaps': sitemaps}, name='django.contrib.sitemaps.views.sitemap'), +] +``` + +## Full Text Search + +- [Posgres Adapter Install](https://www.psycopg.org/psycopg3/docs/basic/install.html#binary-installation) +- [SearchVectorField](https://docs.djangoproject.com/en/5.0/ref/contrib/postgres/search/#performance) +- [Stop words](https://github.com/postgres/postgres/blob/master/src/backend/snowball/stopwords/english.stop) +- [Django Search](https://docs.djangoproject.com/en/5.0/ref/contrib/postgres/search/) + +```bash +brew install docker +docker pull postgres:16.2 +docker run --name=blog_db -e POSTGRES_DB=blog -e POSTGRES_USER=blog -e POSTGRES_PASSWORD= -p 5432:5432 -d postgres:16.2 +python -m pip install "psycopg[binary]" +# dumpdata +# python manage.py dumpdata --indent=2 --output=mysite_data.json +# used this to avoid key issues +python manage.py dumpdata --indent=2 --output=mysite_data.json --natural-foreign --natural-primary + +# migrate data +python manage.py migrate +python manage.py loaddata mysite_data.json +``` + +```python +# settings.py +DATABASES = { + 'default': { + 'ENGINE': 'django.db.backends.postgresql', + 'NAME': config('DB_NAME'), + 'USER': config('DB_USER'), + 'PASSWORD': config('DB_PASSWORD'), + 'HOST': config('DB_HOST', default='localhost'), + 'PORT': config('DB_PORT', default='5432') + } +} + +INSTALLED_APPS = [ + 'django.contrib.postgres', +] +``` + +`.env` + +``` +EMAIL_HOST_USER= +EMAIL_HOST_PASSWORD="" +DEFAULT_FROM_EMAIL=My Blog +DB_NAME=blog +DB_USER=blog +DB_PASSWORD= +DB_HOST=localhost +DB_PORT=5432 +``` + +If search is more than a few hundred rows, you should define a functional index that matches the search vector you are using. See [SearchVectorField](https://docs.djangoproject.com/en/5.0/ref/contrib/postgres/search/#performance) + +```py +# forms.py +class SearchForm(forms.Form): + query = forms.CharField() + +# views.py +def post_search(request): + form = SearchForm() + query = None + results = [] + + if 'query' in request.GET: + form = SearchForm(request.GET) + + if form.is_valid(): + query = form.cleaned_data['query'] + results = ( + Post.published.annotate( + search=SearchVector('title', 'body'), + ).filter(search=query) + ) + + return render( + request, + 'blog/post/search.html', + { + 'form': form, + 'query': query, + 'results': results + } + ) + +# urls.py +path('search/', views.post_search, name='post_search'), +``` + +`templates/blog/post/search.html` + +```html +{% extends 'blog/base.html' %} +{% load blog_tags %} +{% block title %}Search{% endblock %} +{% block content %} + {% if query %} +

Posts containing "{{ query }}"

+

+ {% with results.count as total_results %} + Found {{ total_results }} result{{ total_results|pluralize }} + {% endwith %} +

+ {% for post in results %} +

+ {{ post.title }} +

+ {{ post.body|markdown|truncatewords_html:12 }} + {% empty %} +

There are no results for your query.

+ {% endfor %} +

Search again

+ {% else %} +

Search for posts

+
+ {{ form.as_p }} + +
+ {% endif %} +{% endblock %} +``` + +```python +# views.py +if form.is_valid(): + query = form.cleaned_data['query'] + search_vector = SearchVector('title', 'body') + search_query = SearchQuery(query) + results = ( + Post.published.annotate( + search=search_vector, + rank=SearchRank(search_vector, search_query) + ).filter(search=search_query) + .order_by('-rank') + ) + +# spanish +search_vector = SearchVector('title', 'body', config='spanish') +search_query = SearchQuery(query, config='spanish') + +# weights +search_vector = SearchVector('title', weight='A') + SearchVector('body', weight='B') +results = ( + Post.published.annotate( + search=search_vector, + rank=SearchRank(search_vector, search_query) + ).filter(rank__gte=0.3) + .order_by('-rank') +) +``` + +### Weights + +`D`: `0.1` +`C`: `0.2` +`B`: `0.4` +`A`: `1.0` + +### Trigram Similarity + +- [Postgres Operations](https://docs.djangoproject.com/en/5.0/ref/contrib/postgres/operations/) + +Use `pg_trgm` + +`python manage.py makemigrations --name=trigram_ext --empty blog` + +Edit the migration file + +```python +from django.db import migrations +from django.contrib.postgres.operations import TrigramExtension + +class Migration(migrations.Migration): + + dependencies = [ + ('blog', '0004_post_tags'), + ] + + operations = [ + TrigramExtension(), + ] +``` + +`python manage.py migrate blog` + +```python +if form.is_valid(): + query = form.cleaned_data['query'] + results = ( + Post.published.annotate( + similarity=TrigramSimilarity('title', query), + ).filter(similarity__gt=0.1) + .order_by('-similarity') + ) +``` + +### AI + +- [Chapter 3 Prompt](https://github.com/PacktPublishing/Django-5-by-example/blob/main/Chapter03/prompts/task.md) + +```python +# blog/sitemaps.py +class TagSitemap(Sitemap): + changefreq = 'weekly' + priority = 0.6 + + def items(self): + return Tag.objects.all() + + def location(self, obj): + return reverse('blog:post_list_by_tag', args=[obj.slug]) + +# mysite/urls.py +sitemaps = { + 'posts': PostSitemap, + 'tags': TagSitemap, +} +``` + +## New Project + +```bash +django-admin startproject bookmarks +django-admin startapp account + +``` + +`settings.py` + +```python +INSTALLED_APPS = [ + 'account.apps.AccountConfig', + # ... +] +``` + +`python manage.py migrate` + +## Built-in Authentication Framework + +`django.contrib.auth` + +Middleware: +`AuthenticationMiddleware`: associates users with requests +`SessionMiddleware`: handles current session across requests + +Models: +`User`: represents a user account +`Group`: represents a group of users +`Permission`: represents a permission that can be assigned to a user or group + + +```python +# account/forms.py +from django import forms + +class LoginForm(forms.Form): + username = forms.CharField() + password = forms.CharField(widget=forms.PasswordInput) + +# account/views.py +def user_login(request): + if request.method == 'POST': + form = LoginForm(request.POST) + if form .is_valid(): + cd = form.cleaned_data + user = authenticate( + request, + username=cd['username'], + password=cd['password'] + ) + if user is not None: + if user.is_active: + login(request, user) + return HttpResponse('Authenticated successfully') + else: + return HttpResponse('Disabled account') + else: + return HttpResponse('Invalid login') + else: + form = LoginForm() + return render(request, 'account/login.html', {'form': form}) + +# account/urls.py +from django.urls import path +from . import views + +urlpatterns = [ + path('login/', views.user_login, name='login'), +] + +# bookmarss/urls.py add: +path('account/', include('account.urls')), + +``` + +`authenticate()` returns a user object if the credentials are valid, or `None` if not. +`login()` logs in the user and sets the session cookie. + +`python manage.py createsuperuser` + +- [Default Views](https://docs.djangoproject.com/en/5.0/topics/auth/default/#all-authentication-views) + + +```python +# account/settings.py +LOGIN_REDIRECT_URL = 'dashboard' +LOGIN_URL = 'login' +LOGOUT_URL = 'logout' + +# console email +EMAIL_BACKEND = 'django.core.mail.backends.console.EmailBackend' +``` + +- [Django Auth URLs](https://github.com/django/django/blob/stable/5.0.x/django/contrib/auth/urls.py) + +```python +# forms.py +class UserRegistrationForm(forms.ModelForm): + password = forms.CharField(label='Password', widget=forms.PasswordInput) + password2 = forms.CharField(label='Repeat password', widget=forms.PasswordInput) + + class Meta: + model = get_user_model() + fields = ('username', 'first_name', 'email') + + def clean_password2(self): + cd = self.cleaned_data + if cd['password'] != cd['password2']: + raise forms.ValidationError('Passwords don\'t match.') + + return cd['password2'] + +# views.py +def register(request): + if request.method == 'POST': + user_form = UserRegistrationForm(request.POST) + if user_form.is_valid(): + new_user = user_form.save(commit=False) + new_user.set_password(user_form.cleaned_data['password']) + new_user.save() + return render(request, 'account/register_done.html', {'new_user': new_user}) + else: + user_form = UserRegistrationForm() + return render(request, 'account/register.html', {'user_form': user_form}) +``` + +- [Django Auth get_user_model](https://docs.djangoproject.com/en/5.0/topics/auth/customizing/#django.contrib.auth.get_user_model) + + +`clean_` is called when the form is validated. +`set_password` handles hashing and storing the password. + +Default is `PBKDF2` hashing. Supports: `P8KDF2SHA1`, `argon2`, `bcrypt`, `scrypt`. + +`scrypt` was introduced in Django 4.0 and requires OpenSSL 1.1+ or later and more memory. + +`PASSWORD_HASHERS` setting defines the hashers to use. + +- [Django Auth Passwords](https://docs.djangoproject.com/en/5.0/topics/auth/passwords/) + +Extending the user model with a profile model is simple with a 1:1 relationship. + +- [Django Auth Customizing](https://docs.djangoproject.com/en/5.0/topics/auth/customizing/#substituting-a-custom-user-model) + +Using a custom model will give emore flexibility, but it might also result in more difficult integrations with pluggble applications that interact directly with Django’s user model. + +## Media Files + +`python -m pip install Pillow` + +```python +# settings.py +MEDIA_URL = 'media/' +MEDIA_ROOT = BASE_DIR / 'media' + +# bookmarks/urls.py +if settings.DEBUG: + urlpatterns += static( + settings.MEDIA_URL, + document_root=settings.MEDIA_ROOT + ) + +# bookmarks/forms.py +class UserEditForm(forms.ModelForm): + class Meta: + model = get_user_model() + fields = ['first_name', 'last_name', 'email'] + +class ProfileEditForm(forms.ModelForm): + class Meta: + model = Profile + fields = ['date_of_birth', 'photo'] + +# bookmarks/views.py +Profile.objects.create(user=new_user) + +@login_required +def edit(request): + if request.method == 'POST': + user_form = UserEditForm(instance=request.user, data=request.POST) + profile_form = ProfileEditForm( + instance=request.user.profile, + data=request.POST, + files=request.FILES + ) + if user_form.is_valid() and profile_form.is_valid(): + user_form.save() + profile_form.save() + else: + user_form = UserEditForm(instance=request.user) + profile_form = ProfileEditForm(instance=request.user.profile) + return render( + request, + 'account/edit.html', + {'user_form': user_form, 'profile_form': profile_form} + ) +``` + +Django is very inefficient at servince static files, this is only for development. + +```bash +python manage.py makemigrations +python manage.py migrate +``` + +## Messages + +- [Django Messages](https://docs.djangoproject.com/en/5.0/ref/contrib/messages/) + +`add_message()` or `success()` or `error()` or `warning()` or `info()` or `debug()` + +## OAuth + +- [Django Auth Sources](https://docs.djangoproject.com/en/5.0/topics/auth/customizing/#other-authentication-sources) +- [Django Social App](https://github.com/python-social-auth/social-app-django) +- [Social Backends](https://python-social-auth.readthedocs.io/en/latest/backends/index.html#supported-backends) +- [Allowed Hosts](https://docs.djangoproject.com/en/5.0/ref/settings/#allowed-hosts) +- [Django Estensions](https://django-extensions.readthedocs.io/en/latest/) +- [Google OAuth2](https://developers.google.com/identity/protocols/OAuth2)) + +```python +# settings.py +AUTHENTICATION_BACKENDS = [ + 'django.contrib.auth.backends.ModelBackend', + 'account.authentication.EmailAuthBackend', +] +``` + +```bash +python -m pip install social-auth-app-django +``` + +### SSL + +```bash +python -m pip install django-extensions +python -m pip install werkzeug +python -m pip install pyOpenSSL +python -m pip install python-decouple +``` + +```python +# settings.py INSTALLED_APPS +'django_extensions', + +AUTHENTICATION_BACKENDS = [ + 'django.contrib.auth.backends.ModelBackend', + 'account.authentication.EmailAuthBackend', + 'social_core.backends.google.GoogleOAuth2', +] +``` + +```bash +python manage.py runserver_plus --cert-file cert.crt +``` + +- [Google Dev Key]( https://console.cloud.google.com/projectcreate) +- [Python Social Auth Pipeline](https://python-social-auth.readthedocs.io/en/latest/pipeline.html) + +1. create app +2. authorize consent +3. create credentials with OAuth client ID + + +`.env` + +``` +GOOGLE_OAUTH_KEY= +GOOGLE_OAUTH_SECRET= +``` + +```python +# settings.py + +SOCIAL_AUTH_GOOGLE_OAUTH2_KEY = config('GOOGLE_OAUTH2_KEY') +SOCIAL_AUTH_GOOGLE_OAUTH2_SECRET = config('GOOGLE_OAUTH2_SECRET') + +SOCIAL_AUTH_PIPELINE = [ + 'social_core.pipeline.social_auth.social_details', + 'social_core.pipeline.social_auth.social_uid', + 'social_core.pipeline.social_auth.auth_allowed', + 'social_core.pipeline.social_auth.social_user', + 'social_core.pipeline.user.get_username', + 'social_core.pipeline.user.create_user', + 'account.authentication.create_profile', + 'social_core.pipeline.social_auth.associate_user', + 'social_core.pipeline.social_auth.load_extra_data', + 'social_core.pipeline.user.user_details', +] +``` + +` python manage.py runserver_plus --cert-file cert.crt` + + +```python +# account/authentication.py +def create_profile(backend, user, *args, **kwargs): + """ + Create user profile for social authentication + """ + Profile.objects.get_or_create(user=user) +``` + +## Images + +- [Database Indexes](https://docs.djangoproject.com/en/5.0/ref/models/options/#django.db.models.Options.indexes) +- [Django Many to Many Models](https://docs.djangoproject.com/en/5.0/topics/db/examples/many_to_many/) +- [Image Thumbnails](https://easy-thumbnails.readthedocs.io/en/latest/) + +```txt +requests==2.31.0 +easy-thumbnails==2.8.5 +``` + +```python +# images/models.py +from django.db import models + +from django.conf import settings +from django.db import models + +class Image(models.Model): + user = models.ForeignKey( + settings.AUTH_USER_MODEL, + related_name='images_created', + on_delete=models.CASCADE) + + title = models.CharField(max_length=200) + slug = models.SlugField(max_length=200, blank=True) + url = models.URLField(max_length=2000) + image = models.ImageField(upload_to='images/%Y/%m/%d/') + description = models.TextField(blank=True) + created = models.DateField(auto_now_add=True) + users_like = models.ManyToManyField( + settings.AUTH_USER_MODEL, + related_name='images_liked', + blank=True) + + class Meta: + indexes = [ + models.Index(fields=['-created']), + ] + ordering = ['-created'] + + def __str__(self): + return self.title + + def save(self, *args, **kwargs): + if not self.slug: + self.slug = slugify(self.title) + super().save(*args, **kwargs) + +# images/forms.py +from django import forms +from .models import Image + +class ImageCreateForm(forms.ModelForm): + class Meta: + model = Image + fields = ['title', 'url', 'description'] + widgets = { + 'url': forms.HiddenInput, + } + + +def clean_url(self): + url = self.cleaned_data['url'] + valid_extensions = ['jpg', 'jpeg', 'png'] + extension = url.rsplit('.', 1)[1].lower() + if extension not in valid_extensions: + raise forms.ValidationError('The given URL does not ' + 'match valid image extensions.') + return url + +def save(self, + force_insert=False, + force_update=False, + commit=True): + image = super().save(commit=False) + image_url = self.cleaned_data['url'] + name = slugify(image.title) + extension = image_url.rsplit('.', 1)[1].lower() + image_name = f'{name}.{extension}' + response = requests.get(image_url) + image.image.save( + image_name, + ContentFile(response.content), + save=False) + if commit: + image.save() + + return image + +# images/views.py +from django.contrib import messages +from django.contrib.auth.decorators import login_required +from django.shortcuts import render, redirect +from .forms import ImageCreateForm + +@login_required +def image_create(request): + if request.method == 'POST': + form = ImageCreateForm(data=request.POST) + if form.is_valid(): + cd = form.cleaned_data + new_item = form.save(commit=False) + new_item.user = request.user + new_item.save() + messages.success(request, 'Image added successfully') + + return redirect(new_item.get_absolute_url()) + else: + form = ImageCreateForm(data=request.GET) + return render(request, + 'images/image/create.html', + {'section': 'images', 'form': form} + ) +``` + +Thumbnails + +```bash +python -m pip install easy-thumbnails +python manage.py migrate +``` + +```html +{% load thumbnail %} + + + +``` + +## HTTP Requests + +```python +@require_POST # only accept POST requests +``` + +## Javascript + +- [CSRF](https://docs.djangoproject.com/en/5.0/ref/csrf/#ajax) + +```html + + + + +{% block domready %} +const url = '{% url "images:like" %}'; +var options = { + method: 'POST', + headers: {'X-CSRFToken': csrftoken}, + mode: 'same-origin' +} +document.querySelector('a.like').addEventListener('click', function(e){ + e.preventDefault(); + var likeButton = this; + var formData = new FormData(); + formData.append('id', likeButton.dataset.id); + formData.append('action', likeButton.dataset.action); + options['body'] = formData; + fetch(url, options) + .then(response => response.json()) + .then(data => { + if (data['status'] === 'ok') { + var previousAction = likeButton.dataset.action; + var action = previousAction === 'like' ? 'unlike' : 'like'; + likeButton.dataset.action = action; + likeButton.innerHTML = action; + var likeCount = document.querySelector('span.count .total'); + var totalLikes = parseInt(likeCount.innerHTML); + likeCount.innerHTML = previousAction === 'like' ? totalLikes + 1 : totalLikes - 1; + } + }) +}) +{% endblock %} +``` + +## Pagination + +```python +@login_required +def image_list(request): + images = Image.objects.all() + paginator = Paginator(images, 8) + page = request.GET.get('page') + images_only = request.GET.get('images_only') + try: + images = paginator.page(page) + except PageNotAnInteger: + images = paginator.page(1) + except EmptyPage: + if images_only: + return HttpResponse('') + images = paginator.page(paginator.num_pages) + if images_only: + return render(request, + 'images/image/list_images.html', + { + 'section': 'images', + 'images': images, + }) + return render(request, + 'images/image/list.html', + { + 'section': 'images', + 'images': images, + }) +``` + +```html + +{% extends "base.html" %} +{% block title %}Images bookmarked{% endblock %} +{% block content %} +

Images bookmarked

+
+ {% include "images/image/list_images.html" %} +
+{% endblock %} +{% block domready %} + var page = 1; + var emptyPage = false; + var blockRequest = false; + window.addEventListener('scroll', function(e) { + var margin = document.body.clientHeight - window.innerHeight - 200; + if (window.pageYOffset > margin && !emptyPage && !blockRequest) { + blockRequest = true; + page += 1; + fetch(`?images_only=1&page=${page}`) + .then(response => response.text()) + .then(html => { + if (html === '') { + emptyPage + } else { + var imageList = document.getElementById('image-list'); + imageList.insertAdjacentHTML('beforeend', html); + blockRequest = false; + } + }); + } + }); + const scrollEvent = new Event('scroll'); + window.dispatchEvent(scrollEvent); +{% endblock %} + + + {% load thumbnail %} +{% for image in images %} + +{% endfor %} +``` + +## Many to Many Relationships + +Usually `ManytoManyField` is good enough, when you need additional data an intermediate model is needed. + +Two reasons to use: + +- using the Django user table and don't want to alter it +- storing additional data about the relationship, such as time + +- [Custom User Model](https://docs.djangoproject.com/en/5.0/topics/auth/customizing/#specifying-a-custom-user-model) + +```python +class Contact(models.Model); + user_from = models.ForeignKey( + settings.AUTH_USER_MODEL, + related_name='rel_from_set', + on_delete=models.CASCADE + ) + user_to = models.ForeignKey( + settings.AUTH_USER_MODEL, + related_name='rel_to_set', + on_delete=models.CASCADE + ) + created = models.DateTimeField(auto_now_add=True) + + class Meta: + indexes = [ + models.Index(fields=['-created']), + ] + ordering = ['-created'] + + def __str__(self): + return f'{self.user_from} follows {self.user_to}' + +class Profile(models.Model): + user = models.OneToOneField(settings.AUTH_USER_MODEL, on_delete=models.CASCADE) + date_of_birth = models.DateField(blank=True, null=True) + photo = models.ImageField(upload_to='users/%Y/%m/%d/', blank=True) + user_model = get_user_model() + user_model.add_to_class( + 'following', + models.ManyToManyField( + 'self', + through=Contact, + related_name='followers', + symmetrical=False + ) + ) + + def __str__(self): + return f'Profile of {self.user.username}' +``` + +`symentrical=False` with followers for example, means that because i follow you, doesn't mean you automatically follow me. + +## contenttypes Framework + +- [contenttypes framework](https://docs.djangoproject.com/en/5.0/ref/contrib/contenttypes/) + +`django.contrib.contenttypes` is included by default. Creates a `ContentType` model that stores a reference to the model that is registered in the database. + +```bash +>>> from django.contrib.contenttypes.models import ContentType +>>> image_type = ContentType.objects.get(app_label='images', model='image') +>>> image_type + +>>> image_type.model_class() + +>>> from images.models import Image +>>> ContentType.objects.get_for_model(Image) + +``` + +Three things to setup a generic relationship: + +- A `foreign key` field to `ContentType`: the model relationship +- A field to store the primary key of the related model: usually `PositiveIntegerField` to match Django's automatic primary key fields +- A field to define and mange the generic relation using the two prvious fields: the `contenttypes` framwork offers a `GenericForeignKey` field + + +avoid overlapping actions and save + +```python +# actions/utils.py +import datetime +from django.contrib.contenttypes.models import ContentType +from django.utils import timezone +from .models import Action + +def create_action(user, verb, target=None): + now = timezone.now() + last_minute = now - datetime.timedelta(seconds=60) + similar_actions = Action.objects.filter(user_id=user.id, + verb=verb, + created__gte=last_minute + ) + if target: + target_ct = ContentType.objects.get_for_model(target) + similar_actions = similar_actions.filter( + target_ct=target_ct, + target_id=target.id + ) + + if not similar_actions: + action = Action(user=user, + verb=verb, + target=target + ) + action.save() + return True + return False + +# account/views.py +@login_required +def dashboard(request): + actions = Action.objects.exclude(user=request.user) + following_ids = request.user.following.values_list('id', flat=True) + if following_ids: + actions = actions.filter(user_id__in=following_ids) + actions = actions.select_related('user', 'user__profile').prefetch_related('target')[:10] + + return render(request, + 'account/dashboard.html', + { + 'section': 'dashboard', + 'actions': actions + }) +``` + +- `select_related` is used to avoid N+1 queries with 1:many relationships. Join uses SQL. +- `prefetch_related` is used to avoid N+1 queries with many:many relationships. Join uses python. + +```html + + {% load thumbnail %} +{% with user=action.user profile=action.user.profile %} +
+
+ {% if profile.photo %} + {% thumbnail user.profile.photo 80x80 crop="100%" as im %} + + {{ user.get_full_name }} + + {% endif %} + {% if action.target %} + {% with target=action.target %} + {% if target.image %} + {% thumbnail target.image 80x80 crop="100%" as im %} + + + + {% endif %} + {% endwith %} + {% endif %} +
+
+

+ {{ action.created|timesince }} ago +
+ + {{ user.first_name }} + + {{ action.verb }} + {% if action.target %} + {% with target=action.target %} + {{ target }} + {% endwith %} + {% endif %} +

+
+
+{% endwith %} +``` + +## Using Signals for Denormalizing Counts + +Denormalizing data is making data redundant to optimize for read performance. + +- [Signals](https://docs.djangoproject.com/en/5.0/ref/signals/) +- [App Config](https://docs.djangoproject.com/en/5.0/ref/applications/) + +- Django comes with a signal dispatcher that allows receiver functions to get notified when certain actions occur in the system. +- Decouple logic +- Several methods: + - `pre_save` and `post_save` are sent before and after calling the `save()` method of a model. + - `pre_delete` and `post_delete` are sent before and after calling the `delete()` method of a model or queryset + - `m2m_changed` is sent when a `ManyToManyField` on a model is changed. + +This would retrieve images according to number of likes: + +```python +from django.db.models import Count +from images.models import Image +images_by_popularity = Image.objects.annotate( + total_likes=Count('users_like') +).order_by('-total_likes') +``` + +- It's expensive, it's better to add a field that stores the total counts to images. +- Several ways to imporve performance before denormalizing: + - database indexes + - query optimization + - caching +- signals are synchronous + +```python +# images/models.py + total_likes = models.PositiveBigIntegerField(default=0) + + class Meta: + indexes = [ + models.Index(fields=['-created']), + models.Index(fields=['-total_likes']) + ] + ordering = ['-created'] + +# images/signals.py +from django.db.models.signals import m2m_changed +from django.dispatch import receiver +from .models import Image + +@receiver(m2m_changed, sender=Image.users_like.through) +def users_like_changed(sender, instance, **kwargs): + instance.total_likes = instance.users_like.count() + instance.save() + +# iamges/apps.py +from django.apps import AppConfig + + +class ImagesConfig(AppConfig): + default_auto_field = 'django.db.models.BigAutoField' + name = 'images' + def ready(self): + # import signal handlers + import images.signals +``` + +To sync db state open shell and run: + +```python +from images.models import Image +for image in Image.objects.all(): + image.total_likes = image.users_like.count() + image.save() + +``` + + ## Debug Toolbar + + - [Django Debug Toolbar](https://django-debug-toolbar.readthedocs.io/en/latest/) + - [Django Debug Panels](https://django-debug-toolbar.readthedocs.io/en/latest/panels.html#third-party-panels) + + ```bash +python -m pip install django-debug-toolbar + ``` + + ```python +# settings.py + INSTALLED_APPS = [ + 'debug_toolbar', +] + +MIDDLEWARE = [ + 'debug_toolbar.middleware.DebugToolbarMiddleware', +] + +INTERNAL_IPS = ['127.0.0.1'] + +# urls.py +path('__debug__/', include('debug_toolbar.urls')), +``` + +debug shell + +```bash +python manage.py debugsqlshell +``` + +use the debug shell + +```python +>>> Image.objects.get(id=11) +SELECT "images_image"."id", + "images_image"."user_id", + "images_image"."title", + "images_image"."slug", + "images_image"."url", + "images_image"."image", + "images_image"."description", + "images_image"."created", + "images_image"."total_likes" +FROM "images_image" +WHERE "images_image"."id" = 11 +LIMIT 21 [0.33ms] + +``` + +## Redis + +- [Redis](https://redis.io/) +- [Redis Commands](https://redis.io/docs/latest/commands/) +- [Redis Datatypes](https://redis.io/docs/latest/develop/data-types/) + +```bash +docker pull redis:7.2.4 +docker run -it --rm --name redis -p 6379:6379 redis:7.2.4 +docker exec -it redis sh +redis-cli +SET name "Peter" +GET name +EXISTS name +EXPIRE name 2 +GET name + +SET total 1 +DEL total +GET total + +python -m pip install redis + +python manage.py shell + +>>> import redis +>>> r = redis.Redis(host='localhost', port=6379, db=0) +>>> r.set('foo', 'bar') +True +>>> r.get('foo') +b'bar' +``` + +```python +# settings.py +REDIS_HOST = 'localhost' +REDIS_PORT = 6379 +REDIS_DB = 0 + +# images/views.py +r = redis.Redis( + host=settings.REDIS_HOST, + port=settings.REDIS_PORT, + db=settings.REDIS_DB, +) + +def image_detail(request, id, slug): + image = get_object_or_404(Image, id=id, slug=slug) + total_views = r.incr(f'image:{image.id}:views') + return render(request, + 'images/image/detail.html', + { + 'section': 'images', + 'image': image, + 'total_views': total_views + } + ) + +@login_required +def image_ranking(request): + image_ranking = r.zrange('image_ranking', 0, -1, desc=True)[:10] + image_ranking_ids = [int(id) for id in image_ranking] + most_viewed = list(Image.objects.filter(id__in=image_ranking_ids)) + most_viewed.sort(key=lambda x: image_ranking_ids.index(x.id)) + r.zincrby('image_ranking', 1, image.id) + return render( + request, + 'images/image/ranking.html', + { + 'section': 'images', + 'most_viewed': most_viewed + } + ) +``` + +Redis keys are `object-type:id:field`, which is namespaced. Seperating by colon allows grouped calls. + +## Chat GPT + +### prompt + +**Background:** I've developed a Django application called account, which includes a Profile model. This model extends Django's default authentication User model. + +**Goal:** I aim to use Django signals to automatically create an associated `Profile` object each time a `User` object is created. + +**Here’s some of my existing setup:** + +Definition of the `Profile` model in `account/models.py`: +``` +class Profile(models.Model): + user = models.OneToOneField( + settings.AUTH_USER_MODEL, + on_delete=models.CASCADE + ) + date_of_birth = models.DateField(blank=True, null=True) + photo = models.ImageField( + upload_to='users/%Y/%m/%d/', + blank=True + ) + + def __str__(self): + return f'Profile of {self.user.username}' +``` + +### results + +```python +# account//signals.py +from django.db.models.signals import post_save +from django.contrib.auth import get_user_model +from django.dispatch import receiver +from .models import Profile + +User = get_user_model() + +@receiver(post_save, sender=User) +def create_user_profile(sender, instance, created, **kwargs): + if created: + Profile.objects.create(user=instance) + +# account/apps.py +from django.apps import AppConfig + +class AccountConfig(AppConfig): + name = 'account' + + def ready(self): + import account.signals # noqa: F401 + +# settings..py +from django.apps import AppConfig + +class AccountConfig(AppConfig): + name = 'account' + + def ready(self): + import account.signals # noqa: F401 + +``` + +## images + +```python +# settings.py +MEDIA_URL = 'media/' +MEDIA_ROOT = BASE_DIR / 'media' + +# urls.py +if settings.DEBUG: + urlpatterns += static(settings.MEDIA_URL, + document_root=settings.MEDIA_ROOT) +``` + +## sessions + +- [session settings](https://docs.djangoproject.com/en/5.0/ref/settings/#sessions) + +- anonymous user sessions +- store arbitrary data for the user +- stored on the server side and cookies contain the session id, unleess you use the cookie based session engine +- session middleware manages sending and receiving cookies +- default session engine stores data in the database +- `django.contrib.sessions.middleware.SessionMiddleware` must be in the middleware, added by default + +```python +request.session['key'] = value +request.session.get('key', default) +del request.session['key'] +``` + +### storing session data + +- database sessions +- file based sessions +- cache based sessions +- cached db sessions +- cookie based sessions + +```python +# cart.py +from decimal import Decimal +from django.conf import settings +from shop.models import Product + +class Cart: + def __init__(self, request): + """ + Initialize the cart. + """ + self.session = request.session + cart = self.session.get(settings.CART_SESSION_ID) + if not cart: + # save an empty cart in the session + cart = self.session[settings.CART_SESSION_ID] = {} + self.cart = cart + + def add(self, product, quantity=1, override_quantity=False): + """ + Add a product to the cart or update its quantity. + """ + product_id = str(product.id) + if product_id not in self.cart: + self.cart[product_id] = {'quantity': 0, + 'price': str(product.price)} + if override_quantity: + self.cart[product_id]['quantity'] = quantity + else: + self.cart[product_id]['quantity'] += quantity + self.save() + + def save(self): + self.session.modified = True + + def remove(self, product): + """ + Remove a product from the cart. + """ + product_id = str(product.id) + if product_id in self.cart: + del self.cart[product_id] + self.save() + + def __iter__(self): + """ + Iterate over the items in the cart and get the products + from the database. + """ + product_ids = self.cart.keys() + # get the product objects and add them to the cart + products = Product.objects.filter(id__in=product_ids) + cart = self.cart.copy() + for product in products: + cart[str(product.id)]['product'] = product + for item in cart.values(): + item['price'] = Decimal(item['price']) + item['total_price'] = item['price'] * item['quantity'] + yield item + + def __len__(self): + """ + Count all items in the cart. + """ + return sum(item['quantity'] for item in self.cart.values()) + + def get_total_price(self): + return sum(Decimal(item['price']) * item['quantity'] for item in self.cart.values()) + + def clear(self): + # remove cart from session + del self.session[settings.CART_SESSION_ID] + self.save() +``` + +## context processors + +- [context processsors](https://docs.djangoproject.com/en/5.0/ref/templates/api/#built-in-template-context-processors) +- [request context](https://docs.djangoproject.com/en/5.0/ref/templates/api/#django.template.RequestContext) + +- python function that takes the request object as an argument and returns a dictionary that gets added to the context +- used for global data in templates + +```python +# cart/context_processors.py +from .cart import Cart + +def cart(request): + return {'cart': Cart(request)} + +# settings.py +TEMPLATES = [ + { + # ... + 'OPTIONS': { + 'context_processors': [ + # ... + 'cart.context_processors.cart', + ] + } + } +] + +``` + +## inline admin + +```python +from django.contrib import admin +from .models import Order, OrderItem + +class OrderItemInline(admin.TabularInline): + model = OrderItem + raw_id_fields = ['product'] + +@admin.register(Order) +class OrderAdmin(admin.ModelAdmin): + list_display = ['id', 'first_name', 'last_name', 'email', + 'address', 'postal_code', 'city', 'paid', + 'created', 'updated'] + list_filter = ['paid', 'created', 'updated'] + inlines = [OrderItemInline] +``` + +## Async Tasks + +- workers are task based servers to process tasks asynchronously + - one or many workers can be running and executing tasks in the background + - can queue future tasks + - messages are used to queue tasks in a FIFO order (first in first out) + - when finished workers pickup the next message from the queue + - with multiple, each worker takes the first available message in order when they become available +- queue ensures each broker only gets one task at a time + - producer sends messages to the queue +- message broker used to translate a message to a formal message protocol + - provides reliable storage and guarenteed delivery + - brokers are idle when queues are empty + + +### Celery + +- [celery](https://docs.celeryq.dev/en/stable/index.html) +- [celery intro](https://docs.celeryq.dev/en/stable/getting-started/introduction.html) +- [rabbbitmq](https://www.rabbitmq.com/docs/download) +- [flower](https://flower.readthedocs.io/en/latest/) + +- distrributed task queue +- communicates via messages and requires a message broker to mediatge between the client and worker +- several options for brokers including redis and rabbitmq +- `Rabbit AMQP` is the recommended message worker + +```bash +python -m pip install celery +docker pull rabbitmq:3.13.1-management +docker run -it --rm --name rabbitmq -p 5672:5672 -p 15672:15672 rabbitmq:3.13.1-management +``` + +- [local rabbit admin](http://127.0.0.1:15672/#/) + +default `guest:guest` + +```python +# celery.py +import os +from celery import Celery + +os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'myshop.settings') +app = Celery('myshop') +app.config_from_object('django.conf:settings', namespace='CELERY') +app.autodiscover_tasks() + +# __init__.py +from .celery import app as celery_app + +__all__ = ['celery_app'] +``` + +start a worker + +```bash +celery -A myshop worker -l info +``` + +- The `CELERY_ALWAYS_EAGER` setting allows you to execute tasks locally in a synchronous manner instead of sending them to a queue. This is useful for testing and development. +- convention for task discovery is to define async tasks in tasks module in your application +- recommended to only send ids to tasks and query db to avoid outdated information + +```python +# tasks.py +from celery import shared_task +from django.core.mail import send_mail +from .models import Order + +@shared_task +def order_created(order_id): + """ + Task to send an e-mail notification when an order is + successfully created. + """ + order = Order.objects.get(id=order_id) + subject = f'Order nr. {order.id}' + message = f'Dear {order.first_name},\n\n' \ + f'You have successfully placed an order.' \ + f'Your order ID is {order.id}.' + mail_sent = send_mail(subject, + message, + 'admin@myshop.com', + [order.email]) + return mail_sent + +# order/views.py +order_created.delay(order.id) +``` + +### Flower + +Moniitor Celery tasks + +```bash +python -m pip install flower +celery -A myshop flower +celery -A myshop flower --basic-auth=: +``` + +- http://localhost:5555 + +## Payment Gateway +- [stripe](https://stripe.com/) +- [Stripe Checkout](https://docs.stripe.com/payments/checkout) +- [stripe python](https://github.com/stripe/stripe-python) +- [stripe test apikeys](https://dashboard.stripe.com/test/apikeys) +- [stripe keys](https://docs.stripe.com/keys) +- [stripe session object](https://docs.stripe.com/api/checkout/sessions/create) + +- `pk_test_`: test mode publishable key +- `pk_live_`: live mode publishable key +- `sk_test_`: test mode secret key +- `sk_live_`: live mode secret key + +### test credit cards + +- [stripe dashboard](https://dashboard.stripe.com/test/payments) + +| result | test card number | cvc | expiration date | +|----------------------------------|---------------------|--------------|-----------------| +| success | 4242 4242 4242 4242 | any 3 digits | any future date | +| failed | 4000 0000 0000 0002 | any 3 digits | any future date | +| require 3d secure authentication | 4000 0025 0000 3155 | any 3 digits | any future date | + +### webhook + +- [stripe webhooks](https://dashboard.stripe.com/test/webhooks) +- [stripe webhook events](https://docs.stripe.com/api/events/types) + +```bash +brew install stripe/stripe-cli/stripe +stripe login +stripe listen --forward-to 127.0.0.1:8000/payment/webhook/ +stripe trigger payment_intent.succeeded +``` + +```python +# webhooks.py +import stripe +from django.conf import settings +from django.http import HttpResponse +from django.views.decorators.csrf import csrf_exempt +from orders.models import Order + +@csrf_exempt +def stripe_webhook(request): + payload = request.body + sig_header = request.META['HTTP_STRIPE_SIGNATURE'] + event = None + try: + event = stripe.Webhook.construct_event( + payload, sig_header, settings.STRIPE_WEBHOOK_SECRET + ) + except ValueError as e: + return HttpResponse(status=400) + except stripe.error.SignatureVerificationError as e: + return HttpResponse(status=400) + + if event.type == 'checkout.session.completed': + session = event.data.object + if session.mode == 'payment' and session.payment_status == 'paid': + try: + order = Order.objects.get(id=session.client_reference_id) + except Order.DoesNotExist: + return HttpResponse(status=404) + order.paid = True + order.save() + + return HttpResponse(status=200) + +# orders/models.py + def get_stripe_url(self): + if not self.stripe_id: + return '' + if '_test_' in settings.STRIPE_SECRET_KEY: + path = '/test/' + else: + path = '/' + return f'https://dashboard.stripe.com{path}payments/{self.stripe_id}' + +# orders/admin.py +from django.contrib import admin +from .models import Order, OrderItem +from django.utils.safestring import mark_safe + +def order_payment(obj): + url = obj.get_stripe_url() + if obj.stripe_id: + html = f'{obj.stripe_id}' + return mark_safe(html) + return '' + +order_payment.short_description = 'Stripe payment' + +class OrderItemInline(admin.TabularInline): + model = OrderItem + raw_id_fields = ['product'] + +@admin.register(Order) +class OrderAdmin(admin.ModelAdmin): + list_display = ['id', 'first_name', 'last_name', 'email', + 'address', 'postal_code', 'city', 'paid', + order_payment, 'created', 'updated'] + list_filter = ['paid', 'created', 'updated'] + inlines = [OrderItemInline] +``` + +### Export CSV with Custom Action + +- [outputting csv](https://docs.djangoproject.com/en/5.0/howto/outputting-csv/) +- [django import export library](https://django-import-export.readthedocs.io/en/latest/) +- [import/export with celery](https://github.com/auto-mat/django-import-export-celery) + +```python +# orders/admin.py +def export_to_csv(modeladmin, request, queryset): + opts = modeladmin.model._meta + content_disposition = (f"attachment; filename={opts.verbose_name}.csv") + response = HttpResponse(content_type='text/csv') + response['Content-Disposition'] = content_disposition + writer = csv.writer(response) + fields = [field + for field in opts.get_fields() + if not field.many_to_many and not field.one_to_many] + writer.writerow([field.verbose_name for field in fields]) + + for obj in queryset: + data_row = [] + for field in fields: + value = getattr(obj, field.name) + if isinstance(value, datetime.datetime): + value = value.strftime('%d/%m/%Y') + data_row.append(value) + + writer.writerow(data_row) + + return response + +export_to_csv.short_description = 'Export to CSV' + +@admin.register(Order) +class OrderAdmin(admin.ModelAdmin): + actions = [export_to_csv] +``` + + +### Extend Admin Template + +- [django admin templates](https://github.com/django/django/tree/5.0/django/contrib/admin/templates/admin) + +```python +# orders/views.py +from django.shortcuts import redirect, render, get_object_or_404 +from django.contrib.admin.views.decorators import staff_member_required +from cart.cart import Cart +from .forms import OrderCreateForm +from .models import OrderItem, Order +from .tasks import order_created + +@staff_member_required +def admin_order_detail(request, order_id): + order = get_object_or_404(Order, id=order_id) + + return render(request, + 'admin/orders/order/detail.html', + {'order': order}) + +# orders/urls.py +path('admin/order//', views.admin_order_detail, name='admin_order_detail'), + +# orders/admin.py +from django.urls import reverse + + +def order_detail(obj): + url = reverse('orders:admin_order_detail', args=[obj.id]) + + return mark_safe(f'View') + + +@admin.register(Order) +class OrderAdmin(admin.ModelAdmin): + list_display = ['id', 'first_name', 'last_name', 'email', + 'address', 'postal_code', 'city', 'paid', + order_payment, 'created', 'updated', order_detail] +``` + + +```html + +{% extends "admin/base_site.html" %} + +{% block title %} + Order {{ order.id }} {{ block.super }} +{% endblock %} + +{% block breadcrumbs %} + +{% endblock %} + +{% block content %} +
+

Order {{ order.id }}

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Created{{ order.created }}
Customer{{ order.first_name }} {{ order.last_name }}
E-mail{{ order.email }}
Address + {{ order.address }}, + {{ order.postal_code }} {{ order.city }} +
Total amount${{ order.get_total_cost }}
Status{% if order.paid %}Paid{% else %}Pending payment{% endif %}
Stripe payment + {% if order.stripe_id %} + + {{ order.stripe_id }} + + {% endif %} +
+
+
+

Items bought

+ + + + + + + + + + + {% for item in order.items.all %} + + + + + + + {% endfor %} + + + + + +
ProductPriceQuantityTotal
{{ item.product.name }}${{ item.price }}{{ item.quantity }}${{ item.get_cost }}
Total${{ order.get_total_cost }}
+
+{% endblock %} + +``` + + +### Generating PDF Invoices + +- [Django Outputting PDF](https://docs.djangoproject.com/en/5.0/howto/outputting-pdf/) +- [WeasyPrint](https://doc.courtbouillon.org/weasyprint/stable/first_steps.html) + +```bash +python -m pip install WeasyPrint +mkdir -p orders/static/css # repeat for all static +python manage.py collectstatic +``` + +```python +# myshop/settings.py +STATIC_ROOT = BASE_DIR / 'static' + +# orders/urls.py +path('admin/order//pdf/', views.admin_order_pdf, name='admin_order_pdf'), + +# orders/views.py +from django.shortcuts import redirect, render, get_object_or_404 +from django.contrib.admin.views.decorators import staff_member_required +from django.template.loader import render_to_string +from django.contrib.staticfiles import finders +from django.http import HttpResponse +from cart.cart import Cart +from .forms import OrderCreateForm +from .models import OrderItem, Order +from .tasks import order_created +import weasyprint + + +@staff_member_required +def admin_order_pdf(request, order_id): + order = get_object_or_404(Order, id=order_id) + html = render_to_string('orders/order/pdf.html', {'order': order}) + response = HttpResponse(content_type='application/pdf') + response['Content-Disposition'] = f'filename=order_{order.id}.pdf' + weasyprint.HTML(string=html).write_pdf( + response, + stylesheets=[weasyprint.CSS(finders.find('css/pdf.css'))], + ) + return response + + +# orders/admin.py +def order_pdf(obj): + url = reverse('orders:admin_order_pdf', args=[obj.id]) + return mark_safe(f'PDF') + + +order_detail.short_description = 'Invoice' +``` + + +```html + + + +

My Shop

+

+ Invoice no. {{ order.id }}
+ + {{ order.created|date:"M d, Y" }} + +

+

Bill to

+

+ {{ order.first_name }} {{ order.last_name }}
+ {{ order.email }}
+ {{ order.address }}
+ {{ order.postal_code }}, {{ order.city }} +

+

Items bought

+ + + + + + + + + + + {% for item in order.items.all %} + + + + + + + {% endfor %} + + + + + +
ProductPriceQuantityCost
{{ item.product.name }}${{ item.price }}{{ item.quantity }}${{ item.get_cost }}
Total${{ order.get_total_cost }}
+ + + {% if order.paid %}Paid{% else %}Pending payment{% endif %} + + + + +### Sending Email of PDF + +```python +# payment/tasks.py +from io import BytesIO +import weasyprint +from celery import shared_task +from django.contrib.staticfiles import finders +from django.core.mail import EmailMessage +from django.template.loader import render_to_string +from orders.models import Order +import logging + +logger = logging.getLogger(__name__) + +@shared_task +def payment_completed(order_id): + """ + Task to send an e-mail notification when an order is successfully paid. + """ + logger.info(f"Processing payment completed for order {order_id}") + order = Order.objects.get(id=order_id) + subject = f'My Shop - Invoice no. {order.id}' + message = 'Please, find attached the invoice for your recent purchase.' + email = EmailMessage(subject, + message, + 'admin@myshop.com', + [order.email]) + + html = render_to_string('orders/order/pdf.html', {'order': order}) + out = BytesIO() + stylesheets = [weasyprint.CSS(finders.find('css/pdf.css'))] + weasyprint.HTML(string=html).write_pdf(out, + stylesheets=stylesheets) + + email.attach(f'order_{order.id}.pdf', + out.getvalue(), + 'application/pdf') + + email.send() + +# payment/webhooks.py +import stripe +from django.conf import settings +from django.http import HttpResponse +from django.views.decorators.csrf import csrf_exempt +from orders.models import Order +from .tasks import payment_completed + +@csrf_exempt +def stripe_webhook(request): + payload = request.body + sig_header = request.META['HTTP_STRIPE_SIGNATURE'] + event = None + try: + event = stripe.Webhook.construct_event( + payload, sig_header, settings.STRIPE_WEBHOOK_SECRET + ) + except ValueError as e: + return HttpResponse(status=400) + except stripe.error.SignatureVerificationError as e: + return HttpResponse(status=400) + + if event.type == 'checkout.session.completed': + session = event.data.object + + if session.mode == 'payment' and session.payment_status == 'paid': + try: + order = Order.objects.get(id=session.client_reference_id) + except Order.DoesNotExist: + return HttpResponse(status=404) + order.paid = True + order.stripe_id = session.payment_intent + order.save() + payment_completed.delay(order.id) + + return HttpResponse(status=200) +``` \ No newline at end of file diff --git a/src/markdown-pages/workshops/django/writing-your-first-django-app.md b/src/markdown-pages/workshops/django/writing-your-first-django-app.md new file mode 100644 index 000000000..fb14784cc --- /dev/null +++ b/src/markdown-pages/workshops/django/writing-your-first-django-app.md @@ -0,0 +1,195 @@ +--- +title: Writing your first Django app +date: 2025-03-23 +tags: + - tutorial + - django +--- + +## Install Django + +```bash +asdf set python 3.12.9 +asdf install +python -m venv .venv +. .venv/bin/activate +python -m pip install Django +``` + +## Create Project + +```bash +mkdir djangotutorial +django-admin startproject mysite djangotutorial +``` + +## Run + +```bash +python manage.py runserver +``` + +## Create App + +```bash +python manage.py startapp polls +``` + +## App Basics + +`views.py` - contains view functions +`urls.py` - contains url patterns +`models.py` - contains database models +`settings.py` - contains configuration + +## Database + +SQLITE out of the box. + +```bash +python manage.py migrate +``` + +Add your models to `models.py` + +## Create Migrations for App + +```bash +python manage.py makemigrations polls +# python manage.py sqlmigrate polls 0001 # readable verbose +python manage.py migrate +``` + +## Interactive Shell + +```bash +python manage.py shell +``` + +## Admin + +```bash +python manage.py createsuperuser +``` + +## Views + +Render + +```python +from django.http import HttpResponse +from django.template import loader + +from .models import Question + + +def index(request): + latest_question_list = Question.objects.order_by("-pub_date")[:5] + template = loader.get_template("polls/index.html") + context = { + "latest_question_list": latest_question_list, + } + return HttpResponse(template.render(context, request)) +``` + +Shortcut + +```python +from django.shortcuts import render + +from .models import Question + + +def index(request): + latest_question_list = Question.objects.order_by("-pub_date")[:5] + context = {"latest_question_list": latest_question_list} + return render(request, "polls/index.html", context) +``` + +Get or 404 + +```python +from django.http import Http404 +from django.shortcuts import render + +from .models import Question + + +# ... +def detail(request, question_id): + try: + question = Question.objects.get(pk=question_id) + except Question.DoesNotExist: + raise Http404("Question does not exist") + return render(request, "polls/detail.html", {"question": question}) +``` + +sortcut + +```python +from django.http import Http404 +from django.shortcuts import render + +from .models import Question + + +# ... +def detail(request, question_id): + try: + question = Question.objects.get(pk=question_id) + except Question.DoesNotExist: + raise Http404("Question does not exist") + return render(request, "polls/detail.html", {"question": question}) + +``` + +dynamic urls + +```html +
  • {{ question.question_text }}
  • +``` + +generic views + +```python +from django.db.models import F +from django.http import HttpResponseRedirect +from django.shortcuts import get_object_or_404, render +from django.urls import reverse +from django.views import generic + +from .models import Choice, Question + + +class IndexView(generic.ListView): + template_name = "polls/index.html" + context_object_name = "latest_question_list" + + def get_queryset(self): + """Return the last five published questions.""" + return Question.objects.order_by("-pub_date")[:5] + + +class DetailView(generic.DetailView): + model = Question + template_name = "polls/detail.html" + + +class ResultsView(generic.DetailView): + model = Question + template_name = "polls/results.html" + + +def vote(request, question_id): + # same as above, no changes needed. + # ... +``` + +## Tests + +`tests.py` + +```bash +python manage.py test polls +``` + diff --git a/src/markdown-pages/workshops/pluralsight/working-with-django-models.md b/src/markdown-pages/workshops/pluralsight/working-with-django-models.md new file mode 100644 index 000000000..6e14d45a3 --- /dev/null +++ b/src/markdown-pages/workshops/pluralsight/working-with-django-models.md @@ -0,0 +1,726 @@ +--- +title: Working with Django Models +date: 2025-06-13 +tags: + - course + - pluralsight + - python + - django +--- + +https://app.pluralsight.com/library/courses/django-models/table-of-contents + +https://github.com/codesensei-courses/django-models + +## Models + +- Python classes that are mapped to database tables +- Migrations are python scripts that keep db strucuture in sync with code, autogenerated most of the time +- Models are defined in `models.py` or `models/*.py` +- Models use Fields to define the types + +## Fields + +- Floating points have rounding issues: https://docs.python.org/3.10/tutorial/floatingpoint.html, use DecimalField instead + + +## Postgres + +```bash +python -m pip install psycopg2 +``` + +```python +# settings.py +DATABASES = { + 'default': { + 'ENGINE': 'django.db.backends.postgresql', + 'NAME': 'carved_rock', + } +} +``` + +## Logging + +- Logs all sql +- Bad practice in production + +```python +# settings.py +LOGGING = { + "version": 1, + "handlers": {"console": {"class": "logging.StreamHandler"}}, + "loggers": {"django.db.backends": {"level": "DEBUG"}}, + "root": {"handlers": ["console"]}, +} +``` + +## Basic ORM + +```python +>>> from store.models import Product +>>> p = Product(name="Hiking Boots", price=100, stock_count=50) +>>> p + +>>> p.id +>>> p.save() +(0.002) INSERT INTO "store_product" ("name", "stock_count", "price") VALUES ('Hiking Boots', 50, '100') RETURNING "store_product"."id"; args=('Hiking Boots', 50, Decimal('100')); alias=default +>>> p.id +1 +>>> p.price=80 +>>> p.save() +(0.002) UPDATE "store_product" SET "name" = 'Hiking Boots', "stock_count" = 50, "price" = '80' WHERE "store_product"."id" = 1; args=('Hiking Boots', 50, Decimal('80'), 1); alias=default +>>> p.delete() +(0.002) DELETE FROM "store_product" WHERE "store_product"."id" IN (1); args=(1,); alias=default +(1, {'store.Product': 1}) +``` + +## Databases + +- https://docs.djangoproject.com/en/3.1/ref/databases/ +- https://docs.djangoproject.com/en/3.1/howto/legacy-databases/ + +## Django Model Fields + +- https://docs.djangoproject.com/en/3.1/ref/models/fields/ +- Field class determines + - Database column type (e.g. VARCHAR, INTEGER, DATE) + - How the field is rendered in a form +- Field options + - DB Validation + - Form Rendereing and Validation + - Default value +- Types + - BooleanField + - IntegerField + - FloatField + - DecimalField + - CharField + - TextField + - EmailField + - URLField + - FilePathField + - SlugField + - GenericIPAddressField + - ForeignKey + - ManyToManyField + - OneToOneField + +Adding a new field will result in an error: + +```python +(venv) ➜ carved_rock python manage.py makemigrations +(0.002) + SELECT name, type FROM sqlite_master + WHERE type in ('table', 'view') AND NOT name='sqlite_sequence' + ORDER BY name; args=None; alias=default +(0.000) SELECT "django_migrations"."id", "django_migrations"."app", "django_migrations"."name", "django_migrations"."applied" FROM "django_migrations"; args=(); alias=default +It is impossible to add a non-nullable field 'description' to product without specifying a default. This is because the database needs something to populate existing rows. +Please select a fix: + 1) Provide a one-off default now (will be set on all existing rows with a null value for this column) + 2) Quit and manually define a default value in models.py. +``` + +Adding `description = models.TextField(default="")` will allow it to succeed. + +- `blank=True` allows the field to be empty in the form. Does not require a migration or affect the database. +- `null=True` allows the field to be empty in the database. Requires a migration. +- `unique=True` makes the field unique. Requires a migration. +- `db_index=True` adds an index to the field. Requires a migration. +- `db_column='my_column'` changes the column name in the database. Requires a migration. +- `auto_now=True` for date, use now +- `verbose_name="Bank Acount"` change the name of the form text. +- `help_text="Enter your bank account number"` change the help text for the form. + +```python +class ProductImage(models.Model): + image = models.ImageField() + product = models.ForeignKey('Product', on_delete=models.CASCADE) + + +class Category(models.Model): + name = models.CharField(max_length=100) + product = models.ManyToManyField('Product') +``` + +- For other objects a string is safer to avoid load error orders, for example in a foriegn key. +- One to One is similar to a foreign key with a unique constraint. + +Adding the `__str__` method changes the admin display. + +```python +def __str__(self): + return self.name +``` + +## Managers and QuerySets + +- Managers are used to query the database. +- QuerySets are lazy, the shell will run immediately due to the automatic print +- Because querysets are lazy, you can chaing them to build more complex queries. +- https://docs.djangoproject.com/en/3.1/ref/models/querysets/ + +```python +>>> from store.models import Product +>>> Product.objects.all() +(0.001) SELECT "store_product"."id", "store_product"."name", "store_product"."stock_count", "store_product"."price", "store_product"."description", "store_product"."sku" FROM "store_product" LIMIT 21; args=(); alias=default +, , , ]> +>>> Product.objects.get(pk=1) +(0.002) SELECT "store_product"."id", "store_product"."name", "store_product"."stock_count", "store_product"."price", "store_product"."description", "store_product"."sku" FROM "store_product" WHERE "store_product"."id" = 1 LIMIT 21; args=(1,); alias=default + +>>> Product.objects + +``` + +### Filter + +```python +def category_view(request, name): + products = Product.objects.filter(category__name=name) + + return render(request, "store/category.html", + {'products': products, + 'category_name': name}) +``` + +`category__name` is a lookup. + +```python +>>> Product.objects.filter(name="Kayak") +(0.004) SELECT "store_product"."id", "store_product"."name", "store_product"."stock_count", "store_product"."price", "store_product"."description", "store_product"."sku" FROM "store_product" WHERE "store_product"."name" = 'Kayak' LIMIT 21; args=('Kayak',); alias=default +]> + +>>> Product.objects.filter(name__endswith="k") +(0.001) SELECT "store_product"."id", "store_product"."name", "store_product"."stock_count", "store_product"."price", "store_product"."description", "store_product"."sku" FROM "store_product" WHERE "store_product"."name" LIKE '%k' ESCAPE '\' LIMIT 21; args=('%k',); alias=default +, ]>f + +>>> Product.objects.filter(name__contains="a", price__lt=100) +(0.001) SELECT "store_product"."id", "store_product"."name", "store_product"."stock_count", "store_product"."price", "store_product"."description", "store_product"."sku" FROM "store_product" WHERE ("store_product"."name" LIKE '%a%' ESCAPE '\' AND "store_product"."price" < '100') LIMIT 21; args=('%a%', Decimal('100')); alias=default +]> + +>>> p = Product.objects.filter(name__contains="a", price__lt=100) +>>> p.count() +(0.000) SELECT COUNT(*) AS "__count" FROM "store_product" WHERE ("store_product"."name" LIKE '%a%' ESCAPE '\' AND "store_product"."price" < '100'); args=('%a%', Decimal('100')); alias=default +1 +>>> p.exclude(stock_count__gt=5) +(0.000) SELECT "store_product"."id", "store_product"."name", "store_product"."stock_count", "store_product"."price", "store_product"."description", "store_product"."sku" FROM "store_product" WHERE ("store_product"."name" LIKE '%a%' ESCAPE '\' AND "store_product"."price" < '100' AND NOT ("store_product"."stock_count" > 5)) LIMIT 21; args=('%a%', Decimal('100'), 5); alias=default + + +>>> Product.objects.filter(category__name="Climbing gear") +(0.001) SELECT "store_product"."id", "store_product"."name", "store_product"."stock_count", "store_product"."price", "store_product"."description", "store_product"."sku" FROM "store_product" INNER JOIN "store_category_product" ON ("store_product"."id" = "store_category_product"."product_id") INNER JOIN "store_category" ON ("store_category_product"."category_id" = "store_category"."id") WHERE "store_category"."name" = 'Climbing gear' LIMIT 21; args=('Climbing gear',); alias=default + + +>>> Product.objects.filter(category__name__contains="gear") +(0.000) SELECT "store_product"."id", "store_product"."name", "store_product"."stock_count", "store_product"."price", "store_product"."description", "store_product"."sku" FROM "store_product" INNER JOIN "store_category_product" ON ("store_product"."id" = "store_category_product"."product_id") INNER JOIN "store_category" ON ("store_category_product"."category_id" = "store_category"."id") WHERE "store_category"."name" LIKE '%gear%' ESCAPE '\' LIMIT 21; args=('%gear%',); alias=default +, , ]> + + +``` + +### Relationships + +- Use `_id` insteawd of `` to avoid errors and check if the relationship exists. +- Saving relationships must be in the correct order. + + +```python +# relationshps for new items +>>> i = ProductImage(image="somefile.jpg") +>>> i.product +Traceback (most recent call last): + File "", line 1, in + File "/Users/kayla.altepeter/data/tutorials/django-models-m3-model-classes/carved_rock/venv/lib/python3.12/site-packages/django/db/models/fields/related_descriptors.py", line 271, in __get__ + raise self.RelatedObjectDoesNotExist( +store.models.ProductImage.product.RelatedObjectDoesNotExist: ProductImage has no product.. Did you mean: 'product_id'? +>>> i.product_id +>>> i.product_id is None +True + +# saving relationships +>>> from decimal import Decimal +>>> p = Product(name="example", stock_count=1, price=Decimal(100), sku=("example01")) +>>> i.product = p +>>> i.save() +Traceback (most recent call last): + File "", line 1, in + File "/Users/kayla.altepeter/data/tutorials/django-models-m3-model-classes/carved_rock/venv/lib/python3.12/site-packages/django/db/models/base.py", line 857, in save + self._prepare_related_fields_for_save(operation_name="save") + File "/Users/kayla.altepeter/data/tutorials/django-models-m3-model-classes/carved_rock/venv/lib/python3.12/site-packages/django/db/models/base.py", line 1242, in _prepare_related_fields_for_save + raise ValueError( +ValueError: save() prohibited to prevent data loss due to unsaved related object 'product'. + +>>> p.save() +(0.005) INSERT INTO "store_product" ("name", "stock_count", "price", "description", "sku") VALUES ('example', 1, '100', '', 'example01') RETURNING "store_product"."id"; args=('example', 1, Decimal('100'), '', 'example01'); alias=default +>>> i.save() +(0.002) INSERT INTO "store_productimage" ("image", "product_id") VALUES ('somefile.jpg', 5) RETURNING "store_productimage"."id"; args=('somefile.jpg', 5); alias=default + +``` + +### Aggregates + +```python +>>> from django.db.models import Avg, Count +>>> Product.objects.aggregate(Avg('price')) +(0.000) SELECT (CAST(AVG("store_product"."price") AS NUMERIC)) AS "price__avg" FROM "store_product"; args=(); alias=default +{'price__avg': Decimal('117.200000000000')} + +>>> Category.objects.annotate(Avg('products__price')) +(0.000) SELECT "store_category"."id", "store_category"."name", (CAST(AVG("store_product"."price") AS NUMERIC)) AS "products__price__avg" FROM "store_category" LEFT OUTER JOIN "store_category_products" ON ("store_category"."id" = "store_category_products"."category_id") LEFT OUTER JOIN "store_product" ON ("store_category_products"."product_id" = "store_product"."id") GROUP BY "store_category"."id", "store_category"."name" LIMIT 21; args=(); alias=default +, ]> + +>>> Category.objects.annotate(Avg('products__price')).values() +(0.000) SELECT "store_category"."id", "store_category"."name", (CAST(AVG("store_product"."price") AS NUMERIC)) AS "products__price__avg" FROM "store_category" LEFT OUTER JOIN "store_category_products" ON ("store_category"."id" = "store_category_products"."category_id") LEFT OUTER JOIN "store_product" ON ("store_category_products"."product_id" = "store_product"."id") GROUP BY "store_category"."id", "store_category"."name" LIMIT 21; args=(); alias=default + + +>>> Category.objects.annotate(avg_price=Avg('products__price')).values() +(0.002) SELECT "store_category"."id", "store_category"."name", (CAST(AVG("store_product"."price") AS NUMERIC)) AS "avg_price" FROM "store_category" LEFT OUTER JOIN "store_category_products" ON ("store_category"."id" = "store_category_products"."category_id") LEFT OUTER JOIN "store_product" ON ("store_category_products"."product_id" = "store_product"."id") GROUP BY "store_category"."id", "store_category"."name" LIMIT 21; args=(); alias=default + + +>>> Category.objects.annotate(avg_price=Avg('products__price')).values().order_by('avg_price') +(0.000) SELECT "store_category"."id", "store_category"."name", (CAST(AVG("store_product"."price") AS NUMERIC)) AS "avg_price" FROM "store_category" LEFT OUTER JOIN "store_category_products" ON ("store_category"."id" = "store_category_products"."category_id") LEFT OUTER JOIN "store_product" ON ("store_category_products"."product_id" = "store_product"."id") GROUP BY "store_category"."id", "store_category"."name" ORDER BY 3 ASC LIMIT 21; args=(); alias=default + + +>>> Product.objects.annotate(cat_count=Count('categories')).filter(cat_count__gt=1) +(0.000) SELECT "store_product"."id", "store_product"."name", "store_product"."stock_count", "store_product"."price", "store_product"."description", "store_product"."sku", COUNT("store_category_products"."category_id") AS "cat_count" FROM "store_product" LEFT OUTER JOIN "store_category_products" ON ("store_product"."id" = "store_category_products"."product_id") GROUP BY "store_product"."id", "store_product"."name", "store_product"."stock_count", "store_product"."price", "store_product"."description", "store_product"."sku" HAVING COUNT("store_category_products"."category_id") > 1 LIMIT 21; args=(1,); alias=default +]> + +``` + +### F expressions + +- reference to the value of a field +- Fast due to being executed in the database + +```python +# find all products with the name in the description +>>> from django.db.models import F +>>> Product.objects.filter(description__contains=F('name')) +(0.000) SELECT "store_product"."id", "store_product"."name", "store_product"."stock_count", "store_product"."price", "store_product"."description", "store_product"."sku" FROM "store_product" WHERE "store_product"."description" LIKE '%%' || REPLACE(REPLACE(REPLACE(("store_product"."name"), '\', '\\'), '%%', '\%%'), '_', '\_') || '%%' ESCAPE '\' LIMIT 21; args=(); alias=default + + +# update all prices for a category +>>> Category.objects.get(name='Climbing Gear') +(0.000) SELECT "store_category"."id", "store_category"."name" FROM "store_category" WHERE "store_category"."name" = 'Climbing Gear' LIMIT 21; args=('Climbing Gear',); alias=default + +>>> Category.objects.get(name='Climbing Gear').products.update(price=F('price')*.9) +(0.000) SELECT "store_category"."id", "store_category"."name" FROM "store_category" WHERE "store_category"."name" = 'Climbing Gear' LIMIT 21; args=('Climbing Gear',); alias=default +(0.007) UPDATE "store_product" SET "price" = ("store_product"."price" * 0.9) WHERE "store_product"."id" IN (SELECT U0."id" FROM "store_product" U0 INNER JOIN "store_category_products" U1 ON (U0."id" = U1."product_id") WHERE U1."category_id" = 1); args=(0.9, 1); alias=default +2 + +# add euros +>>> from django.db.models import F, DecimalField, ExpressionWrapper +>>> Product.objects.annotate(price_eur=ExpressionWrapper(F('price')*1.1, output_field=DecimalField())).values() +(0.000) SELECT "store_product"."id", "store_product"."name", "store_product"."stock_count", "store_product"."price", "store_product"."description", "store_product"."sku", (CAST(("store_product"."price" * 1.1) AS NUMERIC)) AS "price_eur" FROM "store_product" LIMIT 21; args=(1.1,); alias=default + +``` + +### Q expressions + +- `~` negates the expression +- `Q` builds expressions +- default quries are all 'AND' +- `|` is an or +- `&` is an and + +```python +>>> from django.db.models import Q +# in stock +>>> in_stock = Q(stock_count__gt=0) + +# no image +>>> no_img = Q(images=None) + +# find all in stock +>>> Product.objects.filter(in_stock) +(0.001) SELECT "store_product"."id", "store_product"."name", "store_product"."stock_count", "store_product"."price", "store_product"."description", "store_product"."sku" FROM "store_product" WHERE "store_product"."stock_count" > 0 LIMIT 21; args=(0,); alias=default +, , , , ]> + +# find all no image +>>> Product.objects.filter(no_img) +(0.000) SELECT "store_product"."id", "store_product"."name", "store_product"."stock_count", "store_product"."price", "store_product"."description", "store_product"."sku" FROM "store_product" LEFT OUTER JOIN "store_productimage" ON ("store_product"."id" = "store_productimage"."product_id") WHERE "store_productimage"."id" IS NULL LIMIT 21; args=(); alias=default +]> + +# find all out of stock +>>> Product.objects.filter(~in_stock) +(0.000) SELECT "store_product"."id", "store_product"."name", "store_product"."stock_count", "store_product"."price", "store_product"."description", "store_product"."sku" FROM "store_product" WHERE NOT ("store_product"."stock_count" > 0) LIMIT 21; args=(0,); alias=default + + +# out of stock or have no images +>>> Product.objects.filter(no_img|~in_stock) +(0.001) SELECT "store_product"."id", "store_product"."name", "store_product"."stock_count", "store_product"."price", "store_product"."description", "store_product"."sku" FROM "store_product" LEFT OUTER JOIN "store_productimage" ON ("store_product"."id" = "store_productimage"."product_id") WHERE ("store_productimage"."id" IS NULL OR NOT ("store_product"."stock_count" > 0)) LIMIT 21; args=(0,); alias=default +]> + +# find out of stock and no image +>>> Product.objects.filter(no_img&~in_stock) +(0.000) SELECT "store_product"."id", "store_product"."name", "store_product"."stock_count", "store_product"."price", "store_product"."description", "store_product"."sku" FROM "store_product" LEFT OUTER JOIN "store_productimage" ON ("store_product"."id" = "store_productimage"."product_id") WHERE ("store_productimage"."id" IS NULL AND NOT ("store_product"."stock_count" > 0)) LIMIT 21; args=(0,); alias=default + + +# store in var +no_img_or_no_stock = no_img|~in_stock +``` + +### Examples + +```python +>>> Product.objects.all()[:5] +(0.000) SELECT "store_product"."id", "store_product"."name", "store_product"."stock_count", "store_product"."price", "store_product"."description", "store_product"."sku" FROM "store_product" LIMIT 5; args=(); alias=default +, , , ]> + +>>> Product.objects.reverse()[0] +(0.000) SELECT "store_product"."id", "store_product"."name", "store_product"."stock_count", "store_product"."price", "store_product"."description", "store_product"."sku" FROM + +>>> Product.objects.reverse().first() +(0.000) SELECT "store_product"."id", "store_product"."name", "store_product"."stock_count", "store_product"."price", "store_product"."description", "store_product"."sku" FROM "store_product" ORDER BY "store_product"."id" DESC LIMIT 1; args=(); alias=default + + +>>> Product.objects.order_by("price") +(0.000) SELECT "store_product"."id", "store_product"."name", "store_product"."stock_count", "store_product"."price", "store_product"."description", "store_product"."sku" FROM "store_product" ORDER BY "store_product"."price" ASC LIMIT 21; args=(); alias=default +, , , ]> + +>>> Product.objects.order_by("price").values() +(0.000) SELECT "store_product"."id", "store_product"."name", "store_product"."stock_count", "store_product"."price", "store_product"."description", "store_product"."sku" FROM "store_product" ORDER BY "store_product"."price" ASC LIMIT 21; args=(); alias=default + + +>>> Product.objects.order_by("price").values_list() +(0.000) SELECT "store_product"."id", "store_product"."name", "store_product"."stock_count", "store_product"."price", "store_product"."description", "store_product"."sku" FROM "store_product" ORDER BY "store_product"."price" ASC LIMIT 21; args=(); alias=default + + +>>> Product.objects.filter(category__name__contains="gear").distinct() +(0.001) SELECT DISTINCT "store_product"."id", "store_product"."name", "store_product"."stock_count", "store_product"."price", "store_product"."description", "store_product"."sku" FROM "store_product" INNER JOIN "store_category_product" ON ("store_product"."id" = "store_category_product"."product_id") INNER JOIN "store_category" ON ("store_category_product"."category_id" = "store_category"."id") WHERE "store_category"."name" LIKE '%gear%' ESCAPE '\' LIMIT 21; args=('%gear%',); alias=default +, ]> + +>>> Product.objects.get(pk=5) +(0.000) SELECT "store_product"."id", "store_product"."name", "store_product"."stock_count", "store_product"."price", "store_product"."description", "store_product"."sku" FROM "store_product" WHERE "store_product"."id" = 5 LIMIT 21; args=(5,); alias=default +Traceback (most recent call last): + File "", line 1, in + File "/Users/kayla.altepeter/data/tutorials/django-models-m3-model-classes/carved_rock/venv/lib/python3.12/site-packages/django/db/models/manager.py", line 87, in manager_method + return getattr(self.get_queryset(), name)(*args, **kwargs) + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + File "/Users/kayla.altepeter/data/tutorials/django-models-m3-model-classes/carved_rock/venv/lib/python3.12/site-packages/django/db/models/query.py", line 633, in get + raise self.model.DoesNotExist( +store.models.Product.DoesNotExist: Product matching query does not exist. +>>> Product.objects.get(pk=2) +(0.000) SELECT "store_product"."id", "store_product"."name", "store_product"."stock_count", "store_product"."price", "store_product"."description", "store_product"."sku" FROM "store_product" WHERE "store_product"."id" = 2 LIMIT 21; args=(2,); alias=default + + +>>> from store.models import Category +>>> c = Category.objects.get(pk=1) +(0.001) SELECT "store_category"."id", "store_category"."name" FROM "store_category" WHERE "store_category"."id" = 1 LIMIT 21; args=(1,); alias=default + +c.product +.ManyRelatedManager object at 0x103914710> + +>>> c.product.all() +(0.001) SELECT "store_product"."id", "store_product"."name", "store_product"."stock_count", "store_product"."price", "store_product"."description", "store_product"."sku" FROM "store_product" INNER JOIN "store_category_product" ON ("store_product"."id" = "store_category_product"."product_id") WHERE "store_category_product"."category_id" = 1 LIMIT 21; args=(1,); alias=default +, ]> + +>>> c.product.filter(price__lt=10) +(0.001) SELECT "store_product"."id", "store_product"."name", "store_product"."stock_count", "store_product"."price", "store_product"."description", "store_product"."sku" FROM "store_product" INNER JOIN "store_category_product" ON ("store_product"."id" = "store_category_product"."product_id") WHERE ("store_category_product"."category_id" = 1 AND "store_product"."price" < '10') LIMIT 21; args=(1, Decimal('10')); alias=default + + +>>> p = Product.objects.get(name="Helmet") +(0.000) SELECT "store_product"."id", "store_product"."name", "store_product"."stock_count", "store_product"."price", "store_product"."description", "store_product"."sku" FROM "store_product" WHERE "store_product"."name" = 'Helmet' LIMIT 21; args=('Helmet',); alias=default +>>> p + +>>> p.category_set +.ManyRelatedManager object at 0x1037a3fb0> +>>> p.category_set.all() +(0.002) SELECT "store_category"."id", "store_category"."name" FROM "store_category" INNER JOIN "store_category_product" ON ("store_category"."id" = "store_category_product"."category_id") WHERE "store_category_product"."product_id" = 3 LIMIT 21; args=(3,); alias=default +, ]> + +>>> i = ProductImage.objects.get(pk=1) +(0.001) SELECT "store_productimage"."id", "store_productimage"."image", "store_productimage"."product_id" FROM "store_productimage" WHERE "store_productimage"."id" = 1 LIMIT 21; args=(1,); alias=default +>>> i.product +(0.000) SELECT "store_product"."id", "store_product"."name", "store_product"."stock_count", "store_product"."price", "store_product"."description", "store_product"."sku" FROM "store_product" WHERE "store_product"."id" = 2 LIMIT 21; args=(2,); alias=default + +>>> i.product_id +2 + +``` + +## Customizing Model Behavior + +- https://docs.djangoproject.com/en/3.1/ref/models/options/ +- Add Meta class to Django models + +### Sorting + +- Can hurt performance +- Sorts always +- `-` is descending order + +```python +class Meta: + ordering = ['price'] + # ordering = ['price', 'name'] # sort by price then name + # ordering = ['-price'] # descending order +``` + +### Renaming + +- rename the plural + +```python +class Meta: + verbose_name_plural = "Categories" + ordering = ['name'] +``` + +### Constraints + +- require non negative price +- requires migration + +```python +class Meta: + constraints = [ + models.CheckConstraint( + check=models.Q(price__gte=0), + name='price_not_negative' + ) + ] +``` + +### Custom Methods + +- add a method to be used in views/etc. +- Look at a pattern called Fat Models, Skinny Views +- Don't create god objects (do everything) +- Overriding save is quite common + +```python +@property +def vat(self): + return Decimal(0.2) * self.price + +def get_absolute_url(self): + return reverse('store:product-detail', kwargs={'pk': self.id}) + +# add a slug field and create if not passed +slug = models.SlugField() + +def save(self, *args, **kwargs): + if not self.slug: + self.slug = slugify(self.name) + return super().save(*args, **kwargs) +``` + +### Custom Managers + +- custom managers are now the default query +- use an alternate name to avoid issues +- https://docs.djangoproject.com/en/5.2/topics/db/managers/ +- + +```python +class ProductIsInStockQuerySet(models.QuerySet): + def in_stock(self): + return self.filter(stock_count__gt=0) + + +class Product(models.Model): + name = models.CharField(max_length=100) + stock_count = models.IntegerField(help_text="How many items are currently in stock?") + price = models.DecimalField(max_digits=6, decimal_places=2) + description = models.TextField(default="", blank=True) + sku = models.CharField(max_length=20, unique=True, verbose_name="Stock Keeping Unit") + slug = models.SlugField() + + # The instock is now the default manager, all queries will use this. This can make it difficult if you need to query out of stock. + # objects = ProductIsInStockQuerySet.as_manager() + # different name for manager + objects = models.Manager() + in_stock = ProductIsInStockQuerySet.as_manager() + + def save(self, *args, **kwargs): + if not self.slug: + self.slug = slugify(self.name) + return super().save(*args, **kwargs) + + class Meta: + ordering = ['price'] + constraints = [ + models.CheckConstraint( + check=models.Q(price__gte=0), + name='price_not_negative' + ) + ] + +# query +Product.in_stock.all() +``` + +### Inheritance + +- https://docs.djangoproject.com/en/5.2/topics/db/models/#model-inheritance +- Three styles available: Abstract, Multi-table, Proxy + + +#### Abstract + +- [Classes for Reuse](https://github.com/jazzband/django-model-utils/tree/master/model_utils) + + +#### Multi-table + +- Avoid model inheritance if possible + +```python +class DigitalProduct(models.Model): + file = models.FileField() + + +class PhysicalProduct(models.Model): + stock_count = models.IntegerField(help_text="How many items are currently in stock?") +``` + +#### Proxy + +- Doesn't change the model +- Add of modify behavior +- Doesn't create a table + +```python +from django.db import models + + +class Person(models.Model): + first_name = models.CharField(max_length=30) + last_name = models.CharField(max_length=30) + + +class MyPerson(Person): + class Meta: + proxy = True + + def do_something(self): + # ... + pass +``` + +## Migrations + +- `showmigrations` will show all, add app to filter +- `sqlmigrate store 002` show the sql for a mgiration +- `makemigrations` will create a migration, `--dry-run` won't run +- `migrate` will apply the migrations, `--fake` will make migrations but not run them +- Good idea to name migrations, renaming would have to be done for dependencies in other migrations, do this immediately to avoid issues +- State is stored in the migrations database +- `makemigrations --merge` will resolve conflicts with migrations with the same number, i.e. two people adding at the same time. Only works for different changes. +- `squashmigrations store 0008` will squash migrations together +- https://docs.djangoproject.com/en/3.1/topics/migrations/ + +```bash +python manage.py showmigrations store +python manage.py migrate store 0002 # go to previous state + +# dependencies +dependencies = [ + ('store', '0001_initial'), +] +``` + +### Squashing + +- Run the squash, test. +- Get to all environments +- Delete old files + +### Custom Migrations + +```bash +python manage.py makemigrations --empty store +``` + +```python +# Generated by Django 5.2.3 on 2025-06-17 02:24 + +from django.db import migrations +from django.utils.text import slugify + + +def slugify_product_title(apps, schema_editor): + Product = apps.get_model('store', 'Product') + for p in Product.objects.filter(slug=""): + p.slug = slugify(p.name) + p.save() + + +def undo_slugify(apps, schema_editor): + pass + + +class Migration(migrations.Migration): + + dependencies = [ + ('store', '0009_product_slug'), + ] + + operations = [ + migrations.RunPython(slugify_product_title, reverse_code=undo_slugify) + ] + + +``` + +### Fixtures + +- `python manage.py dumpdata` will dump the data +- `python manage.py dumpdata -o fixtures/my_data.json` will dump the data +- Not super helpful in the beginning due to a lot of changes. +- https://factoryboy.readthedocs.io/en/stable/, generate data rather than fixtures + +## Optimizing Models + +- Avoid pre-mature optimization +- Use https://django-debug-toolbar.readthedocs.io/en/latest/ to understand queries + +Don't let python do the work. + +```python +# slow, uses a lot of memory +from store.models import Product + +products_in_stock = [] +for p in Product.objects.all(): + if p.stock_count > 0: + products_in_stock.append(p) + +# better, uses less memory +from django.db.models import F +from store.models import Product + +products_in_stock = Product.objects.filter(stock_count__gt=0) +``` + +### QuerySets + +- Querysets are lazy +- They are cached, looping over will not query. +- Slicing results in a new queryset everytime +- Relationships are lazy +- Function calls are not not cached by default, use `@cached_property` from django +- Properties are always cached + +### Avoding Queries + +- Using `select_related` will use an inner join + - `img = ProductImage.objects.select_related('product').get(pk=1)` + - Only single relations are supported +- Using `prefetch_related` will fetch many relationships data + - `products = Product.objects.filter(name__icontains=name).prefetch_related('images')` + - + +### Raw SQL + +- Powerful +- Paramaterized queries using `%s` + - `products = Product.objects.raw('SELECT * FROM store_product WHERE price < %s', [100])` + +### Transactions + +- By default Django runs in auto commit mode, if anything goes wrong the next operation doesn't run + - `'ATOMIC_REQUESTS': True` will run in a transaction, set in the DATABASE settings +- Will use the views to ensure atomic transactions +- For small to medium size it works, slight performance hit +- https://docs.djangoproject.com/en/5.2/topics/db/transactions/ \ No newline at end of file