Five Advanced Django Tips

Introduction

Many of the “Django tips” articles that I see online are geared towards beginners not intermediate or advanced Django developers. I hope to demonstrate some of Django’s depth, specifically around the ORM, and you’ll need to have an intermediate understanding of Django. Let’s start by looking at the example models.

from django.db import models


class Ticker(models.Model):
    symbol = models.CharField(max_length=50, unique=True)


class TickerPrice(models.Model):
    ticker = models.ForeignKey(
        Ticker, on_delete=models.CASCADE, related_name="ticker_prices"
    )
    price = models.DecimalField(max_digits=7, decimal_places=2)
    close_date = models.DateField()

For this article, we’ll use a stock price tracking application as our example. We have a Ticker model to store each stock ticker with its symbol, and a TickerPrice model with a many to one relationship to a Ticker where we’ll store the ticker’s price and close date.

Using Q Objects for Complex Queries

When you filter a queryset, you’re ANDing the keyword arguments together. Q objects allow Django developers to perform lookups with OR. Q objects can be combined together with &, representing AND, or |, representing OR. Let’s look at an example query.

today_and_yesterday_prices = TickerPrice.objects.filter(
    models.Q(close_date=today) | models.Q(close_date=yesterday)
)

In this query, we’re fetching the ticker prices with close dates of today or yesterday. We wrap our close_date keyword arguments with a Q object and join them together with the OR operator, |. We can also combine the OR and AND operators.

today_and_yesterday_greater_than_1000 = TickerPrice.objects.filter(
    models.Q(price__gt=1000),
    (models.Q(close_date=today) | models.Q(close_date=yesterday)),
)

For this query, we’re getting all prices with a close date of today or yesterday with a price greater than 1000. By default, Q objects, similar to keyword arguments, are ANDed together. We can also use the ~ operator to negate Q objects.

today_and_yesterday_greater_than_1000_without_BRK = (
    TickerPrice.objects.filter(
        models.Q(price__gt=1000),
        ~models.Q(ticker__symbol__startswith="BRK"),
        (models.Q(close_date=today) | models.Q(close_date=yesterday)),
    )
)

In this query, we’re fetching all ticker prices greater than 1000 that don’t start with BRK with close dates of either today or yesterday. We added the condition that the ticker’s symbol does not start with BRK (Berkshire Hathaway), which will exclude those from the query.

Optimize Database Calls with Prefetch Related and Select Related

Prefetch related and select related provide us with a mechanism to look up objects related to the model that we’re querying. We use prefetch related when we want to fetch a reverse foreign key or a many to many relationship. We use select related when we want to fetch a foreign key or a one to one relationship.

apple_with_all_prices = Ticker.objects.prefetch_related(
    "ticker_prices"
).get(symbol="AAPL")

In this example, we’re fetching a single ticker, AAPL, and with this ticker, we’re fetching all of the related prices. This helps us optimize our database queries by loading all the related ticker prices instead of fetching them one by one. Without prefetch related, if we looped over ticker_prices.all(), each iteration would result in a database query, but with prefetch related, a loop would result in one database query.

latest_prices = TickerPrice.objects.filter(
    close_date=today
).select_related("ticker")

Select related works similarly to prefetch related except we use select related for different types of relationships. For this case, we’re fetching all ticker prices for today and also fetching the associated ticker. Once again if we loop over latest_prices, referencing a price’s ticker won’t result in an extra database query.

Annotate Querysets to Fetch Specific Values

Annotating a queryset enables us to add attributes to each object in the queryset. Annotations can be a reference to a value on the model or related model or an expression such as a sum or count.

tickers_with_latest_price = Ticker.objects.annotate(
    latest_price=TickerPrice.objects.filter(
        ticker=models.OuterRef("pk")
    )
    .order_by("-close_date")
    .values("price")[:1]
)

This queryset fetches all the tickers and annotates each ticker object with a latest_price attribute. The latest price comes from the most recent related ticker price. The OuterRef allows us to reference the primary key of the ticker object. We use order_by to get the most recent price and use values to select only the price. Finally, the [:1] ensures we retrieve only one TickerPrice object.

We could also query against our annotation.

tickers_with_latest_price = (
    Ticker.objects.annotate(
        latest_price=TickerPrice.objects.filter(ticker=models.OuterRef("pk"))
        .order_by("-close_date")
        .values("price")[:1]
    )
    .filter(latest_price__gte=50)
)

We added an extra filter statement after our annotation. In this query, we fetch all tickers where the latest price is greater than or equal to fifty.

Use Prefetch Objects to Control Your Prefetch Related

Prefetch objects enable Django developers to control the operation of prefetch related. When we pass in a string argument to prefetch related, we’re saying fetch all of the related objects. A prefetch object lets us pass in a custom queryset to fetch a subset of the related objects.

tickers_with_prefetch = Ticker.objects.all().prefetch_related(
    models.Prefetch(
        "ticker_prices",
        queryset=TickerPrice.objects.filter(
            models.Q(close_date=today)
            | models.Q(close_date=yesterday)
        ),
    )
)

In this example, we combine a previous query we made for ticker prices from today or yesterday and pass that as the query set of our prefetch object. We fetch all tickers and with them we fetch all related ticker prices from today and yesterday.

Define Custom Query Sets and Model Managers for Code Reuse

Custom model managers and custom querysets let Django developers add extra methods to or modify the initial queryset for a model. Using these promotes the “don’t repeat yourself” (DRY) principle in software development and promotes reuse of common queries.

import datetime

from django.db import models


class TickerQuerySet(models.QuerySet):
    def annotate_latest_price(self):
        return self.annotate(
            latest_price=TickerPrice.objects.filter(
                ticker=models.OuterRef("pk")
            )
            .order_by("-close_date")
            .values("price")[:1]
        )

    def prefetch_related_yesterday_and_today_prices(self):
        today = datetime.datetime.today()
        yesterday = today - datetime.timedelta(days=1)
        return self.prefetch_related(
            models.Prefetch(
                "ticker_prices",
                queryset=TickerPrice.objects.filter(
                    models.Q(close_date=today)
                    | models.Q(close_date=yesterday)
                ),
            )
        )


class TickerManager(models.Manager):
    def get_queryset(self):
        return TickerQuerySet(self.model, using=self._db)


class Ticker(models.Model):
    symbol = models.CharField(max_length=50, unique=True)

    objects = TickerManager()


class TickerPrice(models.Model):
    ticker = models.ForeignKey(
        Ticker, on_delete=models.CASCADE, related_name="ticker_prices"
    )
    price = models.DecimalField(max_digits=7, decimal_places=2)
    close_date = models.DateField()

In the above code, we’ve created a custom queryset with some of the previously demonstrated queries as methods. We added this new queryset to our custom manager and overrode the default objects manager on the Ticker model with our custom manager. With the custom manager and queryset, we can do the following.

tickers_with_prefetch = (
    Ticker.objects.all().prefetch_related_yesterday_and_today_prices()
)

tickers_with_latest_price = Ticker.objects.all().annotate_latest_price()

Instead of having to write the actual query for each of these examples, we call the methods defined in the custom queryset. This is especially useful if we use these queries in multiple places throughout the codebase.

Final Thoughts

I hope these Django tips shed some light on the more advanced Django features. The Django ORM has a large set of features that can be overwhelming at the beginning, but once you’re past the beginner level, the ORM contains a lot of great functionality that helps maintain a clean codebase and enable complex queries. I encourage you to dive into Django’s documentation, especially around the ORM, as it is well written with good examples.