Skip to content

Introduction to FilterSchema#

2024 iThome Ironman Contest

This is the 26th article in the Django Ninja tutorial series.

Querying is a common API requirement and is essentially a matter of filtering data.

Whether filtering posts, products, or querying users, filtering data based on different conditions and getting results can be said to be an essential feature for most projects.

In a view function, the simplest way to implement querying is to use Django ORM's filtering methods. For example, we can use the filter method to screen a QuerySet based on specific conditions.

This approach is simple and direct, suitable for basic query needs. However, it also has its limitations—as fields and requirements increase, query conditions can become increasingly complex, leading to verbose and hard-to-maintain code.

To solve this problem, Django Ninja provides FilterSchema, allowing us to define and manage query conditions in a more "structured" way.

This article will introduce FilterSchema, implementing and explaining it step-by-step to show you how to use FilterSchema in Django Ninja to achieve a more flexible and modular API query feature.

For all code changes in this article, please refer to this PR.

GitHub Example Project#

👉 Django-Ninja-Tutorial

Traditional Query Methods and Their Problems#

The previous article mentioned the "Get Post List" API. Do you remember the filter by post title feature added in Article 11: Request (Part 3) Query Parameters?

Here is the current state of the code: (Please note the query parameter name title)

...
def get_posts(
    request: HttpRequest,
    title: None | str = Query(None, min_length=2, max_length=10),
) -> QuerySet[Post]:
    """
    Get post list
    """
    posts = Post.objects.all()
    if title:
        posts = posts.filter(
            title__icontains=title).select_related('author')
    return posts

At that time, we used Django ORM's filter method!

posts.filter(title__icontains=title).select_related('author')

The final select_related('author') is to avoid the "N+1" problem and is unrelated to the query logic, so you can ignore it for now.

Potential Problems#

This approach is very intuitive and effective for simple query needs.

However, as the project scale grows and query requirements become complex, the following dilemmas arise:

  • Code Duplication: When you need to perform similar filtering in multiple places, you may find yourself repeating the same filtering logic.
  • Maintenance Difficulty: As filtering conditions increase, your view function can become difficult to maintain. Every time you add a new filtering condition, you may need to modify the code in multiple places.
  • Data Validation and Conversion: You need to manually handle data validation and conversion, which not only increases the likelihood of errors but also increases development complexity.
  • Extensibility: When you need to support more complex filtering conditions, such as range queries and multi-condition combined queries, the method of manually assembling ORM queries can seem very clunky and difficult to manage.

Therefore, we do not recommend directly using ORM's filter method to assemble query conditions in complex query scenarios.

New Requirement: Querying Author Names Simultaneously#

The new requirement is to use the same keyword to simultaneously query the post title or the author name, and display the result if either matches. (Meeting either is sufficient, or both can match)

This requirement is similar to the search feature on the official iThome Ironman Contest website:

However, we can only query 2 types, while it can query 3 types simultaneously: title, description, and contestant nickname.

But the essence is the same.

Implementing With the Traditional Method + Q Object#

If implemented using the traditional method + Django's Q object, the query would look like this:

from django.db.models import Q
...

posts = posts.filter(
    Q(title__icontains=title) | Q(author__name__icontains=title)
).select_related('author')

At this time, it is no longer appropriate for the query parameter to be called title, because it queries two fields. That's fine; we will rename it to query later.

There are two key points in this code:

  1. The query has indeed become longer! If we have new query conditions in the future, then won't it...
  2. What is this Q thing?

Q is Django ORM's Q object, which plays an important role in complex query logic. Therefore, it is necessary to briefly introduce it first.

Introduction to Django Q Object#

To improve the complex structure of the program during multi-condition queries, Django provides the Q object.

The Q object allows us to flexibly organize query conditions and combine them using logical operators (such as &, |). It is extremely useful when dealing with complex condition filtering.

For example, if we want to filter out posts whose titles contain "Ninja" and whose author names contain "Alice", we can write it like this:

posts = posts.filter(
    Q(title__icontains='Ninja') & Q(author__name__icontains='Alice')
)

The above expression is actually equivalent to our common:

posts = posts.filter(
    title__icontains='Ninja', author__name__icontains='Alice'
)

So you usually won't use the Q object for "AND" requirements.

"OR" query conditions are the classic scenario for Q.

Now, changing the condition to—either the post title "or" the author's name contains "Alice". You can use |:

posts = posts.filter(
    Q(title__icontains='Ninja') | Q(author__name__icontains='Alice')
)

The Q object makes queries more flexible and clearer, especially when facing multiple optional conditions.

Improving Queries With FilterSchema#

After understanding that traditional query methods can easily cause verbose code and learning the basics of the Q object, we will introduce today's main character—FilterSchema.

The main feature of FilterSchema provided by Django Ninja is to make query statements more structured and modular, avoiding view functions from becoming verbose and hard to read.

Moreover, similar to the validation methods in Schema, it also realizes the principle of "separation of concerns" to some extent—by extracting query logic from view functions.

However, let's not rush to get there in one step. Allow me to improve the code in stages.

Although this is a bit clumsy, it will give you a deeper understanding of the implementation of FilterSchema and complex queries.

Version 1 "Improvement"#

We first use FilterSchema to implement the aforementioned "New Requirement: Querying Author Names Simultaneously".

Create a new Schema in schemas.py, but this time it is a FilterSchema:

# post/schemas.py
from ninja import Field, FilterSchema, Schema
...

class PostFilterSchema(FilterSchema):
    query: str | None = Field(None, min_length=2, max_length=10)

This FilterSchema is actually used for "query parameters". Therefore, its field (attribute) name is the query parameter name you think the client should use.

Since we want to query both "post title" and "author name" simultaneously, I named it query.

Next, we use it in the view function:

@router.get(...)
@paginate(CustomPagination)
def get_posts(
    request: HttpRequest,
    filters: PostFilterSchema = Query(),  # Use FilterSchema
) -> QuerySet[Post]:
    """
    Get post list
    """
    posts = Post.objects.all()
    if filters.query:
        q = Q(title__icontains=filters.query) | \
            Q(author__username__icontains=filters.query)
        posts = posts.filter(q)
    return posts

PS: The example code in the project is incorrect here, where the second Q query was mistakenly written as "content__icontains". Please be aware of this. I have fixed it in the next branch.

After looking at this new view function, you might not help but think:

Is this a joke? It hasn't become simpler at all!

Exactly, because this is only a "semi-finished product" of FilterSchema, it looks more verbose than not using it.

Nevertheless, there are still some highlights worth noting.

Key Points Analysis#

q = Q(title__icontains=filters.query) | \
    Q(content__icontains=filters.query)
posts = posts.filter(q)

It can be seen from this snippet that Q objects can perform various combination operations independently, and finally be passed to Django's filter method as parameters.

Common Sentence Patterns in Django Ninja#

In this example, this "sentence pattern" of view function parameters is very common in Django Ninja:

filters: PostFilterSchema = Query()

And beginners might easily "misunderstand" it.

Why? Because you might think that the type of filters is PostFilterSchema (which is correct), and then its default value is Query(), because that is how Python functions are defined.

But it is not.

Query() is not the default value of the filters parameter, otherwise, shouldn't its type be Query?

In fact, the assignment = Query() is not for you to read, but for Django Ninja. It is equivalent to telling Django Ninja:

This parameter content should be obtained from HTTP request query parameters, rather than from the body or path.

Thinking about it this way makes it very easy to understand.

Django Ninja will attempt to obtain strings from query parameters, decompose them (if there are multiple query strings), and then pass them one by one to PostFilterSchema for initialization and validation:

  1. Validation fails: Returns 422.
  2. Validation succeeds: Passes the Schema object into the view function as a function parameter (local variable).

Query Results#

Here are the query results using "Alice" as the keyword:

30 posts were found, all from users whose author names contain "Alice".

Summary and Next Steps#

That's all for this article. We have touched upon two new concepts—the Q object and FilterSchema.

We also analyzed the common view function parameter "sentence pattern" in Django Ninja, which is very important for understanding how to use the framework and its conventions.

These concepts take time to digest, but I can assure you that this preparation is worth it.

Rather than diving straight into the advanced usage of FilterSchema, this step-by-step learning approach is more helpful for understanding.

In the next article, you will see why understanding the Q object is important, and how to use FilterSchema to build structured, "separation of concerns" multi-field queries.

See you in the next article.