Skip to content

Custom Pagination#

2024 iThome Ironman Contest

This is the 25th article in the Django Ninja tutorial series.

In the previous article, we introduced Django Ninja's built-in paginators and implemented a simple pagination feature with them.

Although the built-in PageNumberPagination is indeed convenient, in many cases, we still need some customization.

To achieve this, you need to customize a pagination class.

But don't worry, this customization is not built from scratch. Instead, it inherits from the base pagination class provided by Django Ninja and then undergoes our own "processing".

This article will teach you how to do just that.

For all code changes in this article, please refer to this PR.

GitHub Example Project#

👉 Django-Ninja-Tutorial

Custom Requirements#

In addition to basic pagination, we also want to:

  • Allow the client to choose the number of data items displayed per page, with the selectable range restricted to between 1 and 100.
  • Add two new fields in the response to display the current pagination information:
    • Current page (page).
    • Quantity per page (per_page).

This is undoubtedly a very common requirement. We will implement these features by customizing a pagination class.

Without further ado, let's get started!

Implementation: Custom Pagination Class#

Paginators (pagination classes) are typically used project-wide, so they are not suitable to be placed in a Django app directory. However, unlike exception handlers, they cannot be placed in the project's api.py either, as this would cause a circular reference.

Therefore, I created a new Python module, pagination.py, in the project directory NinjaForum.

In this new module, write a pagination class named CustomPagination as follows:

from typing import Any

from django.db.models.query import QuerySet
from ninja import Field, Schema
from ninja.pagination import PaginationBase

class CustomPagination(PaginationBase):
    class Input(Schema):
        page: int = Field(1, ge=1)
        per_page: int = Field(10, ge=1, le=100)

    class Output(Schema):
        items: list
        page: int = Field(examples=[1])
        per_page: int = Field(examples=[10])
        total: int = Field(examples=[100])

    def paginate_queryset(
        self,
        queryset: QuerySet,
        pagination: Input,
        **params: Any,
    ) -> dict[str, Any]:
        start = (pagination.page - 1) * pagination.per_page
        end = start + pagination.per_page
        return {
            'items': queryset[start:end],
            'page': pagination.page,
            'per_page': pagination.per_page,
            'total': queryset.count(),
        }

This pagination class allows us to determine the size and page number of the pagination through query parameters—page and per_page—and adds two new fields of the same name to the response as additional pagination information.

Explanation of Custom Pagination Class#

Although the code seems to have many details, after reading it carefully, you will find it is actually not difficult to understand.

Due to space constraints, we will only highlight a few key points.

Key Point 1: The Overall Structure Comes From the Inherited Class - PaginationBase#

The first question might be: "How would I know how to write a pagination class like this?"

Exactly, we don't know naturally, so we need to look at the official documentation and the source code.

From the official documentation, we learn that we need to inherit from a class called PaginationBase. However, the documentation's description of this class is still a bit brief, so we need to read the source code to get more concrete details.

Then we mock and override some attributes and methods in the class—it is basically just like that.

Key Point 2: Input Schema#

class Input(Schema):
    page: int = Field(1, ge=1)
    per_page: int = Field(10, ge=1, le=100)
You can definitely see that this Schema is used to define and validate URL query parameters related to pagination.

In addition, the Input class will be passed as an argument to the paginate_queryset method as part of implementing the pagination logic.

Each attribute in Input represents a query parameter (limited to pagination-related)—and you can similarly use Field to configure the details!

The Field here is Pydantic's Field, which we introduced in detail in the 18th article. It allows us to set default values, documentation examples, and basic validation rules for each parameter.

In this example, the default value of page is 1, and it must be greater than or equal to 1; the default value of per_page is 10, and it must be between 1 and 100. This ensures that our pagination parameters are always within a reasonable range.

The same logic applies to Output, which determines the format that the HTTP response "should have", equivalent to the Schema of the pagination response.

Key Point 3: The paginate_queryset Method#

This method is the core of all pagination classes, implementing the specific pagination logic.

Its first parameter is self, which shows that it is an "instance method".

The most noteworthy is the second parameter—queryset, which is actually the return value of the view function, and its type must be QuerySet.

paginate_queryset uses the familiar "slicing and indexing" to "slice" the incoming QuerySet. This is a feature Django implements for QuerySet, behaving similarly to Python containers like list and tuple.

When it responds to the client, we get the sliced QuerySet and the custom response format.

Testing Custom Pagination#

After writing the custom class above, the view function only needs one more line: @paginate(CustomPagination). The code is omitted here.

Let's look directly at the result! I used the query parameters /?page=2&per_page=5 (page 2, 5 items per page):

Perfect!

So what happens if per_page is set to more than 100?

{
    "detail": [
        {
            "type": "less_than_equal",
            "loc": [
                "query",
                "per_page"
            ],
            "msg": "Input should be less than or equal to 100",
            "ctx": {
                "le": 100
            }
        }
    ]
}

The answer is a 422 response.

Summary of Pagination Features#

Through these two articles, we demonstrated how to implement pagination in Django Ninja, from simple built-in methods to complex custom pagination classes.

Depending on your project's needs, you can choose the pagination strategy that suits you best, so that every response can be presented to the user in the most appropriate way.

Why Is "Multiple Status Code Response" Not Practical?#

Remember the foreshadowing we left in the 13th and 21st articles?

In "Article 13: Response (I) Django Ninja Handles HTTP Responses," I mentioned:

However, I feel this "multiple status code response" configuration is not very practical in practice. Why? We will discuss this later.

To help you review, "multiple status code response" refers to this usage:

@api.post(
    ...,
    response={200: Token, 401: Message, 402: Message}  # here
)

Then in the view function, we return different responses depending on the situation.

In "Article 21: Error Handling (Part 1) HttpError and Custom HTTP Responses," I said:

This looks good and is very intuitive. I used to write it this way when using Django REST framework.

However, this approach in Django Ninja will hit a snag when using the "pagination decorator".

We will explain this in Article 25, "Pagination (Part 2) Custom Pagination Class."

Here we are!

The reason is very simple, and the key lies in the sentence from "Key Point 3: The paginate_queryset Method" in this article:

The most noteworthy is the second parameter—queryset, which is actually the return value of the view function, and its type must be QuerySet.

Because in the paginate_queryset method, the type of the second parameter must be QuerySet!

Inside paginate_queryset, we treat and operate on this parameter as a QuerySet. If what is passed is not a QuerySet, the pagination logic will fail.

The Conflict Between "Multiple Status Code Response" and the paginate_queryset Method#

However, the return type of a multiple status code response is not necessarily a QuerySet—it is very likely to be a tuple.

Let me give you a simple example to understand. We modify the "Get Post List" API to look like this:

@api.get(
    path="/posts",
    response={200: list[PostResponse], 404: ErrorMessage}
)
@paginate(CustomPagination)
def get_posts(...) -> QuerySet[Post] | tuple[int, dict]:
    posts = Post.objects.all()
    if not posts.exists():
        return 404, {"message": "No posts match the specified criteria"}
    return posts

This example clearly displays the conflict between "multiple status code responses" and the paginator:

  • When the query result is normal, the view function returns a QuerySet (i.e., posts), which is handed to the paginator for pagination, and everything works fine.
  • When no posts are found, the view function attempts to return a tuple—because Django Ninja's "non-200 response" must have a status code, so it is a tuple and not a QuerySet.

This will cause the paginate_queryset method to fail, because it expects to receive a QuerySet, and subsequent internal operations are based on this assumption.

If all APIs in the project do not have pagination, using "multiple status code responses" to handle "non-200" responses is completely feasible.

But if even one API needs pagination, to avoid the conflict described above, this paginated API must switch to the approach also mentioned in the 21st article—raise HttpError.

Considering the overall consistency of the project, the remaining APIs should also adopt the raise HttpError approach.

Since pagination requirements are so common, "multiple status code responses" essentially becomes redundant.