Paginated Requests with the Notion API in Python

Requests that retrieve a list of items in the Notion API use pagination to return all results as the number of values grow bigger. This includes search, querying a database, retrieving blocks, comments, and users. By default, Notion will return 100 items per request, which is also the maximum that can be requested.

Let’s start by taking a look at the simplest response from a paginated endpoint: An empty list.

{
	"object": "list",
	"results": [],
	"next_cursor": null,
	"has_more": false,
	"type": "page_or_database",
	"page_or_database": {}
}

This shows the very basics of a paginated response from the Notion API, which is common for any request that can return more than one item. object is always list for a paginated endpoint, and the actual values will be in the results property; has_more and next_cursor are used when there are many results that will require multiple requests to retrieve. type indicates what types of values might be returned by the response, in this case page_or_database, because it was a request to the search endpoint. Note that even if your search is filtered to return only one type of object, the type will be page_or_database. The final property which in this case is page_or_database is based on the type, and for most types it is empty, however, it is used for property_item responses.

A request that has more than one page will look something like

{
	"object": "list",
	"results": ["big list of results here..."],
  "next_cursor": "dbf03650-6c6a-4b7c-840f-1f3bf2bb8b07",
  "has_more": true,
  "type": "page_or_database",
  "page_or_database": {}
}

To retrieve pages beyond the first page, you pass in next_cursor from the previous request as start_cursor, in the query string for GET requests, and as part of the body for POST requests, same as you would any other parameters for those request types. You can optionally control the size of responses using page_size, on any request. It’s not possible to know how many pages there will be, just that the last page will return false for has_more.

Sample Code #

To retrieve multiple pages from a paginated API, start by making a request for the first page. In the sample I’ve set page_size to 10 so that you don’t need too many items shared with the integration to see pagination!

import os
import requests

# Basic Notion API Setup
NOTION_KEY = os.environ.get("NOTION_KEY")
notion_request_headers = {'Authorization': f"Bearer {NOTION_KEY}", 'Content-Type': 'application/json', 'Notion-Version': '2022-02-22'}

# Paginated Search Requests
search_url = "https://api.notion.com/v1/search"
page_count = 1

# Small page size to demonstrate pagination - default and maximum is 100
params = {"page_size": 10} 

# Make the first request as usual
print(f"Requesting page {page_count}")
search_response = requests.post(search_url, json=params, headers=notion_request_headers)

If that response succeeded, we’ll then collect the results which are the objects we care about, and use has_more from the response to know if we should make another request. When we make the next request, we use next_cursor from the response as start_cursor in the next request, and continue making requests like that until has_more is false.

if search_response.ok:
    search_response_obj = search_response.json()		
    pages_and_databases = search_response_obj.get("results")

    while search_response_obj.get("has_more"):
        page_count += 1
        print(f"Requesting page {page_count}")
        params["start_cursor"] = search_response_obj.get("next_cursor")
        
        search_response = requests.post(search_url, json=params, headers=notion_request_headers)
        if search_response.ok:
            search_response_obj = search_response.json()
            pages_and_databases.extend(search_response_obj.get("results"))

    print(pages_and_databases)

General Guidelines When Using Pagination #

Pagination is a very common pattern in the Notion API, so you’ll want to make sure your code can handle it well!

Notion responses could get quite large, so you should think about if you really want all of the results before going off and retrieving all of them in a loop like this simple example. If you are displaying items in a UI, you may want to request a smaller page size that matches what you need to show in the UI, but if you are retrieving all items you should probably leave it at the default of 100.

The Notion API has rate limits, and in production you would want more error handling especially around 429 responses, however, running this sample single threaded I was unable to trigger the 429 response.