FastAPI – Background tasks with Celery, Redis, and Flower #9

FastAPI is a modern Python web framework that makes it easy to build APIs quickly and efficiently. In this tutorial, we will learn how to work with background tasks in FastAPI using Celery, Redis, and Flower.

Background tasks are handy for offloading time-consuming operations from your main application flow. By using a task queue like Celery and a message broker like Redis, we can easily manage and execute background tasks in FastAPI.

Here’s a step-by-step guide on how to set up background tasks in FastAPI using Celery, Redis, and Flower:

Step 1: Install the required packages

First, we need to install the necessary packages. You can install them using pip:

pip install fastapi[all] celery redis flower

Step 2: Create a Celery instance in your FastAPI app

Create a new Python file, such as celery_app.py, and define a Celery instance there:

from celery import Celery

celery = Celery("tasks", broker="redis://localhost:6379/0")

Step 3: Create a FastAPI app with Celery integration

In your main FastAPI app file, import the Celery instance from celery_app.py and integrate it:

from fastapi import FastAPI
from celery_app import celery

app = FastAPI()

@app.get("/run-task")
def run_background_task():
    task = celery.send_task('tasks.background_task')
    return {"task_id": task.id}

Step 4: Define a background task

Create a new Python file, such as tasks.py, and define a background task there using Celery decorators:

from celery import Celery

celery = Celery('tasks', broker='redis://localhost:6379/0')

@celery.task
def background_task():
    # Write your time-consuming task here
    return "Background task completed successfully"

Step 5: Run Celery worker

Start a Celery worker to process the background tasks:

celery -A tasks worker --loglevel=info

Step 6: Run FastAPI app with Flower

Run your FastAPI app and start the Flower web interface for monitoring the Celery tasks:

uvicorn main:app --reload
flower -A tasks --port=5555

Now you have successfully set up background tasks in FastAPI using Celery, Redis, and Flower. You can trigger the background task by accessing the /run-task endpoint in your FastAPI app and monitor the task execution in the Flower web interface.

This setup allows you to offload time-consuming operations to background tasks, making your FastAPI app more efficient and scalable. Have fun building awesome APIs with FastAPI and Celery!