Background Tasks with Celery

Some operations Bookshelf might need to perform are simply too slow to run inside a normal request/response cycle: sending the welcome email from Sending Email with Flask-Mail, generating a large export of a user's reading history, or processing an uploaded book cover image. Doing any of these synchronously means the user's browser sits there waiting — sometimes for several seconds — before the response finally comes back. Celery is the standard Python tool for moving that kind of slow work out of the request cycle entirely, to run asynchronously in the background.

The Architecture: Broker and Workers

Celery's design has three distinct pieces working together, and understanding how they connect matters more than memorizing any single API call:

1. Your Flask app — when it needs slow work done, it doesn't do the work itself. It packages up the task and pushes it onto a message broker.
2. The message broker — typically Redis (the same tool often used as a cache backend in the previous lesson) — simply holds the queued task until something picks it up.
3. One or more Celery worker processes — separate, long-running processes that watch the broker, pick up tasks as they arrive, and actually execute them, completely independent of any web request.

The web request that queued the task returns to the user almost immediately — it never waits for the worker to actually finish the email or the image processing.

Integrating Celery with the App Factory

A Celery task that touches Bookshelf's database or sends an email still needs access to Flask's app context — the same context covered in Application Context & Globals. A small helper function ties Celery's configuration to Flask's, so tasks running inside a worker process can still reach current_app and the database correctly:

# celery_app.py
from celery import Celery

def make_celery(app):
    celery = Celery(
        app.import_name,
        broker=app.config['CELERY_BROKER_URL'],
        backend=app.config['CELERY_RESULT_BACKEND'],
    )
    celery.conf.update(app.config)

    class ContextTask(celery.Task):
        def __call__(self, *args, **kwargs):
            with app.app_context():
                return self.run(*args, **kwargs)

    celery.Task = ContextTask
    return celery

# app.py
def create_app():
    app = Flask(__name__)
    app.config['CELERY_BROKER_URL'] = 'redis://localhost:6379/0'
    app.config['CELERY_RESULT_BACKEND'] = 'redis://localhost:6379/0'
    return app

app = create_app()
celery = make_celery(app)

The ContextTask override is the key detail: it wraps every task's actual run inside app.app_context(), so code inside a task can call Book.query... or send an email through Flask-Mail exactly as if it were running inside a normal Flask view.

Defining a Task

A task is a plain function wearing the @celery.task decorator. Here, it wraps the Flask-Mail send call from Sending Email with Flask-Mail:

# tasks.py
from celery_app import celery
from extensions import mail
from flask_mail import Message
from models import User

@celery.task
def send_welcome_email_task(user_id):
    user = User.query.get(user_id)
    msg = Message(
        'Welcome to Bookshelf',
        recipients=[user.email],
        body=f'Hi {user.username}, thanks for joining Bookshelf!',
    )
    mail.send(msg)

Calling a Task Asynchronously

The whole point is not calling send_welcome_email_task(user.id) directly — that would run it synchronously, right there in the view, defeating the purpose entirely. Instead, call .delay():

# app.py
@app.route('/register', methods=['POST'])
def register():
    # ...create and save the new user...
    send_welcome_email_task.delay(user.id)
    return redirect(url_for('book_list'))

.delay(user.id) pushes the task onto the Redis broker and returns almost instantly — the view moves on to redirect(...) without waiting for the actual email to send. The email genuinely does get sent, just later, whenever a worker process picks the task up.

Running a Celery Worker

This is the operational detail that's easy to miss: Celery being installed and imported does nothing on its own. A separate, persistently-running worker process has to actually be started, and kept running, alongside the Flask server:

celery -A app.celery worker --loglevel=info

An Honest Note on Complexity

Celery solves a real problem well, but it isn't free: it means a Redis broker to run and keep available, and one or more worker processes to start, monitor, and restart if they crash — real operational surface area that a simple Flask app otherwise wouldn't have. Reach for it once something is genuinely, measurably too slow for a normal request/response cycle — not as a default architectural choice from day one on a project that doesn't need it yet.