FastAPI

FastAPI is the standard Python framework for putting a model behind an HTTP endpoint.

It is asynchronous, which matters because model serving is mostly waiting.

While one request waits on a GPU or an external API, others can be handled.

Pydantic models define the request and response schemas.

Validation is automatic, so a malformed request is rejected before your code runs.

Type hints drive that validation, so the schema and the code cannot drift apart.

Interactive documentation is generated from the same definitions.

That page is often the fastest way for another team to try your endpoint.

Load the model once at startup and keep it in memory.

Loading it per request is the classic mistake and makes everything unusably slow.

Use the lifespan hook so the model is ready before traffic arrives.

Blocking calls inside an async endpoint stall the whole event loop.

Push heavy synchronous work to a thread pool or a background worker.

It runs under Uvicorn, and Gunicorn manages multiple workers in production.

Each worker holds its own copy of the model, which is the memory cost of that concurrency.