OpenAISystem designAI-generated
Discussion / designResponse prompt
Reported interview periodOct 2025–Jul 2026
Design a Multi-User AI Assistant
Design the backend for a production chat assistant used concurrently by many customers. It should stream model output, invoke approved tools, preserve conversation state, and remain useful under model or depend…
Practice focusRequirements - Define APIs for creating conversations, sending messages, streaming events, cancelling work, and retrieving history. - Isolate tenants and authorize every history or tool access. - Orchestrate model calls and tool calls with timeouts, retries, idempotency, and bounded loops. - Enforce rate limits, quotas, content policy, and cost budgets. - Store message lineage, model/tool versions, and audit data without leaking sensitive prompts. - Set explicit latency, scale, durability, and availability targets.…