Build and ship an AI application in four weeks

What a language model can and cannot do, how the API behaves, and a project skeleton with tests and logging that you keep for the whole course.

• How language models work: tokens, context windows, sampling and why outputs vary
• Choosing a model: capability, latency, cost and the trade-offs that matter in production
• The messages API: system, user and assistant turns, temperature and stop conditions
• Structured output: JSON schemas, validation and retry-on-invalid
• Prompt design that holds up: instructions, examples, delimiters and failure cases
• Project setup: repository, environment variables, secrets, logging and a first HTTP endpoint

$ test

Module test: 25 questions on model behaviour, API mechanics and prompt design (pass mark 70%)

$ ship

A command-line assistant that returns validated JSON, with unit tests and request logging

Turn documents into something a model can search, retrieve the right passages, and answer with citations the reader can check.

• Embeddings and similarity search explained with a whiteboard, not a proof
• Document loading and cleaning: PDFs, HTML, tickets and spreadsheets
• Chunking strategies and their effect on answer quality
• Vector stores: indexing, metadata filters and hybrid keyword search
• Retrieval-augmented generation with citations and refusal when the corpus is silent
• Conversation memory: windows, summaries and when to forget

$ test

Module test: 25 questions plus a graded retrieval notebook (recall on a held-out question set)

$ ship

A documentation assistant over a real corpus that cites its sources and declines out-of-scope questions

Let the model act through tools, keep it inside guardrails, and measure whether any of it works before shipping.

• Tool calling: function schemas, argument validation and idempotent tools
• Agent loops: planning, step limits, stopping conditions and human-in-the-loop points
• Guardrails: input validation, output checks, refusals and prompt-injection defence
• Evaluation: golden sets, rubric scoring, model-as-judge and its blind spots
• Regression tests for prompts and tools that run in continuous integration
• Cost and latency budgets: caching, batching and choosing a smaller model on purpose

$ test

Module test: 25 questions plus a graded evaluation harness run against a hidden test set

$ ship

An agent with two tools and an evaluation suite that fails the build when quality drops

Streaming, a front end, deployment, observability and incident basics, then the final exam and the graded project.

• Streaming responses and a minimal web front end
• Deployment: containers, environment configuration and secrets in production
• Tracing and monitoring: token usage, latency, error rates and cost per request
• Safety and privacy in production: data retention, PII handling and audit trails
• Presenting technical work: the demo, the evaluation report and the trade-offs
• Final exam and project review

$ test

Final exam: 60 questions across the four modules, 90 minutes (pass mark 70%)

$ ship

Deploy the final project with monitoring, then submit the repository, the demo and the evaluation report for grading