EP223: Ollama vs vLLM vs SGLang
To use open-weight models on your machine, you have three main options: Ollama, vLLM, and SGLang. But each engine handles requests differently.
Topic - Edition
Architecture
Presentation: SafeChat: Building AI-Powered Safety Systems at Scale in a Real-Time Marketplace
Bruna Pereira explains how DoorDash built a content-agnostic AI moderation platform. She covers replacing costly LLM-only pipelines with a hybrid pattern: using fast internal models to filter obvious cases, LLM…
AWS Releases aws-bench to Evaluate Agents on Cloud Tasks
AWS has released aws-bench, an open-source benchmark for evaluating AI agents on real AWS tasks such as misconfigurations and infrastructure provisioning. Unlike traditional benchmarks, it uses real resources in…