EP223: Ollama vs vLLM vs SGLang
To use open-weight models on your machine, you have three main options: Ollama, vLLM, and SGLang. But each engine handles requests differently.
Daily edition · Architecture
The daily ledger
TILens turns technical updates into a focused daily brief: official releases, trusted reporting, and practitioner analysis, deduplicated and organized by topic.
3 articles · 2 sources ·
Top topics: Architecture
Presentation: SafeChat: Building AI-Powered Safety Systems at Scale in a Real-Time Marketplace
Bruna Pereira explains how DoorDash built a content-agnostic AI moderation platform. She covers replacing costly LLM-only pipelines with a hybrid pattern: using fast internal models to filter obvious cases, LLM…
AWS Releases aws-bench to Evaluate Agents on Cloud Tasks
AWS has released aws-bench, an open-source benchmark for evaluating AI agents on real AWS tasks such as misconfigurations and infrastructure provisioning. Unlike traditional benchmarks, it uses real resources in…