Leo AI and Ollama Bring RTX-Accelerated Local LLMs to Brave Browser Users
Summary
Explains how Brave Browser's Leo AI uses Ollama, llama.cpp, and NVIDIA RTX GPUs to run large language models locally for improved privacy and performance.
Key quotes
By not sending prompts to an outside server for processing, the experience is private and always available.
RTX enables a fast, responsive experience when running AI locally.
The article details the technology stack enabling local AI in the Brave browser, specifically mentioning the role of CUDA and Tensor Cores in accelerating inference via Ollama and llama.cpp.