Opensource

Introduction Over the past month I’ve written two articles on decision models: Jev vs Laya compared the closed-source Jev with the open-source Laya, and Microsoft-Decision-1 Explained added Microsoft to the picture. The question I’ve been asked most since then is: can I train one myself? Yes, you can, and the bar is lower than you might think. Just look at the third-party leaderboard JevBench: as of mid-October, its open-weights board ranks more than 160 systems, the vast majority built by the community and small teams. The ones at the top aren’t mysterious either: H2O-Lightning-4B, first on the composite score, is a fine-tune of Alibaba’s open-source Qwen3.5-4B; Quyet-1.0-Large, first on the capability score, is Gemma-4-31B plus an adapter; and Microsoft’s own Microsoft-Decision-1 is also built on the open-source Qwen3.5-9B. ...

Introduction Last month I wrote Jev vs Laya, comparing TypeSafe AI’s closed-source decision model Jev with the community’s open-source Laya. Back then, “decision models” were a brand-new category: Jev launched on September 15, and Laya was open-sourced three days later. Just three weeks later, the field is crowded: on October 6, OpenAI opened its Decisions API built on GPT-6 Luna; companies such as Liquid AI and Inception have launched their own decision APIs; and the third-party leaderboard JevBench now ranks more than 140 open-weight systems. On October 9, Microsoft joined the race with Microsoft-Decision-1. ...

In my previous article, I migrated the OpenWebUI database to Azure Database for PostgreSQL. In my day-to-day use of OpenWebUI, I kept running into two problems: No access to the latest information: Models have a knowledge cutoff. When I ask about recent releases, news, or prices, they either can’t answer or get it wrong. Pasted links are never opened: When I paste the URL of an article, a document, or a GitHub page into the chat and ask the model to summarize, translate, or compare it, the model doesn’t read the link at all. Sometimes it even guesses content that “looks plausible” from the URL alone. Both problems can be solved with OpenWebUI’s web capabilities. In this article, I use Microsoft’s new Microsoft Web IQ as the web search engine, and let the model automatically detect URLs in messages and read the pages before answering. All it takes is creating a Web IQ resource in the Azure portal and then configuring OpenWebUI in its admin UI. No code to write and no extra services to deploy. ...

Introduction In September 2026, a new kind of model that “doesn’t talk” suddenly became the talk of the AI world. On September 15, TypeSafe AI, founded by former OpenAI researcher Diogo Almeida, released Jev. It doesn’t chat, write articles, or generate code. It takes a piece of state and a set of predefined questions, and directly returns structured decisions with calibrated probabilities. ...

As Large Language Models (LLMs) are widely deployed in dialogue systems, intelligent assistants, and Agent scenarios, the core challenge for inference systems has shifted from “can it run” to “how to run with lower latency, higher throughput, and greater stability”. Against this backdrop, PD Disaggregated (Prefill / Decode Decoupling) has gradually become a key architectural concept in large-scale online inference systems. This article will systematically explain what PD Disaggregated is, why it is needed, and the core advantages it brings to LLM inference systems from the perspective of model inference execution flow, without relying on any specific inference framework. ...

Large language models often encounter GPU memory bottlenecks during inference deployment: Model parameters (P) can reach hundreds of GB and must remain resident in GPU memory. Input/output data (D) changes dynamically with each request but is often coupled with parameters on the same device, leading to imbalanced memory usage and limited scalability. To solve this problem, we can leverage the vLLM framework to implement Parameter/Data (P/D) Separation, improving the flexibility and throughput of inference systems. ...

What is VibeVoice? VibeVoice is a research framework released by Microsoft Research for long-form, multi-speaker, conversational speech synthesis. Target scenarios include entire podcast episodes, audio dramas, or interviews: it can maintain speaker consistency within a single generation and handle natural turn-taking. The model family includes multiple scales (e.g., 1.5B, 7B, etc.) and is available on Hugging Face as microsoft/VibeVoice-1.5B, along with model cards, weights, installation guides, and responsible use notes. It addresses several pain points of traditional TTS systems, such as: ...