Cloudflare Drop
522 by coloneltcb | 280 comments on Hacker News.
news
Friday, 10 July 2026
New best story on Hacker News: Show HN: Getting GLM 5.2 running on my slow computer
Show HN: Getting GLM 5.2 running on my slow computer
707 by vforno | 170 comments on Hacker News.
A few days ago I found myself trying out GLM 5.2 and was really positively impressed. The capabilities and security I was getting from this LLM are similar to those I've gotten from models like Claude or GPT, and this really surprised me. But then I thought, "I wonder how it would work on a normal computer like mine," and above all, "I wonder if it would work without going into OOM on a computer like mine." So I started working with the help of agents to test this possibility. I started converting the model to int4, understanding MTP usage, and if possible implementing DSA for long context. How it responds in int4 and whether the quality is maintained or not. Until I got to the point, on my computer with 32GB of RAM, I was able to communicate with GLM 5.2 with times that, of course, aren't high in cold start, but even then, we're talking about 0.1 tok/s, but that wasn't important to me. The important thing was the journey to reach this goal. I just wanted it to work at all costs, even slowly. So I created Colibrì, which was born from a very simple idea, to be honest, but tested in every way, where a 744B Mixture-of-Experts model activates only ~40B parameters per token—and only ~11 GB of those change from token to token (the routed experts). So: The dense part (attention, shared experts, embeddings—~17B params) stays resident in RAM at int4 (~9.9 GB); The 21,504 routed experts (75 MoE layers × 256 experts + the MTP head, ~19 MB each at int4) live on disk (~370 GB) and are streamed on demand, with a per-layer LRU cache, an optional pinned hot-store, and the OS page cache as a free L2. The engine is a single C file (c/glm.c, ~1,300 lines) plus small headers. No BLAS, no Python at runtime, no GPU.No GPU or serious hardware because I don't have that hardware so I can't test it on hardware that is more powerful than my computer.Colibrì is a one-person project, written and tested entirely on a 12-core laptop with 25 GB of RAM — the numbers above are the ceiling of what I can measure at home. Any feedback is welcome! (and if anyone wanted to participate in the project I would be delighted) Repo: https://ift.tt/DdO4kAg
707 by vforno | 170 comments on Hacker News.
A few days ago I found myself trying out GLM 5.2 and was really positively impressed. The capabilities and security I was getting from this LLM are similar to those I've gotten from models like Claude or GPT, and this really surprised me. But then I thought, "I wonder how it would work on a normal computer like mine," and above all, "I wonder if it would work without going into OOM on a computer like mine." So I started working with the help of agents to test this possibility. I started converting the model to int4, understanding MTP usage, and if possible implementing DSA for long context. How it responds in int4 and whether the quality is maintained or not. Until I got to the point, on my computer with 32GB of RAM, I was able to communicate with GLM 5.2 with times that, of course, aren't high in cold start, but even then, we're talking about 0.1 tok/s, but that wasn't important to me. The important thing was the journey to reach this goal. I just wanted it to work at all costs, even slowly. So I created Colibrì, which was born from a very simple idea, to be honest, but tested in every way, where a 744B Mixture-of-Experts model activates only ~40B parameters per token—and only ~11 GB of those change from token to token (the routed experts). So: The dense part (attention, shared experts, embeddings—~17B params) stays resident in RAM at int4 (~9.9 GB); The 21,504 routed experts (75 MoE layers × 256 experts + the MTP head, ~19 MB each at int4) live on disk (~370 GB) and are streamed on demand, with a per-layer LRU cache, an optional pinned hot-store, and the OS page cache as a free L2. The engine is a single C file (c/glm.c, ~1,300 lines) plus small headers. No BLAS, no Python at runtime, no GPU.No GPU or serious hardware because I don't have that hardware so I can't test it on hardware that is more powerful than my computer.Colibrì is a one-person project, written and tested entirely on a 12-core laptop with 25 GB of RAM — the numbers above are the ceiling of what I can measure at home. Any feedback is welcome! (and if anyone wanted to participate in the project I would be delighted) Repo: https://ift.tt/DdO4kAg
Thursday, 9 July 2026
New best story on Hacker News: GPT-5.6
GPT-5.6
683 by logickkk1 | 484 comments on Hacker News.
https://ift.tt/Clgd95A https://ift.tt/hckO6TP
683 by logickkk1 | 484 comments on Hacker News.
https://ift.tt/Clgd95A https://ift.tt/hckO6TP
Wednesday, 8 July 2026
Subscribe to:
Posts (Atom)
-
U.S. Postal Service starts nationwide electric vehicle fleet, buying 9,250 EVs 444 by lxm | 336 comments on Hacker News.
-
Learn Postgres at the Playground – Postgres compiled to WASM running in browser 543 by samwillis | 144 comments on Hacker News.
-
NSA, NIST, and post-quantum crypto: my second lawsuit against the US government 486 by trulyrandom | 143 comments on Hacker News.