Show HN: Llama 3.1 70B on a single RTX 3090 via NVMe-to-GPU bypassing the CPU
358 by xaskasdf | 93 comments on Hacker News.
Hi everyone, I'm kinda involved in some retrogaming and with some experiments I ran into the following question: "It would be possible to run transformer models bypassing the cpu/ram, connecting the gpu to the nvme?" This is the result of that question itself and some weekend vibecoding (it has the linked library repository in the readme as well), it seems to work, even on consumer gpus, it should work better on professional ones tho
-
By BY THE LEARNING NETWORK from NYT The Learning Network https://ift.tt/2Gh2Q2k via IFTTT
-
By BY PETER BAKER from NYT en Español https://ift.tt/2U8g3PZ via IFTTT
-
U.S. Postal Service starts nationwide electric vehicle fleet, buying 9,250 EVs 444 by lxm | 336 comments on Hacker News.
