colibri
JustVugg · Kit
A hierarchical inference engine that runs large mixture-of-experts models by streaming experts from NVMe storage while keeping dense layers in RAM. It supports…
New49.1k
JustVugg · Kit
A hierarchical inference engine that runs large mixture-of-experts models by streaming experts from NVMe storage while keeping dense layers in RAM. It supports…