Sam: Your model's 'cold' experts are being streamed directly from your NVMe drive.
Kai: [beat] This is Kai.
Sam: And I'm Sam.
Kai: And you're listening to Trending Repos for Thursday, November 5th. We have a packed show for you.
Kai: Coming up: a way to run absolutely massive AI models on a regular home computer.
Sam: A tool named 'Heretic' that promises to uncensor LLMs, no retraining required.
Kai: And a deceptively simple Git utility that could solve a huge headache for AI agent developers.
Kai: Okay, let's dive in. How do you run an enormous model on hardware you actually own? A project called `colibri` from GitHub user JustVugg thinks it has the answer.
Sam: I saw this one. The claim is a 'hierarchical inference engine'. That sounds like a PhD thesis, not a weekend project. What does it actually do?
Kai: [excited] It's brilliant! It's for Mixture-of-Experts models. So instead of needing a datacenter's worth of VRAM, it puts the model's 'experts' into tiers.
Sam: Tiers them how?
Kai: The hottest, most-used experts live on your GPU in VRAM. The 'warm' ones that are used less often sit in your system RAM. And the 'cold' ones—the rarely used specialists—are streamed on-demand from your NVMe drive.
Sam: Hold on. Streaming model chunks from a storage drive during inference? The latency must be astronomical. This can't possibly be fast.
Kai: It's a trade-off! Look, it won't be as fast as an H100 cluster, but it makes running these models possible where before it was impossible. And it's written in C with CUDA and Vulkan backends, so it's as close to the metal as you can get.
Sam: Close to the metal and close to a space heater. Your GPU is doing inference, your CPU is managing I/O... this is going to make your machine scream.
Kai: But think what this unlocks! You can grab a massive open MoE model that only the cloud giants could run before, and actually experiment with it yourself. You can finally tinker with the state of the art. The repo is `JustVugg/colibri`.
Sam: Tinker, or just wait. It's an early-stage C project from a solo dev. That's a lot of trust to put in code that's going to be hitting every piece of your hardware at once.
Kai: It's open source, Sam. You can read the code. This is how democratization happens—making the impossible, possible. Even if it's a little slow at first.
Sam: Okay, so what happens after you get a model running? There's a project called `heretic` from user `p-e-w` that claims to modify LLM behavior, without any fine-tuning.
Kai: No training? How? Is this like a complex prompting strategy?
Sam: It's more invasive than that. It seems to directly patch the model's weights or activation layers during runtime to suppress or amplify certain concepts. The README mentions removing censorship, reducing sycophancy, suppressing 'slop'...
Kai: [excited] Oh, I love that. No more 'As a large language model...'? Just tell me the answer! That's a feature, not a bug.
Sam: Kai, the project is named `heretic`. They are not being subtle. The primary use case they're advertising is 'removing censorship'.
Kai: Which can mean a lot of things! It could mean getting a straight answer about a controversial topic for research, or just getting a model to be less... preachy.
Sam: Or it could mean removing every single safety guardrail designed to prevent the model from generating dangerous, hateful, or just wildly incorrect information. This is a jailbreak-as-a-service tool.
Kai: It's a research tool! Understanding how to surgically alter model behavior is critical for alignment research. You have to know how it breaks to know how to build it better.
Sam: I'm sure that's what everyone who downloads it will use it for. [sarcastic] Purely academic pursuits. The name alone is a giant red flag about its intended purpose.
Kai: I think you're being cynical. It's a powerful new technique for model control.
Sam: [sighs] A powerful technique for chaos. It's literally in the name.
Kai: Alright, let's get to our last one. It's a bit more down to earth, but incredibly useful. It's called `worktrunk`, from `max-sixty`, and it's a CLI for managing Git worktrees.
Sam: [dry] Git worktree management. Okay. I know what Git is, but I feel like `worktree` is one of those commands ninety-nine percent of devs have never used.
Kai: Exactly! But it's perfect for a new, specific problem: managing parallel AI agent workflows. Imagine you have three AI agents all trying to write code in the same repo at the same time.
Sam: Sounds like a nightmare of merge conflicts. Constant `git stash`, `git checkout`, fighting over the same files...
Kai: Right. But `git worktree` lets you check out multiple branches into different directories at the same time, all linked to the same `.git` history. `worktrunk` is a simple wrapper around that, making it trivial to create, list, and clean up a worktree for each agent.
Sam: So each agent gets its own clean working directory, can commit to its own branch, and you don't have them stepping on each other's toes. That's... actually really smart.
Kai: It's a sharp tool for a sharp problem! It's not trying to reinvent version control, just making an existing, underused feature accessible for a modern use case.
Sam: I like this one. No grand claims, just a developer (`max-sixty`) who had a problem and built a clean solution. It's practical. Refreshing, honestly.
Kai: So let's do a quick recap.
Sam: We started with `colibri`, for running huge AI models by splitting them across your GPU, RAM, and even your storage drive.
Kai: Then `heretic`, the controversial tool for patching language models on the fly.
Sam: And we finished with `worktrunk`, a clever little CLI for managing Git, specifically for AI agent development.
Kai: Before we go, Sam, did you see the GitHub Archive Program just finished its latest data deposit into the Arctic World Archive?
Sam: In Svalbard, right? So `worktrunk` and `heretic` are now officially preserved for a thousand years on photosensitive film in a decommissioned coal mine?
Kai: [laughs] Exactly. So future civilizations can either build amazing AI agent workflows or... unleash unspeakable horrors. A real coin toss.
Kai: And that's our show, thanks for listening.
Sam: This has been Trending Repos. We'll be back tomorrow. Try not to set your laptop on fire.
This show is made with AI: the hosts’ voices are synthetic and the scripts are AI-assisted. Every story links to its original source.