CakeMix LLM Setup

CakeMix uses a 12b Mistral Instruct Q3_K_M LLM (6GB) by default:

Get it on HuggingFace.co

Additional LLMs that work well:

Model Name Pros / Cons Size Download
Mistral-Nemo-Instruct-2407 Q4_K_M
  • ✔ High accuracy
  • ✔ Good performance
  • ✔ Great stability
  • ✘ Slightly slower on 8GB VRAM video cards
 7.48GB Get It
Ellaria-9B Q4_K_M *
  • ✔ Great performance
  • ✔ Good stability
  • ✔ More "forward" vibe
  • ✘ Sometimes gets confused
  • ✘ Can not be distributed commercially for NSFW
 5.76GB Get It

🍰 Get CakeMix

If you have no idea what this is, find out here!