llmfit

llmfit is a terminal-based utility that analyzes your system's hardware to identify which large language models will run effectively on your specific configuration. It provides an interactive TUI and CLI to score models based on quality, speed, and memory fit while supporting various backends like Ollama, llama.cpp, and MLX. Users can also perform hardware simulations to test how different model configurations would perform on target system specifications.

llmfit is a terminal-based utility designed to bridge the gap between hardware capabilities and Large Language Model (LLM) requirements. By automatically detecting system specifications—including CPU, RAM, and various GPU architectures—it scores and ranks models based on their fit, speed, and quality, effectively solving the problem of manual trial-and-error in local model deployment. The project employs a sophisticated scoring engine that accounts for dynamic quantization, Mixture-of-Experts (MoE) architectures, and context-length constraints, providing users with actionable insights into which models will perform optimally on their specific hardware.