# Expectations for Kimi-class large locals: step by step

# Context for “Expectations for Kimi-class large locals”

Hardware reality and license reading first. Software is the part of a local AI stack that changes fastest. A setup that worked last month can break after a driver update, a runtime upgrade, or a change in how a model is packaged. A short, written procedure keeps those changes boring.

# A minimal procedure

1. Note the operating system, GPU driver, runtime version and model file before changing anything.
2. Make one change: a new runtime, a new quantisation, a new context length, or a new model.
3. Run the same ten to twenty prompts you always use, in the languages your users actually write in.
4. Compare answers and resource use against the previous run.
5. Keep or roll back, and record the decision with a date.

# Installation choices

Native installs are simplest on a single machine and avoid GPU passthrough issues. Containers help when several services share a host or when you need reproducible deployments; mount model volumes rather than baking multi-gigabyte weights into images. On Windows, WSL2 works well but its virtual disk can grow quietly, so watch free space. On macOS, decide whether you will use llama.cpp builds or MLX and keep the directory layout tidy.

# Settings worth documenting

• Default temperature and system prompt for the team, with a reason for any exception.
• Context length and number of parallel slots per model.
• Which features are enabled: tool calls, speculative decoding, vision input.
• Where logs go and how long they are kept.

# When something goes wrong

Collect the exact error text, the command or UI action that triggered it, and resource use at that moment. Many reports turn out to be out-of-memory conditions, a wrong prompt template, or a partially downloaded file. Search a shared failure wiki before posting; add the fix once you find it.

# Checklist for a new machine

• Install the GPU driver first and confirm the card is visible to the system.
• Install one runtime only, and verify it can see the GPU.
• Pull one small model, run a short prompt, and note memory use.
• Move the model directory to a disk with enough free space and update the path.
• Add a second model only after the first one works reliably.
• Write the steps you followed into a shared document while they are fresh.

This order makes problems easy to isolate. When several tools and models are installed at once, it becomes much harder to tell which change caused a failure.

Further reading from MicroAI One: VRAM and model-size table, and Dify + Ollama on Windows.

# Bottom line

Local AI rewards patience and record keeping more than raw spending. Decide what the system must do, measure it on your own hardware and documents, write down every change with a date and an owner, and review the setup after each significant upgrade. Small, documented steps are easier to debug, easier to hand over and easier to defend when someone asks why a particular choice was made. When in doubt, test on a smaller scale first and expand only when the evidence supports it.

# Quick reminders

• Treat retrieved text and uploaded files as untrusted.
• Test a restore, not just a backup.
• Publish what the assistant will refuse to do.
• Measure on your own prompts before quoting any speed.

Published by MicroAI One (本地AI 1号网), a Shenzhen team working on local LLM hardware, software and deployment.