ubergarm ca52a825db
feat: add --threads-mtmd for independent multimodal thread count (#1797)
Add `-tm` / `--threads-mtmd` to control CPU thread count used during
multimodal image/audio processing (mmproj encoding), separate from the
main LLM thread count.

This allows running the LLM on GPU with minimal CPU threads (e.g. `-t 1`)
to reduce sync overhead, while using many threads (e.g. `-tm 16`) for
CPU-bound mmproj encoding with `--no-mmproj-offload`.

Fallback chain when `-tm` is not specified:
 1. `--threads-batch` (-tb) — multimodal encoding is a batch/prefill-like
    operation, so it makes sense to track with batch thread count
 2. `--threads` (-t) — final default

Works with both mtmd-cli and llama-server.

AI: ubergarm/Qwen3.6-27B-GGUF MTP IQ4_KS 15.113 GiB (4.752 BPW) + pi.dev
2026-05-13 17:44:43 +03:00
..
2026-04-23 09:05:39 +02:00
2024-07-27 07:55:01 +02:00
2024-07-27 07:55:01 +02:00
2025-06-19 10:24:53 +03:00
2026-04-23 09:05:39 +02:00
2025-12-15 08:27:20 +01:00
2024-08-12 15:14:32 +02:00
2023-03-29 20:21:09 +03:00
2024-07-27 07:55:01 +02:00