Xuan-Son Nguyen f5c6ae1827

mtmd, server: add "placeholder bitmap" for counting tokens , add */input_tokens API (#23913) ...

* mtmd: add "placeholder bitmap" for counting tokens w/o preprocessing

* fast path skip preproc for placeholder

* fix build

* correct the api

* add server endpoint + tests

* add object name

* update docs

* add proxy handling

* fix build

* fix audio input path

* use is_placeholder in process_mtmd_prompt()

* nits

* nits (2)

* docs: clarify chat/completions/input_tokens is not official

* fix merge problem

2026-06-06 11:06:51 +02:00

..

cogvlm.cpp

mtmd: add clip_graph::build_mm() (#20751)

2026-03-19 13:11:39 +01:00

conformer.cpp

mtmd, server: add "placeholder bitmap" for counting tokens , add */input_tokens API (#23913)

2026-06-06 11:06:51 +02:00

deepseekocr2.cpp

mtmd: Add DeepSeekOCR 2 Support (#20975)

2026-05-29 16:13:51 +02:00

deepseekocr.cpp

mtmd: Add DeepSeekOCR 2 Support (#20975)

2026-05-29 16:13:51 +02:00

dotsocr.cpp

mtmd: support dots.ocr (#17575)

2026-04-09 12:16:38 +02:00

exaone4_5.cpp

mtmd, server: add "placeholder bitmap" for counting tokens , add */input_tokens API (#23913)

2026-06-06 11:06:51 +02:00

gemma4a.cpp

mtmd: add Gemma 4 audio conformer encoder support (#21421)

2026-04-12 14:15:26 +02:00

gemma4ua.cpp

mtmd, model: allow skip build_vit() (#24077)

2026-06-03 17:10:35 +02:00

gemma4uv.cpp

mtmd, model: allow skip build_vit() (#24077)

2026-06-03 17:10:35 +02:00

gemma4v.cpp

mtmd: fix gemma 4 projector pre_norm (#23822)

2026-05-28 20:58:55 +02:00

glm4v.cpp

mtmd, server: add "placeholder bitmap" for counting tokens , add */input_tokens API (#23913)

2026-06-06 11:06:51 +02:00

granite4-vision.cpp

model, mtmd: Granite4 Vision (#23545)

2026-06-05 17:44:59 +02:00

granite-speech.cpp

mtmd, server: add "placeholder bitmap" for counting tokens , add */input_tokens API (#23913)

2026-06-06 11:06:51 +02:00

hunyuanvl.cpp

mtmd, model : merge HunyuanOCR into HunyuanVL and fix OCR vision precision (#23329)

2026-05-21 00:35:37 +02:00

internvl.cpp

clip: move model cgraphs into their own files (#17965)

2025-12-12 21:14:48 +01:00

kimik25.cpp

mtmd, server: add "placeholder bitmap" for counting tokens , add */input_tokens API (#23913)

2026-06-06 11:06:51 +02:00

kimivl.cpp

clip: move model cgraphs into their own files (#17965)

2025-12-12 21:14:48 +01:00

llama4.cpp

mtmd: add clip_graph::build_mm() (#20751)

2026-03-19 13:11:39 +01:00

llava.cpp

model, mtmd: Granite4 Vision (#23545)

2026-06-05 17:44:59 +02:00

mimovl.cpp

mtmd, server: add "placeholder bitmap" for counting tokens , add */input_tokens API (#23913)

2026-06-06 11:06:51 +02:00

minicpmv.cpp

mtmd : support MiniCPM-V 4.6 (#22529)

2026-05-06 21:54:09 +02:00

mobilenetv5.cpp

mtmd: add clip_graph::build_mm() (#20751)

2026-03-19 13:11:39 +01:00

models.h

model, mtmd: Granite4 Vision (#23545)

2026-06-05 17:44:59 +02:00

nemotron-v2-vl.cpp

mtmd : Add Nemotron Nano 12B v2 VL support (#19547)

2026-02-14 14:07:00 +01:00

paddleocr.cpp

model: Add PaddleOCR-VL model support (#18825)

2026-02-19 17:05:25 +01:00

pixtral.cpp

mtmd: add clip_graph::build_mm() (#20751)

2026-03-19 13:11:39 +01:00

qwen2vl.cpp

mtmd, server: add "placeholder bitmap" for counting tokens , add */input_tokens API (#23913)

2026-06-06 11:06:51 +02:00

qwen3a.cpp

mtmd: add chunks and fix preproc for qwen3a (#23073)

2026-05-15 19:32:47 +02:00

qwen3vl.cpp

mtmd, server: add "placeholder bitmap" for counting tokens , add */input_tokens API (#23913)

2026-06-06 11:06:51 +02:00

siglip.cpp

mtmd: Add DeepSeekOCR Support (#17400)

2026-03-25 19:57:40 +01:00

step3vl.cpp

model : support step3-vl-10b (#21287)

2026-04-08 09:51:31 +02:00

whisper-enc.cpp

mtmd, server: add "placeholder bitmap" for counting tokens , add */input_tokens API (#23913)

2026-06-06 11:06:51 +02:00

yasa2.cpp

mtmd: Add support for Reka Edge 2603 (#21616)

2026-04-21 20:02:49 +02:00

youtuvl.cpp

mtmd: add clip_graph::build_mm() (#20751)

2026-03-19 13:11:39 +01:00