[{"data":1,"prerenderedAt":4},["ShallowReactive",2],{"readme:wemm-embedding":3},"\u003Ch1>WeMM-Embedding: WeChat Multi-Modal Embedding\u003C\u002Fh1>\u003Cp align=\"center\">\n  \u003Cb>English\u003C\u002Fb> | \u003Ca href=\"https:\u002F\u002Fgithub.com\u002FTencent\u002Fwemm-embedding\u002Fblob\u002FHEAD\u002FREADME_zh.md\" rel=\"nofollow ugc noopener\">中文\u003C\u002Fa>\n\u003C\u002Fp>\u003Cp align=\"center\">\n  \u003Ca href=\"https:\u002F\u002Fhuggingface.co\u002Fcollections\u002Ftencent\u002Fwemm-embedding\" rel=\"nofollow ugc noopener\">\n    \u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002F🤗-Hugging%20Face-yellow\" alt=\"Hugging Face\" \u002F>\n  \u003C\u002Fa>\n  \u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F2608.24053\" rel=\"nofollow ugc noopener\">\n    \u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002F📄-Technical%20Report-red\" alt=\"Technical Report\" \u002F>\n  \u003C\u002Fa>\n  \u003Ca href=\"https:\u002F\u002Fgithub.com\u002FTencent\u002Fwemm-embedding\u002Fblob\u002FHEAD\u002FLICENSE\" rel=\"nofollow ugc noopener\">\n    \u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FLicense-Apache%202.0-blue.svg\" alt=\"License\" \u002F>\n  \u003C\u002Fa>\n\u003C\u002Fp>\u003Cp>WeMM-Embedding is a family of universal multimodal embedding models developed by the WeChat Vision team. It provides unified representations for text, images, videos, visual documents, and interleaved multimodal inputs, achieving state-of-the-art performance across multiple benchmarks covering diverse tasks and domains.\u003C\u002Fp>\n\u003Cp align=\"center\">\n  \u003Ca href=\"https:\u002F\u002Fgithub.com\u002FTencent\u002Fwemm-embedding\u002Fblob\u002FHEAD\u002Fassets\u002Fperformance-overview.pdf\" rel=\"nofollow ugc noopener\">\n    \u003Cimg src=\"https:\u002F\u002Fraw.githubusercontent.com\u002FTencent\u002Fwemm-embedding\u002FHEAD\u002Fassets\u002Fperformance-overview.png\" width=\"100%\" alt=\"WeMM-Embedding Performance Overview\" \u002F>\n  \u003C\u002Fa>\n\u003C\u002Fp>\u003Ch2>Model Zoo\u003C\u002Fh2>\n\u003Ctable>\n\u003Cthead>\n\u003Ctr>\n\u003Cth>Model\u003C\u002Fth>\n\u003Cth>Matryoshka dimensions\u003C\u002Fth>\n\u003Cth>Hugging Face\u003C\u002Fth>\n\u003C\u002Ftr>\n\u003C\u002Fthead>\n\u003Ctbody>\u003Ctr>\n\u003Ctd>WeMM-Embedding-2B\u003C\u002Ftd>\n\u003Ctd>\u003Ccode>64, 128, 256, 512, 1024, 2048\u003C\u002Fcode>\u003C\u002Ftd>\n\u003Ctd>\u003Ca href=\"https:\u002F\u002Fhuggingface.co\u002Ftencent\u002FWeMM-Embedding-2B\" rel=\"nofollow ugc noopener\">🤗 Link\u003C\u002Fa>\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctr>\n\u003Ctd>WeMM-Embedding-4B\u003C\u002Ftd>\n\u003Ctd>\u003Ccode>64, 128, 256, 512, 1024, 2560\u003C\u002Fcode>\u003C\u002Ftd>\n\u003Ctd>\u003Ca href=\"https:\u002F\u002Fhuggingface.co\u002Ftencent\u002FWeMM-Embedding-4B\" rel=\"nofollow ugc noopener\">🤗 Link\u003C\u002Fa>\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctr>\n\u003Ctd>WeMM-Embedding-9B\u003C\u002Ftd>\n\u003Ctd>\u003Ccode>64, 128, 256, 512, 1024, 2048, 4096\u003C\u002Fcode>\u003C\u002Ftd>\n\u003Ctd>\u003Ca href=\"https:\u002F\u002Fhuggingface.co\u002Ftencent\u002FWeMM-Embedding-9B\" rel=\"nofollow ugc noopener\">🤗 Link\u003C\u002Fa>\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003C\u002Ftbody>\u003C\u002Ftable>\n\u003Cp>All models support text, images, videos, visual documents, and interleaved multimodal inputs. Embeddings are obtained from the last-layer hidden state at the dedicated \u003Ccode><embedding>\u003C\u002Fcode> token position, followed by L2 normalization. Audio input is not currently supported.\u003C\u002Fp>\n\u003Ch2>Installation\u003C\u002Fh2>\n\u003Cpre>\u003Ccode class=\"language-bash\">pip install -r requirements.txt\n\u003C\u002Fcode>\u003C\u002Fpre>\n\u003Ch2>Transformers\u003C\u002Fh2>\n\u003Cp>We recommend using \u003Ccode>transformers==5.2.0\u003C\u002Fcode> for inference and reproducibility, as newer versions may differ in preprocessing behavior.\u003C\u002Fp>\n\u003Cpre>\u003Ccode class=\"language-bash\">python examples\u002Ftransformers_inference.py \\\n  --model \u002Fpath\u002Fto\u002FWeMM-Embedding-2B \\\n  --image \u002Fpath\u002Fto\u002Fimage.jpg \\\n  --video \u002Fpath\u002Fto\u002Fvideo.mp4 \\\n  --dimension 2048\n\u003C\u002Fcode>\u003C\u002Fpre>\n\u003Cp>The example produces independent text, image, and video embeddings. Omit \u003Ccode>--dimension\u003C\u002Fcode> for the full embedding dimension.\u003C\u002Fp>\n\u003Ch2>Sentence Transformers\u003C\u002Fh2>\n\u003Cpre>\u003Ccode class=\"language-bash\">python examples\u002Fsentence_transformers_inference.py \\\n  --model \u002Fpath\u002Fto\u002FWeMM-Embedding-2B \\\n  --image \u002Fpath\u002Fto\u002Fimage.jpg \\\n  --video \u002Fpath\u002Fto\u002Fvideo.mp4 \\\n  --dimension 2048\n\u003C\u002Fcode>\u003C\u002Fpre>\n\u003Cp>\u003Ccode>SentenceTransformer\u003C\u002Fcode> loads the model directly, so a Hugging Face model id such as \u003Ccode>tencent\u002FWeMM-Embedding-2B\u003C\u002Fcode> also works in place of a local path. Text, image, and video inputs go through \u003Ccode>SentenceTransformer.encode()\u003C\u002Fcode>, and MRL is selected with \u003Ccode>--dimension\u003C\u002Fcode>.\u003C\u002Fp>\n\u003Ch2>Serving\u003C\u002Fh2>\n\u003Cp>Tested versions: vLLM \u003Ccode>0.27.0\u003C\u002Fcode> and SGLang \u003Ccode>0.5.9\u003C\u002Fcode>.\u003C\u002Fp>\n\u003Cp>vLLM:\u003C\u002Fp>\n\u003Cpre>\u003Ccode class=\"language-bash\">MODEL_PATH=\u002Fpath\u002Fto\u002FWeMM-Embedding-2B\nvllm serve \"$MODEL_PATH\" \\\n  --runner pooling \\\n  --chat-template \"$MODEL_PATH\u002Fembedding_chat_template.jinja\"\n\u003C\u002Fcode>\u003C\u002Fpre>\n\u003Cp>SGLang:\u003C\u002Fp>\n\u003Cpre>\u003Ccode class=\"language-bash\">MODEL_PATH=\u002Fpath\u002Fto\u002FWeMM-Embedding-2B\npython scripts\u002Fpatch_sglang_video.py\npython -m sglang.launch_server \\\n  --model-path \"$MODEL_PATH\" \\\n  --is-embedding \\\n  --enable-precise-embedding-interpolation\n\u003C\u002Fcode>\u003C\u002Fpre>\n\u003Cp>Equivalent one-command wrappers are available in \u003Ccode>scripts\u002Fserve_vllm.sh\u003C\u002Fcode> and \u003Ccode>scripts\u002Fserve_sglang.sh\u003C\u002Fcode>.\u003C\u002Fp>\n\u003Ch2>Matryoshka Embeddings\u003C\u002Fh2>\n\u003Cp>For a supported dimension \u003Ccode>d\u003C\u002Fcode>, truncate the full embedding and normalize it again:\u003C\u002Fp>\n\u003Cpre>\u003Ccode class=\"language-python\">embedding = torch.nn.functional.normalize(embedding[..., :d], dim=-1)\n\u003C\u002Fcode>\u003C\u002Fpre>\n\u003Cp>On MMEB-v2, the 2B model at 256 dimensions retains 98.7% of its full-dimensional image and video performance.\u003C\u002Fp>\n\u003Ch2>Evaluation\u003C\u002Fh2>\n\u003Ch3>MMEB-v2\u003C\u002Fh3>\n\u003Cp>Results on 78 datasets from Table 1 of the \u003Ca href=\"https:\u002F\u002Fgithub.com\u002FTencent\u002Fwemm-embedding\u002Fblob\u002FHEAD\u002Fassets\u002FWeMM_Embedding_tech_report.pdf\" rel=\"nofollow ugc noopener\">technical report\u003C\u002Fa>. Image and video tasks use Hit@1, while visual-document tasks use NDCG@5. Higher is better.\u003C\u002Fp>\n\u003Ctable>\n\u003Cthead>\n\u003Ctr>\n\u003Cth>Model\u003C\u002Fth>\n\u003Cth align=\"right\">Size\u003C\u002Fth>\n\u003Cth align=\"right\">AVG\u003C\u002Fth>\n\u003Cth align=\"right\">Image\u003C\u002Fth>\n\u003Cth align=\"right\">Video\u003C\u002Fth>\n\u003Cth align=\"right\">VisDoc\u003C\u002Fth>\n\u003C\u002Ftr>\n\u003C\u002Fthead>\n\u003Ctbody>\u003Ctr>\n\u003Ctd>VLM2Vec\u003C\u002Ftd>\n\u003Ctd align=\"right\">2B\u003C\u002Ftd>\n\u003Ctd align=\"right\">4\u003C\u002Ftd>\n\u003Ctd align=\"right\">\u003C\u002Ftd>\n\u003Ctd align=\"right\">\u003C\u002Ftd>\n\u003Ctd align=\"right\">\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003C\u002Ftbody>\u003C\u002Ftable>\n",1788652558844]