Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] internvl2+lmdeploy部署推理视频,有没有控制采样频率,前处理的参数?目前的推理时间过长了 #2489

Open
PancakeAwesome opened this issue Sep 20, 2024 · 1 comment

Comments

@PancakeAwesome
Copy link

Motivation

Describe the bug
What the bug is, and how to reproduce, better with screenshots(描述bug以及复现过程,最好有截图)
internvl2+lmdeploy部署推理视频,有没有控制采样频率,前处理的参数?目前的推理时间过长了

Your hardware and system info
Write your system info like CUDA version/system/GPU/torch version here(在这里给出硬件信息和系统信息,如CUDA版本,系统,GPU型号和torch版本等)
lmdeploy 0.5.3
vllm 0.6.1
ms-swift 2.4.1

Additional context
Add any other context about the problem here(在这里补充其他信息)

谢谢

Related resources

No response

Additional context

No response

@irexyc
Copy link
Collaborator

irexyc commented Sep 20, 2024

https://github.com/InternLM/lmdeploy/blob/main/docs/en/multi_modal/internvl.md (video multi-round conversation)

lmdeploy 本身没有处理视频的操作。抽帧是在外面做的,具体一个视频抽几是用户来做的。lmdeploy 能做的是对每张图片设置max_dynamic_patch 一张图 patch 的数量跟长宽比有关,一个 patch 在 internvl2 中占用 256 个 input_token。这个参数可以控制每张图最大的 patch 数量。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants