Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

请问MiniCPM V2_6的视频微调如何设置视频的采样帧数 #604

Open
mycroft1603 opened this issue Oct 7, 2024 · 4 comments
Open

请问MiniCPM V2_6的视频微调如何设置视频的采样帧数 #604

mycroft1603 opened this issue Oct 7, 2024 · 4 comments

Comments

@mycroft1603
Copy link

如题，请问是设置这个max_slice_nums参数吗？我看minicpm V2.6在swift中的template中有这个参数，训练过程中也有提示。
截屏2024-10-05 23 45 29
截屏2024-10-05 23 45 44
@spidercatfly
Copy link

Same question

@LDLINGLINGLING
Copy link
Collaborator

你好，这个max_slice_nums并非最切分帧数的设置参数，而是图片切割的参数。在微调模型时，需要先将视频进行抽帧处理成多图，再进行训练。推理时保持和之前相似的抽帧逻辑

@mycroft1603
Copy link
Author

你好，这个max_slice_nums并非最切分帧数的设置参数，而是图片切割的参数。在微调模型时，需要先将视频进行抽帧处理成多图，再进行训练。推理时保持和之前相似的抽帧逻辑
我看这个链接中有提到直接使用视频进行微调：modelscope/ms-swift#1613

截屏2024-10-09 15 06 37 我尝试之后也确实是成功的，只是目前不知道如何设置采样的帧数。 所以您的意思是最好还是手动将视频采样为多图进行多图SFT吗？

@LDLINGLINGLING
Copy link
Collaborator

您好，你这个使用的是swift的框架，这个框架我不太确定有没有将帧采样的频率接口暴露出来

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@LDLINGLINGLING @mycroft1603 @spidercatfly