Distributed inference example for llava_next

[VladOS95-cyber]

Add distributed inference example for LLaVA-NeXT-Video-7B-hf

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case. Link: [Community Contributions] examples on distributed inference using 🤗 Accelerate #3078
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

@sayakpaul @a-r-r-o-w @muellerzr

[VladOS95-cyber]

Hi @sayakpaul @a-r-r-o-w! This PR is ready for review, please, take a look.

[sayakpaul]

Thanks for working on this so quickly. Left some comments.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[sayakpaul]

I think there's some gap between the objectives. I thought we agreed on doing distributed captioning?

[VladOS95-cyber]

Hey @sayakpaul! Please, take a look on recent changes. Does it look ok to you?

[sayakpaul]

Thanks for the updates, I left some minor comments. I think this is close to merge.

Could you additionally show some sample outputs (i.e., video - generated caption pairs)?

[sayakpaul]

@muellerzr I think this is now ready for your review.

[VladOS95-cyber]

Thanks for the updates, I left some minor comments. I think this is close to merge.

Could you additionally show some sample outputs (i.e., video - generated caption pairs)?

@sayakpaul Thank you for your review! Do you mean by providing some examples in description to the script? Just in case, as example:
{ "prompt": "USER: <video>\nGenerate caption ASSISTANT:", "video": "/Users/{user_name}/.cache/huggingface/hub/datasets--malterei--LLaVA-Video-small-swift/snapshots/7d712657d4907c4f29c2c6fa17afaa7289a362a7/videos/4255049031.mp4", "generated_text": [ "USER: \nGenerate caption ASSISTANT: \"Practicing his moves: A man in a white shirt and jeans demonstrates his agility and balance on a basketball court, with a water bottle nearby, ready for hydration.\"" ] }

[muellerzr]

Thanks for doing this! As long as @sayakpaul is happy I'm happy 🤗

I did leave one suggestion/feedback, we need to destroy process groups now at the end else torch will get angry at us

[VladOS95-cyber]

Hey @sayakpaul! Do you have any additional comments or suggestions?

[sayakpaul]

Thanks for working on this!

[VladOS95-cyber]

Hi @muellerzr! I think this PR is ready to be merged or there is something else?

[VladOS95-cyber]

Hi @muellerzr! Just a kind reminder in order to not lose this PR!

[VladOS95-cyber]

Hi @sayakpaul @muellerzr! Just a kind reminder, I suppose this PR was missed to be merged. Please, take a look when you are available.

[sayakpaul]

Could I ask you to be a bit more respectful about pinging maintainers? You have pinged @muellerzr already once. So, that should be more than okay.

[VladOS95-cyber]

Could I ask you to be a bit more respectful about pinging maintainers? You have pinged @muellerzr already once. So, that should be more than okay.

Hi, sure, sry fo that, I just wanted to avoid moving this PR into stale or smth.