Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issues with accessing models from Nvidia #852

Open
chezhia opened this issue Dec 26, 2024 · 0 comments
Open

Issues with accessing models from Nvidia #852

chezhia opened this issue Dec 26, 2024 · 0 comments

Comments

@chezhia
Copy link

chezhia commented Dec 26, 2024

Bug Report for ScrapeGraphAI - Nvidia Model Configuration

Describe the bug:

ScrapeGraphAI encounters an error when attempting to use Nvidia API models with the ChatNVIDIA class. The expected format for specifying the model in the configuration is "modelprovider/modelname", such as "nvidia/nemotron-4-340b-instruct". However, the internal code in abstract_graph.py separates the model provider and model name before passing them to the ChatNVIDIA class, resulting in the model not being found.

To Reproduce:

  1. Install ScrapeGraphAI and its dependencies, ensuring the langchain_nvidia_ai_endpoints package is included for Nvidia model support (pip install scrapegraphai[other-language-models]).
  2. Configure ScrapeGraphAI to use a Nvidia API model by setting the model key in the llm section of the configuration to the format "modelprovider/modelname". Here's an example:
graph_config = {"llm": {
        "model": "nvidia/nemotron-4-340b-instruct",
        "api_key": "YOUR_KEY",
        "temperature": 1,
        "format": "json"
    }}

Run a scraping script that utilizes the configured ScrapeGraphAI instance.

Observe the following error message in the traceback:

ValueError: Model nemotron-4-340b-instruct is unknown, check 'available_models'

Additional context:

You can access models from Meta and Mistral from Nvidia apis, so using model_provider to determine the API might not be applicable.

Possible solutions:

Modify the code in abstract_graph.py to preserve the original format of "modelprovider/modelname" when passing it to the ChatNVIDIA class. But also add additional parameter such as model_source = 'Nvidia' to determine if NVIDIA's API should be called using ChatNVIDIA.

Update the documentation to clearly explain the expected format for specifying Nvidia models in the configuration and the potential consequences of using an incorrect format.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant