v0.121.2

v0.121.2 - 2024-01-11

Fixed

• Ollama response schema for repeated calls with Ollama 0.1.19 completion models. Thanks @jakedetels for the bugfix!

v0.121.1

Fixed

• Ollama response schema for repeated calls with Ollama 0.1.19

v0.121.0

Added

• Synthia prompt template

Changed

• breaking change: Renamed parentCallId function parameter to callId to enable options pass-through.
• Better output filtering for detailed-object log format (e.g. via modelfusion.setLogFormat("detailed-object"))

v0.120.0

v0.120.0 - 2024-01-09

Added

• OllamaCompletionModel supports setting the prompt template in the settings. Prompt formats are available under ollama.prompt.*. You can then call .withTextPrompt(), .withInstructionPrompt() or .withChatPrompt() to use a standardized prompt.

  const model = ollama
    .CompletionTextGenerator({
      model: "mistral",
      promptTemplate: ollama.prompt.Mistral,
      raw: true, // required when using custom prompt template
      maxGenerationTokens: 120,
    })
    .withTextPrompt();

v0.119.1

Fixed

• Incorrect export. Thanks @mloenow for the fix!

v0.119.0

Added

• Schema-specific GBNF grammar generator for LlamaCppCompletionModel. When using jsonStructurePrompt, it automatically uses a GBNF grammar for the JSON schema that you provide. Example:

  const structure = await generateStructure(
    llamacpp
      .CompletionTextGenerator({
        // run openhermes-2.5-mistral-7b.Q4_K_M.gguf in llama.cpp
        promptTemplate: llamacpp.prompt.ChatML,
        maxGenerationTokens: 1024,
        temperature: 0,
      })
      // automatically restrict the output to your schema using GBNF:
      .asStructureGenerationModel(jsonStructurePrompt.text()),
  
    zodSchema(
      z.array(
        z.object({
          name: z.string(),
          class: z
            .string()
            .describe("Character class, e.g. warrior, mage, or thief."),
          description: z.string(),
        })
      )
    ),
  
    "Generate 3 character descriptions for a fantasy role playing game. "
  );

v0.118.0

Added

• LlamaCppCompletionModel supports setting the prompt template in the settings. Prompt formats are available under llamacpp.prompt.*. You can then call .withTextPrompt(), .withInstructionPrompt() or .withChatPrompt() to use a standardized prompt.

  const model = llamacpp
    .CompletionTextGenerator({
      // run https://huggingface.co/TheBloke/OpenHermes-2.5-Mistral-7B-GGUF with llama.cpp
      promptTemplate: llamacpp.prompt.ChatML,
      contextWindowSize: 4096,
      maxGenerationTokens: 512,
    })
    .withChatPrompt();

Changed

• breaking change: renamed response to rawResponse when using fullResponse: true setting.
• breaking change: renamed llamacpp.TextGenerator to llamacpp.CompletionTextGenerator.

Removed

• breaking change: removed .withTextPromptTemplate on LlamaCppCompletionModel.

v0.117.0

Added

• Predefined Llama.cpp GBNF grammars:

  • llamacpp.grammar.json: Restricts the output to JSON.
  • llamacpp.grammar.jsonArray: Restricts the output to a JSON array.
  • llamacpp.grammar.list: Restricts the output to a newline-separated list where each line starts with - .

• Llama.cpp structure generation support:

  const structure = await generateStructure(
    llamacpp
      .TextGenerator({
        // run openhermes-2.5-mistral-7b.Q4_K_M.gguf in llama.cpp
        maxGenerationTokens: 1024,
        temperature: 0,
      })
      .withTextPromptTemplate(ChatMLPrompt.instruction()) // needed for jsonStructurePrompt.text()
      .asStructureGenerationModel(jsonStructurePrompt.text()), // automatically restrict the output to JSON
  
    zodSchema(
      z.object({
        characters: z.array(
          z.object({
            name: z.string(),
            class: z
              .string()
              .describe("Character class, e.g. warrior, mage, or thief."),
            description: z.string(),
          })
        ),
      })
    ),
  
    "Generate 3 character descriptions for a fantasy role playing game. "
  );

v0.116.1

v0.116.1

v0.116.0

Added

• Semantic classifier. An easy way to determine a class of a text using embeddings. Example:

  import { SemanticClassifier, openai } from "modelfusion";
  
  const classifier = new SemanticClassifier({
    embeddingModel: openai.TextEmbedder({
      model: "text-embedding-ada-002",
    }),
    similarityThreshold: 0.82,
    clusters: [
      {
        name: "politics" as const,
        values: [
          "isn't politics the best thing ever",
          "why don't you tell me about your political opinions",
          "don't you just love the president",
          "don't you just hate the president",
          "they're going to destroy this country!",
          "they will save the country!",
        ],
      },
      {
        name: "chitchat" as const,
        values: [
          "how's the weather today?",
          "how are things going?",
          "lovely weather today",
          "the weather is horrendous",
          "let's go to the chippy",
        ],
      },
    ],
  });
  
  console.log(await classifier.classify("don't you love politics?")); // politics
  console.log(await classifier.classify("how's the weather today?")); // chitchat
  console.log(
    await classifier.classify("I'm interested in learning about llama 2")
  ); // null