Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat (docs): @friendliai/ai-provider v0.2 update #4222

Merged
merged 3 commits into from
Jan 2, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion content/docs/02-foundations/02-providers-and-models.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -50,15 +50,16 @@ The open-source community has created the following providers:

- [Ollama Provider](/providers/community-providers/ollama) (`ollama-ai-provider`)
- [ChromeAI Provider](/providers/community-providers/chrome-ai) (`chrome-ai`)
- [AnthropicVertex Provider](/providers/community-providers/anthropic-vertex-ai) (`anthropic-vertex-ai`)
- [FriendliAI Provider](/providers/community-providers/friendliai) (`@friendliai/ai-provider`)
- [Portkey Provider](/providers/community-providers/portkey) (`@portkey-ai/vercel-provider`)
- [Cloudflare Workers AI Provider](/providers/community-providers/cloudflare-workers-ai) (`workers-ai-provider`)
- [OpenRouter Provider](/providers/community-providers/openrouter) (`@openrouter/ai-sdk-provider`)
- [Crosshatch Provider](/providers/community-providers/crosshatch) (`@crosshatch/ai-provider`)
- [Mixedbread Provider](/providers/community-providers/mixedbread) (`mixedbread-ai-provider`)
- [Voyage AI Provider](/providers/community-providers/voyage-ai) (`voyage-ai-provider`)
- [Mem0 Provider](/providers/community-providers/mem0)(`@mem0/vercel-ai-provider`)
- [LLamaCpp Provider](/providers/community-providers/llama-cpp) (`llamacpp-ai-provider`)
- [AnthropicVertex Provider](/providers/community-providers/anthropic-vertex-ai) (`anthropic-vertex-ai`)

## Model Capabilities

Expand Down
41 changes: 30 additions & 11 deletions content/providers/03-community-providers/08-friendliai.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -82,13 +82,13 @@ This feature is available with both `generateText` and `streamText` functions.

For a deeper understanding of how to effectively use regex patterns with LLMs, check out our detailed guide in the [Structured Output LLM Agents](https://friendli.ai/blog/structured-output-llm-agents) blog post.

```ts highlight="1,6"
```ts highlight="6"
import { friendli } from '@friendliai/ai-provider';
import { generateText } from 'ai';

const { text } = await generateText({
model: friendli('meta-llama-3.1-8b-instruct', {
regex: '[\n ,.?!0-9\uac00-\ud7af]*',
regex: new RegExp('[\n ,.?!0-9\uac00-\ud7af]*'),
}),
prompt: 'Who is the first king of the Joseon Dynasty?',
});
Expand All @@ -106,17 +106,18 @@ import { generateObject } from 'ai';
import { z } from 'zod';

const { object } = await generateObject({
model: friendli('meta-llama-3.1-8b-instruct'),
model: friendli('meta-llama-3.1-70b-instruct'),
schemaName: 'CalendarEvent',
schema: z.object({
brand: z.string(),
model: z.string(),
car_type: z.enum(['sedan', 'suv', 'truck', 'coupe']),
name: z.string(),
date: z.string(),
participants: z.array(z.string()),
}),
prompt:
'Generate a JSON with the brand, model and car_type of the most iconic car from the 90s',
system: 'Extract the event information.',
prompt: 'Alice and Bob are going to a science fair on Friday.',
});

console.log(JSON.stringify(object, null, 2));
console.log(object);
```

### Example: Using built-in tools
Expand Down Expand Up @@ -146,16 +147,34 @@ for await (const textPart of result.textStream) {

### Example: Generating text with Dedicated Endpoints

To use a custom model via a dedicated endpoint, you can use the `friendli.dedicated` instance with the endpoint id, e.g. `zbimjgovmlcb`
To use a custom model via a dedicated endpoint, you can use the `friendli` instance with the endpoint id, e.g. `zbimjgovmlcb`

```ts
import { friendli } from '@friendliai/ai-provider';
import { generateText } from 'ai';

const { text } = await generateText({
model: friendli.dedicated('YOUR_ENDPOINT_ID'),
model: friendli('YOUR_ENDPOINT_ID'),
prompt: 'What is the meaning of life?',
});

console.log(text);
```

You can use the code below to force requests to dedicated endpoints. By default, they are auto-detected.

```ts highlight="5,6,7"
import { friendli } from '@friendliai/ai-provider';
import { generateText } from 'ai';

const { text } = await generateText({
model: friendli("YOUR_ENDPOINT_ID", {
endpoint: "dedicated",
});
prompt: 'What is the meaning of life?',
});

console.log(text);
```

FriendliAI language models can also be used in the `streamText`, `generateObject`, `streamObject`, and `streamUI` functions.
Expand Down
Loading