ComfyUI SDK

A powerful TypeScript SDK for ComfyUI that provides type-safe workflow management, multi-instance support, and real-time execution monitoring.

Features

• 🔥 Type-safe Workflow Building: Build and validate workflows at compile time
• 🌐 Multi-Instance Support: Load balance across multiple ComfyUI instances
• 🔄 Real-time Monitoring: WebSocket integration for live execution updates
• 🛠️ Extension Support: Built-in support for ComfyUI-Manager and Crystools
• 🔒 Authentication Ready: Basic, Bearer and Custom auth support for secure setups

Quick Start

Table of Contents

1. Features
2. Installation
3. Core Concepts
4. Basic Usage
5. Advanced Usage
6. API Reference
7. Configuration
8. Error Handling
9. Best Practices
10. Contributing
11. License

Installation

Install the package using npm:

npm install @saintno/comfyui-sdk

Or using bun:

bun add @saintno/comfyui-sdk

Core Concepts

• ComfyApi: The main client for interacting with a single ComfyUI instance.
• ComfyPool: A manager for multiple ComfyUI instances, providing load balancing.
• PromptBuilder: A utility for constructing workflows with type-safe inputs and outputs.
• CallWrapper: A wrapper for API calls that provides comprehensive event handling and execution control.

Basic Usage

Setting up a ComfyApi instance

import { ComfyApi } from "@saintno/comfyui-sdk";

const api = new ComfyApi("http://localhost:8188", "client-id");
api.on("log", (ev) => console.log(ev.detail)); // Debug logs
await api
  .init(
    1000 // Retry 1000 times before giving up, default 10 [optional]
    1000 // Retry every 1000ms, default 1000ms [optional]
  ) // Initialize websocket
  .waitForReady(); // Wait for the client to be ready

Creating a PromptBuilder

import { PromptBuilder } from "@saintno/comfyui-sdk";
import workflowJson from "./workflow.json"; // Get from `Save (API Format)` or `Export (API Format)` from ComfyUI Web

const promptBuilder = new PromptBuilder(
  workflowJson,
  ["checkpoint", "positive", "negative", "seed", "steps"], // Input keys
  ["images"] // Output keys
)
  .setInputNode("checkpoint", "4.inputs.ckpt_name")
  .setInputNode("positive", "6.inputs.text")
  .setInputNode("negative", "7.inputs.text")
  .setInputNode("seed", "3.inputs.seed")
  .setInputNode("steps", "3.inputs.steps")
  .setInputNode("size", ["7.inputs.width", "7.inputs.height"]) // Bind multiple values to a single key
  .setOutputNode("images", "9");

Executing a Workflow

To avoid ComfyUI's caching and ensure different results with each call, use unique seed values.

import { CallWrapper } from "@saintno/comfyui-sdk";

const workflow = promptBuilder
  .input(
    "checkpoint",
    "SDXL/realvisxlV40_v40LightningBakedvae.safetensors",
    /**
     * Use the client's osType to encode the path
     *
     * For example, if the client's `osType` is "nt" (Windows), the path should be encoded as below
     * "SDXL\\realvisxlV40_v40LightningBakedvae.safetensors"
     */
    api.osType // This is optional, but recommended it you want to support multiple platforms
  )
  .input("positive", "A beautiful landscape")
  .input("negative", "blurry, text")
  .input("seed", 42)
  .input("steps", 20);

new CallWrapper(api, workflow)
  .onStart((promptId) => console.log(`Task ${promptId} started`))
  .onProgress((info, promptId) => console.log(`Task ${promptId} progress:`, info))
  .onFinished((data, promptId) => console.log(`Task ${promptId} finished:`, data))
  .run();

You can use api.getCheckpoints() to get the list of available checkpoints.

const client = new ComfyApi("http://localhost:8188");
await client.getCheckpoints().then(console.log);

Advanced Usage

ComfyUI Manager

Support interact with ComfyUI-Manager extesion, require ComfyUI-Manager to be installed in the ComfyUI instance.

const api = new ComfyApi("http://localhost:8189").init();
/**
 * Should wait for the client to be ready for checking supported extensions
 */
await api.waitForReady();

if (api.ext.manager.isSupported) {
  /**
   * Get the list of all extensions
   */
  await api.ext.manager.getExtensionList().then(console.log);
  //More methods are available, check the `api.ext.manager` reference for more details
}

ComfyUI Crystools

Support listen event from ComfyUI-Crystools extension for tracking system resources, require ComfyUI-Crystools to be installed in the ComfyUI instance.

const api = new ComfyApi("http://localhost:8189").init();
/**
 * Should wait for the client to be ready for checking supported extensions
 */
await api.waitForReady();

if (api.ext.monitor.isSupported) {
  /**
   * Listen to the system monitor event
   */
  api.ext.monitor.on("system_monitor", (ev) => {
    console.log(ev.detail);
  });
  /**
   * Current monitoring data
   */
  console.log(api.ext.monitor.monitorData);
}

Connection Pooling

import { ComfyPool, EQueueMode } from "@saintno/comfyui-sdk";

const pool = new ComfyPool(
  [new ComfyApi("http://localhost:8188", "node-1"), new ComfyApi("http://localhost:8189", "node-2")],
  //   "PICK_ZERO", Picks the client which has zero queue remaining. This is the default mode. (For who using along with ComfyUI web interface)
  //   "PICK_LOWEST", Picks the client which has the lowest queue remaining.
  //   "PICK_ROUTINE", Picks the client in a round-robin manner.
  EQueueMode.PICK_ZERO
);

pool.run(async (api, clientIdx) => {
  // Your workflow execution logic here
});

// Or execute multiple jobs in parallel
pool.batch([
  (api) => executeWorkflow(api, params1),
  (api) => executeWorkflow(api, params2)
  // ...
]);

Authentication

const api = new ComfyApi("http://localhost:8188", "client-id", {
  credentials: {
    type: "basic", // Can be "basic", "bearer_token" and "custom"
    username: "your-username",
    password: "your-password"
  }
});

API Reference

ComfyApi

• constructor(host: string, clientId?: string, opts?: { credentials?: BasicCredentials })
• queuePrompt(number: number, workflow: object): Promise<QueuePromptResponse | false>
• getQueue(): Promise<QueueResponse>
• getHistories(maxItems?: number): Promise<HistoryResponse>
• getSystemStats(): Promise<SystemStatsResponse>
• uploadImage(file: Buffer | Blob, fileName: string, config?: { override?: boolean, subfolder?: string }): Promise<{ info: ImageInfo; url: string } | false>

ComfyPool

• constructor(clients: ComfyApi[], mode: EQueueMode = EQueueMode.PICK_ZERO)
• run<T>(job: (client: ComfyApi, clientIdx?: number) => Promise<T>, weight?: number): Promise<T>
• batch<T>(jobs: Array<(client: ComfyApi, clientIdx?: number) => Promise<T>>, weight?: number): Promise<T[]>

PromptBuilder

• constructor(prompt: T, inputKeys: I[], outputKeys: O[])
• setInputNode(input: I, key: DeepKeys<T>): this
• setOutputNode(output: O, key: DeepKeys<T>): this
• input<V = string | number | undefined>(key: I, value: V): PromptBuilder<I, O, object>

CallWrapper

• constructor(client: ComfyApi, workflow: T)
• onPreview(fn: (ev: Blob, promptId?: string) => void): this
• onPending(fn: (promptId?: string) => void): this
• onStart(fn: (promptId?: string) => void): this
• onFinished(fn: (data: Record<keyof T["mapOutputKeys"], any>, promptId?: string) => void): this
• onFailed(fn: (err: Error, promptId?: string) => void): this
• onProgress(fn: (info: NodeProgress, promptId?: string) => void): this
• run(): Promise<Record<keyof T["mapOutputKeys"], any> | undefined | false>

Configuration

The library supports various configuration options, including:

• Setting up multiple ComfyUI instances
• Configuring authentication credentials
• Choosing execution modes for the connection pool

Refer to the individual class constructors for specific configuration options.

Error Handling

The library provides comprehensive error handling through the event system. Use the onFailed method of CallWrapper to catch and handle errors during execution.

Best Practices

1. Always use PromptBuilder to ensure type safety when constructing workflows.
2. Utilize ComfyPool for managing multiple ComfyUI instances to improve reliability and load distribution.
3. Implement proper error handling using the provided event callbacks.
4. Use batch method of ComfyPool for parallel execution of multiple workflows.

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

1. Fork the repository
2. Create your feature branch (git checkout -b feature/AmazingFeature)
3. Commit your changes (git commit -m 'Add some AmazingFeature')
4. Push to the branch (git push origin feature/AmazingFeature)
5. Open a Pull Request

License

This project is licensed under the MIT License. See the LICENSE file for details.

---

This README provides a comprehensive overview of the ComfyUI API Client library, including detailed usage examples, API references, and best practices. It should give users a solid understanding of how to use and configure the library for their needs.