diff --git a/2_typescript/3_plugins/1_tools-provider/status-reports-and-warnings.md b/2_typescript/3_plugins/1_tools-provider/status-reports-and-warnings.md index 6a6524d..3e2722f 100644 --- a/2_typescript/3_plugins/1_tools-provider/status-reports-and-warnings.md +++ b/2_typescript/3_plugins/1_tools-provider/status-reports-and-warnings.md @@ -47,47 +47,3 @@ The following example shows how to implement a tool that waits for a specified n ``` Note status updates and warnings are only visible to the user. If you want the model to also see those messages, you should return them as part of the tool's return value. - -## Handling Aborts - -A prediction may be aborted by the user while your tool is still running. In such cases, you should handle the abort gracefully by handling the `AbortSignal` object passed as the second parameter to the tool's implementation function. - -```lms_code_snippet - title: "src/toolsProvider.ts" - variants: - TypeScript: - language: typescript - code: | - import { tool, Tool, ToolsProviderController } from "@lmstudio/sdk"; - import { z } from "zod"; - - export async function toolsProvider(ctl: ToolsProviderController) { - const tools: Tool[] = []; - - const fetchTool = tool({ - name: `fetch`, - description: "Fetch a URL using GET method.", - parameters: { url: z.string() }, - implementation: async ({ url }, { signal }) => { - const response = await fetch(url, { - method: "GET", - signal, // <-- Here, we pass the signal to fetch to allow cancellation - }); - if (!response.ok) { - return `Error: Failed to fetch ${url}: ${response.statusText}`; - } - const data = await response.text(); - return { - status: response.status, - headers: Object.fromEntries(response.headers.entries()), - data: data.substring(0, 1000), // Limit to 1000 characters - }; - }, - }); - tools.push(fetchTool); - - return tools; - } -``` - -You can learn more about `AbortSignal` in the [MDN documentation](https://developer.mozilla.org/en-US/docs/Web/API/AbortSignal). diff --git a/2_typescript/3_plugins/4_custom-configuration/config-ts.md b/2_typescript/3_plugins/4_custom-configuration/config-ts.md index 15a7d66..01bfe9e 100644 --- a/2_typescript/3_plugins/4_custom-configuration/config-ts.md +++ b/2_typescript/3_plugins/4_custom-configuration/config-ts.md @@ -51,7 +51,7 @@ By default, the plugin scaffold will create a `config.ts` file in the `src/` dir .build(); ``` -If you've added your config schematics manual, you will also need to register the configurations in your plugin's `index.ts` file. +If you've added your config schematics manually, you will also need to register the configurations in your plugin's `index.ts` file. This is done by calling `context.withConfigSchematics(configSchematics)` and `context.withGlobalConfigSchematics(globalConfigSchematics)` in the `main` function of your plugin. diff --git a/2_typescript/3_plugins/create-your-first-plugin.md b/2_typescript/3_plugins/create-your-first-plugin.md new file mode 100644 index 0000000..fe2a7a4 --- /dev/null +++ b/2_typescript/3_plugins/create-your-first-plugin.md @@ -0,0 +1,24 @@ +--- +title: "Create Your First Plugin" +sidebar_title: "Create Your First Plugin" +description: "Create your first LM Studio plugin using TypeScript and the LM Studio SDK" +index: 2 +--- + +```lms_private_beta +Plugin support is currently in private beta. [Join the beta here](https://forms.gle/ZPfGLMvVC6DbSRQm9). +``` + +To create a LM Studio plugin, + +1. Open LM Studio. +2. Press `⌘` `Shift` `R` on Mac or `Ctrl` `Shift` `R` on Windows/Linux. +3. Follow the instructions. + +If you don't know where to start, try "Tools Provider". You can add more "hooks" later as you need them. + +Anatomy + +Run in dev + +Next should read about ctl (control LM Studio) diff --git a/2_typescript/3_plugins/ctl-object.md b/2_typescript/3_plugins/ctl-object.md new file mode 100644 index 0000000..59c9b0b --- /dev/null +++ b/2_typescript/3_plugins/ctl-object.md @@ -0,0 +1,61 @@ +--- +title: "The `ctl` Object" +description: "Learn more about the `ctl` object available in LM Studio plugin hooks" +index: 3 +--- + +```lms_private_beta +Plugin support is currently in private beta. [Join the beta here](https://forms.gle/ZPfGLMvVC6DbSRQm9). +``` + +When you implement a plugin hook, such as a tools provider, a prompt preprocessor, or a generator, you will receive a `ctl` object as the first argument. + +The `ctl` object provides various methods and properties that allow you to interact with the LM Studio environment, access configuration, manage tools, and handle abort signals. + +The exact methods available are dependent on the type of plugin hook you are implementing. Below are some common methods and properties you might find useful: + +## LM Studio API Access + +Normally when using the LM Studio SDK, you will create, when make SDK, get Client from + +Use `ctl.client` to interact with the LM Studio API: + +```ts +// List loaded models +const models = await ctl.client.llm.listLoaded(); +console.log(`Found ${models.length} loaded models`); +``` + +## Configuration + +Get per-chat and application-wide configuration for your plugin: + +```ts +const config = ctl.getPluginConfig(configSchematics); +const globalConfig = ctl.getGlobalPluginConfig(globalConfigSchematics); +const value = config.get("fieldKey"); +``` + +For more details on creating and using configurations, see [Custom Configurations](./custom-configuration). + +## File System + +### `getWorkingDirectory()` + +Returns the working directory path where your plugin should place any generated files. + +## Cancellation Handling + +Use `ctl.abortSignal` with async operations to support cancellation: + +```ts +// Use with fetch requests +const response = await fetch("https://api.example.com/data", { + signal: ctl.abortSignal, +}); + +// Or register a callback for cleanup +ctl.onAborted(() => { + console.log("Operation cancelled - cleaning up..."); +}); +``` diff --git a/2_typescript/3_plugins/index.md b/2_typescript/3_plugins/index.md index 5a2fd0f..23905cf 100644 --- a/2_typescript/3_plugins/index.md +++ b/2_typescript/3_plugins/index.md @@ -30,7 +30,7 @@ lms dev Your plugin will appear in LM Studio's plugin list. Development mode automatically rebuilds and reloads your plugin when you make code changes. -You only need `lms dev` during development. When the plugin is installed, LM Studio automatically runs them as needed. Learn more about distributing and installing plugins in the [Sharing Plugins](./plugins/sharing) section. +You only need `lms dev` during development. When the plugin is installed, LM Studio automatically runs them as needed. Learn more about distributing and installing plugins in the [Sharing Plugins](./plugins/publish-plugins) section. ## Next Steps diff --git a/2_typescript/3_plugins/tutorial-fetch-web-page-plugin.md b/2_typescript/3_plugins/tutorial-fetch-web-page-plugin.md new file mode 100644 index 0000000..4256c0a --- /dev/null +++ b/2_typescript/3_plugins/tutorial-fetch-web-page-plugin.md @@ -0,0 +1,148 @@ +--- +title: "Tutorial: Make a Fetch Web Page Plugin" +sidebar_title: "Tutorial: Make a Fetch Web Page Plugin" +description: "Make a simple LM Studio plugin that allows the LLM to fetch and read web pages" +index: 7 +--- + +```lms_private_beta +Plugin support is currently in private beta. [Join the beta here](https://forms.gle/ZPfGLMvVC6DbSRQm9). +``` + +In this tutorial, we will create a simple LM Studio plugin that allows the LLM to fetch and read web pages. This plugin will demonstrate how to use tools providers, handle aborts, and define custom configurations. + +## Step 1: Create a New Plugin + +1. Open LM Studio. +2. Press `⌘` `Shift` `R` on Mac or `Ctrl` `Shift` `R` on Windows/Linux to open the plugin creation wizard. +3. Select "Tools Provider" as the starting point. +4. Give the plugin a name, such as "fetch-web-page". +5. Click "Next" and select a directory to create the plugin in. LM Studio will create a new directory within the selected directory. +6. Open the newly created plugin directory in your code editor. + +## Step 2: Install `npm` Package: `html-to-text` + +In this plugin, we will use the `html-to-text` package to convert HTML content to plain text. To install this package, run the following command in your plugin directory: + +```bash +cd /path/to/your/plugin +npm install html-to-text +npm install --save-dev @types/html-to-text +``` + +When your plugins is installed, we will automatically install the npm dependencies for the user. Your user do not need to have Node.js/npm installed. + +Learn more about `npm` dependencies in the [Using `npm` Dependencies](./dependencies) section. + +## Step 3: Define Configurations + +For this plugin, we would like to allow the user to configure the User-Agent header used when fetching web pages. This is done by first defining the configuration schema. + +Replace the code in `src/config.ts`: + +```ts +import { createConfigSchematics } from "@lmstudio/sdk"; + +/** + * This is the schematics of the per-chat configuration for your plugin. Configurations of this + * type will be saved with each chat and can be different for each chat. + */ +export const configSchematics = createConfigSchematics().build(); + +/** + * This is the schematics of the application-wide configuration for your plugin. Configurations of + * this type will be saved globally and will be the same for all chats. This is useful for things + * like global settings or API keys that should be consistent across all chats. + */ +export const globalConfigSchematics = createConfigSchematics() + .field( + "userAgent", // The key of the field. + "string", + { + displayName: "User Agent", + subtitle: + "The User-Agent header to use when fetching web pages. This helps identify your requests to web servers.", + }, + "Mozilla/5.0 (compatible; LMStudio-FetchWebsite/1.0)", // Default Value + ) + .build(); +``` + +## Step 4: Implement the Tools Provider + +A tools provider is a function that when called, will return a list of tools that the LLM can use. The name and the description of the tool is provided to the LLM, and LLM will choose which tool to use based on the task at hand. + +Replace the code in `src/toolsProvider.ts` with the following: + +```ts +import { tool, Tool, ToolsProviderController } from "@lmstudio/sdk"; +import { z } from "zod"; +import { globalConfigSchematics } from "./config"; +import { convert as convertHtmlToText } from "html-to-text"; + +// See details about defining tools in the documentation: +// https://lmstudio.ai/docs/typescript/agent/tools + +export async function toolsProvider(ctl: ToolsProviderController) { + // Access the global configuration for the plugin. + const globalConfig = ctl.getGlobalPluginConfig(globalConfigSchematics); + + const tools: Tool[] = []; + + const fetchWebsiteTool = tool({ + name: "fetchWebsite", + description: "Fetch the content of a web page given its URL.", + parameters: { url: z.string() }, + implementation: async ({ url }) => { + const response = await fetch(url, { + headers: { + "User-Agent": globalConfig.get("userAgent"), + }, + }); + if (!response.ok) { + return `Error: Unable to fetch the URL. Status code: ${response.status}`; + } + const rawHtml = await response.text(); + // Use the html-to-text package to convert HTML to plain text. + return convertHtmlToText(rawHtml); + }, + }); + tools.push(fetchWebsiteTool); + + return tools; +} +``` + +## Step 5: Run the Plugin in Development Mode + +To use a plugin, you can either install the plugin, or run it in the "development mode". To run the plugin in development mode, run the following command in your plugin directory: + +```bash +lms run dev +``` + +It will start the plugin development server and reload the plugin automatically when you make code changes. + +Once the development server is running, you can enable the plugin in LM Studio. Load a model, and try to ask it to fetch a web page, for example: + +``` +What is the latest post on https://lmstudio.ai/blog? +``` + +The LLM model should automatically use the `fetchWebsite` tool to fetch the content of the web page and provide an answer based on the content. + +If the model is having a hard time forming a correct tool call, you can try to use a model that has better tool calling capabilities, such as `gpt-oss-20b`. You can download the model using the command: + +```bash +lms get openai/gpt-oss-20b +``` + +## Step 6: Distributing the Plugin + +To share the plugin with other users, you can publish using the command: + +```bash +lms push +``` + +Learn more about distributing and installing plugins in the [Sharing Plugins](./publish-plugins) section.