Skip to content

Conversation

@markb-trustifi
Copy link

This PR resolves issue #663.
Right now Computer tool should implement each UI action as a separate method. These methods also don't have RunContext parameter opposite to regular tools. It makes impossible passing user context when executing UI actions.

The fix adds:

One common method for all UI actions with RunContext (contains user context) and ComputerUseCallItem (contains UI actions).
The fix keeps backward compatibility with the regular methods definition.

Invoke method implementation example:

const { computerTool, Computer, asInvokeComputer, isInvokeComputer  } = require("@openai/agents");
function makeComputer(): Computer {
	return asInvokeComputer({
		environment: 'browser',
		dimensions: [1, 1],
		invoke: async (runContext, toolCall) => {
			const action = toolCall.action;
			const ws = runContext.context.ws;
			const {promise, resolve, reject} = Promise.withResolvers();

			ws.once("message", (raw) => {
				let msg;
				try {
					msg = JSON.parse(raw);
				} catch (e) {
					return reject(new Error("Invalid message."));
				}
				if (!msg.screenshot) {
					return reject(new Error("Screenshot not found."));
				}
				resolve(msg.screenshot);
			});

			ws.send(JSON.stringify(action));

			return promise;
		}
	});
}
const comp = makeComputer();
const tool = computerTool({ computer: comp });

@changeset-bot
Copy link

changeset-bot bot commented Dec 12, 2025

🦋 Changeset detected

Latest commit: 69748c6

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 5 packages
Name Type
@openai/agents-core Minor
@openai/agents-extensions Minor
@openai/agents-openai Minor
@openai/agents-realtime Minor
@openai/agents Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant