feat(frontend): Render base64 images in artifact updates #109
Conversation
Updates frontend/src/script.ts to handle artifact updates containing file parts with base64 data (`bytes` field) and an image mimeType. Constructs a data URI and renders an <img> tag to display the image inline, improving usability for agents returning visual data.
Summary of ChangesHello @bustosjuan, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request significantly enhances the A2A Inspector's frontend by enabling the direct rendering of base64-encoded images within Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a feature to render base64 images directly in the A2A Inspector's frontend chat interface within artifact-update messages. The code changes involve modifying the AgentResponseEvent interface to include bytes and name properties for file parts, updating the artifact-update case in the event listener to handle base64 image rendering, adding a requiresHtmlRendering flag, and modifying the appendMessage function to sanitize HTML content. I have identified a potential security issue related to sanitizing the mime type and suggest adding a missing break statement in the switch case.
refactor(frontend): Use regex validation for image MIME types Replaces DOMPurify sanitization with regex validation for image MIME types used in base64 data URIs, addressing review feedback. Ensures only known safe image types (png, jpeg, gif, webp, svg+xml, bmp) are rendered inline, preventing potential manipulation or errors from overly strict sanitization. Uses the validated MIME type directly. DOMPurify is still used for sanitizing displayed text like alt attributes and fallback messages.
|
@bustosjuan Can you resolve the merge conflict? |
Resolved conflicts in frontend/src/script.ts by adopting main's FileContent type refactor while retaining the new regex-based validation and rendering logic for base64 images.
2 participants
Description
This PR introduces a new feature to the A2A Inspector's frontend chat interface: the ability to render images sent as base64-encoded strings directly within
artifact-updatemessages.Context
The A2A Inspector helps developers debug A2A agents. Agents often return visual artifacts, like plots, which can be encoded directly in the payload using base64.
Problem Addressed
Currently, if an agent sends an image artifact using the
bytesfield (base64) within afilepart, the inspector displays a generic "File received (image/png): Open Link" message. This forces the developer to manually copy the base64 data from the raw JSON (viewable in the debug console or message modal) and decode it externally to see the image. This process is cumbersome and disrupts the debugging workflow.Solution Implemented
The frontend TypeScript code (
frontend/src/script.ts) handling incomingagent_responseevents forartifact-updatehas been updated. It now checks if afilepart contains both abytesfield and an imagemimeType.If these conditions are met, the code dynamically constructs an
<img>tag with adata:URI source (data:[mimeType];base64,[bytes]) and renders it directly in the chat message bubble. Basic styling ensures the image fits within the message bounds. Thefile.namefield is used for thealttext. Existing logic for handling URIs or other part types remains unchanged.Benefits
Testing Done
artifact-updatemessages containing image data encoded in thefile.bytesfield (base64) withmimeType: "image/png".(Make sure to check the boxes in the actual PR template on GitHub)
CONTRIBUTINGGuide.bash scripts/format.shfrom the repository root to format)