How to Render Markdown and Images With ChatGPT Plugins

When developing a ChatGPT plugin, you might want ChatGPT to render returned items as links or images inline with it’s response. For example, if you create a ChatGPT plugin which returns information about products on your website, you might want ChatGPT to also render correct links to products referenced. Fortunately, integrating this functionality into a plugin is relatively easy.

ChatGPT Supports Markdown

If you take the time to disect the ChatGPT UI a bit, you’ll notice that it uses Tailwind. More specifically, the ChatGPT response component uses the prose HTML class. This class comes specifically from the Tailwind CSS Typography Plugin which renders markdown contained within the child element of the given HTML node.

In addition to markdown, ChatGPT is capable of rendering formatted and highlighted code snippets, as well as images. You can try this out yourself by prompting ChatGPT with a link to an image, and asking it to provide the proper markdown representation of the image for you:

You can exploit ChatGPT’s rendering abilities to render your own images and links from directly within the UI.

Rendering Links and Images

The best way to render links and images from within the UI is by returning full URLs from your plugin API and prompting ChatGPT to render them for you. Typically, ChatGPT will take care of formatting links for you; however, if you have a specific output format in mind, you should take advantage of the description fields in your OpenAPI spec to tell ChatGPT how to display the output of specific endpoints. You can read more about how the OpenAPI spec affects the ChatGPT prompt in our Plugin Prompt Guide.

As an example, this plugin returns fully qualified URLs to businesses on https://acquire.com. ChatGPT takes those URLs and wraps them in valid markdown links before returning them to the user:

This plugin didn’t require any additional prompting. Convincing ChatGPT to render images can be a bit trickier. The best way to achieve this is to return an image URL directly from your plugin API, and then prompt ChatGPT to render the output as a markdown image:

This example in particular explicitly tells ChatGPT to wrap all image URLs in a markdown representation to “trick” it into rendering them directly. You might find that ChatGPT pushes back a bit if you ask it directly to render the image for you. The best strategy for controlling the format of your plugin outputs are to ask ChatGPT to write valid markdown representations of your outputs.

After rendering links, the ChatGPT response will contain Open Graph representations of each returned URL. For the best experience possible, ensure that any links rendered from within the ChatGPT UI correctly configure the Open Graph meta tags.

Rendering Tables

An under utilized feature of ChatGPT’s ability to render markdown is it’s ability to render tables. For example, the PortfolioPilot plugin takes advantage of this fact to present information about a given portfolio to a user:

This is accomplished by passing the data in a structured format to ChatGPT, and then instructing it to render data as a table in markdown.

Conclusion

You can exploit ChatGPT’s native markdown rendering abilities to return images, links, and more directly from your plugin. In the future, I predict the plugin development experience will evolve to support richer UIs. I believe plugin developers will have more control over what ChatGPT can display. However, for now, markdown support provides a decent experience for your plugin users.