AvalonAvalon
GitHub

Agent Optimization Plugin

Optimize your Avalon site for AI agents, search engines, and social sharing with automatic JSON-LD, llms.txt, and markdown content negotiation.

Overview

The @useavalon/agent-optimization plugin enhances your Avalon site for AI agents and search engines by providing:

  • JSON-LD Structured Data — Automatic Schema.org WebPage injection
  • llms.txt Generation — Machine-readable site index for AI crawlers
  • Markdown Content Negotiation — Serve markdown to agents that request it
  • Sitemap Generation — XML sitemaps for search engines

Installation

The plugin is included in the Avalon monorepo. Add it to your Vite config:

// vite.config.ts
import { defineConfig } from 'vite';
import { avalon } from '@useavalon/avalon';
import { agentOptimization } from '@useavalon/agent-optimization';

export default defineConfig({
  plugins: [
    avalon(),
    agentOptimization({
      siteUrl: 'https://example.com',
      siteName: 'My Avalon Site',
    }),
  ],
});

Configuration

interface AgentOptimizationConfig {
  /** Base URL of your site (required for absolute URLs) */
  siteUrl: string;
  
  /** Site name for structured data */
  siteName?: string;
  
  /** Enable JSON-LD injection (default: true) */
  jsonLd?: boolean;
  
  /** Enable llms.txt generation (default: true) */
  llmsTxt?: boolean;
  
  /** Enable markdown content negotiation (default: true) */
  markdown?: boolean;
  
  /** Enable sitemap generation (default: true) */
  sitemap?: boolean;
  
  /** Routes to exclude from llms.txt and sitemap */
  exclude?: string[];
}

JSON-LD Structured Data

The plugin automatically injects Schema.org JSON-LD for pages with metadata:

<script type="application/ld+json">
{
  "@context": "https://schema.org",
  "@type": "WebPage",
  "url": "https://example.com/blog/hello-world",
  "name": "Hello World",
  "description": "My first blog post.",
  "image": "/og-image.png"
}
</script>

How It Works

  1. The plugin intercepts HTML responses
  2. Extracts <title> and <meta name="description"> from the HTML
  3. Builds a WebPage JSON-LD object
  4. Injects it before </head>

Manual JSON-LD

For custom structured data types, use the exported helpers:

import { 
  buildWebPageJsonLd, 
  buildWebSiteJsonLd, 
  injectJsonLd 
} from '@useavalon/agent-optimization';

// WebPage JSON-LD
const pageJsonLd = buildWebPageJsonLd(
  { title: 'My Page', description: 'Description', openGraph: { image: '/img.png' } },
  'https://example.com/my-page'
);

// WebSite JSON-LD
const siteJsonLd = buildWebSiteJsonLd('https://example.com', 'My Site');

// Inject into HTML
const html = injectJsonLd(originalHtml, pageJsonLd);

llms.txt Generation

The plugin generates /llms.txt — a machine-readable index of your site for AI agents:

# My Avalon Site
> A multi-framework islands architecture site.

## Pages

- [Home](https://example.com/): Welcome to our site
- [About](https://example.com/about): Learn about us
- [Blog](https://example.com/blog): Latest articles

## Documentation

- [Getting Started](https://example.com/docs/getting-started): Quick start guide
- [API Reference](https://example.com/docs/api): Full API documentation

Full Content Version

The plugin also generates /llms-full.txt with complete page content in markdown format, useful for AI agents that want to index your entire site.

Configuration

agentOptimization({
  siteUrl: 'https://example.com',
  llmsTxt: {
    enabled: true,
    title: 'My Site',
    description: 'Site description for AI agents',
    sections: [
      { name: 'Pages', pattern: '^/$|^/about' },
      { name: 'Documentation', pattern: '^/docs/' },
      { name: 'Blog', pattern: '^/blog/' },
    ],
  },
});

Markdown Content Negotiation

When a client sends Accept: text/markdown, the plugin converts HTML responses to clean markdown:

curl -H "Accept: text/markdown" https://example.com/docs/getting-started

Response:

---
title: "Getting Started"
description: "Quick start guide for Avalon"
---

# Getting Started

Welcome to Avalon! This guide will help you...

## Installation

Run the following command:

\`\`\`bash
npm install @useavalon/avalon
\`\`\`

How It Works

  1. Checks the Accept header for text/markdown
  2. Renders the page as HTML normally
  3. Converts HTML to markdown using smart extraction
  4. Adds YAML frontmatter from page metadata
  5. Returns with Content-Type: text/markdown

Sitemap Generation

The plugin generates /sitemap.xml for search engines:

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <url>
    <loc>https://example.com/</loc>
    <lastmod>2024-01-15</lastmod>
    <changefreq>weekly</changefreq>
    <priority>1.0</priority>
  </url>
  <url>
    <loc>https://example.com/docs/getting-started</loc>
    <lastmod>2024-01-10</lastmod>
    <changefreq>monthly</changefreq>
    <priority>0.8</priority>
  </url>
</urlset>

API Reference

Exports

// Plugin
export { agentOptimization } from '@useavalon/agent-optimization';

// JSON-LD helpers
export { buildWebPageJsonLd, buildWebSiteJsonLd, injectJsonLd } from '@useavalon/agent-optimization';

// Markdown conversion
export { htmlToMarkdown, buildFrontMatter, shouldServeMarkdown } from '@useavalon/agent-optimization';

// llms.txt generation
export { buildLlmsTxt, buildLlmsFullTxt } from '@useavalon/agent-optimization';

// Types
export type { PageMetadata, AgentOptimizationConfig } from '@useavalon/agent-optimization';

buildWebPageJsonLd

function buildWebPageJsonLd(metadata: PageMetadata, url: string): object

Builds a Schema.org WebPage JSON-LD object from page metadata.

injectJsonLd

function injectJsonLd(html: string, jsonLd: object): string

Injects a JSON-LD script tag into HTML before </head>.

htmlToMarkdown

function htmlToMarkdown(html: string): string

Converts HTML to clean markdown, extracting content from #app, <main>, or <body>.

Best Practices

  1. Always set siteUrl — Required for absolute URLs in structured data and sitemaps.

  2. Include page metadata — JSON-LD is only injected for pages with title or description.

  3. Use semantic HTML — The markdown converter works best with proper heading hierarchy and semantic elements.

  4. Exclude private routes — Use the exclude option for admin pages or authenticated routes.

agentOptimization({
  siteUrl: 'https://example.com',
  exclude: ['/admin/*', '/api/*', '/preview/*'],
});

Related