LLM Streaming

@humanspeak/svelte-markdown handles real-time streaming from Large Language Models (LLMs) like ChatGPT, Claude, Gemini, and other AI assistants out of the box. As tokens arrive via Server-Sent Events (SSE) or WebSocket connections, you can write them directly to the component instance with writeChunk() and the rendered markdown updates instantly.

This is also the path for rendering streaming agent output that mixes markdown with rich HTML (designs, dashboards, interactive artifacts). Built-in sanitization runs per token as it is emitted, so a partial <script> or <a href="javascript:..."> arriving mid-stream is sanitized before render. Half-open tags buffer until the parser sees enough to emit a well-formed token, so progressive HTML renders without flicker.

How It Works

LLM APIs stream responses token-by-token. Each token is a small chunk of text — sometimes a word, sometimes a partial word, sometimes punctuation or whitespace. The typical integration pattern is:

1. The LLM API sends tokens via Server-Sent Events (SSE) or a streaming HTTP response.
2. Your application accumulates tokens into a growing markdown string, either manually or through writeChunk().
3. SvelteMarkdown re-parses and re-renders the full source on each update.
4. Svelte’s fine-grained reactivity ensures only the changed DOM nodes are updated.

The component works reactively by default. For best streaming performance, pass streaming={true} to enable smart token diffing — the component re-parses the full source for correctness but only updates DOM nodes for tokens that actually changed, keeping per-update cost under 2ms regardless of document size.

Note: streaming is automatically disabled when async extensions (e.g., Mermaid) are used, since async walkTokens callbacks are incompatible with the synchronous diffing path. A console warning is logged in this case.

Basic Usage

Preferred: imperative chunk writes

<script lang="ts">
    import SvelteMarkdown from '@humanspeak/svelte-markdown'
    import type { StreamingChunk } from '@humanspeak/svelte-markdown'

    let markdown:
        | {
              writeChunk: (chunk: StreamingChunk) => void
              resetStream: (nextSource?: string) => void
          }
        | undefined

    async function streamFromAPI() {
        const response = await fetch('/api/chat', { method: 'POST', body: '...' })
        if (!response.body) throw new Error('No response body')
        const reader = response.body.getReader()
        const decoder = new TextDecoder()

        markdown?.resetStream('')

        while (true) {
            const { done, value } = await reader.read()
            if (done) break
            markdown?.writeChunk(decoder.decode(value, { stream: true }))
        }
    }
</script>

<SvelteMarkdown bind:this={markdown} source="" streaming={true} />

<script lang="ts">
    import SvelteMarkdown from '@humanspeak/svelte-markdown'
    import type { StreamingChunk } from '@humanspeak/svelte-markdown'

    let markdown:
        | {
              writeChunk: (chunk: StreamingChunk) => void
              resetStream: (nextSource?: string) => void
          }
        | undefined

    async function streamFromAPI() {
        const response = await fetch('/api/chat', { method: 'POST', body: '...' })
        if (!response.body) throw new Error('No response body')
        const reader = response.body.getReader()
        const decoder = new TextDecoder()

        markdown?.resetStream('')

        while (true) {
            const { done, value } = await reader.read()
            if (done) break
            markdown?.writeChunk(decoder.decode(value, { stream: true }))
        }
    }
</script>

<SvelteMarkdown bind:this={markdown} source="" streaming={true} />

writeChunk() accepts:

• string for append mode
• { value, offset } for websocket-style offset patches

The first successful write locks the stream into one mode until resetStream(), a streamId change, or a source prop reset. If an offset patch skips ahead, missing positions are padded with spaces.

With websocket-style offset chunks

markdown?.writeChunk({ value: 'Hel', offset: 0 })
markdown?.writeChunk({ value: 'lo', offset: 3 })
markdown?.writeChunk({ value: ' world', offset: 5 })

markdown?.writeChunk({ value: 'Hel', offset: 0 })
markdown?.writeChunk({ value: 'lo', offset: 3 })
markdown?.writeChunk({ value: ' world', offset: 5 })

Offset chunks use overwrite semantics, not insert semantics. Each patch writes value starting at offset, preserves any trailing content after the overwritten span, and pads skipped gaps with spaces.

This means out-of-order websocket delivery is fine:

markdown?.writeChunk({ value: ' world', offset: 5 })
markdown?.writeChunk({ value: 'Hello', offset: 0 })

markdown?.writeChunk({ value: ' world', offset: 5 })
markdown?.writeChunk({ value: 'Hello', offset: 0 })

There is no delete or truncate behavior in offset mode, and offsets must be non-negative safe integers.

Like append mode, the first successful write locks the stream shape until resetStream(), a streamId change, or a source prop reset.

Resetting Between Messages

The streaming buffer, the incremental parser, and the input-mode lock are per-component-instance state. They outlive any single message. If one instance is reused for a second stream without being reset, the new stream starts on top of the previous message’s buffer.

This is easy to hit in a chat transcript, because Svelte reuses the component instance whenever it is not keyed by message identity:

<!-- ⚠️ Broken: one recycled instance, no reset between messages -->
{#each messages as message}
    <SvelteMarkdown bind:this={markdown} source="" streaming={true} />
{/each}

<!-- ⚠️ Broken: one recycled instance, no reset between messages -->
{#each messages as message}
    <SvelteMarkdown bind:this={markdown} source="" streaming={true} />
{/each}

Holding source="" for the whole conversation means the source prop never changes, so nothing ever triggers the implicit reset. The symptoms differ by mode:

• Append mode — the next message renders as previous message + new message.
• Offset mode — writes overwrite in place without truncating, so the previous message’s tail survives past the end of the new one. It does not self-correct until the new message grows longer than the old one.
• Either mode — the previous stream’s input-mode lock is still in force, so the first chunk of the new stream is dropped with a warning if it uses the other chunk type.

Pass a streamId that changes per message. Whenever its value changes, the component drops the buffer, any pending unflushed chunk, the parser, and the mode lock, then rebaselines on the current source:

<!-- ✅ Correct: streamId identifies the stream -->
{#each messages as message}
    <SvelteMarkdown bind:this={markdown} source="" streaming={true} streamId={message.id} />
{/each}

<!-- ✅ Correct: streamId identifies the stream -->
{#each messages as message}
    <SvelteMarkdown bind:this={markdown} source="" streaming={true} streamId={message.id} />
{/each}

streamId accepts a string or number and is ignored when streaming is false.

There are three equivalent ways to start a clean stream, in rough order of preference:

1. streamId={message.id} — declarative, and works even when the instance is recycled.
2. A keyed each — {#each messages as message (message.id)} gives each message its own instance, so there is nothing to reset. Use this when the key identifies the message rather than a slot in a virtual list.
3. markdown.resetStream() — imperative; call it before the new stream’s first writeChunk().

Note: resetting by changing source is not safe to combine with a same-tick writeChunk(). writeChunk() runs synchronously, while a prop-driven reset runs in an effect, so the write would land on the old buffer and then be wiped. streamId and resetStream() both take effect immediately and have no such ordering hazard.

SDK Examples

Choose between chunked mode (imperative writeChunk() API) and concat mode (reactive source += chunk pattern):

With the Anthropic SDK (Claude)

<script>
    import SvelteMarkdown from '@humanspeak/svelte-markdown'
    import Anthropic from '@anthropic-ai/sdk'

    let markdown

    async function streamResponse(prompt) {
        const client = new Anthropic()

        const stream = client.messages.stream({
            model: 'claude-sonnet-4-20250514',
            max_tokens: 1024,
            messages: [{ role: 'user', content: prompt }]
        })

        markdown?.resetStream('')

        for await (const event of stream) {
            if (
                event.type === 'content_block_delta' &&
                event.delta.type === 'text_delta'
            ) {
                markdown?.writeChunk(event.delta.text)
            }
        }
    }
</script>

<SvelteMarkdown bind:this={markdown} source="" streaming={true} />

<script>
    import SvelteMarkdown from '@humanspeak/svelte-markdown'
    import Anthropic from '@anthropic-ai/sdk'

    let markdown

    async function streamResponse(prompt) {
        const client = new Anthropic()

        const stream = client.messages.stream({
            model: 'claude-sonnet-4-20250514',
            max_tokens: 1024,
            messages: [{ role: 'user', content: prompt }]
        })

        markdown?.resetStream('')

        for await (const event of stream) {
            if (
                event.type === 'content_block_delta' &&
                event.delta.type === 'text_delta'
            ) {
                markdown?.writeChunk(event.delta.text)
            }
        }
    }
</script>

<SvelteMarkdown bind:this={markdown} source="" streaming={true} />

With the OpenAI SDK (ChatGPT)

<script>
    import SvelteMarkdown from '@humanspeak/svelte-markdown'
    import OpenAI from 'openai'

    let markdown

    async function streamResponse(prompt) {
        const client = new OpenAI()

        const stream = await client.chat.completions.create({
            model: 'gpt-4o',
            messages: [{ role: 'user', content: prompt }],
            stream: true
        })

        markdown?.resetStream('')

        for await (const chunk of stream) {
            const delta = chunk.choices[0]?.delta?.content
            if (delta) {
                markdown?.writeChunk(delta)
            }
        }
    }
</script>

<SvelteMarkdown bind:this={markdown} source="" streaming={true} />

<script>
    import SvelteMarkdown from '@humanspeak/svelte-markdown'
    import OpenAI from 'openai'

    let markdown

    async function streamResponse(prompt) {
        const client = new OpenAI()

        const stream = await client.chat.completions.create({
            model: 'gpt-4o',
            messages: [{ role: 'user', content: prompt }],
            stream: true
        })

        markdown?.resetStream('')

        for await (const chunk of stream) {
            const delta = chunk.choices[0]?.delta?.content
            if (delta) {
                markdown?.writeChunk(delta)
            }
        }
    }
</script>

<SvelteMarkdown bind:this={markdown} source="" streaming={true} />

With fetch and Server-Sent Events

<script>
    import SvelteMarkdown from '@humanspeak/svelte-markdown'

    let markdown

    async function streamFromAPI(prompt) {
        const response = await fetch('/api/chat', {
            method: 'POST',
            body: JSON.stringify({ prompt }),
            headers: { 'Content-Type': 'application/json' }
        })

        if (!response.ok) throw new Error(`HTTP ${response.status}`)

        const reader = response.body.getReader()
        const decoder = new TextDecoder()

        markdown?.resetStream('')

        try {
            while (true) {
                const { done, value } = await reader.read()
                if (done) break
                markdown?.writeChunk(decoder.decode(value, { stream: true }))
            }
        } finally {
            reader.releaseLock()
        }
    }
</script>

<SvelteMarkdown bind:this={markdown} source="" streaming={true} />

<script>
    import SvelteMarkdown from '@humanspeak/svelte-markdown'

    let markdown

    async function streamFromAPI(prompt) {
        const response = await fetch('/api/chat', {
            method: 'POST',
            body: JSON.stringify({ prompt }),
            headers: { 'Content-Type': 'application/json' }
        })

        if (!response.ok) throw new Error(`HTTP ${response.status}`)

        const reader = response.body.getReader()
        const decoder = new TextDecoder()

        markdown?.resetStream('')

        try {
            while (true) {
                const { done, value } = await reader.read()
                if (done) break
                markdown?.writeChunk(decoder.decode(value, { stream: true }))
            }
        } finally {
            reader.releaseLock()
        }
    }
</script>

<SvelteMarkdown bind:this={markdown} source="" streaming={true} />

Performance Characteristics

We measured render performance across different streaming speeds and chunking strategies using the interactive streaming demo:

All render times stay well under the 16.7ms frame budget (60fps), meaning the browser has time to paint every frame without jank. Even at 100 characters per second in character mode (a worst-case scenario far beyond real LLM speeds), average render time remains under 5ms.

How Render Time Scales

Render time grows linearly with document length because the full markdown source is re-parsed on each update. For a typical LLM response (~2,000 characters), the overhead is negligible:

• 0-500 chars: <1ms per render
• 500-1,000 chars: ~2-3ms per render
• 1,000-2,000 chars: ~5-7ms per render
• 2,000+ chars: ~7-10ms per render

For very long documents (10,000+ characters), consider the optimization strategies below.

Best Practices

1. Prefer Word-Level Chunking

If you control the chunking strategy (e.g., in a custom SSE endpoint), emit tokens at word boundaries rather than individual characters. This reduces the total number of re-renders while producing the same visual result:

// Server-side: buffer tokens and emit at word boundaries
let buffer = ''
for await (const token of llmStream) {
    buffer += token
    if (buffer.endsWith(' ') || buffer.endsWith('\n')) {
        controller.enqueue(encoder.encode(buffer))
        buffer = ''
    }
}
if (buffer) controller.enqueue(encoder.encode(buffer))

// Server-side: buffer tokens and emit at word boundaries
let buffer = ''
for await (const token of llmStream) {
    buffer += token
    if (buffer.endsWith(' ') || buffer.endsWith('\n')) {
        controller.enqueue(encoder.encode(buffer))
        buffer = ''
    }
}
if (buffer) controller.enqueue(encoder.encode(buffer))

2. Use Token Caching for Chat History

When displaying a conversation with multiple messages, previously completed messages are re-rendered on every update. Enable token caching so that completed messages skip re-parsing:

<script>
    import SvelteMarkdown from '@humanspeak/svelte-markdown'

    let messages = $state([])
</script>

{#each messages as message}
    <!-- Completed messages hit the token cache automatically -->
    <SvelteMarkdown source={message.content} />
{/each}

<script>
    import SvelteMarkdown from '@humanspeak/svelte-markdown'

    let messages = $state([])
</script>

{#each messages as message}
    <!-- Completed messages hit the token cache automatically -->
    <SvelteMarkdown source={message.content} />
{/each}

3. Debounce for Extremely Fast Streams

If your LLM stream is unusually fast (100+ tokens/second) and you notice frame drops, you can batch updates using requestAnimationFrame:

let pending = ''
let rafScheduled = false

function onToken(token) {
    pending += token
    if (!rafScheduled) {
        rafScheduled = true
        requestAnimationFrame(() => {
            source += pending
            pending = ''
            rafScheduled = false
        })
    }
}

let pending = ''
let rafScheduled = false

function onToken(token) {
    pending += token
    if (!rafScheduled) {
        rafScheduled = true
        requestAnimationFrame(() => {
            source += pending
            pending = ''
            rafScheduled = false
        })
    }
}

This coalesces multiple tokens into a single render per frame, reducing total renders from ~100/sec to ~60/sec while maintaining smooth visual output.

Estimating Streaming Costs

When building production LLM streaming UIs, understanding token costs is as important as render performance. Each streamed token has a price that varies by model, provider, and whether it’s an input or output token. ModelPricing.ai provides a pricing estimation API that covers all major LLM providers — useful for displaying real-time cost tracking alongside your streamed responses, setting usage budgets, or building cost-aware model selection into your application.

Try It Live

Experiment with different streaming speeds, jitter, chunk modes, and websocket-style offset patch simulation in the interactive LLM streaming demo.

Related

• Security — XSS defaults applied per token during streaming
• Token Caching — cache parsed tokens for instant re-renders
• Getting Started — installation and basic usage
• SvelteMarkdown API — full prop reference
• ModelPricing.ai — LLM pricing estimation API for cost tracking