AI Gateway - AI Gateway custom costs support cache tokens
<p>AI Gateway custom costs now support cache-read and cache-write token rates. This lets custom cost metrics reflect negotiated cache pricing across providers.</p> <p>Add <code>per_cache_read_token</code> or <code>per_cache_write_token</code> to the <code>cf-aig-custom-cost</code> header:</p> <figure class="nb-code-figure" data-nb-lang="json"><pre class="astro-code astro-code-themes github-light github-dark nb-shiki-c6xiwz" tabindex="0" data-language="json" data-nb-lang="json"><code><span class="line"><span class="nb-shiki-140thh">{</span></span> <span class="line"><span class="nb-shiki-dzsirb"> "per_token_in"</span><span class="nb-shiki-140thh">: </span><span class="nb-shiki-dzsirb">0.000001</span><span class="nb-shiki-140thh">,</span></span> <span class="line"><span class="nb-shiki-dzsirb"> "per_token_out"</span><span class="nb-shiki-140thh">: </span><span class="nb-shiki-dzsirb">0.000002</span><span class="nb-shiki-140thh">,</span></span> <span class="line"><span class="nb-shiki-dzsirb"> "per_cache_read_token"</span><span class="nb-shiki-140thh">: </span><span class="nb-shiki-dzsirb">0.0000001</span><span class="nb-shiki-140thh">,</span></span> <span class="line"><span class="nb-shiki-dzsirb"> "per_cache_write_token"</span><span class="nb-shiki-140thh">: </span><span class="nb-shiki-dzsirb">0.0000005</span></span> <span class="line"><span class="nb-shiki-140thh">}</span></span></code></pre></figure> <p>Cache-token pricing activates when either cache rate is present. An omitted cache rate defaults to <code>per_token_in</code>. If both cache rates are omitted, AI Gateway preserves the existing input and output calculation.</p> <p>Providers can include cache tokens within input tokens or report them separately. AI Gateway automatically accounts for these differences and prevents double-counting.</p> <p>For more information, refer to <a href="https://developers.cloudflare.com/ai-gateway/configuration/custom-costs/">Custom costs</a>.</p>