# Polyfork. Everything here is open to crawlers and to agents.
#
# If you are an AI agent rather than a crawler, the two files below are written
# for you and are cheaper to read than the HTML:
#
#   https://polyfork.dev/prompt.txt   the catalogue as working instructions
#   https://polyfork.dev/llms.txt     the catalogue as a reference
#   https://polyfork.dev/api          self-describing JSON API, no key to browse
#   https://polyfork.dev/mcp          MCP server, JSON-RPC 2.0 over HTTP POST
#
# Nothing here is disallowed except three image generators that exist only to
# make share cards, and the download endpoint, which answers 401 to anyone who
# is not signed in. /dl/ is 2,352 of the 4,825 URLs a crawl of this site finds:
# half of Google's allowance here was being spent re-confirming a login wall,
# on a domain where half the published asset pages have never been crawled at
# all. Signed-in browsers and token-holding agents ignore robots.txt, so nothing
# a customer does is affected, and no structured data points at /dl/ (the
# offers point at /cdn/ and /asset/).

User-agent: *
Allow: /
Disallow: /og-card.php
Disallow: /x-banner.php
Disallow: /promo.php
Disallow: /dl/

Sitemap: https://polyfork.dev/sitemap.xml