Trey Piepmeier Profile Photo

Trey Piepmeier

treypiepmeier.com

  • About
  • Archive
  • Photos
    Oct 10, 2023 ∞

    Blocking all the things with robots.txt

    • Adactio: Journal—Permission
    • Sites scramble to block ChatGPT web crawler after instructions emerge | Ars Technica
    • GitHub - svendahlstrand/plugin-custom-robots

    Like so:

    User-agent: GPTBot
    Disallow: /
    
    User-agent: Googlebot
    Disallow: /
    

Hosted by Micro.blog. Designed with ♥ by Matt Langford.