Shopify robots txt: Editing It Replaces Everything

Most advice about editing robots.txt on Shopify says roughly this: since Online Store 2.0 you can add a robots.txt.liquid template and put your own rules in. That is true. However, it leaves out the word that matters, and the omission is expensive.

You are not adding a rule. You are replacing the entire file.

What Shopify’s default robots.txt actually contains

First, it helps to see the thing you would be overwriting. We pulled the live robots.txt from a production Shopify store on 28 October 2026. Here is what it holds:

Those rules are not decorative. For example, they keep crawlers out of /admin, /cart/, /checkout, /checkouts/, /orders, /account and /services. In addition, they block a set of URL parameters that would otherwise generate endless duplicate pages, including preview_theme_id and preview_script_id.

Then there is a second block most merchants have never seen. Shopify writes a dedicated section for adsbot-google, explicitly allowing it into products, collections, pages and blogs while still disallowing checkout, orders and services. Consequently, if you overwrite the file and don’t reproduce that block, you may affect how Google crawls your paid landing pages.

A caveat on that count, because it matters later. Two groups is what the served file contained. Older descriptions of Shopify’s robots.txt list several more, including Nutch, AhrefsBot, MJ12bot and Pinterest. Those groups are still what the Liquid object returns, which is precisely the problem covered below.

The word in Shopify’s documentation is “replaces”

This is not a trap we inferred. On the contrary, Shopify states it directly in its Customize robots.txt documentation.

The wording is unambiguous: “Adding the robots.txt.liquid template replaces Shopify’s generated robots.txt output.” The template also ships in no theme by default, because the generated file already works for most stores.

So the mental model most merchants carry is wrong. They picture appending a line to an existing file. In reality, they are handing Shopify a blank page and taking responsibility for everything that was on the old one.

Shopify’s own documentation “strongly recommend[s]” building on the provided Liquid objects rather than writing rules from scratch. Note the actual reason, though: Shopify updates its default rules regularly, and a template built on the objects keeps receiving those updates, whereas hand-typed plain text does not.

The Liquid objects do not reproduce the current file

This is the part that turns a manageable task into a genuinely risky one, and it is the reason the previous version of this article was too relaxed.

Using robots.default_groups does not guarantee you keep today’s rules. Shopify says so on the same page: “The robots object provides default rule groups for custom templates, but rendered template output doesn’t always match Shopify’s generated file exactly.”

In practice the gap is wide. A bug report on Shopify’s developer forum documents robots.default_groups returning an older rule set than the file Shopify actually serves. The reported differences run in both directions:

• Present in the served file, missing from the Liquid output: the agentic-commerce directives (the UCP/MCP and agents.md pointers), the explicit Allow rules, and newer disallows such as /sf_*, /services and /cart.js.
• Only in the Liquid output: disallows for /search and /policies/, plus user-agent groups for Nutch, AhrefsBot, MJ12bot and Pinterest that the current served file no longer carries.

Read that second bullet carefully. /search flips from crawlable to blocked purely because a template exists. Nothing the merchant wrote caused it, and nothing they can write inside the loop undoes it.

Shopify has acknowledged the report and said it is reviewing the object and the documentation. Until that lands, the honest position is that no method currently guarantees you preserve the served defaults. Therefore the diff below is not a tidy final check. It is the step the whole job depends on.

What Shopify started shipping that you would also lose

Here is the part that makes this more urgent than it was a year ago.

Shopify has begun injecting an agent-commerce preamble into storefront robots.txt files. On stores where it is live, the file opens with commented instructions for AI shopping agents: a link to an agents.md file, a UCP discovery endpoint, a UCP/MCP endpoint, and rules agents must follow to transact.

In other words, your robots.txt is no longer only a crawler-control file. Increasingly, it is also how your store introduces itself to AI shopping assistants. Therefore an overwrite doesn’t just risk your Google crawling. It can also remove the block that tells agent-based shoppers how to buy from you.

Few merchants know this block exists, because nobody put it there on purpose. That is exactly why it is easy to delete by accident.

How to edit it without losing anything

If you genuinely need a custom rule, the safe sequence takes about five minutes.

1. Save the current output first. Visit yourstore.com/robots.txt and save the whole file. This is your reference copy, and it is the step people skip.
2. Create the template. Add robots.txt.liquid to your theme’s templates directory. Note that it must be a Liquid template, since Shopify does not accept a JSON template here.
3. Build on the Liquid objects, but do not trust them. Loop over robots.default_groups and add your rules around it, because that keeps you connected to Shopify’s future updates. However, as above, the object’s output will not match today’s served file.
4. Diff the result line by line. This is the critical step, not a formality. Compare your rendered output against the copy you saved, and expect real differences rather than hoping for none. Re-add by hand anything the object dropped, and delete anything stale it reintroduced — the /search and /policies/ disallows being the ones most likely to bite.
5. Re-check after theme work. The template lives in your theme. As a result, switching or duplicating themes can leave it behind.

When you actually need to edit it, and when you don’t

Most stores never need to. Shopify’s defaults are sensible, and the cases that genuinely justify a custom template are narrow.

Reasonable reasons include blocking a specific crawler you have a problem with, adding a second sitemap for a subdirectory setup, or blocking a parameter pattern a third-party app has introduced.

Poor reasons are more common. Above all, do not reach for robots.txt to keep a page out of Google’s index. As Google’s own robots.txt documentation explains, blocking a URL prevents crawling, not indexing, and a blocked page can still appear in results from external links alone. If you want a page out of the index, use noindex and let the crawler in to see it. We cover that distinction in detail in our guide to why robots.txt doesn’t keep Shopify search pages out of Google.

Similarly, do not use it to hide thin collection pages. Fix them, consolidate them, or give them a reason to exist instead.

The honest summary

Editing robots.txt on Shopify is not dangerous because the syntax is hard. Rather, it is dangerous because the action is not what it looks like. A merchant adding one rule believes they are making a small, additive change. Meanwhile the platform treats it as a full handover of a 94-line file they have never read.

Check your current output before you touch anything. Then, if you still need the change, build on the Liquid objects rather than from an empty file. Finally, if you want to know what search engines can actually reach on your store without reading crawl rules yourself, Plug In SEO audits access across your catalogue and surfaces the pages that have quietly become unreachable.