Create A Robots.txt File On New Local Sites

Feature requests
1

In Local, I had a site in dev for over a month. Then, two weeks ago, I created the site in Flywheel so that I could push up the local site and deploy it. All of this went well for this rebuild, until I started working the SEO and ads.

With the help of the Flywheel team, it was discovered that creating a site on Flywheel will in fact create the robots.txt file, but Local doesn’t create it. So, when I pushed up the site from Local to Flywheel, the robots.txt file was deleted.

The down stream effect for SEO and ads is that a missing robots file on WordPress will automatically redirect from /robots.txt to /?robots=1. All of the Google ads for this website stopped running immediately, and haven’t been able to run them for 2 weeks. The problem just got fixed today, so we are expecting to run the ads again.

Even though you’ll never need a robots file on Local, please create one anyways. I even checked the local site to see if it was redirecting like the live site was, and it didn’t redirect. Leading one to believe that the robots file is present. Yet, according to the Flywheel logs, the robots file was deleted when we pushed up from Local.

Please fix this.

3

Hi @carbondigitalus - thanks for using Local, and thank you for the feedback!

You are correct, Local does not create a robots.txt file when a new site is created in the app, while new sites created on the Flywheel platform do have this file.

I apologize for the issues this caused with your ads! We can discuss adding a copy of this file locally with the engineering team.

In the meantime - are you using MagicSync (our “Select files” experience in Connect) when pushing to Local? See my screenshot below:

image
image2692×1702 238 KB

I can choose to push “All Files”, which replaces everything on Flywheel with a copy of what is on my machine. This would result in robots.txt being deleted… however, if I choose Select Files, I have the option to uncheck robots.txt and leave that file untouched. Hopefully this addresses your original concern in the meantime!

4

Hey sir. I’ve always used the “All Files” including the database option, because I never knew that the files created were different. Now that I know, I can be more selective, but it’s easier and more secure, if we are pushing up all files.