Unable to import database when pulling from WP Engine

Support
,
1

Bug Summary

I’m unable to sync the database to a site on WP Engine. When I include the database I get the following:

Screen Shot 2022-04-18 at 7.10.12 PM
Screen Shot 2022-04-18 at 7.10.12 PM744×718 46.1 KB

The plugin files and such are pulled just fine. It’s only the database that has stopped working.

Environment Info

Describe your environment.

  • What Operating System are you using? MacOS
  • What versions of site software (Nginx, Apache, PHP, MySQL) is used?
    • PHP: 7.4.1
    • MySQL: 8.0.16
  • What version of Local is installed? 6.3.1+5826

Supporting info

Please provide your Local Log. See this Community Forum post for instructions on how to do so:

local-lightning.log (788.8 KB)

2

I tried deleting the autoload.sql file to make sure that wasn’t causing a problem, but that didn’t help. I can see that the SQL file is in fact being generated properly.

3

My team has also been encountering this issue. We’re all using Version 7.1.2+6410.

I’ve attached the log file (with identifying details redacted).

local-lightning.log (45.5 KB)

1 Like
4

Hey @tylersticka

Have you tried resetting API credentials?

Does downloading a backup and direct importing work?

Is your team all using a shared network? Are you all on the same OS/OS version? i.e. all using Windows 10 or all using Mac?

5

We are not on a shared network. We are all on macOS Ventura. We have not tried resetting API credentials, mostly because this doesn’t seem like an authentication issue and some of the team members experiencing this issue set up their authentication fairly recently.

After trying a zillion things, I just found a partial solution, but it’s only been confirmed on my machine. I didn’t want to download a full WP Engine site backup (the site has a lot of large files, which were already successfully synced), so instead I downloaded just a SQL backup. It was barely too large for Adminer to import directly, but I was able to open the site shell and run mysql -u root -proot local < path/to/file.sql. Then, I restarted the site in Local, clicked away to a different Local site and back again to trigger the “domain name is wrong” bar with an offer to “fix it,” and after that process finished we seem to be synced again.

But even if that works for others on my team, that’s not a great solution for ongoing development, and it doesn’t really explain why this has started happening within the last year (especially since, according to old backups, our database is smaller than it was a year ago).

6 
it doesn’t really explain why this has started happening within the last year

So it sounds like this has been an ongoing issue for you all rather than something that just happened recently after an update?

When your team is running into this issue is it intermittent across various sites or is it happening with a single specific site? Are there any particular steps to replicate the error for example or does it just seem to be random?

It seems like this site in particular might be struggling due to some size limitations. While we don’t have a hard limit on the size of a site that can be pulled or pushed with Local, sites that are 5GB or larger can occasionally struggle due to some timeout limitations. Things like network speed can play a part but there are a number of variables that affect this. So if you have sites with large media libraries, big databases with massive tables/rows, extra backups, or 100+ plugins, etc these may have intermittent luck with working on Local Connect. Not to say they can’t be pushed or pulled, but it may take a few attempts either way and require some extra patience.

However, depending on your answers to the above we’re happy to continue looking into this if it’s more of a widespread intermittent issue. If you’re able to replicate it again, can you go to the Help Center in your Local App, Scroll Down, and click the Download Local Log button? This bundles up several logs and provides us with a lot more diagnostic information for troubleshooting.

7

So it sounds like this has been an ongoing issue for you all rather than something that just happened recently after an update?

Yes, we noticed that it was happening a few weeks ago. Then we updated to 7.1.0, but ran into a different now-fixed issue. Updating to 7.1.2 restored the original issue we were experiencing.

When your team is running into this issue is it intermittent across various sites or is it happening with a single specific site?

It’s happening with a specific site. The site is very large, but that isn’t a new development, and we never had trouble using Local with it until recently. It also happens even if we’re only syncing the database (no files).

Are there any particular steps to replicate the error for example or does it just seem to be random?

It happens consistently when syncing the database. Files sync just fine. We can see in Adminer that most of the database syncs, it just seems to be choking at a certain point.

While we don’t have a hard limit on the size of a site that can be pulled or pushed with Local, sites that are 5GB or larger can occasionally struggle due to some timeout limitations.

This is a large site, but it’s odd that the error always happens when processing the /app/public/_wpeprivate/autoload.sql SQL file, and persists even if we’re only syncing the database, which is far below 5GB in size.

Here’s the specific error encountered in the log I passed along earlier:

{"thread":"main","class":"ImporterWPE","stack":"Error: ERROR 1062 (23000) at line 1966: Duplicate entry '?' for key 'token'\n    at %%appPath%%/main/_helpers/importSQLFile.js:1:2560\n    at Generator.next (<anonymous>)\n    at a (%%appPath%%/main/_helpers/importSQLFile.js:1:742)","level":"error","message":"Unable to import /Users/HOMEDIR/Local Sites/SITENAME/app/public/_wpeprivate/autoload.sql","timestamp":"2023-08-08T22:30:34.670Z"}

I’d love to know what’s on line 2560 of main/_helpers/importSQLFile.js, but I’m not sure where %%appPath%% is or if its viewable.

Also, following up on your earlier question:

Does downloading a backup and direct importing work?

I tried downloading a full site backup and importing it to see if this would also work, but it did not. Local reports that the process “succeeded,” but the resulting site displays various “missing file” errors and the like.

So right now, the only method that works for us is:

  1. Create a partial backup in WP Engine that only includes the site’s database.
  2. Use Local Connect to sync the site.
  3. Once it displays the “Uh-oh!” message mentioning /app/public/_wpeprivate/autoload.sql, open the site shell and import the WPE database backup using the command mysql -u root -proot local < path/to/file.sql.

If you’re able to replicate it again, can you go to the Help Center in your Local App, Scroll Down, and click the Download Local Log button? This bundles up several logs and provides us with a lot more diagnostic information for troubleshooting.

If/when we replicate it again, I will reach out to WP Engine support with these details (the log contains customer details I’m not able to share on a public forum).

8

Thank you for all the great detail! We can start taking a look. I sent you a direct message to discuss some of this further as well.

9

Hello, I’m another developer on Tyler’s team, and I’m running into the same issue. I’m running Local v7.1.2+6410 on Mac OS Ventura.

When I try to sync from the site, I get the “Uh-oh unable to import” due to autoload.sql error shown in the screenshot above.

After that error, Local shows a green success bar with “Site pulled successfully” but when I try to access the site, it shows the language selection and seems to have a blank database.

When I followed Tyler’s instructions to download a database backup and manually import that from the command line, it still fails. The terminal command succeeds, then after restarting my site, Local.app prompts me that my database URL doesn’t match my Local site URL, with a button to “Fix this.” When I click “Fix this” I get the following error: “Uh-oh! Unable to change site domain to host.”

Screenshot 2023-08-21 at 3.32.12 PM
Screenshot 2023-08-21 at 3.32.12 PM1040×896 140 KB

At this point, I’m dead in the water and cannot get a local environment running for this site at all, and every time I try it’s hours of waiting at progress bars. I’d really appreciate assistance.

If you’d like logs, I can provide them via DM or email.

10

This file basically spawns a MySQL process to import a database dump. The first part of the error is more important for our needs:

This part of the error is coming directly from the spawned MySQL process. The specific issue is that Mysql is trying to insert data that has a duplicate key, in this case, a question mark. That doesn’t make sense, and hints at maybe a data issue in the SQL file.

It sounds like you’ve already downloaded the autoload.sql file. When you examine the lines around the line mentioned in the above error, 1966, what do you see?

Obviously don’t share any sensitive info from the raw DB here, but some things I’d be looking for when looking at the sql dump:

  • What table is this value being inserted into?
  • What is the value/key being inserted? Does it look like a regular string, or is it some garbled binary characters?
  • What DB versions are in use? On the remote, and on your Local sites. If they are different versions, that might cause issues.

I’d love to hear about what you find!

1 Like
11

This topic was automatically closed after 730 days. New replies are no longer allowed.