Uh-oh! Unable to provision site on large multisite setup

Bug Summary

When trying to start LocalWP multisite sites with many sub-sites (approx more than 100), the connection to the db frequently fails (s. screenshot). The db actually starts, but it seems to me that it just takes too long for LocalWP, which times out and doesn’t start the site. Sometimes I have to retry 10 - 50 times before the site eventually starts.

Steps to reproduce

  1. Create a multisite setup with more than 100 sub-sites (not sure about the actual treshold).
  2. (Stop and) start the site.

Environment Info

Describe your environment.

  • MacOs: 14.6.1 (23G93)
  • What versions of site software is used: Nginx, PHP: 8.2, MySQL; 8.0.16
  • What version of Local is installed: Version 9.1.1+6738

Supporting info

{“level”:“error”,“message”:“Database did NOT respond to ping after 15 tries.”,“service”:{},“serviceBinVersion”:{},“stack”:“Error: Command failed: %%userDataPath%%/lightning-services/mysql-8.0.16+6/bin/darwin/bin/mysqladmin ping\n\u0007mysqladmin: connect to server at ‘localhost’ failed\nerror: ‘Can’t connect to local MySQL server through socket ‘%%site.runData%%/mysql/mysqld.sock’ (2)’\nCheck that mysqld is running and that the socket: ‘%%site.runData%%/mysql/mysqld.sock’ exists!\n\n at ChildProcess.exithandler (node:child_process:422:12)\n at ChildProcess.emit (node:events:517:28)\n at ChildProcess.emit (node:domain:489:12)\n at maybeClose (node:internal/child_process:1098:16)\n at ChildProcess._handle.onexit (node:internal/child_process:303:5)”,“thread”:“main”,“timestamp”:“2024-12-18T08:08:03.604Z”}
{“class”:“SiteProcessManagerService”,“level”:“error”,“message”:“Unable to start site.”,“stack”:“Error: Command failed: %%userDataPath%%/lightning-services/mysql-8.0.16+6/bin/darwin/bin/mysqladmin ping\n\u0007mysqladmin: connect to server at ‘localhost’ failed\nerror: ‘Can’t connect to local MySQL server through socket ‘%%site.runData%%/mysql/mysqld.sock’ (2)’\nCheck that mysqld is running and that the socket: ‘%%site.runData%%/mysql/mysqld.sock’ exists!\n\n at ChildProcess.exithandler (node:child_process:422:12)\n at ChildProcess.emit (node:events:517:28)\n at ChildProcess.emit (node:domain:489:12)\n at maybeClose (node:internal/child_process:1098:16)\n at ChildProcess._handle.onexit (node:internal/child_process:303:5)”,“thread”:“main”,“timestamp”:“2024-12-18T08:08:03.605Z”}
{“class”:“Process”,“level”:“warn”,“message”:“2024-12-18T08:07:55.663210Z 0 [System] [MY-010116] [Server] %%userDataPath%%/lightning-services/mysql-8.0.16+6/bin/darwin/bin/mysqld (mysqld 8.0.16) starting as process 1363”,“process”:“mysql”,“thread”:“main”,“timestamp”:“2024-12-18T08:08:34.960Z”}
{“class”:“Process”,“level”:“warn”,“message”:“2024-12-18T08:07:55.666485Z 0 [Warning] [MY-013243] [Server] --character-set-server: The character set UTF8MB3 is deprecated and will be removed in a future release. Please consider using UTF8MB4 instead.”,“process”:“mysql”,“thread”:“main”,“timestamp”:“2024-12-18T08:08:34.960Z”}
{“class”:“Process”,“level”:“warn”,“message”:“2024-12-18T08:07:55.667078Z 0 [Warning] [MY-010159] [Server] Setting lower_case_table_names=2 because file system for %%site.runData%%/mysql/data/ is case insensitive”,“process”:“mysql”,“thread”:“main”,“timestamp”:“2024-12-18T08:08:34.960Z”}
{“class”:“Process”,“level”:“warn”,“message”:“2024-12-18T08:08:05.222866Z 0 [Warning] [MY-010068] [Server] CA certificate ca.pem is self signed.”,“process”:“mysql”,“thread”:“main”,“timestamp”:“2024-12-18T08:08:34.961Z”}
{“class”:“Process”,“level”:“warn”,“message”:“2024-12-18T08:08:05.248840Z 0 [System] [MY-010931] [Server] %%userDataPath%%/lightning-services/mysql-8.0.16+6/bin/darwin/bin/mysqld: ready for connections. Version: ‘8.0.16’ socket: ‘%%site.runData%%/mysql/mysqld.sock’ port: 10189 MySQL Community Server - GPL.”,“process”:“mysql”,“thread”:“main”,“timestamp”:“2024-12-18T08:08:34.961Z”}

Hi @peroks

I don’t believe we have any built-in thresholds, but you might be finding some unintentionally, as over 100 sub-sites is quite the undertaking for a multisite install.

Some questions here to look into this further:

  • If you create a new, blank site in Local, are you able to start and open that without issue?

  • Do you have any other single sites or multisites in Local?

  • Does your device have plenty of disk space?

  • Do you use any external storage for your Local sites?

Could you share your full Local Log here for us as well? There are some different ways to access and share Local Logs. For us to be able to troubleshoot thoroughly, please click the Download Local Logs button from the Support tab in Local. This will generate a zip archive that contains the Local log along with some other diagnostic information to help quickly zero in on any issues that Local is encountering.

Hi Nick, thanks for getting back to me so quickly.

If you create a new, blank site in Local, are you able to start and open that without issue?

Yes.

Do you have any other single sites or multisites in Local?

Yes, I have a lot of sites in LocalWP - single and multisite - and they all start up fine with the exception of two very large multisites, both with over 100 sub-sites.

The issue I reported only occurs on very large multisites. In the error messages, you see that Database did NOT respond to ping after 15 tries. In the Mac Activity Monitor, I see that the db actually starts, but probably not fast enough for LocalWP, which stops pinging it after 15 unsuccessful tries.

I think this issue was introduced with LocalWP 8 or 9. It did not appear in earlier versions.

Does your device have plenty of disk space?

Yes.

Do you use any external storage for your Local sites?

No.

Okay, thank you for the extra details here, @peroks! I’ll check with our Local Devs on this.

Following up here @peroks! We weren’t able to replicate this issue solely by creating the same number of subsites, but it seems like the root of the issue might be related to the database size. Our Devs are looking into some optimizations they might be able to make in database handling that could help resolve this down the road. For now, we’ll leave this as a Bug for others to watch and follow. We don’t have a firm ETA on when any changes might be implemented but we’ll provide updates here!

2 Likes

@peroks

Does it matter if you are running MySQL 5.7.28 instead of 8.0.16? I had problems with a multisite (not that big though) a while ago when I tried to migrate to MySQL 8.0.16. So I gave up on trying to update MySQL for the multisite as I was short of time.

1 Like