Wordpress Content Crawler plugin cannot fetch HTML page

Support
, ,
4

There doesn’t appear to be much info in the Local log, but the PHP error log has a number of errors that look like:

[08-Oct-2021 23:22:43 UTC] WPCC (info): Request error (Details: URL: https://www.google.com/, Message: cURL error 60: SSL certificate problem: unable to get local issuer certificate (see https://curl.haxx.se/libcurl/c/libcurl-errors.html))
(Exception: #0 C:\Users\BonSuperman\Local Sites\test\app\public\wp-content\plugins\wp-content-crawler\app\vendor\guzzlehttp\guzzle\src\Handler\CurlFactory.php(155): GuzzleHttp\Handler\CurlFactory::createRejection(Object(GuzzleHttp\Handler\EasyHandle), Array)
#1 C:\Users\BonSuperman\Local Sites\test\app\public\wp-content\plugins\wp-content-crawler\app\vendor\guzzlehttp\guzzle\src\Handler\CurlFactory.php(105): GuzzleHttp\Handler\CurlFactory::finishError(Object(GuzzleHttp\Handler\CurlHandler), Object(GuzzleHttp\Handler\EasyHandle), Object(GuzzleHttp\Handler\CurlFactory))
#2 C:\Users\BonSuperman\Local Sites\test\app\public\wp-content\plugins\wp-content-crawler\app\vendor\guzzlehttp\guzzle\src\Handler\CurlHandler.php(43): GuzzleHttp\Handler\CurlFactory::finish(Object(GuzzleHttp\Handler\CurlHandler), Object(GuzzleHttp\Handler\EasyHandle), Object(GuzzleHttp\Handler\CurlFactory))
#3 C:\Users\BonSuperman\Local Sites\test\app\public\wp-content\plugins\wp-content-crawler\app\vendor\guzzlehttp\guzzle\src\Handler\Proxy.php(28): GuzzleHttp\Handler\CurlHandler->__invoke(Object(GuzzleHttp\Psr7\Request), Array)
#4 C:\Users\BonSuperman\Local Sites\test\app\public\wp-content\plugins\wp-content-crawler\app\vendor\guzzlehttp\guzzle\src\Handler\Proxy.php(51): GuzzleHttp\Handler\Proxy::GuzzleHttp\Handler\{closure}(Object(GuzzleHttp\Psr7\Request), Array)
#5 C:\Users\BonSuperman\Local Sites\test\app\public\wp-content\plugins\wp-content-crawler\app\vendor\guzzlehttp\guzzle\src\PrepareBodyMiddleware.php(37): GuzzleHttp\Handler\Proxy::GuzzleHttp\Handler\{closure}(Object(GuzzleHttp\Psr7\Request), Array)
#6 C:\Users\BonSuperman\Local Sites\test\app\public\wp-content\plugins\wp-content-crawler\app\vendor\guzzlehttp\guzzle\src\Middleware.php(35): GuzzleHttp\PrepareBodyMiddleware->__invoke(Object(GuzzleHttp\Psr7\Request), Array)
#7 C:\Users\BonSuperman\Local Sites\test\app\public\wp-content\plugins\wp-content-crawler\app\vendor\guzzlehttp\guzzle\src\RedirectMiddleware.php(54): GuzzleHttp\Middleware::GuzzleHttp\{closure}(Object(GuzzleHttp\Psr7\Request), Array)
#8 C:\Users\BonSuperman\Local Sites\test\app\public\wp-content\plugins\wp-content-crawler\app\vendor\guzzlehttp\guzzle\src\Middleware.php(59): GuzzleHttp\RedirectMiddleware->__invoke(Object(GuzzleHttp\Psr7\Request), Array)
#9 C:\Users\BonSuperman\Local Sites\test\app\public\wp-content\plugins\wp-content-crawler\app\vendor\guzzlehttp\guzzle\src\HandlerStack.php(71): GuzzleHttp\Middleware::GuzzleHttp\{closure}(Object(GuzzleHttp\Psr7\Request), Array)
#10 C:\Users\BonSuperman\Local Sites\test\app\public\wp-content\plugins\wp-content-crawler\app\vendor\guzzlehttp\guzzle\src\Client.php(351): GuzzleHttp\HandlerStack->__invoke(Object(GuzzleHttp\Psr7\Request), Array)
#11 C:\Users\BonSuperman\Local Sites\test\app\public\wp-content\plugins\wp-content-crawler\app\vendor\guzzlehttp\guzzle\src\Client.php(162): GuzzleHttp\Client->transfer(Object(GuzzleHttp\Psr7\Request), Array)
#12 C:\Users\BonSuperman\Local Sites\test\app\public\wp-content\plugins\wp-content-crawler\app\vendor\guzzlehttp\guzzle\src\Client.php(182): GuzzleHttp\Client->requestAsync('GET', Object(GuzzleHttp\Psr7\Uri), Array)
#13 C:\Users\BonSuperman\Local Sites\test\app\public\wp-content\plugins\wp-content-crawler\app\vendor\fabpot\goutte\Goutte\Client.php(174): GuzzleHttp\Client->request('GET', 'https://www.goo...', Array)
#14 C:\Users\BonSuperman\Local Sites\test\app\public\wp-content\plugins\wp-content-crawler\app\vendor\symfony\browser-kit\Client.php(404): Goutte\Client->doRequest(Object(Symfony\Component\BrowserKit\Request))
#15 C:\Users\BonSuperman\Local Sites\test\app\public\wp-content\plugins\wp-content-crawler\app\Objects\Crawling\Bot\AbstractBot.php(548): Symfony\Component\BrowserKit\Client->request('GET', 'https://www.goo...')
#16 C:\Users\BonSuperman\Local Sites\test\app\public\wp-content\plugins\wp-content-crawler\app\Objects\Crawling\Bot\AbstractBot.php(393): WPCCrawler\Objects\Crawling\Bot\AbstractBot->getResponseText('GET', 'https://www.goo...', false, 'https')
#17 C:\Users\BonSuperman\Local Sites\test\app\public\wp-content\plugins\wp-content-crawler\app\Objects\Crawling\Bot\DummyBot.php(31): WPCCrawler\Objects\Crawling\Bot\AbstractBot->request('https://www.goo...', 'GET', Array)
#18 C:\Users\BonSuperman\Local Sites\test\app\public\wp-content\plugins\wp-content-crawler\app\Test\Tests\SourceCodeTest.php(42): WPCCrawler\Objects\Crawling\Bot\DummyBot->request('https://www.goo...', 'GET', Array)
#19 C:\Users\BonSuperman\Local Sites\test\app\public\wp-content\plugins\wp-content-crawler\app\Test\Base\AbstractTest.php(110): WPCCrawler\Test\Tests\SourceCodeTest->createResults(Object(WPCCrawler\Test\Data\TestData))
#20 C:\Users\BonSuperman\Local Sites\test\app\public\wp-content\plugins\wp-content-crawler\app\Test\Test.php(153): WPCCrawler\Test\Base\AbstractTest->run()
#21 C:\Users\BonSuperman\Local Sites\test\app\public\wp-content\plugins\wp-content-crawler\app\Services\PostService.php(133): WPCCrawler\Test\Test::respondToTestRequest(Array)
#22 C:\Users\BonSuperman\Local Sites\test\app\public\wp-includes\class-wp-hook.php(303): WPCCrawler\Services\PostService->WPCCrawler\Services\{closure}('')
#23 C:\Users\BonSuperman\Local Sites\test\app\public\wp-includes\class-wp-hook.php(327): WP_Hook->apply_filters('', Array)
#24 C:\Users\BonSuperman\Local Sites\test\app\public\wp-includes\plugin.php(470): WP_Hook->do_action(Array)
#25 C:\Users\BonSuperman\Local Sites\test\app\public\wp-admin\admin-ajax.php(187): do_action('wp_ajax_wcc_tes...')
#26 {main})

I’ve never seen this specific error before, but here are some observations:

  • This plugin appears to be using Guzzle which is an HTTP client for PHP. This is significant because it means that the plugin is rolling its own HTTP requests as opposed to using the WordPress HTTP API.
  • Because this plugin isn’t using the WordPress HTTP API, it’s obtaining the SSL certificate from whatever the system’s SSL implementation is using.
  • The questions to answer are:
    • Where is that cert set?
    • How to update it to the latest version?

I don’t have the answers to those last two questions yet since I haven’t really dug that deeply into how Local configures SSL under Windows, but I have it on my list of things to investigate further!

For reference, I think the solution will be similar to this topic: Expired SSL certificate - #9 by ben.turner

However, that topic is focused on Mac, whereas this issue is specific to Windows.