I went in to do some additional testing and tried to replicate this, but I haven’t been able to. It looks like the fileinfo extension was added to the PHP service, but maybe the installation on your machine is using an older version of the PHP service.
The Local team is working towards making updating services easier, but in the meantime, @jonc, can you try updating your PHP services by following this FAQ:
Let me know if that gets the fileinfo module added for newly created sites!