This is the process that I use, as well. But lately, there seems to be a problem that can be seriously annoying. The latest versions of WordPress want to automatically update core, by default. I don’t know how many times I’ve used @Clay’s method only to relaunch the website after the older version install and found the latest version has been reinstalled. It’s not enough to remove all traces of the current version (files and database). It’s also important to tell WordPress to NOT automatically update its core files.
Add the WP_AUTO_UPDATE_CORE define to your wp-config.php file before starting the downgrade, or at the end of the installation of the old version, WordPress will update it to the latest version.
Specifically, add the following line to wp-config.php.
define( 'WP_AUTO_UPDATE_CORE', false );