So I finally tracked down about 6 different errors that were keeping my VPS running Ubuntu 14.04 instead of something more recent. One of the main errors was due to a system misconfig by my VPS host OVH. After getting that fixed, vexed with frustration and carelessly impatient, I upgraded all the installed packages and ended up taking all the sites I host on this VPS down for the last 10 hours.
So, aside from promising that I'll be more careful, keep better backups(IE: ones that are more easily restorable) what did I need to do to fix it? Well, quite a few things. I'll start with what will probably be most people's problem, broken Nginx PHP-FPM. After the upgrade I was given the wonderful and not-especially-helpful error message:
**502 Bad Gateway** nginx/1.10.0 (Ubuntu)
But at least there was some meaningful info in ngninx's logs which pointed to nginx not being able to hook into php. On 14.04 I had php 5.x installed, which got removed and replaced with php 7.0. The fix for this was to update the location directives on each of the sites that I have php running on.
Your config may vary, but it will have a line that looks something like this:
That just needs to point to the new path, which by default should be /var/run/php, not just /var/run, and the updated php version:
That got things working, ish for me. After that I was just getting a blank page and an HTTP 200. No errors anywhere, no problems from anything, just a totally blank page. This is because of some php includes not being added to your site config. Make sure Nginx knows to use the existing fastcgi config file by adding the following to the site location directive:
After that I had a mobile device detector plugin installed on here which crashed as soon as Nginx started succesfully, I had to go in to the wp-content/plugins folder and delete the affected plugin. After deleting the plugin from the disk, I was able to start Nginx, log-in to the WP site and remove the plugin from the list of installed plugins.
Jetpack wouldn't work properly though, so analytics and stuff were broken. All that took was installing php7.0-xml and restarting Nginx for a 421,039th time.
Great, php nonsense fixed. I also had some trouble because I have HSTS enabled on two of the sites, and the upgrade broke the letsencrypt installation. So when trying to fix everything above, I was also fighting with getting my server to allow me to connect over http, not https. I ended up deleting the config for the most problematic site, one that had an expired cert, removing the included version of letsencrypt with Ubuntu 16.04 is an older version that's not forward compatible, redoing the site's cert from scratch, and restoring the https config in the site's config file for Nginx, restarting, and good.