So some of you may be wondering what happened to the server and then the forum today/yesterday.
TLDR; Note to self, if it ain't broke, don't fix it. If you want to sleep, don't be a programmer.
Our server's kernel was out of date, had some issues and then crashed forcing us to do a hard reboot. (Nothing was really lost though, our backups are much more frequent) This was a pretty quick fix, I just had to replace some of the corrupted files with ones from our backups.
The forum was a bit of a bigger situation that took a bit more time and effort to fix. After the main server crashed, I figured it would be a good idea to update the forum's server as well (sounds like a reasonable idea). I expected a few minutes of downtime while we were updating due to restarts but I had no idea what was about to happen. After running the correct commands to update and waiting a bit, it asked me to restart the server. When it came back, I knew I had fucked up.
The forum is built on PHP 5.6 and is not updated for php 7.0. Ubuntu 16.04 only has the repository for PHP 7.0, by using a PPA to get PHP 5.6 I was able to get the forum to run but not communicate with the database server. After getting the database server to work I realized that it was no longer on PHP 5.6 and was back on 7.0 (which broke the forum again). I decided the best way to fix all the issues was to go back to Ubuntu 14.04 (I know it works) however they don't make it easy to downgrade at all. I took a snapshot of the server as a restore point incase it all went tits up and then clicked on the reinstall button on OVH's control panel.
I couldn't believe my eyes, it said that I had a snapshot and I had to delete it before I could reinstall, what good is a restore point if you can use it to restore????
I spent bunch of quality time finding all the bits of data that I would need to save to setup the new OS. I deleted the snapshot, clicked the reinstall button and crossed my fingers. About 10 minutes later I got my email telling me that my new OS was ready, I setup the webserver, database server, ssl, keys and then proceeded to import all the data into the server. The forum content was easy, just copied the files over, however, when it came time to importing the data from the database, I was met with an unpleasant flood of errors. It turns out that after updating to Ubuntu it modified the database a bit (changed the engine type of the tables that store system data) I had to manually change each one that created an error one at a time because it would get to that line and the quit.
The final curve ball was a bit of a surprise to me, when I was trying to fix the issue I turned on debug mode (shows more detail about errors including stuff like deprecation messages). That caused it to just insert all the error messages just anywhere in the forum making it unusable.
It is now 6 in the morning and I have yet to go to sleep from the previous server crash and before that I was attempting to port forward through 2 routers, a modem and then reverse proxy through https, little did I know that both firewalls on the routers were blocking non http ports, the reverse proxy didn't support https and the webserver wasn't even running even though it said it was.
I have school in about 3 hours so I'm going to get some sleep.
Night Ya'll!
- Crafty
