I know it is easy for me to say seeing as I would not have to do any of the work but I would definitely favor moving on to a spigot/bukkit server. I mean unless there is less than 6-8 people online and playing the lag is was so bad earlier today I spent well over 10 minutes in a black void, no movement no nothing for both my primary homes where I play...
Anyhow if it just wouldn't be manageable due to IRL etc. or other reasons I totally understand.

Unfortunately during the "bad hours" things did indeed turn to real mush like any other night. The hours I mention in this post all refer to Canada (Ottawa) time.

It seems from roughly 2pm to 8pm things get insanely bad; that's what I've noticed for several nights now anyway. Tonight was perhaps maybe possibly a bit better (?), but it was still riding on the edge of unplayable from my perspective. Things were down to 7 TPS.

I'd be hesitant to just write it off as player count, because a) the difference is drastic -- going from 6 to 12 players can't solely be the cause of such a big change, and b) tonight it has clearly performed better with 16 players at around 9pm than it performed with 11 players at around 6pm.

Once it started improving again (after around 8pm), it was definitely starting to act more like the magical improvement we saw earlier today. It wasn't nearly as good yet, but it did appear to be recovering in a way that didn't happen in previous days, made most evident by the occasional return to 20 TPS. I don't think the server has really hit 20 TPS (or even came close) in the last month until your recent magical fix.

Anyway, just thoughts and observations. I guess you guys already have logs and stats that can tell you as much without my fuzzy descriptions. So a summary:
Good hours = mind-blowing good. Drop any thought of spigot like a brick if CM can hold onto that performance.
Bad hours = still super bad and really frustrating. Personal suspicion about how the performance drop seems to sync up with (real world) time of day more than player count.

And all that said, playing today on Crafty while it was good was some of the funnest Minecrafting I've had. It was sooo good; especially with raids, and caves just packed with never-ending mobs.