Lets be honest, the chances of Mojang fixing the lag problem is like finding a skeleton with full diamond armor at night, and I've only ever seen one in my years of playing Minecraft. So with that said, this conversation will continue to come up unless the whole community and Crafty decide to change the server. And even then, someone will find a new problem to talk about. Yes this lag is so bad that many active and great players are leaving and considering not coming back, not I of course. This is a really nice community and I would hate to see everyone leave because of something that the server, nor the owner is not responsible for. There really is no solid answer for us to do besides play the lag infected game, or leave....

@BlueAce56 Lets be honest, the chances of Mojang fixing the lag problem is like finding a skeleton with full diamond armor at night, and I've only ever seen one in my years of playing Minecraft. So with that said, this conversation will continue to come up unless the whole community and Crafty decide to change the server. And even then, someone will find a new problem to talk about. Yes this lag is so bad that many active and great players are leaving and considering not coming back, not I of course. This is a really nice community and I would hate to see everyone leave because of something that the server, nor the owner is not responsible for. There really is no solid answer for us to do besides play the lag infected game, or leave....

It’s apparently been fixed
https://bugs.mojang.com/plugins/servlet/mobile#issue/MC-151772

Unfortunately during the "bad hours" things did indeed turn to real mush like any other night. The hours I mention in this post all refer to Canada (Ottawa) time.

It seems from roughly 2pm to 8pm things get insanely bad; that's what I've noticed for several nights now anyway. Tonight was perhaps maybe possibly a bit better (?), but it was still riding on the edge of unplayable from my perspective. Things were down to 7 TPS.

I'd be hesitant to just write it off as player count, because a) the difference is drastic -- going from 6 to 12 players can't solely be the cause of such a big change, and b) tonight it has clearly performed better with 16 players at around 9pm than it performed with 11 players at around 6pm.

Once it started improving again (after around 8pm), it was definitely starting to act more like the magical improvement we saw earlier today. It wasn't nearly as good yet, but it did appear to be recovering in a way that didn't happen in previous days, made most evident by the occasional return to 20 TPS. I don't think the server has really hit 20 TPS (or even came close) in the last month until your recent magical fix.

Anyway, just thoughts and observations. I guess you guys already have logs and stats that can tell you as much without my fuzzy descriptions. So a summary:
Good hours = mind-blowing good. Drop any thought of spigot like a brick if CM can hold onto that performance.
Bad hours = still super bad and really frustrating. Personal suspicion about how the performance drop seems to sync up with (real world) time of day more than player count.

And all that said, playing today on Crafty while it was good was some of the funnest Minecrafting I've had. It was sooo good; especially with raids, and caves just packed with never-ending mobs.

@Jkeller4000 Adding on to what Crafty said, Mojang has historically neglected the benefits of focusing on performance and (ideally, but not likely) shifting over to a multi threaded approach to the game logic. There's no way you can tweak the variables of the vanilla server to get the max performance possible without radically changing the server jar's programming (with something like nSpigot), unless you want to severely diminish the server experience.

We live in an era where we have 8 or even 16 cores in consumer grade CPUs, and for every vanilla and normal Spigot server, every single core in their CPU is twiddling their thumbs around except for one, which is busy handling chunk loading, entity processing, command blocks, chunk and world serialization, and literally every other aspect about game logic aside from chat (???). This is just old lazy code carried over from the Notch days, and Mojang, despite developing one of the biggest games of all time and having the resources to do so, still hasn't done jack to better the performance of their game.

The only practical solutions here are to wait for Mojang to get their shit together and fix the many issues they've introduced with 1.14, or replace the server software with something that actually focuses on high performance like Spigot.