The CPU you are using was released 5 years ago midway through 2012.
However, the large amount of ram you have I wonder what flags you are using to launch the server.
I doubt garbage collection is running for a full 4.8 seconds, but it’s possible.
I suggest you follow @Aikar 's suggestions for a base level. Optimized Startup Flags for consistent Garbage Collection