What's the best way to handle Lapito's Galacticraft's Official Server's Lag Spikes?

sfxworks · October 28, 2018, 7:30am

[] [Server thread/INFO] [nucleus]: Server ran the command: /gc
[] [] []: ================== Server Statistics ===================
[] [] []: Current TPS: §c9.74§7/20.00
[] [] []: Average TPS for session: §c9.16§7/20.00
[] [] []: Server Uptime: 1 hour, 53 minutes, 19 seconds
[] [] []: JVM Uptime: 1 hour, 55 minutes, 47 seconds
[] [] []:  
[] [] []: Maximum allocatable memory: 25600 MB
[] [] []: Allocated memory: 25600 MB
[] [] []: Utilised memory: 22044 MB §7(§786§7% of allocated, §786§7% of max)
[] [] []: Free (but allocated) memory: 3556 MB
[] [] []:  
[] [] []: World: world
[] [] []: Type: overworld, Generator: default, Entities: 1962, Loaded chunks: 3585
[] [] []:  
[] [] []: World: Boss_world
[] [] []: Type: overworld, Generator: flat, Entities: 0, Loaded chunks: 256
[] [] []:  
[] [] []: World: DIM-1509
[] [] []: Type: Oberon, Generator: default, Entities: 0, Loaded chunks: 256
[] [] []:  
[] [] []: World: BackupSpawn
[] [] []: Type: overworld, Generator: default, Entities: 4, Loaded chunks: 257
[] [] []:  
[] [] []: World: DIM-16
[] [] []: Type: Saturn, Generator: default, Entities: 0, Loaded chunks: 256
[] [] []:  
[] [] []: World: DIM-1500
[] [] []: Type: Io, Generator: default, Entities: 0, Loaded chunks: 256
[] [] []:  
[] [] []: World: DIM_SPACESTATION_Kepler22b_8
[] [] []: Type: Kepler22b Space Station, Generator: default, Entities: 0, Loaded chunks: 256
[] [] []:  
[] [] []: World: DIM_SPACESTATION9
[] [] []: Type: Space Station, Generator: default, Entities: 0, Loaded chunks: 256
[] [] []:  
[] [] []: World: DIM-1501
[] [] []: Type: Europa, Generator: default, Entities: 0, Loaded chunks: 256
[] [] []:  
[] [] []: World: DIM_SPACESTATION24
[] [] []: Type: Space Station, Generator: default, Entities: 0, Loaded chunks: 256
[] [] []:  
[] [] []: World: DIM_SPACESTATION_Mercury_26
[] [] []: Type: Mercury Space Station, Generator: default, Entities: 0, Loaded chunks: 256
[] [] []:  
[] [] []: World: DIM_SPACESTATION_Mars_16
[] [] []: Type: Mars Space Station, Generator: default, Entities: 0, Loaded chunks: 256
[] [] []:  
[] [] []: World: DIM-18
[] [] []: Type: Neptune, Generator: default, Entities: 0, Loaded chunks: 256
[] [] []:  
[] [] []: World: DIM-17
[] [] []: Type: Uranus, Generator: default, Entities: 0, Loaded chunks: 256
[] [] []:  
[] [] []: World: DIM_SPACESTATION_Mars_17
[] [] []: Type: Mars Space Station, Generator: default, Entities: 0, Loaded chunks: 256
[] [] []:  
[] [] []: World: DIM_SPACESTATION23
[] [] []: Type: Space Station, Generator: default, Entities: 0, Loaded chunks: 256
[] [] []:  
[] [] []: World: DIM_SPACESTATION_Venus_18
[] [] []: Type: Space Station, Generator: default, Entities: 0, Loaded chunks: 256
[] [] []:  
[] [] []: World: DIM_SPACESTATION35
[] [] []: Type: Space Station, Generator: default, Entities: 0, Loaded chunks: 256
[] [] []:  
[] [] []: World: DIM_SPACESTATION27
[] [] []: Type: Space Station, Generator: default, Entities: 0, Loaded chunks: 256
[] [] []:  
[] [] []: World: DIM-31
[] [] []: Type: Venus, Generator: default, Entities: 0, Loaded chunks: 256
[] [] []:  
[] [] []: World: DIM_SPACESTATION31
[] [] []: Type: Space Station, Generator: default, Entities: 0, Loaded chunks: 256
[] [] []:  
[] [] []: World: DIM_SPACESTATION_Kepler22b_32
[] [] []: Type: Kepler22b Space Station, Generator: default, Entities: 0, Loaded chunks: 256
[] [] []:  
[] [] []: World: DIM_SPACESTATION34
[] [] []: Type: Space Station, Generator: default, Entities: 2, Loaded chunks: 256
[] [] []:  
[] [] []: World: DIM_SPACESTATION29
[] [] []: Type: Space Station, Generator: default, Entities: 0, Loaded chunks: 256
[] [] []:  
[] [] []: World: DIM-29
[] [] []: Type: Mars, Generator: default, Entities: 0, Loaded chunks: 256
[] [] []:  
[] [] []: World: DIM-1505
[] [] []: Type: Callisto, Generator: default, Entities: 0, Loaded chunks: 283
[] [] []:  
[] [] []: World: DIM_SPACESTATION4
[] [] []: Type: Space Station, Generator: default, Entities: 0, Loaded chunks: 256
[] [] []:  
[] [] []: World: DIM_SPACESTATION13
[] [] []: Type: Mercury Space Station, Generator: default, Entities: 0, Loaded chunks: 256
[] [] []:  
[] [] []: World: DIM-13
[] [] []: Type: Mercury, Generator: default, Entities: 0, Loaded chunks: 260
[] [] []:  
[] [] []: World: DIM_SPACESTATION_Mars_12
[] [] []: Type: Mars Space Station, Generator: default, Entities: 0, Loaded chunks: 256
[] [] []:  
[] [] []: World: DIM-1504
[] [] []: Type: Triton, Generator: default, Entities: 0, Loaded chunks: 261
[] [] []:  
[] [] []: World: DIM_SPACESTATION30
[] [] []: Type: Space Station, Generator: default, Entities: 0, Loaded chunks: 256
[] [] []:  
[] [] []: World: DIM-30
[] [] []: Type: Asteroids, Generator: default, Entities: 0, Loaded chunks: 271
[] [] []:  
[] [] []: World: DIM_SPACESTATION28
[] [] []: Type: Space Station, Generator: default, Entities: 0, Loaded chunks: 256
[] [] []:  
[] [] []: World: DIM-28
[] [] []: Type: Moon, Generator: default, Entities: 7, Loaded chunks: 263
[] [] []:  
[] [] []: World: DIM_SPACESTATION21
[] [] []: Type: Space Station, Generator: default, Entities: 0, Loaded chunks: 256
[] [] []:  
[] [] []: World: DIM_SPACESTATION_Mars_19
[] [] []: Type: Mars Space Station, Generator: default, Entities: 0, Loaded chunks: 256
[] [] []:  
[] [] []: World: DIM-20
[] [] []: Type: Ceres, Generator: default, Entities: 147, Loaded chunks: 318
[] [] []:  
[] [] []: World: DIM-22
[] [] []: Type: Kepler 22b, Generator: default, Entities: 179, Loaded chunks: 429
[] [] []:  
[] [] []: World: DIM-19
[] [] []: Type: Pluto, Generator: default, Entities: 0, Loaded chunks: 256
[] [] []:  
[] [] []: World: DIM4598
[] [] []: Type: nc_wasteland, Generator: default, Entities: 0, Loaded chunks: 256
[] [] []:  
[] [] []: World: DIM_SPACESTATION_Mercury_22
[] [] []: Type: Mercury Space Station, Generator: default, Entities: 0, Loaded chunks: 256
[] [] []:  
[] [] []: World: DIM_SPACESTATION20
[] [] []: Type: Space Station, Generator: default, Entities: 0, Loaded chunks: 256
[] [] []:  
[] [] []: World: DIM-21
[] [] []: Type: Eris, Generator: default, Entities: 0, Loaded chunks: 256
[] [] []:  
[] [] []: World: DIM2
[] [] []: Type: Storage Cell, Generator: default, Entities: 0, Loaded chunks: 256
[] [] []:  
[] [] []: World: DIM-1507
[] [] []: Type: Rhea, Generator: default, Entities: 0, Loaded chunks: 256
[] [] []:  
[] [] []: World: DIM_SPACESTATION_Venus_6
[] [] []: Type: Venus Space Station, Generator: default, Entities: 0, Loaded chunks: 256
[] [] []:  
[] [] []: World: DIM_SPACESTATION15
[] [] []: Type: Space Station, Generator: default, Entities: 0, Loaded chunks: 256
[] [] []:  
[] [] []: World: DIM-15
[] [] []: Type: Jupiter, Generator: default, Entities: 0, Loaded chunks: 256
[] [] []:  
[] [] []: World: Server_Spawn_ss
[] [] []: Type: overworld, Generator: default, Entities: 0, Loaded chunks: 272
[] [] []:  
[] [] []: World: DIM-1503
[] [] []: Type: Deimos, Generator: default, Entities: 0, Loaded chunks: 274
[] [] []:  
[] [] []: World: DIM-343800852
[] [] []: Type: Spectre, Generator: default, Entities: 0, Loaded chunks: 256
[] [] []:  
[] [] []: World: DIM-1502
[] [] []: Type: Phobos, Generator: default, Entities: 281, Loaded chunks: 426
[] [] []:  
[] [] []: World: DIM_SPACESTATION11
[] [] []: Type: Space Station, Generator: default, Entities: 0, Loaded chunks: 256
[] [] []:  
[] [] []: World: DIM_SPACESTATION_Venus_10
[] [] []: Type: Venus Space Station, Generator: default, Entities: 0, Loaded chunks: 256
[] [] []:  
[] [] []: World: DIM-1
[] [] []: Type: the_nether, Generator: default, Entities: 35, Loaded chunks: 184
[] [] []:  
[] [] []: World: DIM1
[] [] []: Type: the_end, Generator: default, Entities: 0, Loaded chunks: 2
[] [] []:  
[] [] []: World: DIM_SPACESTATION_Jupiter_3
[] [] []: Type: Jupiter Space Station, Generator: default, Entities: 0, Loaded chunks: 256
[] [] []:  
[] [] []: World: DIM-1506
[] [] []: Type: Ganymede, Generator: default, Entities: 0, Loaded chunks: 256
[] [] []:  
[] [] []: World: DIM_SPACESTATION_Mercury_7
[] [] []: Type: Mercury Space Station, Generator: default, Entities: 0, Loaded chunks: 256
[] [] []:  
[] [] []: World: DIMnull
[] [] []: Type: Titan, Generator: default, Entities: 0, Loaded chunks: 256
[] [] []:  
[] [] []: World: DIM_SPACESTATION5
[] [] []: Type: Space Station, Generator: default, Entities: 0, Loaded chunks: 256
[] [] []:  
[] [] []: World: DIM-1510
[] [] []: Type: Titania, Generator: default, Entities: 0, Loaded chunks: 256
[] [] []:  
[] [] []: World: DIM-1511
[] [] []: Type: Iapetus, Generator: default, Entities: 0, Loaded chunks: 256
[] [] []: ======================== « » =========================

We’ve got so many worlds, with ExtraPlanets. I’ve done everything from configuring sponge to setting the nice value for the java process and all spawned threads to -20. And that doesn’t even include the various issues I’ve raised on git in the process. We’ve got it running on a server that can turbo up to 4.6GHz with plenty of RAM to boot. Yet we are still hovering on 12 TPS with lag spikes. I have no idea what else you would need so here’s a good chunk of everything.

Our Global config

gist.github.com

https://gist.github.com/sfxworks/624a97a4a8fd29f87ec406d5c04a8618

gistfile1.txt


# 1.0
# 
# # If you need help with the configuration or have any questions related to Sponge,
# # join us at the IRC or drop by our forums and leave a post.
# 
# # IRC: #sponge @ irc.esper.net ( https://webchat.esper.net/?channel=sponge )
# # Forums: https://forums.spongepowered.org/
#

This file has been truncated. show original

A profiler

pie_flavor · October 28, 2018, 11:15pm

Have you got a timings report for us?

sfxworks · October 29, 2018, 12:27am

Is that not spark’s profiler?If not, what command would I need to run to generate the proper timing report for this board? It would help if you defined such a thing.

pie_flavor · October 29, 2018, 12:30am

To enable timings tracking: sponge timings on. To generate a report: sponge timings report.

sfxworks · October 29, 2018, 12:33am

Ah thanks. I will let this run for an hour I suppose and then provide a report then.

sfxworks · October 29, 2018, 1:32am

Here is our latest timing report.

https://timings.aikar.co/?id=6352a6a868194d7b85837dc7e90b2a35

pie_flavor · October 29, 2018, 1:51am

Looks like an obtuse amount of TiC slimes are spawning - a hundred and ninety thousand of them. Other than that, AE2 looks like a significant factor in the lagspikes - it’s taking almost 35% of each tick.

sfxworks · October 29, 2018, 1:59am

Oh man, do you have any suggestions for a plugin or mod to get rid of these? Or limit them? I mean, we have slime islands but man. One Hundred and Ninety Thousand??

sfxworks · October 29, 2018, 2:09am

Update: we have opted to use https://minecraft.curseforge.com/projects/bad-mobs to manage the slimes. Thank you so much for your help so far.

Hmm… That and a few other configs are not generating a new…

Does sponge have a native way of managing this @pie_flavor?

pie_flavor · October 29, 2018, 4:13am

Each plugin manages configuration generation itself. There’s default settings and all but actual loading and saving is handled by the plugin.

sfxworks · November 2, 2018, 5:46am

Lifesaver

sfxworks · November 2, 2018, 5:47am

Is there a guide on how to read those / identify the reports in the sponge timings?