Auto-save causes server to stop responding

[barnabwhy]

Expected behavior

Auto-save behaves normally and works without causing the server to hang.

Observed/Actual behavior

The server hangs for upwards of 10 seconds, sometimes over 30 which causes players to time out.

Steps/models to reproduce

Run a server for long enough for it to auto-save.

[18:35:32 ERROR]: --- DO NOT REPORT THIS TO PAPER - THIS IS NOT A BUG OR A CRASH  - 1.21-DEV-fe42d87 (MC: 1.21) ---
[18:35:32 ERROR]: The server has not responded for 10 seconds! Creating thread dump
[18:35:32 ERROR]: ------------------------------
[18:35:32 ERROR]: Server thread dump (Look for plugins here before reporting to Paper!):
[18:35:32 ERROR]: ------------------------------
[18:35:32 ERROR]: Current Thread: Server thread
[18:35:32 ERROR]:       PID: 39 | Suspended: false | Native: false | State: RUNNABLE
[18:35:32 ERROR]:       Stack:
[18:35:32 ERROR]:               [email protected]/sun.nio.ch.UnixFileDispatcherImpl.write0(Native Method)
[18:35:32 ERROR]:               [email protected]/sun.nio.ch.UnixFileDispatcherImpl.write(UnixFileDispatcherImpl.java:65)
[18:35:32 ERROR]:               [email protected]/sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:137)
[18:35:32 ERROR]:               [email protected]/sun.nio.ch.IOUtil.write(IOUtil.java:102)
[18:35:32 ERROR]:               [email protected]/sun.nio.ch.IOUtil.write(IOUtil.java:72)
[18:35:32 ERROR]:               [email protected]/sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:300)
[18:35:32 ERROR]:               [email protected]/sun.nio.ch.ChannelOutputStream.writeFully(ChannelOutputStream.java:68)
[18:35:32 ERROR]:               [email protected]/sun.nio.ch.ChannelOutputStream.write(ChannelOutputStream.java:105)
[18:35:32 ERROR]:               [email protected]/java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:125)
[18:35:32 ERROR]:               [email protected]/java.io.BufferedOutputStream.implFlush(BufferedOutputStream.java:252)
[18:35:32 ERROR]:               [email protected]/java.io.BufferedOutputStream.flush(BufferedOutputStream.java:240)
[18:35:32 ERROR]:               [email protected]/java.util.zip.DeflaterOutputStream.flush(DeflaterOutputStream.java:290)
[18:35:32 ERROR]:               [email protected]/java.io.BufferedOutputStream.implFlush(BufferedOutputStream.java:253)
[18:35:32 ERROR]:               [email protected]/java.io.BufferedOutputStream.flush(BufferedOutputStream.java:240)
[18:35:32 ERROR]:               [email protected]/java.io.DataOutputStream.flush(DataOutputStream.java:131)
[18:35:32 ERROR]:               [email protected]/java.io.FilterOutputStream.close(FilterOutputStream.java:184)
[18:35:32 ERROR]:               net.minecraft.nbt.NbtIo.writeCompressed(NbtIo.java:222)
[18:35:32 ERROR]:               net.minecraft.nbt.NbtIo.writeCompressed(NbtIo.java:174)
[18:35:32 ERROR]:               net.minecraft.world.level.saveddata.SavedData.save(SavedData.java:40)
[18:35:32 ERROR]:               net.minecraft.world.level.storage.DimensionDataStorage.lambda$save$0(DimensionDataStorage.java:129)
[18:35:32 ERROR]:               net.minecraft.world.level.storage.DimensionDataStorage$$Lambda/0x0000003002d80470.accept(Unknown Source)
[18:35:32 ERROR]:               [email protected]/java.util.HashMap.forEach(HashMap.java:1429)
[18:35:32 ERROR]:               net.minecraft.world.level.storage.DimensionDataStorage.save(DimensionDataStorage.java:127)
[18:35:32 ERROR]:               net.minecraft.server.level.ServerLevel.saveLevelData(ServerLevel.java:1350)
[18:35:32 ERROR]:               net.minecraft.server.level.ServerLevel.save(ServerLevel.java:1314)
[18:35:32 ERROR]:               net.minecraft.server.MinecraftServer.saveAllChunks(MinecraftServer.java:961)
[18:35:32 ERROR]:               net.minecraft.server.MinecraftServer.saveAllChunks(MinecraftServer.java:948)
[18:35:32 ERROR]:               net.minecraft.server.MinecraftServer.saveEverything(MinecraftServer.java:995)
[18:35:32 ERROR]:               net.minecraft.server.MinecraftServer.tickServer(MinecraftServer.java:1614)
[18:35:32 ERROR]:               net.minecraft.server.MinecraftServer.runServer(MinecraftServer.java:1308)
[18:35:32 ERROR]:               net.minecraft.server.MinecraftServer.lambda$spin$0(MinecraftServer.java:329)
[18:35:32 ERROR]:               net.minecraft.server.MinecraftServer$$Lambda/0x0000003001ccdf48.run(Unknown Source)
[18:35:32 ERROR]:               [email protected]/java.lang.Thread.runWith(Thread.java:1596)
[18:35:32 ERROR]:               [email protected]/java.lang.Thread.run(Thread.java:1583)
[18:35:32 ERROR]: ------------------------------
[18:35:32 ERROR]: --- DO NOT REPORT THIS TO PAPER - THIS IS NOT A BUG OR A CRASH ---
[18:35:32 ERROR]: ------------------------------
[18:35:36 WARN]: Can't keep up! Is the server overloaded? Running 14332ms or 286 ticks behind

Plugin and Datapack List

> plugins
[18:45:17 INFO]: Server Plugins (10):
[18:45:17 INFO]: Bukkit Plugins:
[18:45:17 INFO]:  - AxiomPaper, FileCleaner, floodgate, Geyser-Spigot, ImageOnMap, MapUpdater, ProtocolLib, SeeMore, spark, ViaVersion

> datapack list
[18:45:31 INFO]: There are 3 data pack(s) enabled: [vanilla (built-in)], [file/bukkit (world)], [paper (built-in)]
[18:45:31 INFO]: There are no more data packs available

Paper version

> version
[18:42:44 INFO]: Checking version, please wait...
[18:42:44 INFO]: This server is running Paper version 1.21-DEV-map-render-optimisation@fe42d87 (2024-07-21T15:09:30Z) (Implementing API version 1.21-R0.1-SNAPSHOT)
Unknown version
Previous version: 1.20.6-DEV-9c80169 (MC: 1.20.6)

Other

running #11000 on 1.21, issue is unrelated to the map rendering optimisations
wasn't an issue when running 1.20.6 build 147

[electronicboy]

That off-branch fork PR is ~50 commits behind master, missing important changes, such as incremental chunk saving, async world data saving, etc

[barnabwhy]

That off-branch fork PR is ~50 commits behind master, missing important changes, such as incremental chunk saving, async world data saving, etc

😭 i had rebased it earlier, i guess i didn't have all upstream changes synced