The island was not always empty. Every new world gets a random mix of abandoned buildings, ruins and caves.
Abandoned buildings
Barn, chapel, farmhouse, fisherman's hut, inn, manor house, smithy, toll house, guard hut and watermill.
Ruins
Bastion, bridge, city gate, cloister, coastal watch, crypt, market, shrine, signal post and tower stump. The vegetation on ruins changes with the seasons.
Caves
Split Fang, Mossy Spiral, Three Gates, Basalt Ridge, Wind-Scoured, Broken Crown, Reedwater Refuge, Silverbrook Refuge and two labyrinths. Caves have chambers, dripping water, a pool and cave spiders that leave you alone.