I think you need to be a specific distance away from the monument. Just enough for them to spawn but not enough for them to despawn. Every time this happens, I'm able to hover over it and it quickly goes back to a reasonable limit.