One of the more significant annoyances in game are perhaps the Chaos tribes after the invasion (Varg and Skaeling). Even after Chaos is defeated they continue to send full stacks against the world. And with their own homeland being so hard to enter, it makes them a major annoyance in the late game.
My idea to deal with them is to give them major diplomacy penalties with each other after the invasion ends, along with major public order penalties to all regions save their original home regions. This would also be somewhat lore friendly; showing the fracturing and infighting of the tribes after the unifying force of chaos undivided is gone.
This way, we'd still have the occasional raid, but the late game focus could then shift to the other large factions.
What are your ideas?