Remove the purple and blue sub factions and have 3 spaced waves of significantly more powerfull chaos armies, each one arriving at a random one of 6 or so, fixed points based across the map.
Might be near you or might be far away.
The waves 1: 7 weak armies 2: 7 mid tier armies 3: 7 strong armies .first turn they arrive only have half stack or half health so if near you can try get em quick
Have to plan for them get 3 turns warning before each wave. Not location though
Have increased stats for when they fight other ai factions.
But I think having to dread a random location would be alot more thrilling than the anoying and persistent weak armies from the same places.
Infact having one randomly selected ai faction as well with more buffs than other ai, when facing ai, would allow for a more interesting and varied endgame. Having to deal with some new, decent lategame armies that wouldn't always be the same factions with chaos being a threat that's going to hurt your empire regardless of how good you are.