Since the game pushes doomstacking and to always fill up to the full 20 units, what's the point of vanilla garrisons, which are low tier and low in number? They're pure AR fodder in no time. They contribute nothing but free XP to the attacker on top of him getting the settlement anyway. At the same time it encourages spamming the garrison building which turns lategame into one siege after another. Boring!
So here's what needs to happen:
1.Remove garrisons from all minor settlements. They're useless and only contribute to the enemy anyway
2.Make the garrison building's troops much stronger and more numerous while also reducing an invader's movement speed by 50% and disable underground movement in the region they stand in so they become actual useful barriers. Towers also need an extended firing arc, more firepower and the gate needs to be more resistant to getting broken
3.Make the garrison building chain lower growth and economical output by 25% in the whole province so you actually have to weigh its usefulness against its economic impact and building more than one becomes expensive. Also double its building time and have it destroyed whenever a settlement with it is successfully assaulted forcing a rebuild
4.Reduce the siege timer when an army camps inside a settlement to immediate starvation if it gets besieged, but reduce attrition rates from 50% to 10%
That way building garrisons strategically becomes actually important.