So I’m Warhammer so far, your local garrisons come from two major sources: The town center and a defensive upgrade for that faction. Many factions also have a spellcaster provided once you upgrade their spellcaster hero building to a certain point.
In Rome 2, your town center had a pile of units it provided, as well as any building where you hired troops, and some government buildings also lended extra men and the like. There were far more buildings contributing defenders, and pretty much if you could hire from that building, it provided some.
I much prefer the Rome 2 method and provided defenders. This makes town management much more varied, as you have to balance how many defenders you want in the area against just spamming the same 1-2 money makers in every town across the Empire. With this would also come a sprinkling of other ways to make money, so not all would be lost.
The closest current comparison is the Reikland exclusive building the Reikfort. It not only allows you to recruit Reiksgard locally, but also provides a pair of Reiksgard defenders. I’d prefer this become the new standard for most buildings, so having more then a single building granting you global access to a troop isn’t the best course of action for so many of these troop buildings