Actually I had thought the same thing about the armory, and also about the capitol. Once they are used there is no reason for them to continue to exist.
The US Capitol can create carts to build anything. Why did the Europeans not receive a benefit accordingly?
Block buffs should be replaced with something worthwhile. Instead of blocking, it makes drops cost less experience, and the upgrade would simulate a situation like local government.
Spy upgrade should be a passive ability that consumes coins per second. (Many coins /s).
There should be new improvements or bonuses depending on the civilization, for example Ottomans. The imams assure the creation of troops and the arrival of the military envoys will be garrisoned in the capitilio.
They could cover the shortcomings of each civilization in this way.