I don’t know why this is not in the game yet, to have a specific key to select military or villagers.
There should be a hotkey when you press it your drag selection only selects villagers just like in AOE2 with ALT. In AOE2 I have military only drag selection but when I want to select only villagers I press the ALT key and drag.
When you are getting raided and you want to micro is almost not possible, I either have to take time to select the villagers or double click all of them witch is not efficient.
This feature is on AOE2 so I don’t get it how is not in AOE4 too.