Villagers gathering boar:
Select villagers and press ‘G’ to seek shelter
Villagers run into nearby tower
Click tower and press ‘D’ for them to return to work
Villagers go back and stand idle next to boar
This is the same if you use the ‘Return all villagers to work (from Seek Shelter)’ hotkey.
Move Ger slightly further away in case it’s interferring somehow. Same result
Playing as Mongols on Arabia