When holding the seek shelter key (instead of just pressing and quickly releasing it), villagers decrease their speed considerably, turning them more vulnerable than they already are by default.
Bug spotted via Return of Rome game mode, using build 162286 on Windows 10 (Steam).