When you have a villager selected and right click an enemy farm it gives you the harvest farm symbol on your cursor, however you cannot harvest from it.
The villager will pause and then start attacking it.
However if you have your own farm nearby then the villager will not attack the enemy farm but instead start farming from your own farm.