I’ve discovered that if u save the game after destroying the last column and getting your reinforcement notification, if you for some reason load back to that save the quest for destroying the Market town show in the top left corner. So u effectively can’t do the task. Also the voice ques don’t start either.
I tried for several hours to destroy the troops outside when first going to the castle, with just the reinforcement units until realising that something isn’t adding up. Hopefully no one else will have this same experience.