Extremely slow construction times

Removed initial builder by Town Bell button can cause extremely slow build rates. This bug sometimes happens because of other similar actions as well, but only the Town Bell button provides reliable reproduction.