Chariot archers are a bit overpowered, but not really broken. The problem is magnified a bit by matchmaking - if you haven’t played AOE/ROR competitively before (or recently, anyway) and you’re playing vs people who have, then yes, you’ll probably lose a lot to chariots. If it wasn’t chariots it would be fast cavalry or a tool rush.
As someone who’s decent but by no means an expert, a few thoughts from playing people at a similar level:
If you go cavalry and I go chariot archers, all else being equal I have to wall in or die. You can build cav as soon as you hit bronze, I have to wait for wheel to research. Even if I get some CAs out, in low numbers cav beat CAs. Only in larger numbers does it swing the other way.
You can wall too. If you’re worried about speedy chariots avoiding your army and hitting your villagers, some selective walling can go along way. Yes I can get siege, but that removes the speed advantage.
If your cav rush is unsuccessful, try adding some slingers into the mix when you’re preparing to defend against a counterattack. Cav + slingers is pretty good against pure CA armies. Later bronze, try mixing in some compies - they’re cheap and do good damage.
Stonethrowers. A few games where I’ve been facing a significantly larger CA army, a couple of STs have swung things in my favour. They may be more or less effective depending on your micro and your opponent’s, but they can make all the difference.
Also, for games that are decided in bronze or early iron, it doesn’t matter whether a unit costs wood or gold. Neither is inherently more valuable in most games, it’s just a matter of how you allocate your villagers.
(Note: I don’t know how much of all this holds true at expert level, but hey, most of us will never get there anyway. You can definitely be competitive against most players without spamming CAs.)