The Actual Factors
Samurai can beat all other infantry with equal resources, it’s fast attack rate and bonus damage makes it dangerous to Huskarls and Berserks, it can patrol easily into units, has a very fast creation speed and it pairs well with Kataparuto trebuchets. I can go on forever about them and that isn’t a joke. Here’s SotL’s video about them.
The problem is that no one is going to make infantry/slow UU against Japanese. They will make archer UU to snipe them from range or calvary UU to hit and run but most likely just generic units. It’s a tough sell most of the time because they die vs archers, too slow vs calvary, and are tough to mass vs generic units. Also your opponent needs to make their UU to let samurai have their full potential which they obviously won’t do most of the time. In theory, it’s a cool idea for a unit because you prevent your opponent from making their UU which is usually very good. In reality, generic units are the standard so it doesn’t really do much.
The only civ that allows the samurai to shine is against the goths, which is the only true “infantry” civ. Japanese militia line destroy goth militia line and samurai annihilate huskarls.