Unfortunately the AI behavior will not change fundamental as the developers do not give full access to AI files. At the moment AI can be modified superficially, but it does not change anything drastically. Unfortunately the AI behavior in AoE 4 is rigidly programmed. There is a mod that improves the situation, but it can not solve all the major problems of AI. Also the AI doesn’t use most of the abilities of units and buildings . The AI is pretty limited at the moment.
Mod Mods Single - Age of Empires
Mod theme Jakcie A.I (Actual AI improvements) - #85 by JakcieChan1178
Thread discussing AI in general The AI is terrible - #183 by VisionedTuba52
In my main thread I described all the bugs I found in the new patch .
By the way you can get the AI to play instead of yourself and you can test the AI.
It’s easy to check. You must first install the mod Mods Single - Age of Empires 1 ( AI Battle) and then select the mode (where you select modes like Nomad) then set the command 1 if you want to be played by AI (AI takes control of the player, it is a cooperative mode with AI) or set the command 8 but then you disappear from the game at the beginning and only AI vs AI plays. This way you can check both directly AI script and observe from the side. I think you will find it useful 
By the way there is also a speed increase in the lower right corner of the screen (near the minicart) so that the testing process was much faster.
And if you want to play in co-op against the AI do not put cheater difficulty because the AI that will play in co-op with you will not have bonuses cheater difficulty and the enemy AI will have bonuses.
You can also use any AI mod remove and play with the original AI.
https://forums.ageofempires.com/uploads/default/original/3X/8/2/82c275908efa7b2d318d7dc77bf47d37d5702e6c.jpeg
Me and @JakcieChan1178 are trying to improve the AI in every way we can .
I have also sent a lot of bug reports to the developers but many of them have not been fixed. Let’s hope that in the next patches the AI will be improved.