Like you said, assymetry is the key here -LVT-1 starts off weaker vs Ka Mi, M2A4 is much better than Ha Go, the M3 and Chi Ha Kai are on par, and the M8 and Chi Ha have some comparable HEAT capabilities, although in all these cases, the US vehicles are 25-48% at the top end faster, and have a faster acquisition time due to a faster turret traverse.
I think you underestimate maneouverability of tanks based on power to weight ratios, and this is where the US vehicles really shine. They are very agile and can both turn and climb very well. The same cannot be said for any of the Japanese vehicles, which are sluggish and very slow to accelerate.
The tanks speed in a straight line is only very situationally relevant on relatively flat ground, something that most Gaijin maps have in very very limited amounts.
EDIT, the LVT-1 has 6-12mm of armour vs the Ka Mi’s 10-12mm not including the turret mantlet. Both vehicles have the same top speed. The LVT has a power to weight ratio of 16.7 hp/t, whilst the Ka Mi has 12.63 hp/t - so less agile, the turret traverse on the LVT would be between 14-16 degrees per sec on manual traverse, arguably faster if powered compared to the 10 degrees per sec in the Ka Mi, the vertical guidance between elevation and depression would be very similar on both, and the ROF will be a big differentiator - the LVT is a HMG so can hose onto, suppress and overwhelm the Ka Mi with approx 576 rpm. The Ka Mi has a ROF of 11rpm, and so each is likely to kill each other with a solid hit, but if the Ka Mi misses its shot or the shot is somehow ineffectve/absorbed by the space/overpenetrates, its going to take another 5 or so sec to reload its next shot, whilst the LVT is hosing it with .50 cal.
Finally the pen of the .50 Cal AP round is approx 26mm at 500m and 30mm at around 100m so once close enough this thing can kill any of the Japanese tanks in game from the side and rear and the early ones from the front.