The crux of the issues with most tank destroyers is that they lack the versatility of tanks, and are generally worse choices in most situations, except when engaging other tanks.
An example:
In Normandy, the US and Germans both have access to tank destroyers.
The US has access to the M10 and M18, with both having a similar design; 76 mm gun in an open-top turret, no co-axial or hull machine guns except for an AA oriented .50 cal used by the (Exposed for the M18) commander. They are accentuated on mobility, with a powerful gun in a turret, at the cost of armor and vulnerability to anything coming from above.
The Germans have access mainly to the Panzer IV/70 (and the Stug III G as well, although as a premium), which use a Pak42 75 mm gun with a co-axial MG42 in a low profile, sloped heavy frontal armor casemate design. You get excellent armor against anything the US has combined with a very good gun in a small package, at the cost of mobility and lack of ability to engage anything that isn’t directly in front of you.
In both cases, you get great design against other tanks, that generally fare very poorly against infantry (which is what you’ll encounter 90% of the time) due to the lack of MG and ammo and also make you substantially more vulnerable against almost everything else compared to a standard tank (US open-top and German lack of turret), and you’ll rarely get the type of terrain that actually favors you to exploit the few pros you get.
You’re much better off to get a Jumbo, Sherman 76 or Panther/Tiger which all have both good guns and/or plenty of machine guns to use, much greater amount of armor and still plenty good against other tanks. Also, TD are generally unlocked late in the current linear progression system, so you already have much better alternatives already available once you do get them, nullifying any kind of early unlock advantage they may have had.
TL:DR: There is no point in taking a TD versus a regular ol’ tank.
The best way to fix this is to give TD’s a unique advantage in spawn availability; by being separate from tank slots (Similarly how fighters and attack aircraft/bombers are in separate slot categories), TD’s now can be used as an additional tool to counter enemy tanks without sacrificing a precious tank slot in the process that can be used to support against infantry and everything else.
So there would be actually a good use for it, as it would be possible for a player to use a TD instead of a tank, which would generally be more freely available regardless of the number of deployed tanks. Or for a player to have both a TD and a tank, and if all tanks slots are used, can simply revert to a TD instead.