So, I've never owned FMJs, so I can only speak to the specific complaints I've overheard, and not from experience.
One complaint is the high GPI in relation to the spine. With a straight carbon or alu shaft of the same spine, you could have the same TAW with a lighter shaft and heavier point, increasing your FOC.
One complaint that I've never really explored in any detail is that over time FMJs weaken. Something about bent or dented aluminum causing microfractures in the carbon, or something, IDK, just that a given FMJ might be weaker than a comprably-spined carbon or aluminum arrow. Or that as the shaft flexes through paradox over a large number of shots the dissimilar materials eventually begin to work against each other causing microfractures or adhesive failures or something along those lines. One of the haters can correct me on this. Really, this seems more like commenters arguing hypotheticals rather than actual experience.
The last one is more a criticism of the HIT system than FMJs per se, that hard hits without shock collars will force the HIT insert deeper into the shaft, causing it to splinter apart. I use 5mm Axis with Ethics components and avoid any HIT issues.
Out of curiosity, what induced you to chose FMJ shafts over straight carbon or aluminum? I went with the 5mm Axis because of the price, the generally positive reviews, and the available the Ethics components. I think my next set - whenever I get them, not this year - will be Victory VAPs, and I'll see what these micro-diameters are all about.