It's hard to say as they both handle badly, but I would say it's the difference between the standard Freelancer and the Freelancer MAX.
The MAX technically has bigger engines, but the added mass makes it fly badly compared to the standard version.
The Starfarer and Gemini have the exact same engines and dimensions, but the armor makes it weigh more. Not a huge amount, but about 7,000kg. It doesn't fly much worse, but it's noticeable. The biggest issue isn't the main engine thrust, it's the maneuvering thrusters. Since they're so small, they have trouble with the added mass. Same issue with the MAX.
I wouldn't make a decision on that alone, they both fly like bricks. But the Gemini flies like a brick that someone coated in bronze. Nothing I'd worry about. I'd make the decision based on the extra money it costs and lower cargo space VS the benefits listed above. (I'm not even 100% on a lower cargo space as the RSI website lists them as the same)