(Your magic defense is still low, so saving against the spell doesn't happen that often.) That's not unusual, and intentional.
* Fire and Thunder do 20-40 damage on a failed save with heal potions restoring 30 HP out of battle (less in WSC/Origins), that means each time the spell is used, you're down a heal poition.
Having enemies cast spells early might be a bit of an issue, actually.