Sometimes you can chock it up to Unity just being iffy, or Unity Editor overhead causing fps issues.
Pretty sure it's just the norm at this point, and you should probably only do performance testing with an actual build rather than within the editor itself.