Hi,
that's one way yes.
Basically, you will need to split your game into levels when it will become too big to load. that's especialy true on mobile. you can't have the user wait 30 seconds to load everything... so that's when you start splitting everything into smaller scenes and lead them when required, either by completly reloading everything or by additively loading.
but from your questions, I think you are misunderstanding what prefabs are. Prefabs are reusable components, you can use them in different scenes in your project as well as in one scene, several times. It's very powerful, but not so much linked to levels that much. It's just a very convenient ( and a must know) feature of unity.
bye,
Jean