Hi,
Have you tried particles?
bullets is really a matter of context. if you need a missile that has a trajectory, then simply fire a real physics gameObject with a collider etc etc, no need for raycasting then. also, if your missile has a complex shape, make sure you use a simple collider like a capsule or a box, else it will increase computation costs. Also play with the physics layer system to only have your missile collider work in a specific layer instead of trying to handl everything in the scene. that will help as well.
if your bullet is a straight line and you know the outcome even before firing, then don't use physics at all, simply animate an object. this will be simpler. and raycast only once, when the user press fire.
don't have a bullet raycasting as it flies, I can't think of an exmaple where it's needed, simply use a collider for this. That's exactly what colliders are meant to solve.
bye,
Jean