The difference between a sea-level and vacuum-optimized rocket engine mainly comes down to how they handle atmospheric pressure and the nozzle attached to the engine. Sea-level engines are designed to operate within Earth's atmosphere, where air pressure is high. They have smaller nozzles to prevent over-expansion of exhaust gases, which could reduce thrust if the exhaust spreads too thin.
Vacuum-optimized engines, on the other hand, operate in the vacuum of space, where there's no atmospheric pressure. They feature larger nozzles (often about 2 to 3 times bigger in terms of diameter) to allow exhaust gases to expand fully, maximizing thrust efficiency. In space, without atmospheric pressure, larger nozzles optimize the flow and energy of the gases, whereas small nozzles would limit the exhaust's expansion and waste potential thrust.
SpaceX's Falcon 9 with its Merlin engines is a good example of this, where the first stage which operates primarily in the atmosphere uses smaller sea level optimised nozzles ,and the Upper Stage which only operates in the vacuum of space, uses a single Merlin engine with a far larger Vacuum optimised nozzle.
Image Credit: Everyday Astronaut (X/Twitter)