Many automotive parts are designed to accept a certain amount of surface "flash rust" with no deleterious effects. Disc brake rotors come readily to mind (removed by brake pad interaction).
Counterintuitively a thin layer of rust can actually curtail further rust in many cases. There are some commercial building exterior steel girders that are designed to form an initial layer of rust.
I'm pretty anal so I typically clean away same in the engine bay if I can reasonably reach it. If I can't I just don't worry about it (well, I try not to

).