This could indicate either a problem with the fan, which has accumulated dirt over time, or one of the internal components of the PSU. Switch on the PC and listen carefully to confirm that it’s the PSU fan and not the CPU fan or the hard disk. Usually, the noise will stop once the fan picks up speed, and you can ignore it temporarily.
It’s a good idea, however, to clean out the dirt around the PSU fan using a PC vacuum. This will increase the working life of the PSU fan as well as the PSU itself. If the fan stops working, the PSU will generate heat and cause more trouble. So a little prevention will save you a lot of headaches later.
If the sound is not from the fan but from within the PSU itself, then you may be able to service it. A PSU has no ‘user-serviceable’ parts, and it’s best left to a competent technician, although in most cases of component failure, you will have to replace the PSU.