Step 1: Remove Cover Screw in the Back

It's hidden under the sticker, use a torx screwdriver for that.

Step 2: Remove HDD Cover and HDD

This step is not absolutely necessary but I prefer it this way. Also avoids ripping off the plastic loop attached to the HDD.

Step 3: Remove the Bottom Cover

Remove the bottom cover by forcing it upwards at the edges (PS4 flipped over)

Step 4: Here the Annoying Buzzer

This is where the sound comes from.

We could cover the hole at the top with tape, that will decrease volume of the beep.

To disable it completely it will need to be removed.

Step 5: Unsolder the Buzzer

Get yourself a cheap soldering iron if you don't have one and remove the buzzer by heating the solder joints alternatively and forcing the it away from the board.

Step 6: Reassembly the PS4

Reassemble the PS4 in reverse order.

Enjoy totally silent startup!