PSP 6.20 downgrader. . . is there such a thing?  

Which PSP version do you have? PSP 1xxx, 2xxx or 3xxx? If you have a PSP-3000 then its going to be pretty tricky to downgrade without bricking as its not pandora compatible (as far as I remember).

(This method will take you back to 5.00m33-6 ==> a custom firmware)
You're (best) safest bet is to invest in a pandora battery (they start from like $4.99 if you look hard enough).

(BEFORE YOU BEGIN: search Magic Stick Memory Guide 5.00m33-6)

Once you have that, you need to download PSP Grader v006. (I assume you have a memory stick which is larger than 128MB). After downloading and installing PSP Grader, connect your psp to your pc. Open the window which explores the contents of your memory stick. BACK UP all of the files from your memory stick onto your computer, just in case the install decides to mess itself up.

After backing up, disconnect the psp from your computer. On your psp, go to system settings and reformat your memory stick (only works with the PSP-1000 range, if you have a PSP-2000, you MUST use mspformat).

After reformatting, plug your psp back into the computer. Find the files that you have downloaded, and open 'Pre-made 5.00 m33-4). Drag all the files to your memory stick, clicking yes to all copy procedures.

Install PSP Grader. Once installed, open it. Don't worry about the top stuff, you will be interested in what it says under "Select IPL"; for that you must select none. Uncheck "format memory stick". Where it says "PSP memory stick", choose the drive where your psp is located.

At the bottom, click "Inject IPL only". A window will open showing the progress. Click ok at the bottom when it is finished. The Magic Memory stick is now created :)

Installing the custom firmware:
Unplug the psp, shut down by holding down the power button (NOT standby) until the psp is off completely. Remove the battery and disconnect any cables. Keep the Magic Memory stick inserted.
Hold the 'Up' button on the D-Pad. Insert the pandora battery WHILE STILL HOLDING the 'Up' button. The PSP should automatically turn on. A menu will appear; select 'Install 5.00 M33'.
Once it has installed, reinsert your normal battery. Then format your memory stick like you did for the preparations earlier. You can now restore all of the files you backed up onto your Memory Stick.

Turn on your PSP. Go to Settings/Network Update and complete the update to obtain the latest version!

Phew! It's a long task, but this is the safest way (that I have tried) to do it. I hope it helps..give us a shout on how it went

I think there is now a downgrader available in pspslimhacks.com