Should I delete a computer virus or quarantine it?

Whenever I run my computer anti-virus software, it recommends that I quarantine viruses. But, personally, I don't want them anywhere on my computer! I just want to destroy them and get them as far away from my computer as possible. Is there any harm in deleting a virus or trojan from one's computer? Do viruses always have to be quarantined first? Awhile back, I was looking around inside of one of my older spyware zapper programs (Spybot) and came across a file filled with hundreds of zipped "viruses". Very spooky indeed! I guess spyware anti-virus programs "zip" the virus or trojan and that is how they quarantine the problem file. But, that doesn't sound very secure to me. What happens if one of those files is mistakenly unzipped by a novice user? Wouldn't that simply release the virus onto the computer again to do more damage? Wouldn't it just be better to delete the virus once and for all, instead of just putting it into quarantine?