Pet frienly wasp killer?

Tweet

I believe that a family of wasps is trying to take up residence on our balcony int he cracks of the wood. I could just call maintenance but they are just going to hand me a can of wasp spray. I would rather not spray anything that toxic even near my home since we have pets. Lord know what could happen if the fumes did get into the house! Is there any pet friendly and possibly eco-friendly way to get rid of wasps?