The first thing you have to do is remove their food sources - any seed, dog/cat food etc. If you have to store it there, get plastic bins with screw tops to keep the mice out. If you don't mice will return again and again.

Beyond that I have no advice, since I don't hesitate in killing the plague carrying little *hits.

-Zeke