go to the hardware store and pick up a package of terro. it's a liquid that you put on cardboard tabs wherever you find those things. they take it back to their nest and it kills them all off.

when i bought my house last september, it had been sitting for about 8 months, and when it started warming up, they started crawling out from under the sink. terro was the only thing that worked.