That darn cat, again

I weeded my gardens last night, finally. The grass was trying to re-take the large garden. The two flower gardens were not too bad. But there are many gaps in the lines of corn, beans, peas, etc. The neighbor cat is in the garden several times a day digging potty holes and digging out my little vegetable plants. He always picks a different spot to dig, too! Any suggestions about how to scare him off permanently? (without antagonizing the neighbors--they are nice, otherwise).