Ok, where's a guy draw the line? (hacking related)

I just installed Network ICE in my computer yesterday, and have already had two different attackers hit my comp that I'd have had no clue existed if I had not installed this proggie. The question I have is: At what point do you use countermeasures to deter someone from poking around in your PC? One of the two did intrude (at least that's what the program said) Which I guess means they got through the firewall, and could've screwed with me. What kinds of measures can I use without resorting to hacking them right back, and for that matter, at this point, why the hell not hack them back?