It depends on what you're selling. If the affiliate manager is saying it's got a 25% conversion rate, and after 100 clicks you've got a 5% or less. Dump it.
But if the AM is saying it's got a 10% and you've got a 5% after 100 clicks, you haven't got a statistically significant sample size to make a sound judgement yet. You'll need a larger sample size to see that first result was just a fluke or if the real rate of conversion is lower than the AM is saying (Helpful Hint: it always is. The question is, by how much?)
For me, I usually don't feel comfortable dumping a campaign until I've gone through 500 or 1000 clicks, and I feel pretty good that I've got a solid landing page and a good keyword list. Then, if after 500 or 1000 clicks it's still crap, I can blame the product, and not my own ineptness, and move on.
I've got a campaign that for the past 18 months has made me a consistent $9 or 10 thousand a month, but that I took significant losses on for the first month, through about 5,000 clicks.
But I stuck with it because the campaign had a 120-day cookie and I knew that it was working for other people. And, like clockwork, in the second month the sales picked up as visitors returned and made purchases. In fact, when I got Google slapped in late July I ended up making more money in the August (with no adword spend) while I worked to relaunch the campaign, than I had in with the adwords going in July. (But then September was a poor month, because I had to get the adwords running again, and returning cookie sales were dipping from my previous spend.)
If I'd ditched that campaign after only 100 or 1000 clicks, I would have missed out on a real winner.
Anyway, for me, there are so many variables at work in a campaign that I don't feel comfortable dumping a campaign after only 100 clicks. But, of course, I've also lost plenty of money in testing some real dogs.