probability - How many rolls do I need to determine if my dice are fair?

Roughly how many times do I need to roll a 6-sided die to feel confident that it's giving "fair" results? What about a 10-sided or 20-sided die?

Note that I will be actually manually rolling physical dice, this isn't just a textbook exercise. I'd like to minimize how long it takes me to perform this experiment with each die :)

I know this depends on my expected "confidence level" (95%? 99%?) If I choose a 95% confidence, for example, does that imply that 1 out of 20 fair dice will fail this test? Or that a single fair dice would fail the test 1 out of 20 times? If so, that sounds fairly high.

Are there standard techniques for doing this kind of a test?

Edit: It is beyond the scope of the math-focused question I've asked here, but I've explained more about the overall testing scenario over on the stats site here: https://stats.stackexchange.com/questions/14301/designing-a-test-for-a-psychic-who-says-he-can-influence-dice-rolls/14302#14302