soy and thyroid issues?

I eat a lot of soy products, and I recently read about how too much soy can lead to an underactive thyroid?  Should I be worried? What are your thoughts on this?

I have hypothryroidism (underactive thyroid) and I take medicine daily for it.  According to some sources I should be avoiding soy all together.  But most of the sources that are backed by medical studies say that if you are taking your medicine then you don't need to avoid soy.  It could vary by the individual but I eat soy daily and it doesn't affect my blood tests, my thyroid levels have been stable for 12 months and counting.

Also I should note that I didn't eat ANY soy leading up to being diagnosed, so --no, soy didn't cause my thyroid to stop putting out.

Then you'll really like this guy.

Soy is for dorks