According to this survey, about 26% of Americans can speak a language other than English well enough to hold a conversation:
http://www.gallup.com/poll/1825/abou...-language.aspx
According to that same survey, about 55% of those bilingual Americans speak Spanish, about 17% speak French and about 10% speak German.
In other words, about 4.4% of Americans can speak French (26% X 17%). Multiply that by 250 million Americans (not the whole population but I'm excluding small children) gives you about 11 million French-speaking Americans.
According to this presentation from Gallaudet, there are about 250,000 to 500,000 ASL speakers in the US and Canada combined.
http://research.gallaudet.edu/Presen...04-04-07-1.pdf
In other words, the number of French speaking Americans greatly exceeds the number of ASL speaking Americans. Likewise, the number of German speaking Americans also greatly exceeds the number of ASL speaking Americans.