If you're in Canada, I'd recommend going with Scottish Gaelic. Ignore all that doom-and-gloom crap about statistics. I don't mean to be rude, but I get so sick of people boiling this wonderful, living language down to numbers and third-party observations of how active its community is.
I've learned this lanaguage to an advanced level and while I do wish there were more speakers worldwide, I don't go around thinking I wish I had learned Irish instead because of the number of speakers. Like you, I wanted to learn Gaelic because it's my ancestral lanaguage, and even though I've both Irish and Scottish blood too, I'm very glad I went with Scottish Gaelic.
I've used a Canadian long-distance learning program for the past 4 years and have been to Cape Breton to study Gaelic at the Gaelic College. I can honestly say there are some really great people in the Canadian Gaelic community, in Nova Scotia/Cape Breton primarily, but elsewhere in Canada. There's definitely more community and support for it in Canada than for Irish, and there's been a real push in the past few years to encourage growth in the Canadian Gaelic community, and their efforts are seeing some fruit. There are more classes available, more summer programs and weekend workshops, more events and ceilidhs, and just more people involved in general than when I started studying Gaelic 4 years ago. (I wish I was a Canadian so I could take advantage of these!) And there is plenty of literature, traditional and contemporary, in Scottish Gaelic, and more and more is becoming available each year, in print and digital.
Scottish Gaelic is a little less complex grammatically in regard to Irish, but both a pretty distinct from English, which can be intimidating. But they are learnable languages. Best of luck with which ever one you choose!
Some links for Scottish Gaelic:
BBC's online beginner's course in Gaelic (not the greatest but it will get you started)http://www.bbc.co.uk/alba/foghlam/beag_air_bheag/
A more comprehensive self-study online course--I've done this entire course with my online classes:http://www.taic.me.uk/taic.htm
I took four online classes with this program, and I can recommend their beginners class up to the first advanced course:http://www.gaelicacademy.ca/
The Gaelic College in Cape Breton--they have a summer program that you may wish to check out:http://gaeliccollege.edu/
Sìol Enterprises, a Scottish Gaelic book supplier in Canada:http://www.gaelicbooks.com/
The Gaelic Council of Nova Scotia: http://www.gaelic.ca/