Follow your long-term interest, I say; both are tough sledding for native speakers of Romance languages, so you'll need a few good overarching reasons to want to communicate.

For a useful comparison of the respective learning processes, check out John Pasden's Sinosplice:

http://www.sinosplice.com/life/archives/2008/06/25/learning-curves-chinese-vs-japanese

For what it's worth, my girlfriend learned Japanese and Chinese as second languages and is fonder of Japanese, but she's been at Chinese longer so burnout could be a factor.