Absolutely Al - in fact I used to always prepare for multi-track recordings by laying down a click track first, to hold everything together metronomically, until I was talking about doing some stuff with a drummer friend of mine - he turned the air blue and insisted I should "let the music breathe more". He was right, I think.
I cannot remember a session or even a recording session where the speed did not vary a bit.
My only suggestion if anyone finds it hard to play along with less rigid rhythms is to keep going over your part with the "master" track in your headphones. If you don't fight it, then eventually you'll find yourself remembering automatically where the "hiccups" are in the master and find that you're playing exactly with them rather than against them. Being involved in multi-track recording is a funny business when all is said and done, but often great when it's all put together.