It seems that sometimes you just cannot connect with a game if a particular person is in it. killedthadeus and I were never able to hook up. I did, however get some good racing in with teal and another guy (sorry I forgot the gamertag).
In doing so, I did some web research on connectivity issues.
It seems that when you go to a populated place, you are more likely to be running into server congestion and partitioned into separate servers (and hence not see each other in free drive mode). In our case I could even enter the DSP club house and not see the other 2 guys there, nor could they see me!
So here are the tips that will mostly alleviate difficulty in finding and staying connected with each other.
Three methods for connecting.
1. Go to your club. Pros are obvious. Cons are that our club is in Honolulu, which might have more server congestion than some other remote corner of the island.
2. Someone goes to a less busy part of the island. Everyone there hunts him down using the "Key Locations" filter on the map, and selecting friends. If you can warp to the person's location, and get him listed in the local free ride area, you can press left on the d-pad to get to the free ride players and LOCK that player, meaning he's going to be locked in the same session with you so you don't lose him.
3. Start a PRIVATE multiplayer race, and use the XBox dashboard friends panel to specifically invite people to the race.
I did get into a funky state where I could see the cars of killedthadeus and naples driving around in the parking lot of the club house, but they were REALLY REALLY choppy (like one update per second at best). And although I could see their cars chopping around, I couldn't see them in free ride.
Bottom line is that when TDU connects, racing is a lot of fun,but it seems getting all of us together is more trouble than its worth sometimes

Randy