Getting legal advice from a forum is a REALLY bad idea, as you can see from the replies.
That said, I was in your shoes once. My advice to you is do not reply to any electronic communication that this person or their lawyer might send you. If the person wants to truly pursue you legally because he really does have a case, then his lawyer will send you a letter in the mail.
Don't bother getting a lawyer yourself until you get a Cease and Desist letter (or similar) from their lawyer via Postal Mail. If his lawyer cannot figure out where to send the letter, then you probably have no worries.
What you are experiencing is, unfortunately, a popular tactic some people take to quash competition or to get a domain (they do not legally have the right to) for cheap or nothing. I'm not saying this is your case, but it could be and this is why you need to pursue this with a cool head and wait for postal correspondence and refrain from any electronic communication with the accuser or his lawyer BEFORE you take any legal action yourself or engage in any communications with them.
Also, just because someone trademarked a word, name, etc, it does not mean the TM will hold up in a court. Many people have had to drop their TM by court order when the TM was contested. Unfortunately, it takes money to contest a TM. But my point is just because they own a TM does not mean the TM was approved by the USPTO properly in the first place. The USPTO does not check all legal angles before approving a TM, therefore some TMs actually hold no legal value even though they are active and registered.