i wouldnt use the steam friends thing for anything.
also yea connect ip:port, since it is a ded server you might have it on 27016 which you would have to type in, also sometimes if you dont type in the default port it wont let you connect until you do.
also try having him add the server to favorites including the port number