Unity relay server without matchmaking

Hi,

I have just started playing with Unet and wondering if I can use the relay server without unity matchmaking?

what I have in mind is

e.g. A player starts as a host and it's ip and port number get stored on a server. Then the host player can invite other players to join the game by sending them a request.

I imagine if I try to connect to host directly from the client, I would run into issues with NAT. So, how can I set this up to use the Unity relay server?