This may be a discovery issue.
For syncthing to connect to a given device via a relay, it must know on which relay the given device is available on. Connecting two devices to the same relay doesn’t fix this: The relay server isn’t chatty by itself and won’t tell you about other connected peers.
Syncthing usually resolves this via discovery: Each device publishes the relay(s) it is available on to discovery. Then other devices retrieve this information from discovery and connect to them - via any of the available methods, which may include relays.
Since you are apparently running offline, global discovery likely does not work. You can manually override discovery by setting the relay address in the remote device’s addresses field (not only the local devices listen address).
Otherwise, self hosting global discovery and pointing your instance to that should work as well, enabling the “dynamic” remote addresses to work.