Help with L2TP VPN

I cannot get the RADIUS L2TP VPN to connect to any devices.

I’ve tried it on Windows 10, Android (OnePlus 8T) and a Galaxy S9+

This is the guide I followed: https://www.youtube.com/watch?v=-hFDmJi98mo, I’ve tried simple and complex keys, neither work.

My network is UAP-AC-Lite > USG-3P > Modem > WAN. There are no port forwardings blocking any of the RADIUS ports.

Controller Version: 6.2.26

The USG logs from when trying to connect with an android:

Sep 13 19:53:45 SecurityGateway xl2tpd[28014]: Connection established to 192.168.1.46, 42008.  Local: 18404, Remote: 18903 (ref=0/0).  LNS session is 'default'
Sep 13 19:53:45 SecurityGateway xl2tpd[28014]: Call established with 192.168.1.46, PID: 31297, Local: 29497, Remote: 17901, Serial: -1103168722
Sep 13 19:53:46 SecurityGateway pppd[31297]: pppd 2.4.4 started by root, uid 0
Sep 13 19:53:46 SecurityGateway pppd[31297]: Connect: ppp0 <-->
Sep 13 19:53:46 SecurityGateway zebra[758]: interface ppp0 index 19 <POINTOPOINT,NOARP,MULTICAST> added.
Sep 13 19:53:49 SecurityGateway zebra[758]: interface ppp0 mtu changed from 1500 to 1400
Sep 13 19:54:19 SecurityGateway pppd[31297]: rc_send_server: no reply from RADIUS server setup.ubnt.com:1812
Sep 13 19:54:19 SecurityGateway pppd[31297]: Peer andrew failed CHAP authentication
Sep 13 19:54:19 SecurityGateway zebra[758]: interface ppp0 mtu changed from 1400 to 1500
Sep 13 19:54:19 SecurityGateway pppd[31297]: Connection terminated: no multilink.
Sep 13 19:54:19 SecurityGateway zebra[758]: interface ppp0 index 19 deleted.
Sep 13 19:54:19 SecurityGateway xl2tpd[28014]: handle_avps: Bad exit status handling attribute 1 (Result Code) on mandatory packet.
Sep 13 19:54:19 SecurityGateway xl2tpd[28014]: network_thread: recvfrom returned error 146 (Connection refused)

I get the same error, when trying from a hotspot just with a external IP.

Any help would greatly appreciated, I don’t know what else to try.