If I've read this correctly, then everything is probably working the way it should...

If you configure your home PC and mobile devices as though they are they physically in the office - i.e. use the 192.168.2... addresses for everything - then they should all work *when the VPN is established*.

(Simplistically), connecting the VPN makes you appear to be on the office LAN. In trying to access the 82...... public address, you're trying to loop around at the router end (i.e. go out and come back in again). I think some models actually allow this, but it wouldn't be a good idea in your scenario. (If it *did* work with the 82... address, it would still work if you forget to establish the VPN; there would be no indication that your data wasn't being protected)

Of course, it's entirely possible that I haven't read this correctly