My reverse-ssh tunnel is using keepalives but they're not helping

My reverse-ssh tunnel is using keepalives but they're not helping

I have an ssh client machine picard behind multiple unreliable internet connections - all with NAT.
I have my server time, reliable with a static IP.
I want to be able to access picard thorugh time. I've done this before:
ssh -i <unencrypted key> -N -R 19999:localhost:22 user@my.domain
This works, but if there is a problem it exits and does not restart, and it doesn't start on boot, so now I add a sydtemd service to run:
/bin/bash -c "while true; do /usr/bin/ssh -i <unencrypted key> -o ServerAliveInterval=10 -v -o ServerAliveCountMax=6 -N -R 19999:localhost:22 user@my.domain; sleep 5; done"
while true ... sleep 5 re-runs ssh if it exits
-o ServerAliveInterval=10 sends a keep-alive every 10 secnods
-o ServerAliveCountMax=6 exits if 6 keep-alives go out with no response
-v keeps debug info in /var/log/messages through systemd
On the server side I added a couple of lines to sshd_config:
KeepAlive yes
ClientAliveInterval 10
ClientAliveCountMax 6
Same idea as the client - break the connection after 60s of inactivity.
Unfortunately it seems to take a lot longer than a minute to restart:
< tunnel is up and keepalives are coming in >
Jun  7 17:31:02 picard bash[135]: debug1: client_input_global_request: rtype keepalive@openssh.com want_reply 1
Jun  7 17:31:12 picard bash[135]: debug1: client_input_global_request: rtype keepalive@openssh.com want_reply 1
Jun  7 17:31:15 picard bash[135]: debug1: client_input_channel_open: ctype forwarded-tcpip rchan 2 win 2097152 max 32768
Jun  7 17:31:15 picard bash[135]: debug1: client_request_forwarded_tcpip: listen localhost port 19998, originator 127.0.0.1 port 38267
Jun  7 17:31:15 picard bash[135]: debug1: connect_next: host localhost ([127.0.0.1]:22) in progress, fd=4
Jun  7 17:31:15 picard bash[135]: debug1: channel 0: new [127.0.0.1]
Jun  7 17:31:15 picard bash[135]: debug1: confirm forwarded-tcpip
Jun  7 17:31:15 picard bash[135]: debug1: channel 0: connected to localhost port 22
Jun  7 17:31:20 picard systemd-logind[137]: New session 1 of user main_username.
< I break eth0 and plug it back in after NM sees it's down >
< eth0 is back up within a few seconds >
< nothing happens with my ssh connection for a LONG time >
Jun  7 17:54:16 picard bash[135]: Write failed: Broken pipe
Jun  7 17:54:22 picard bash[135]: OpenSSH_6.1p1, OpenSSL 1.0.1c-fips 10 May 2012
Jun  7 17:54:22 picard bash[135]: debug1: Reading configuration data /etc/ssh/ssh_config
Jun  7 17:54:22 picard bash[135]: debug1: /etc/ssh/ssh_config line 50: Applying options for *
Jun  7 17:54:22 picard bash[135]: debug1: Connecting to my.domain [123.234.123.234] port 22.
Jun  7 17:54:22 picard bash[135]: debug1: Connection established.
Jun  7 17:54:23 picard bash[135]: debug1: identity file /home/test/.ssh/id_rsa type 1
Jun  7 17:54:23 picard bash[135]: debug1: identity file /home/test/.ssh/id_rsa-cert type -1
Jun  7 17:54:23 picard bash[135]: debug1: Remote protocol version 2.0, remote software version OpenSSH_5.8p1 Debian-1ubuntu3
Jun  7 17:54:23 picard bash[135]: debug1: match: OpenSSH_5.8p1 Debian-1ubuntu3 pat OpenSSH_5*
Jun  7 17:54:23 picard bash[135]: debug1: Enabling compatibility mode for protocol 2.0
Jun  7 17:54:23 picard bash[135]: debug1: Local version string SSH-2