On Mon, Feb 12, 2018 at 10:11:05AM +0000, James Chapman wrote:
> Since L2TP hooks on sockets opened by userspace using sk_user_data, we
> may race with other socket families that attempt to use the same
> socket.
>
> This problem was discovered by syzbot using AF_KCM. KCM has since been
> modified to use only TCP sockets to avoid hitting this issue but we
> should prevent such races in L2TP anyway.
>
> Fixes: c8fffcea0a079 ("l2tp: Refactor l2tp core driver to make use of the
> common UDP tunnel function")
> Reported-by: syzbot+8865eaff7f9acd593...@syzkaller.appspotmail.com
>
> Kernel BUG at net/l2tp/l2tp_ppp.c:176!
> invalid opcode: 0000 [#1] SMP KASAN
> Dumping ftrace buffer:
> (ftrace buffer empty)
> Modules linked in:
> CPU: 1 PID: 3503 Comm: syzkaller938388 Not tainted 4.15.0-rc7+ #181
> Hardware name: Google Google Compute Engine/Google Compute Engine
> RIP: 0010:pppol2tp_sock_to_session net/l2tp/l2tp_ppp.c:176 [inline]
> RIP: 0010:pppol2tp_sendmsg+0x512/0x670 net/l2tp/l2tp_ppp.c:304
> RSP: 0018:ffff8801d4887438 EFLAGS: 00010293
> RAX: ffff8801bfef2180 RBX: ffff8801bff88440 RCX: ffffffff84ffbca2
> RDX: 0000000000000000 RSI: ffff8801d4887598 RDI: ffff8801bff88820
> RBP: ffff8801d48874a8 R08: 0000000000000000 R09: 1ffff1003a910e17
> R10: 0000000000000003 R11: 0000000000000001 R12: ffff8801bfff9bc0
> R13: 0000000000000000 R14: 0000000000008000 R15: 0000000000000000
> FS: 0000000001194880(0000) GS:ffff8801db300000(0000) knlGS:0000000000000000
> CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> CR2: 0000000020ea0000 CR3: 00000001bfecf001 CR4: 00000000001606e0
> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> Call Trace:
> sock_sendmsg_nosec net/socket.c:628 [inline]
> sock_sendmsg+0xca/0x110 net/socket.c:638
> kernel_sendmsg+0x47/0x60 net/socket.c:646
> sock_no_sendpage+0x1cc/0x280 net/core/sock.c:2581
> kernel_sendpage+0xbf/0xe0 net/socket.c:3349
> kcm_write_msgs+0x404/0x1b80 net/kcm/kcmsock.c:646
> kcm_sendmsg+0x148d/0x22d0 net/kcm/kcmsock.c:1035
> sock_sendmsg_nosec net/socket.c:628 [inline]
> sock_sendmsg+0xca/0x110 net/socket.c:638
> ___sys_sendmsg+0x767/0x8b0 net/socket.c:2018
> __sys_sendmsg+0xe5/0x210 net/socket.c:2052
> SYSC_sendmsg net/socket.c:2063 [inline]
> SyS_sendmsg+0x2d/0x50 net/socket.c:2059
> entry_SYSCALL_64_fastpath+0x23/0x9a
> RIP: 0033:0x440159
> RSP: 002b:00007ffe74df8288 EFLAGS: 00000217 ORIG_RAX: 000000000000002e
> RAX: ffffffffffffffda RBX: ffffffffffffffff RCX: 0000000000440159
> RDX: 0000000000000000 RSI: 00000000201fcfc8 RDI: 0000000000000005
> RBP: 00000000006ca018 R08: 0000000000000000 R09: 0000000000000000
> R10: 0000000000000000 R11: 0000000000000217 R12: 0000000000401ac0
> R13: 0000000000401b50 R14: 0000000000000000 R15: 0000000000000000
> Code: c5 61 70 fc 48 8b 7d d0 e8 7c c2 5b fd 84 c0 74 0d e8 b3 61 70 fc 48 89
> df e8 3b 49 2f ff 41 bd f7 ff ff ff eb 86 e8 9e 61 70 fc <0f> 0b 41 bd 95 ff
> ff ff e9 74 ff ff ff e8 ec 32 a8 fc e9 77 fb
> RIP: pppol2tp_sock_to_session net/l2tp/l2tp_ppp.c:176 [inline] RSP:
> ffff8801d4887438
> RIP: pppol2tp_sendmsg+0x512/0x670 net/l2tp/l2tp_ppp.c:304 RSP:
> ffff8801d4887438
> ---
> net/l2tp/l2tp_core.c | 21 ++++++++++++++++++---
> net/l2tp/l2tp_ppp.c | 8 ++++++--
> 2 files changed, 24 insertions(+), 5 deletions(-)
>
> diff --git a/net/l2tp/l2tp_core.c b/net/l2tp/l2tp_core.c
> index 194a7483bb93..de7dce64173f 100644
> --- a/net/l2tp/l2tp_core.c
> +++ b/net/l2tp/l2tp_core.c
> @@ -1216,6 +1216,7 @@ static void l2tp_tunnel_destruct(struct sock *sk)
>
>
> /* Disable udp encapsulation */
> + write_lock_bh(&sk->sk_callback_lock);
> switch (tunnel->encap) {
> case L2TP_ENCAPTYPE_UDP:
> /* No longer an encapsulation socket. See net/ipv4/udp.c */
> @@ -1229,7 +1230,8 @@ static void l2tp_tunnel_destruct(struct sock *sk)
>
> /* Remove hooks into tunnel socket */
> sk->sk_destruct = tunnel->old_sk_destruct;
> - sk->sk_user_data = NULL;
> + rcu_assign_sk_user_data(sk, NULL);
> + write_unlock_bh(&sk->sk_callback_lock);
>
> /* Remove the tunnel struct from the tunnel list */
> pn = l2tp_pernet(tunnel->l2tp_net);
> @@ -1583,6 +1585,20 @@ int l2tp_tunnel_create(struct net *net, int fd, int
> version, u32 tunnel_id, u32
> }
> #endif
>
> + /* Assign socket sk_user_data. Must be done with
> + * sk_callback_lock. Bail if sk_user_data is already assigned.
> + */
> + write_lock_bh(&sk->sk_callback_lock);
> + if (sk->sk_user_data) {
> + err = -EALREADY;
> + write_unlock_bh(&sk->sk_callback_lock);
> + kfree(tunnel);
> + tunnel = NULL;
> + goto err;
> + }
> + rcu_assign_sk_user_data(sk, tunnel);
> + write_unlock_bh(&sk->sk_callback_lock);
> +
I'd rather use this code only to replace the direct ->sk_user_data
assignment (in the 'else' branch). It looks strange to assign it here
using the new locking scheme and let setup_udp_tunnel_sock() re-assign
it later, with the same value but without locking.
Of course, if every user of ->sk_user_data was updated to the new
locking scheme, that should kill the race without having to update
setup_udp_tunnel_sock(). But only KCM locks ->sk_callback_lock for now
and, as you pointed out, it doesn't act on UDP sockets anymore.
Therefore setting ->sk_user_data here rather than at its original place
only brings confusion in my opinion.
And using rcu_assign_sk_user_data() is a bit confusing too. We never treat
->sk_user_data as RCU protected in the rest of the code.
> /* Mark socket as an encapsulation socket. See net/ipv4/udp.c */
> tunnel->encap = encap;
> if (encap == L2TP_ENCAPTYPE_UDP) {
> @@ -1594,8 +1610,6 @@ int l2tp_tunnel_create(struct net *net, int fd, int
> version, u32 tunnel_id, u32
> udp_cfg.encap_destroy = l2tp_udp_encap_destroy;
>
> setup_udp_tunnel_sock(net, sock, &udp_cfg);
> - } else {
> - sk->sk_user_data = tunnel;
> }
