Add the range check back to do_proc_ulong_conv_ms_jiffies that commit
b96b5c6708 ("sysctl: Replace do_proc_do{int,ulong,uint}vec with
do_proc_vec") incorrectly removed. Append "_minmax" to the end of
do_proc_ulong_conv_ms_jiffies so it is clear that there should be a
range check.
Fixes: b96b5c6708 ("sysctl: Replace do_proc_do{int,ulong,uint}vec with do_proc_vec")
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Signed-off-by: Joel Granados <joel.granados@kernel.org>
Add the range check to do_proc_int_conv_ms_jiffies_minmax that commit
d174174c67 ("sysctl: replace SYSCTL_INT_CONV_CUSTOM macro with
functions") incorrectly removed.
Fixes: d174174c67 ("sysctl: replace SYSCTL_INT_CONV_CUSTOM macro with functions")
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Signed-off-by: Joel Granados <joel.granados@kernel.org>
Fahad Alharbi reported that recv(0, MSG_PEEK) triggers busy-wait
in unix_stream_read_generic() if recv() is blocking and the last
skb in the queue is MSG_OOB skb.
In such a situation, TCP returns 0 immediately regardless of
blocking or non-blocking.
Let's follow the behaviour.
Fixes: 314001f0bf ("af_unix: Add OOB support")
Reported-by: Fahad Alharbi <fahad@codepure.com>
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://patch.msgid.link/20260902202202.892676-3-kuniyu@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Fahad Alharbi reported that blocking recv(MSG_PEEK) could hog CPU
due to OOB skb.
In the following cases, manage_oob() skips OOB skb(s) and returns
NULL for the last recv(MSG_PEEK):
socketpair(AF_UNIX, SOCK_STREAM, 0, sk);
1) skb -> OOB skb -> NULL
send(sk[0], "ab", 2, MSG_OOB);
recv(sk[1], buf, 0, MSG_PEEK);
2) skb -> consumed OOB skb -> NULL
send(sk[0], "ab", 2, MSG_OOB);
recv(sk[1], buf, 1, MSG_OOB);
recv(sk[1], buf, 0, MSG_PEEK);
3) consumed OOB skb -> OOB skb -> NULL
send(sk[0], "a", 1, MSG_OOB);
recv(sk[1], buf, 0, MSG_OOB);
send(sk[0], "b", 1, MSG_OOB);
recv(sk[1], buf, 1, MSG_PEEK);
Then, @copied is 0 in unix_stream_read_generic() (zero-length buffer,
or non-OOB skb is not yet consumed), and unix_stream_data_wait() is
called.
However, it returns immediately because @last is not updated in
unix_stream_read_generic(), and the thread busy-waits for a new skb.
Let's update @last in manage_oob().
For MSG_PEEK, @last is updated with the skipped OOB, and for the
non-peek case, @last matches the returned value (when !copied)
because OOB is unlinked.
Note that manage_oob() is inlined and no stack canary is added.
Fixes: 22dd70eb2c ("af_unix: Don't peek OOB data without MSG_OOB.")
Reported-by: Fahad Alharbi <fahad@codepure.com>
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://patch.msgid.link/20260902202202.892676-2-kuniyu@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Currently, vxcan_dellink() unregisters both local and peer devices
synchronously under RTNL.
Once RTNL is removed, it can be called concurrently from different
netns.
Let's use xchg() and unregister_netdevice_queue_net() to support
per-netns device unregistration.
This way, each device is queued for destruction only once by
the winner of the race.
Note that the extra netdev_hold() ensures that @peer obtained by
the first xchg() is not freed during the subsequent access to
netdev_priv(peer). The 2nd xchg() overwrites @dev to balance
the refcount.
Tested:
1. Create two vxcan pairs (vxcan1-2, vxcan3-4) between two netns
(ns1 & ns2).
# ip netns add ns1
# ip netns add ns2
# ip -n ns1 link add vxcan1 type vxcan peer vxcan2 netns ns2
# ip -n ns1 link add vxcan3 type vxcan peer vxcan4 netns ns2
2. Run bpftrace to check if the same process does NOT
unregister the paired vxcan devices
# bpftrace -e '#include <linux/netdevice.h>
kprobe:free_netdev {
$dev = (struct net_device *)arg0;
printf("PID: %d | DEV: %s%s\n", pid, $dev->name, kstack());
}'
3. Remove vxcan2 in ns2 and check bpftrace output
# ip -n ns2 link del vxcan2
PID: 1524 | DEV: vxcan2
free_netdev+5
netdev_run_todo+4798
rtnl_dellink+1507
rtnetlink_rcv_msg+1791
netlink_rcv_skb+504
...
PID: 453 | DEV: vxcan1
free_netdev+5
netdev_run_todo+4798
process_scheduled_works+2538
worker_thread+1906
kthread+806
ret_from_fork+805
ret_from_fork_asm+17
4. Remove ns2 (thus vxcan4) and check bpftrace output
# ip netns del ns2
PID: 12 | DEV: vxcan4
free_netdev+5
netdev_run_todo+4798
default_device_exit_batch+2271
ops_undo_list+993
cleanup_net+1122
process_scheduled_works+2538
worker_thread+1906
kthread+806
ret_from_fork+805
ret_from_fork_asm+17
...
PID: 462 | DEV: vxcan3
free_netdev+5
netdev_run_todo+4798
process_scheduled_works+2538
worker_thread+1906
kthread+806
ret_from_fork+805
ret_from_fork_asm+17
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://patch.msgid.link/20260731231755.2474376-1-kuniyu@google.com
[mkl: fix indention struct vxcan_priv::peer_tracker]
Signed-off-by: Marc Kleine-Budde <mkl@pengutronix.de>
When a lower device is unregistered, amt_device_event() tries
to unregister its upper AMT device, but it has two problems.
1. amt_lookup_upper_dev() looks up an upper device in the
lower device's netns only
2. amt_device_event() unregisters a single upper device only
If AMT device is created on a lower device in another netns,
removing the lower device triggers the splat below and gets
stuck until all upper devices are removed. [0]
The cross-netns setup seems unintentional considering 1. and
the following points:
* amt_link_setup() sets dev->netns_immutable to true
* skb_scrub_packet() is not called in the fast path
* iproute2 binary fails to find cross-netns lower device via
link-netns:
# ip -n ns1 link add amt0 link-netns ns2 type amt dev veth1
Cannot find device "veth1"
Instead of supporting it properly and preparing for per-netns
netdev unreg, let's forbid cross-netns setup.
Note that the problem 2. needs a separate fix.
[0]:
WARNING: net/core/dev.c:12518 at unregister_netdevice_many_notify+0x1cce/0x2250, CPU#48: ip/2031
Modules linked in:
CPU: 48 UID: 0 PID: 2031 Comm: ip Not tainted 7.2.0-rc5+ #27 PREEMPT(full)
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-debian-1.17.0-1 04/01/2014
RIP: 0010:unregister_netdevice_many_notify (net/core/dev.c:12518)
Code: 89 ef e8 d5 52 ae fe e9 d0 f4 ff ff 48 8d 3d f9 3b 9c 02 48 c7 c6 c0 0b 63 84 ba ab 1f 00 00 67 48 0f b9 3a e9 65 ff ff ff 90 <0f> 0b 90 eb 81 48 8d 3d f6 3b 9c 02 48 c7 c6 c0 0b 63 84 ba e2 1f
RSP: 0018:ffffc90004abf160 EFLAGS: 00010212
RAX: ffff888104d38260 RBX: ffff88800b0911b8 RCX: dffffc0000000000
RDX: 0000000000000000 RSI: 0000000000000008 RDI: ffffffff85b9f880
RBP: ffffc90004abf2d0 R08: ffffffff85b9f887 R09: 1ffffffff0b73f10
R10: dffffc0000000000 R11: fffffbfff0b73f11 R12: ffff88800b091d08
R13: ffff88800b091178 R14: dffffc0000000000 R15: ffff88800b091000
FS: 00007f555b86c600(0000) GS:ffff8881942a0000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000562107d489c0 CR3: 0000000109a40002 CR4: 0000000000372ef0
Call Trace:
<TASK>
rtnl_dellink (net/core/rtnetlink.c:3632 net/core/rtnetlink.c:3674)
rtnetlink_rcv_msg (net/core/rtnetlink.c:7112)
netlink_rcv_skb (net/netlink/af_netlink.c:2556)
netlink_unicast (net/netlink/af_netlink.c:1319)
netlink_sendmsg (net/netlink/af_netlink.c:1900)
____sys_sendmsg (net/socket.c:775)
__sys_sendmsg (net/socket.c:2738)
do_syscall_64 (arch/x86/entry/syscall_64.c:63)
entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:121)
...
unregister_netdevice: waiting for veth0 to become free. Usage count = 7
ref_tracker: netdev@ffff88800d7496d8 has 3/3 users at
__netdev_adjacent_dev_insert (./include/linux/netdevice.h:4525 ./include/linux/netdevice.h:4554 net/core/dev.c:8791)
__netdev_upper_dev_link (net/core/dev.c:8879 net/core/dev.c:8963)
netdev_upper_dev_link (net/core/dev.c:9009)
amt_newlink (drivers/net/amt.c:3321)
Fixes: b9022b53ad ("amt: add control plane of amt interface")
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Reviewed-by: Taehee Yoo <ap420073@gmail.com>
Link: https://patch.msgid.link/20260807020326.2519445-1-kuniyu@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Kyle Zeng reported that GC could free a dead SCC partially.
The scenario is as follows:
1) Create two SCCs:
X -. A <-> B
^--'
2) Run the following concurrently:
2-1) send() sk-B to sk-B from sk-X
2-2) close() both A and B
At 2-1), there is a small window where unix_add_edges()
publishes a new edge (B <-> B) to GC but its skb is not queued
by skb_queue_tail().
If 2-2) completes before skb_queue_tail() and GC is triggered,
it judges A <-> B as dead, but B is not freed because GC cannot
collect the not-yet-queued skb holding the B <-> B edge.
X -. A <-> B -. This edge is visible
^--' ^..' but skb is not
This itself is not a problem since the next GC run will judge
B as dead as well and free it finally.
X -. A <.> B -.
^--' ^--'
However, X's SCC forces the next GC to call unix_walk_scc_fast(),
and it iterates over A through B's scc_entry.
Let's unlink scc_entry before freeing the vertex in unix_del_edge().
Fixes: 4090fa373f ("af_unix: Replace garbage collection algorithm.")
Reported-by: Kyle Zeng <kylebot@openai.com>
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Reviewed-by: Kyle Zeng <kylebot@openai.com>
Fixes: 4090fa373f ("af_unix: Replace garbage collection algorithm.").
Link: https://patch.msgid.link/20260804002155.2233594-1-kuniyu@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
pfcp_net_exit_rtnl() iterates pfcp devices whose sockets
are in the dying netns and queues them for destruction.
So the devices may reside in different netns.
Let's use unregister_netdevice_queue_net() to support per-netns
device unregistration.
list_del() is changed to list_del_init() to avoid queueing the
same device twice.
Even after pfcp_net_exit_rtnl() queues a cross-netns pfcp device,
pfcp_dellink() could be called concurrently for it (once RTNL is
removed). In such a case, __rtnl_net_unlock() will perform the
unregistration.
We can see pfcp0 below is unregistered by the per-netns work
instead of cleanup_net().
# bpftrace -e '#include <linux/netdevice.h>
kprobe:pfcp_dev_uninit {
$dev = (struct net_device *)arg0;
printf("PID: %d | DEV: %s%s\n", pid, $dev->name, kstack());
}
kprobe:pfcp_net_exit_rtnl {
printf("PID: %d%s\n", pid, kstack());
}' &
# ip netns add ns1
# ip netns add ns2
# ip -n ns1 link add pfcp0 link-netns ns2 type pfcp
# ip netns del ns2
PID: 12
pfcp_net_exit_rtnl+5
ops_undo_list+702
cleanup_net+1122
process_scheduled_works+2538
...
PID: 462 | DEV: pfcp0
pfcp_dev_uninit+5
unregister_netdevice_many_notify+7129
unregister_netdevice_many_net+1050
rtnl_net_work_func+136
process_scheduled_works+2538
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Reviewed-by: Simon Horman <horms@kernel.org>
Link: https://patch.msgid.link/20260731224406.2444121-3-kuniyu@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
struct pfcp_dev.net is the netns where the backend pfcp socket
resides.
struct pfcp_dev is linked to the pfcp_net.pfcp_dev_list of
the socket's netns.
During netns dismantle or module unload, pfcp_net_exit_rtnl()
iterates the list and queues devices for destruction regardless
of the devices' netns.
Thus, once RTNL is removed, the list can be modified concurrently
from different netns due to device removal.
Let's protect it with per-netns mutex.
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Reviewed-by: Simon Horman <horms@kernel.org>
Link: https://patch.msgid.link/20260731224406.2444121-2-kuniyu@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
geneve_exit_rtnl_net() iterates geneve devices whose sockets
are in the dying netns and queues them for destruction.
So the devices may reside in different netns.
Let's use unregister_netdevice_queue_net() to support per-netns
device unregistration.
list_del() is changed to list_del_init() to avoid queueing the
same device twice.
Even after geneve_exit_rtnl_net() queues a cross-netns geneve
device, geneve_dellink() can be called concurrently for it.
In such a case, __rtnl_net_unlock() will perform the unregistration.
Note that geneve uses register_pernet_subsys() instead of _device(),
so default_device_exit_batch() guarantees that the async per-netns
works are flushed before ->exit().
Tested:
1. Create geneve device across two netns.
# ip netns add ns1
# ip netns add ns2
# ip -n ns1 link add geneve0 link-netns ns2 type geneve external
2. Run bpftrace to check that geneve_uninit() is called between
->exit_rtnl() and ->exit().
# bpftrace -e '#include <linux/netdevice.h>
kprobe:geneve_uninit {
$dev = (struct net_device *)arg0;
printf("PID: %d | DEV: %s%s\n", pid, $dev->name, kstack());
}
kprobe:geneve_exit_rtnl_net,
kprobe:geneve_exit_net {
printf("PID: %d%s\n", pid, kstack());
}'
3. Remove the netns where the geneve socket resides
# ip netns del ns2
Now, we can see geneve0 is unregistered by per-netns work
instead of cleanup_net() and it finishes before ->exit() to
avoid WARN_ON_ONCE(!list_empty(&gn->sock_list)) there.
PID: 571
geneve_exit_rtnl_net+5
ops_undo_list+702
cleanup_net+1122
process_scheduled_works+2538
...
PID: 1047 | DEV: geneve0
geneve_uninit+5
unregister_netdevice_many_notify+7129
unregister_netdevice_many_net+1050
rtnl_net_work_func+136
process_scheduled_works+2538
...
PID: 571
geneve_exit_net+5
ops_undo_list+1064
cleanup_net+1122
process_scheduled_works+2538
...
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://patch.msgid.link/20260731164612.2148830-4-kuniyu@google.com
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
struct geneve_dev.net is the netns where the backend geneve
socket resides.
struct geneve_dev is linked to the geneve_net.geneve_list of
the socket's netns.
During netns dismantle or module unload, geneve_exit_rtnl_net()
iterates the list and queues devices for destruction regardless
of devices' netns.
Moreover, a socket can be shared by multiple geneve devices in
different netns, and geneve_open() and geneve_stop() modify
geneve_sock.vni_list and geneve_net.sock_list.
Thus, once RTNL is removed, the three lists can be modified
concurrently from different netns due to device removal and
link-up/down.
Let's protect them with per-netns mutex.
geneve_newlink() is still protected by rtnl_net_lock()s, so
acquiring gn->lock twice in geneve_find_dev() and
geneve_configure() is not a problem.
Note that udp_tunnel_notify_add_rx_port() is moved outside of
the mutex, otherwise gn->lock -> utn->lock ordering would trigger
AB-BA deadlock in geneve_offload_rx_ports(), which acquires
gn->lock under utn->lock. Even without gn->lock, geneve_sock_add()
and geneve_offload_rx_ports() are still serialised with (per-netns)
RTNL, so there is no race.
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://patch.msgid.link/20260731164612.2148830-3-kuniyu@google.com
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
Currently, geneve->sock[46].hlist[46] is unliked from
geneve_sock.vni_list in geneve_stop() and geneve_sock.refcnt is
decremented for each socket later in __geneve_sock_release().
The following patch will introduce a mutex in geneve_net to
protect geneve_sock.{refcnt,vni_list}.
However, udp_tunnel_notify_del_rx_port() must be outside of the
lock to avoid AB-BA deadlock.
To make the change cleaner, let's move hlist_del_init_rcu()
from geneve_stop() to __geneve_sock_release().
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://patch.msgid.link/20260731164612.2148830-2-kuniyu@google.com
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
When a lower device is unregistered, its upper ipvlan devices
must also be unregistered. However, these upper devices may
reside in different netns than the lower device.
Let's use unregister_netdevice_queue_net() to support per-netns
device unregistration for ipvlan.
The new dying flag in struct ipvl_dev is used to avoid a race
that ipvlan_link_delete() is called while its lower device is
being removed in ipvlan_device_event().
If dying is true in ipvlan_link_delete(), the ipvlan device is
already destructed but not yet unregistered. In this case,
unregistration will be done in __rtnl_net_unlock() of the
->dellink() caller.
Tested:
1. Create veth in ns1 and two ipvlan devices in ns2 and ns3.
# ip netns add ns1
# ip netns add ns2
# ip netns add ns3
# ip -n ns1 link add veth0 type veth peer veth1
# ip -n ns2 link add ipvl2 link veth0 link-netns ns1 type ipvlan mode l2
# ip -n ns3 link add ipvl3 link veth0 link-netns ns1 type ipvlan mode l2
2. Run bpftrace to check that veth is unregistered first but
wait ipvlan to be unregistered
# bpftrace -e '#include <linux/netdevice.h>
kprobe:ipvlan_uninit,
kprobe:veth_dellink,
kprobe:free_netdev {
$dev = (struct net_device *)arg0;
printf("PID: %d | DEV: %s%s\n", pid, $dev->name, kstack());
}'
3. Remove the lower veth0 in ns1.
# ip -n ns1 link del veth0
We can see that veth0 is freed after unregistering ipvl2 and ipvl3
in per-netns work because ipvl_port holds refcount of veth0.
PID: 2010 | DEV: veth0
veth_dellink+5
rtnl_dellink+1213
rtnetlink_rcv_msg+1791
...
PID: 440 | DEV: ipvl2
ipvlan_uninit+5
unregister_netdevice_many_notify+7129
unregister_netdevice_many_net+1050
rtnl_net_work_func+136
process_scheduled_works+2538
...
PID: 440 | DEV: ipvl2
free_netdev+5
netdev_run_todo+4798
process_scheduled_works+2538
...
PID: 440 | DEV: ipvl3
ipvlan_uninit+5
unregister_netdevice_many_notify+7129
unregister_netdevice_many_net+1050
rtnl_net_work_func+136
process_scheduled_works+2538
...
PID: 2010 | DEV: veth0
free_netdev+5
netdev_run_todo+4798
rtnl_dellink+1507
rtnetlink_rcv_msg+1791
...
PID: 440 | DEV: ipvl3
free_netdev+5
netdev_run_todo+4798
process_scheduled_works+2538
...
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://patch.msgid.link/20260703001009.1572444-15-kuniyu@google.com
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
struct ipvl_port is shared between a lower device and its upper
ipvlan devices.
All upper devices are linked to ipvl_port.ipvlans.
Once RTNL is removed, the list can be modified concurrently from
different netns due to device removal.
Let's protect it with a per-port mutex.
NETDEV_PRECHANGEUPPER and NETDEV_CHANGEUPPER are explicitly
skipped to avoid deadlock for netdev_upper_dev_unlink() called
from NETDEV_UNREGISTER.
Note that __ipvtap_dellink_ptr is added for CONFIG_IPVLAN=y
but CONFIG_TAP=m and CONFIG_IPVTAP=m.
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://patch.msgid.link/20260703001009.1572444-14-kuniyu@google.com
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
ipvlan_uninit() for the last ipvlan device resets the lower device's
rx_handler_data to NULL.
Once RTNL is removed, ipvlan_init() would race with ipvlan_uninit(),
which could leak a newly allocated ipvl_port.
ipvlan_init() ipvlan_uninit()
| |- if (refcount_dec_and_test(old_port))
... |- ipvlan_port_destroy(old_port)
| '
|- refcount_inc_not_zero(old_port) <-- fails
|- ipvlan_port_create(phy_dev) .
|- new_port = kzalloc() |
|- phy_dev->rx_handler_data = new_port
|- phy_dev->rx_handler_data = NULL
...
`- kfree(old_port);
Let's synchronise the two by holding the lower device's netdev_lock().
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://patch.msgid.link/20260703001009.1572444-13-kuniyu@google.com
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
struct ipvl_port is shared between a lower device and its upper
ipvlan devices.
While each upper device can always access ipvl_port safely via
ipvlan_dev.port, the lower device relies on RTNL to access it
via net_device.rx_handler_data.
Once RTNL is removed, the lower device cannot read ipvl_port safely
in ipvlan_device_event() because the port could be freed concurrently
and net_device.rx_handler_data is set to NULL if the last ipvlan
device in another namespace is unregistered.
Let's convert ipvl_port.count to refcount_t and use RCU along with
refcount_inc_not_zero() in ipvlan_device_event().
netdev_put() in ipvlan_port_destroy() is also moved down after
cancel_work_sync(), which is the last user of port->dev.
Note that ipvlan->port is now set in ipvlan_init() so that it can
be used in ipvlan_uninit(), instead of ipvlan_port_get_rtnl()
(rtnl_dereference()).
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://patch.msgid.link/20260703001009.1572444-12-kuniyu@google.com
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
bareudp_exit_rtnl_net() iterates bareudp devices whose sockets
are in the dying netns and queues them for destruction.
So the devices may reside in different netns.
Let's use unregister_netdevice_queue_net() to support per-netns
device unregistration.
list_del() is changed to list_del_init() to avoid queueing the
same device twice.
Even after bareudp_exit_rtnl_net() queues a cross-netns bareudp
device, bareudp_dellink() could be called concurrently for it
(once RTNL is removed). In such a case, __rtnl_net_unlock() will
perform the unregistration.
Note that bareudp uses register_pernet_subsys() instead of _device(),
so default_device_exit_batch() guarantees that the async per-netns
works are flushed before ->exit().
Tested:
1. Create bareudp device across two netns.
# ip netns add ns1
# ip netns add ns2
# ip -n ns1 link add bareudp0 link-netns ns2 type bareudp \
dstport 9292 ethertype ipv4
2. Run bpftrace to check that bareudp_uninit() is called between
->exit_rtnl() and ->exit().
# bpftrace -e '#include <linux/netdevice.h>
kprobe:bareudp_uninit {
$dev = (struct net_device *)arg0;
printf("PID: %d | DEV: %s%s\n", pid, $dev->name, kstack());
}
kprobe:bareudp_exit_rtnl_net,
kprobe:bareudp_exit_net {
printf("PID: %d%s\n", pid, kstack());
}'
3. Remove the netns where the bareudp socket resides
# ip netns del ns2
Now, we can see bareudp0 is unregistered by per-netns work
instead of cleanup_net() and it finishes before ->exit() to
avoid WARN_ON_ONCE(!list_empty(&bn->bareudp_list)) there.
PID: 576
bareudp_exit_rtnl_net+5
ops_undo_list+702
cleanup_net+1122
process_scheduled_works+2538
...
PID: 470 | DEV: bareudp0
bareudp_uninit+5
unregister_netdevice_many_notify+7129
unregister_netdevice_many_net+1050
rtnl_net_work_func+136
process_scheduled_works+2538
...
PID: 576
bareudp_exit_net+5
ops_undo_list+1064
cleanup_net+1122
process_scheduled_works+2538
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://patch.msgid.link/20260703001009.1572444-11-kuniyu@google.com
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
struct bareudp_dev.net is the netns where the backend bareudp
socket resides.
struct bareudp_dev is linked to the bareudp_net.bareudp_list of
the socket's netns.
During netns dismantle or module unload, bareudp_exit_rtnl_net()
iterates the list and queues devices for destruction regardless
of the devices' netns.
Thus, once RTNL is removed, the list can be modified concurrently
from different netns due to device removal.
Let's protect it with per-netns mutex.
bareudp_newlink() is still protected by rtnl_net_lock()s, so
acquiring bn->lock twice in bareudp_find_dev() and
bareudp_configure() is not a problem.
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://patch.msgid.link/20260703001009.1572444-10-kuniyu@google.com
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
Currently, veth_dellink() unregisters both local and peer devices
synchronously under RTNL.
Once RTNL is removed, it can be called concurrently from different
netns.
Let's use xchg() and unregister_netdevice_queue_net() to support
per-netns device unregistration.
This way, each device is queued for destruction only once by
the winner of the race.
Note that the extra netdev_hold() ensures that @peer obtained by
the first xchg() is not freed during the subsequent access to
netdev_priv(peer). The 2nd xchg() overwrites @dev to balance
the refcount.
Tested:
1. Create two veth pairs (veth1-2, veth3-4) between two netns
(ns1 & ns2).
# ip netns add ns1
# ip netns add ns2
# ip -n ns1 link add veth1 type veth peer veth2 netns ns2
# ip -n ns1 link add veth3 type veth peer veth4 netns ns2
2. Run bpftrace to check if the same process does NOT
unregister the paired veth devices
# bpftrace -e '#include <linux/netdevice.h>
kprobe:free_netdev {
$dev = (struct net_device *)arg0;
printf("PID: %d | DEV: %s%s\n", pid, $dev->name, kstack());
}'
3. Remove veth2 in ns2 and check bpftrace output
# ip -n ns2 link del veth2
PID: 2194 | DEV: veth2
free_netdev+5
netdev_run_todo+4798
rtnl_dellink+1507
rtnetlink_rcv_msg+1791
...
PID: 448 | DEV: veth1
free_netdev+5
netdev_run_todo+4798
process_scheduled_works+2538
...
4. Remove ns2 (thus veth4) and check bpftrace output
# ip netns del ns2
PID: 571 | DEV: veth4
free_netdev+5
netdev_run_todo+4798
default_device_exit_batch+2271
ops_undo_list+993
cleanup_net+1122
process_scheduled_works+2538
...
PID: 441 | DEV: veth3
free_netdev+5
netdev_run_todo+4798
process_scheduled_works+2538
...
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://patch.msgid.link/20260703001009.1572444-9-kuniyu@google.com
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
For per-netns device unregistration, the list passed to
unregister_netdevice_many() must contain devices from a single
netns only (once all callers are converted).
Let's move collected devices in the following functions to
net->dev_unreg_head and let __rtnl_net_unlock() pass them to
unregister_netdevice_many().
* default_device_exit_batch()
* ops_exit_rtnl_list()
* __rtnl_kill_links()
This allows incremental conversion of each driver to support
per-netns device unregistration without affecting the normal
kernel where CONFIG_DEBUG_NET_SMALL_RTNL is disabled.
Note that this change unbatches synchronize_rcu() etc in
unregister_netdevice_many(), but we can later split it into
multiple stages to batch them again.
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://patch.msgid.link/20260703001009.1572444-8-kuniyu@google.com
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
When we need to unregister a netdev in a different netns, we will
delegate its unregistration to per-netns work.
There are three types of such cross-netns devices:
1. Paired devices (e.g., netkit, veth, vxcan)
-> Unregistering one device also deletes its peer, which
may reside in another netns.
2. Tunnel devices (e.g., bareudp, geneve, etc)
-> Destroying a netns removes devices in another netns if
their backend sockets reside in the dying netns
3. Stacked devices (e.g., ipvlan, macvlan, etc)
-> Removing the lower device also removes multiple upper
devices, each of which may reside in different namespaces.
In these cases, we will use unregister_netdevice_queue_net() to
queue such potential cross-netns devices for destruction.
Each driver must not call both unregister_netdevice_queue_net()
and unregister_netdevice_queue() for the same device. See the
subsequent veth/bareudp/ipvlan patches for how they avoid double
queueing.
unregister_netdevice_queue_net() takes net and dev. If dev resides
in the net, it simply calls unregister_netdevice_queue().
If dev_net(dev) is different from the net, it enqueues the device
to dev_net(dev)->dev_unreg_head and schedules the per-netns work.
When __rtnl_net_unlock() is called from the per-netns work (or another
thread already holding the lock), unregister_netdevice_many_net()
collects the queued devices and calls unregister_netdevice_many()
to perform the actual unregistration.
During netns dismantle, rtnl_net_flush_workqueue() is called at the
end of default_device_exit_batch() to ensure that cross-netns
devices in the other alive netns are unregistered.
Once RTNL is removed, a device could be moved to another netns while
being queued to net->dev_unreg_head.
__dev_change_net_namespace() handles this race by acquiring
net->dev_unreg_lock of both the old and new netns after dev_set_net()
and moving the device between their dev_unreg_head lists.
Since dev_set_net() and unregister_netdevice_queue_net() are
synchronised by netdev_lock(), the device is either queued to the
old netns's dev_unreg_head and then moved, or queued directly to
the new netns.
Note that unregister_netdevice_move_net() does not need to call
rtnl_net_queue_work() because __dev_change_net_namespace() is
(supposed to be) called with rtnl_net_lock(). (Not all callers
hold it yet, but the race does not happen until all callers
are converted and RTNL is removed.)
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://patch.msgid.link/20260703001009.1572444-7-kuniyu@google.com
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
Currently, netdev_run_todo() processes pending devices from multiple
namespaces in a batch.
To expand the per-netns RTNL coverage for NETDEV_UNREGISTER, let's
acquire __rtnl_net_lock() in netdev_wait_allrefs_any().
Note that netdev_run_todo() itself will need to be namespacified
before RTNL is removed.
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://patch.msgid.link/20260703001009.1572444-6-kuniyu@google.com
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
The biggest blocker to per-netns RTNL is netdev unregistration.
It starts within a single netns (e.g., during a device lookup or
netns dismantle), but it can eventually involve multiple namespaces,
such as when upper ipvlan devices reside in different netns.
This prevents us from acquiring multiple rtnl_net_lock()s beforehand.
When we encounter such a cross-netns device, we must delegate the
unregistration to the work of the netns where the device actually
resides.
Let's add per-netns rtnl_work to support the deferred netdev
unregistration.
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://patch.msgid.link/20260703001009.1572444-4-kuniyu@google.com
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
When rtnl_link_unregister() is called during module unload, it
calls __rtnl_kill_links() for every netns.
__rtnl_kill_links() collects all devices of the unloaded module
and passes them to unregister_netdevice_many().
Let's move unregister_netdevice_many() to rtnl_link_unregister()
to unregister all devices across netns in a single batch.
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://patch.msgid.link/20260703001009.1572444-3-kuniyu@google.com
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
There are a few cases where rtnl_net_lock() is not properly
held in rtnl_newlink().
When either of IFLA_NET_NS_PID / IFLA_NET_NS_FD / IFLA_TARGET_NETNSID
is specified but IFLA_LINK_NETNSID is not, sock_net(skb->sk) is used
as link_net in rtnl_newlink_link_net().
In addition, the do_setlink() path uses sock_net(skb->sk) and one
from the three netns attributes while rtnl_link_get_net_capable()
returns only one of four.
Let's add sock_net(skb->sk) to rtnl_nets in rtnl_newlink().
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://patch.msgid.link/20260703001009.1572444-2-kuniyu@google.com
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
Now, RTM_DELRULE no longer needs RTNL, and the only RTNL dependant
in RTM_NEWRULE is fib_unmerge(), which is called for the first
IPv4 rule.
Let's add fib_rules_ops.need_rtnl() and hold RTNL only for the
first IPv4 rule.
Tested:
The script below creates 1K rules in parallel in 4K netns, and
it got 20x/30x faster for IPv4/IPv6.
#!/bin/bash
N=4096
F=rules.txt
for i in $(seq $N); do ip netns add ns-$i; done
printf 'rule add from all table %d\n' {1..1024} > $F
for v in 4 6; do
echo "=== IPv${v} ==="
time { for i in $(seq $N); do nsenter \
--net=/var/run/netns/ns-$i ip -$v -batch $F & done; wait; }
done
for i in $(seq $N); do ip netns del ns-$i; done
rm -f $F
Without this series:
# ./test.sh
=== IPv4 ===
real 0m22.752s
user 0m7.834s
sys 92m46.721s
=== IPv6 ===
real 0m35.181s
user 0m8.635s
sys 142m30.479s
With this series:
# ./test.sh
=== IPv4 ===
real 0m0.918s
user 0m5.675s
sys 2m7.024s
=== IPv6 ===
real 0m1.214s
user 0m7.917s
sys 4m19.489s
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://patch.msgid.link/20260629181226.1929658-10-kuniyu@google.com
Reviewed-by: Ido Schimmel <idosch@nvidia.com>
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
We will no longer hold RTNL for RTM_NEWRULE and RMT_DELRULE
except for the first IPv4 RTM_NEWRULE.
Let's covnert __dev_get_by_name() in fib_nl2rule_rtnl() to
dev_get_by_name_rcu() and rename it to fib_nl2rule_locked().
Note that dev_get_by_name_rcu() must be called inside ops->lock
to serialise fib_rules_event() by __dev_change_net_namespace().
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://patch.msgid.link/20260629181226.1929658-9-kuniyu@google.com
Reviewed-by: Ido Schimmel <idosch@nvidia.com>
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
Now, fib_rule structs are protected by per-fib_rules_ops mutex.
Let's drop ASSERT_RTNL_NET() and rtnl_dereference().
Note that fib_rules_event() iterates over net->rules_ops without
net->rules_mod_lock, but this is fine because all fib_rule users
are built-in and concurrent fib_rules_unregister() does not happen.
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://patch.msgid.link/20260629181226.1929658-8-kuniyu@google.com
Reviewed-by: Ido Schimmel <idosch@nvidia.com>
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
fib_newrule() will drop RTNL except for the first IPv4 rule.
net->ipv4.fib_table_hash[] will be read with no protection,
but this is fine because fib_table is not destroyed until
netns dismantle except for the merged main/local table.
fib_unmerge() will continue to be called under RTNL, so other
readers (fib_flush() and fib_info_notify_update()) just have
to care about the concurrent hlist_add().
IPv6 and IPMR/IP6MR also take this strategy and use RCU helpers
to avoid data race against concurrent hlist_add().
Let's not use lockdep_rtnl_is_held() and rcu_dereference_rtnl()
for net->ipv4.fib_table_hash[].
Note that commit a7e5353123 ("fib_trie: Make fib_table rcu
safe") started to use the _safe version in fib_flush(), but it
is not needed thanks to RTNL.
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://patch.msgid.link/20260629181226.1929658-5-kuniyu@google.com
Reviewed-by: Ido Schimmel <idosch@nvidia.com>
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
fib_newrule() will drop RTNL except for the first IPv4 rule.
Then, fib4_rule_configure() could call fib_empty_table() and create
a new IPv4 fib_table without RTNL.
Currently, net->ipv4.fib_table_hash[] is only protected by RTNL.
As a prep, let's protect net->ipv4.fib_table_hash[] with a dedicated
spinlock.
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://patch.msgid.link/20260629181226.1929658-4-kuniyu@google.com
Reviewed-by: Ido Schimmel <idosch@nvidia.com>
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
IPv4 local and main route tables are merged by default to avoid
unnecessary rule lookups.
When the first IPv4 rule is created, fib_unmerge() splits the
two tables.
However, fib4_rule_configure() currently always calls fib_unmerge(),
and even fetching a table via fib_get_table() requires RTNL (or RCU).
We will drop RTNL from fib_newrule() if not needed.
Let's call fib_unmerge() only once for the first rule.
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://patch.msgid.link/20260629181226.1929658-3-kuniyu@google.com
Reviewed-by: Ido Schimmel <idosch@nvidia.com>
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
When CONFIG_IP_MULTIPLE_TABLES is enabled but no rule is added,
fib_lookup() performs route lookup directly on two tables.
Since the first lookup does not properly bail out, the result
of an error route in the merged local/main table could be
overwritten by another route in the default table:
# unshare -n
# ip link set lo up
# ip route add 192.168.0.0/24 dev lo table 253
# ip route add unreachable 192.168.0.0/24
# ip route get 192.168.0.1
192.168.0.1 dev lo table default uid 0
cache <local>
Once a random rule is added, the error route is respected:
# ip rule add table 0
# ip rule del table 0
# ip route get 192.168.0.1
RTNETLINK answers: No route to host
Let's fix the inconsistent behaviour.
Fixes: f4530fa574 ("ipv4: Avoid overhead when no custom FIB rules are installed.")
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Reviewed-by: Ido Schimmel <idosch@nvidia.com>
Link: https://patch.msgid.link/20260619212753.3367244-1-kuniyu@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
syzbot reported use-after-free of net->ipv4.rules_ops. [0]
It can be reproduced with these commands:
while true; do
ip netns add ns1
ip -n ns1 link set dev lo up
ip -n ns1 address add 192.0.2.1/24 dev lo
ip -n ns1 link add name dummy1 up type dummy
ip -n ns1 address add 198.51.100.1/24 dev dummy1
ip -n ns1 rule add ipproto tcp sport 12345 table 12345
ip -n ns1 fou add port 5555 ipproto 47 local 192.0.2.1 peer 198.51.100.2 peer_port 54321
ip netns del ns1
done
The cited commit moved fib4_rules_exit() earlier to ->exit_rtnl(),
but the kernel socket destroyed in ->exit() could eventually reach
__fib_lookup().
I left fib4_rules_exit() in ->exit_rtnl() because fib4_rule_delete()
calls fib_unmerge(), which requires RTNL.
However, when ->delete() is called, ->configure() has already been
called, thus fib_unmerge() in ->delete() has no effect.
Let's remove fib_unmerge() in fib4_rule_delete() and move
fib4_rules_exit() to ->exit().
Many thanks to Ido Schimmel for providing the nice repro very quickly.
Note that we can make fib_rules_ops.delete() return void once
net-next opens.
[0]:
BUG: KASAN: slab-use-after-free in fib_rules_lookup+0x15e/0xeb0 net/core/fib_rules.c:321
Read of size 8 at addr ffff88804ec4c680 by task kworker/u8:21/12641
CPU: 0 UID: 0 PID: 12641 Comm: kworker/u8:21 Not tainted syzkaller #0 PREEMPT(full)
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 05/09/2026
Workqueue: netns cleanup_net
Call Trace:
<TASK>
dump_stack_lvl+0xe8/0x150 lib/dump_stack.c:120
print_address_description+0x55/0x1e0 mm/kasan/report.c:378
print_report+0x58/0x70 mm/kasan/report.c:482
kasan_report+0x117/0x150 mm/kasan/report.c:595
fib_rules_lookup+0x15e/0xeb0 net/core/fib_rules.c:321
__fib_lookup+0x106/0x210 net/ipv4/fib_rules.c:96
ip_route_output_key_hash_rcu+0x294/0x2720 net/ipv4/route.c:2811
ip_route_output_key_hash+0x18d/0x2a0 net/ipv4/route.c:2702
__ip_route_output_key include/net/route.h:169 [inline]
ip_route_output_flow+0x2a/0x150 net/ipv4/route.c:2929
ip4_datagram_release_cb+0x89d/0xbe0 net/ipv4/datagram.c:118
release_sock+0x206/0x260 net/core/sock.c:3861
inet_shutdown+0x2b1/0x390 net/ipv4/af_inet.c:950
udp_tunnel_sock_release+0x6d/0x80 net/ipv4/udp_tunnel_core.c:197
fou_release net/ipv4/fou_core.c:562 [inline]
fou_exit_net+0x17d/0x1f0 net/ipv4/fou_core.c:1230
ops_exit_list net/core/net_namespace.c:199 [inline]
ops_undo_list+0x43d/0x8d0 net/core/net_namespace.c:252
cleanup_net+0x572/0x810 net/core/net_namespace.c:702
process_one_work kernel/workqueue.c:3314 [inline]
process_scheduled_works+0xa8e/0x14e0 kernel/workqueue.c:3397
worker_thread+0xa47/0xfb0 kernel/workqueue.c:3478
kthread+0x389/0x470 kernel/kthread.c:436
ret_from_fork+0x514/0xb70 arch/x86/kernel/process.c:158
ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245
</TASK>
Fixes: 759923cf03 ("ipv4: fib: Convert fib_net_exit_batch() to ->exit_rtnl().")
Reported-by: syzbot+965506b59a2de0b6905c@syzkaller.appspotmail.com
Closes: https://lore.kernel.org/all/6a315824.b0403584.28d0ff.0000.GAE@google.com/
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://patch.msgid.link/20260616191359.4142661-1-kuniyu@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Currently, IPv4 routes are flushed in ->exit_batch() after
all devices are unregistered.
Unlike IPv6, IPv4 routes are not added from the fast path,
so we can flush routes before default_device_exit_batch().
Let's call ip_fib_net_exit() from ->exit_rtnl() to save
one RTNL locking dance.
ip_fib_net_exit() must use list_del_rcu() for fib_table
for the fast path on dying dev.
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Reviewed-by: Ido Schimmel <idosch@nvidia.com>
Reviewed-by: Eric Dumazet <edumazet@google.com>
Link: https://patch.msgid.link/20260612063225.455191-6-kuniyu@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
We will call ip_fib_net_exit() from ->exit_rtnl().
All fib_table will be destroyed before devices are unregistered.
During device unregistration, inetdev_destroy() could call
fib_del_ifaddr(), which calls fib_magic(RTM_DELROUTE).
fib_magic() calls fib_new_table(), but we do not want to create
a new table after ip_fib_net_exit() destroys all tables.
As a prep, let's add check_net() before fib_trie_table() in
fib_new_table().
fib_trie_table() is also called from fib_trie_unmerge(), but
fib_get_table() fails first in fib_unmerge(), so the same
problem does not occur there.
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Reviewed-by: Ido Schimmel <idosch@nvidia.com>
Reviewed-by: Eric Dumazet <edumazet@google.com>
Link: https://patch.msgid.link/20260612063225.455191-5-kuniyu@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
We will call ip_fib_net_exit() from ->exit_rtnl().
However, some paths will still access net->ipv4.fib_table_hash
after ->exit_rtnl().
For example, fib_flush() is called from fib_disable_ip() for
NETDEV_UNREGISTER.
Let's move kfree(net->ipv4.fib_table_hash) and fib4_notifier_exit()
from ip_fib_net_exit() to its caller.
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Reviewed-by: Ido Schimmel <idosch@nvidia.com>
Reviewed-by: Eric Dumazet <edumazet@google.com>
Link: https://patch.msgid.link/20260612063225.455191-4-kuniyu@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
We will call ip_fib_net_exit() from ->exit_rtnl().
Since the exit callbacks are called in the following order,
1. ->pre_exit()
~~~ synchronize_rcu() ~~~
2. ->exit_rtnl() : ip_fib_net_exit()
3. ->exit() : fib_proc_exit() / nl_fib_lookup_exit()
4. ->exit_batch() : fib4_semantics_exit()
the reverse order of fib_net_init() would get messed up.
Let's move fib_proc_exit() and nl_fib_lookup_exit() to ->pre_exit().
This is fine because procfs/netlink access from userspace cannot
occur at this point and synchronize_rcu() is not needed.
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Reviewed-by: Ido Schimmel <idosch@nvidia.com>
Reviewed-by: Eric Dumazet <edumazet@google.com>
Link: https://patch.msgid.link/20260612063225.455191-3-kuniyu@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Even when fib_table_flush() is called with flush_all true, it does
not flush all fib_info due to this condition:
!(fi->fib_flags & RTNH_F_DEAD) && !fib_props[fa->fa_type].error)
This creates an implicit ordering between default_device_exit_batch()
and fib_net_exit_batch().
fib_table_flush(flush_all=true) must be called after all devices
are NETDEV_UNREGISTERed, which is after nexthop_flush_dev() marks
RTNH_F_DEAD.
This would cause memory leak if the order were reversed.
fib_table_flush() does not skip non-dead error routes when flush_all
is true:
!flush_all &&
!(fi->fib_flags & RTNH_F_DEAD) && fib_props[fa->fa_type].error
Let's merge the two conditions not to skip all non-dead fib_info
during netns dismantle.
Note that we could further apply !flush_all to the basic table
id check and the rtmsg_fib() call in the loop.
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Reviewed-by: Ido Schimmel <idosch@nvidia.com>
Reviewed-by: Eric Dumazet <edumazet@google.com>
Link: https://patch.msgid.link/20260612063225.455191-2-kuniyu@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
ip6mr does not have rtnetlink interface for MFC unlike ipmr,
which uses dev_get_by_index_rcu() to set struct mfcctl.mfcc_parent.
ip6mr_mfc_add() and ip6mr_mfc_delete() are called under RTNL
from ip6_mroute_setsockopt() only.
There are no RTNL dependant, but ip6_mroute_setsockopt() reuses
RTNL just for mrt->mfc_hash and mrt->mfc_cache_list.
Let's replace RTNL with a new per-netns mutex.
Later, ip6mr_notifier_ops and ipmr_seq will be moved under
CONFIG_IPV6_MROUTE.
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://patch.msgid.link/20260604224712.3209821-15-kuniyu@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
When ip6mr_free_table() is called from ip6mr_rules_init() or
ip6mr_net_init(), the netns is not yet published.
Thus, no device should have been registered, and
mroute_clean_tables() will not call mif6_delete(), so
unregister_netdevice_many() is unnecessary.
unregister_netdevice_many() does nothing if the list is empty,
but it requires RTNL due to the unconditional ASSERT_RTNL()
at the entry of unregister_netdevice_many_notify().
Let's remove unnecessary RTNL and ASSERT_RTNL() and instead
add WARN_ON_ONCE() in ip6mr_free_table().
Note that we use a local list for the new WARN_ON_ONCE() because
dev_kill_list passed from ip6mr_rules_exit_rtnl() may have some
devices when other ops->init() fails after ipmr durnig setup_net().
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://patch.msgid.link/20260604224712.3209821-14-kuniyu@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
ip6mr_net_ops uses ->exit_batch() to acquire RTNL only once
for dying network namespaces.
ip6mr does not depend on the ordering of ->exit_rtnl() and
->exit_batch() of other pernet_operations (unlike fib_net_ops).
Once ip6mr_free_table() is called and all devices are
queued for destruction in ->exit_rtnl(), later during
NETDEV_UNREGISTER, ip6mr_device_event() will not see anything
in vif table and just do nothing.
Let's convert ip6mr_net_exit_batch() to ->exit_rtnl().
We will remove RTNL and unregister_netdevice_many() in
ip6mr_rules_init().
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://patch.msgid.link/20260604224712.3209821-13-kuniyu@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
This is a prep commit to convert ip6mr_net_exit_batch() to
->exit_rtnl().
Let's move unregister_netdevice_many() in ip6mr_free_table()
to its callers.
Now ip6mr_rules_exit() can do batching all tables per netns.
Note that later we will remove RTNL and unregister_netdevice_many()
in ip6mr_rules_init().
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://patch.msgid.link/20260604224712.3209821-12-kuniyu@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
fib_rules_unregister() removes ops from net->rules_ops under
spinlock, calls ops->delete() for each rule, and frees the ops.
ip6mr_rules_ops_template does not have ->delete(), and any
operation does not require RTNL there.
Let's move fib_rules_unregister() from ip6mr_rules_exit() to
ip6mr_net_exit().
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://patch.msgid.link/20260604224712.3209821-10-kuniyu@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Since default_device_exit_batch() is called after ->exit_rtnl(),
idev->mc_ifc_work could finally call mroute6_is_socket() under RCU
while ->exit_rtnl() is running. [0]
With CONFIG_IPV6_MROUTE_MULTIPLE_TABLES=n, ip6mr_fib_lookup() does
not check if net->ipv6.mrt6 is NULL. If ip6mr_net_exit_batch()
set net->ipv6.mrt6 to NULL and freed it, the mrt->mroute_sk access
could result in null-ptr-deref or use-after-free.
Let's prepare for that situation by applying RCU rule to ip6mr
table similarly.
!check_net(net) is added in ip6mr_cache_unresolved() and
mroute_clean_tables() to synchronise the two by mfc_unres_lock
so that ip6mr_cache_unresolved() will not queue skb after
mroute_clean_tables() purged &mrt->mfc_unres_queue.
rcu_read_lock() in reg_vif_xmit() is moved up to cover
ip6mr_fib_lookup() as with ipmr.
Link: https://lore.kernel.org/netdev/20260407184202.34cfe2d6@kernel.org/ #[0]
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://patch.msgid.link/20260604224712.3209821-9-kuniyu@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Commit 22e36ea9f5 ("inet: allow ip_valid_fib_dump_req() to
be called with RTNL or RCU") introduced the rtnl_held field in
struct fib_dump_filter to switch __dev_get_by_index() and
dev_get_by_index_rcu() depending on the caller's context.
This field served as an interim measure while we were incrementally
converting all callers of ip_valid_fib_dump_req() to RCU.
Now that all users (IPv4, IPv6, ipmr, ip6mr, and MPLS) have
been converted to RCU, the field is no longer necessary.
Let's remove it.
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://patch.msgid.link/20260604224712.3209821-8-kuniyu@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
ip6mr_rtm_dumproute() calls mr_table_dump() or mr_rtm_dumproute(),
and mr_rtm_dumproute() finally calls mr_table_dump().
mr_table_dump() calls the passed function, _ip6mr_fill_mroute().
_ip6mr_fill_mroute() is a wrapper for ip6mr_fill_mroute() to cast
struct mr_mfc * to struct mfc6_cache *.
ip6mr_fill_mroute() can already be called safely under RCU.
Let's convert ip6mr_rtm_dumproute() to RCU.
Now there is no user of the rtnl_held field in struct
fib_dump_filter, and the next patch will remove it.
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://patch.msgid.link/20260604224712.3209821-7-kuniyu@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
ip6mr_rtm_getroute() calls __ip6mr_get_table(), ip6mr_cache_find(),
and ip6mr_fill_mroute().
Once created, struct mr_table is not freed until netns dismantle,
so it's safe under RCU.
ip6mr_cache_find() iterates mrt->mfc_hash with rhl_for_each_entry_rcu().
struct mr_mfc is freed with call_rcu(), so this is also safe under
RCU.
ip6mr_fill_mroute() calls mr_fill_mroute(), which properly uses
RCU helpers.
Let's call them under RCU and register ip6mr_rtm_getroute() with
RTNL_FLAG_DOIT_UNLOCKED.
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://patch.msgid.link/20260604224712.3209821-6-kuniyu@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
We will convert ip6mr_rtm_getroute() to RCU in the following patch,
where __ip6mr_get_table() will be called under RCU.
nlmsg_new() uses GFP_KERNEL and needs to be called before holding
rcu_read_lock().
As a prep, let's move nlmsg_new() before __ip6mr_get_table().
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://patch.msgid.link/20260604224712.3209821-5-kuniyu@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
mr6_msgsize() calculates skb size needed for ip6mr_fill_mroute().
The size differs based on mrt->maxvif.
We will drop RTNL for ip6mr_rtm_getroute() and mrt->maxvif may
change under RCU.
To avoid -EMSGSIZE, let's calculate the size with the maximum
value of mrt->maxvif, MAXMIFS.
struct rtnexthop is 8 bytes and MAXMIFS is 32, so the maximum delta
is 256 bytes, which is small enough.
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://patch.msgid.link/20260604224712.3209821-4-kuniyu@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
These fields in struct mr_table are updated in ip6_mroute_setsockopt()
under RTNL:
* mroute_do_pim
* mroute_do_assert (MRT6_PIM is under RTNL while MRT6_ASSERT is lockless)
* mroute_do_wrvifwhole
However, ip6_mroute_getsockopt() does not hold RTNL and read the first
two fields locklessly, and ip6_mr_forward() reads all the three under
RCU.
Let's use WRITE_ONCE() and READ_ONCE() for them.
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://patch.msgid.link/20260604224712.3209821-3-kuniyu@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
By default, a GENEVE device bind()s its underlying UDP socket(s) to
the IPv4 or IPv6 wildcard address because there is no way to specify
a specific local IP address to bind() to.
This prevents deploying multiple GENEVE devices on a multi-homed host
where each device should be isolated and bound to a different local IP
address on the same UDP port.
Let's introduce new options, IFLA_GENEVE_LOCAL and IFLA_GENEVE_LOCAL6,
to allow specifying a local IPv4/IPv6 address for the backend UDP
socket.
By default, when collect metadata mode (IFLA_GENEVE_COLLECT_METADATA)
is enabled, both IPv4 and IPv6 sockets are created. However, if a
source address is specified via the new attributes, only a single
socket corresponding to that specific address family is created.
Accordingly, geneve_find_sock() and geneve_find_dev() are updated to
take the source address into account, ensuring that multiple devices
and sockets configured with different source addresses can coexist
without conflict.
In addition, the source address is validated in geneve_xmit_skb()
and geneve6_xmit_skb(), so the BPF prog must set it in bpf_tunnel_key.
With this change, multiple GENEVE devices can be successfully created
and bound to their respective local IP addresses:
(*) "local" is the keyword for IFLA_GENEVE_LOCAL / IFLA_GENEVE_LOCAL6
# for i in $(seq 1 2);
do
ip link add geneve4_${i} type geneve local 192.168.0.${i} external
ip addr add 192.168.0.${i}/24 dev geneve4_${i}
ip link set geneve4_${i} up
ip link add geneve6_${i} type geneve local 2001:9292::${i} external
ip addr add 2001:9292::${i}/64 dev geneve6_${i} nodad
ip link set geneve6_${i} up
done
# ip -d l | grep geneve
9: geneve4_1: <BROADCAST,MULTICAST,UP,LOWER_UP> ...
geneve external id 0 local 192.168.0.1 ...
10: geneve6_1: <BROADCAST,MULTICAST,UP,LOWER_UP> ...
geneve external id 0 local 2001:9292::1 ...
11: geneve4_2: <BROADCAST,MULTICAST,UP,LOWER_UP> ...
geneve external id 0 local 192.168.0.2 ...
12: geneve6_2: <BROADCAST,MULTICAST,UP,LOWER_UP> ...
geneve external id 0 local 2001:9292::2 ...
# ss -ua | grep geneve
UNCONN 0 0 192.168.0.2:geneve 0.0.0.0:*
UNCONN 0 0 192.168.0.1:geneve 0.0.0.0:*
UNCONN 0 0 [2001:9292::2]:geneve *:*
UNCONN 0 0 [2001:9292::1]:geneve *:*
Note that even if the local address is explicitly configured with
the wildcard address, kernel does not dump it except for devices with
IFLA_GENEVE_COLLECT_METADATA. This is consistent with the behaviour
of is_tnl_info_zero(), which treats the wildcard remote address as not
configured.
## ynl example.
# ./tools/net/ynl/pyynl/cli.py \
--spec ./Documentation/netlink/specs/rt-link.yaml \
--do newlink --create \
--json '{"ifname": "geneve0",
"linkinfo": {"kind":"geneve",
"data": {"local": "0.0.0.0",
"collect-metadata": true}}}'
# ./tools/net/ynl/pyynl/cli.py \
--spec ./Documentation/netlink/specs/rt-link.yaml \
--do getlink \
--json '{"ifname": "geneve0"}' --output-json | \
jq .linkinfo.data.local
"0.0.0.0"
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://patch.msgid.link/20260602190436.139591-6-kuniyu@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
When collect metadata mode (IFLA_GENEVE_COLLECT_METADATA) is
enabled, the GENEVE device creates both IPv4 and IPv6 sockets
and bind()s them to wildcard addresses.
The next patch allows creating only one socket bound to a
specific address even when the collect metadata mode is
enabled.
Then, we need a flag to distinguish dualstack GENEVE devices
to detect local address conflict.
Let's add the dualstack flag to struct geneve_config.
IFLA_GENEVE_COLLECT_METADATA processing is moved up in
geneve_nl2info() for the next patch to overwrite dualstack
to false while keeping collect_md true.
Note that IFLA_GENEVE_REMOTE and IFLA_GENEVE_REMOTE6 does not
set cfg->dualstack to false since is_tnl_info_zero() ignores
the wildcard remote address:
# ip link add geneve0 type geneve external remote 0.0.0.1
Error: Device is externally controlled, so attributes (VNI, Port, and so on) must not be specified.
# ip link add geneve0 type geneve external remote 0.0.0.0
# ss -ua | grep geneve
UNCONN 0 0 0.0.0.0:geneve 0.0.0.0:*
UNCONN 0 0 *:geneve *:*
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://patch.msgid.link/20260602190436.139591-5-kuniyu@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
This is a prep patch to make a subsequent patch clean.
We will need to access geneve_dev->cfg.info.key.u.{ipv4,ipv6}.src
in geneve_find_sock() later and extend conditions there.
Let's pass down struct geneve from geneve_sock_add() to
geneve_find_sock() and flatten the conditional logic.
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://patch.msgid.link/20260602190436.139591-4-kuniyu@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
This is a prep patch to make a subsequent patch clean.
We will need to access geneve_dev->cfg.info.key.u.{ipv4,ipv6}.src
in geneve_create_sock() later.
Let's pass down struct geneve_dev from geneve_sock_add() to
geneve_create_sock() instead of individual config fields.
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Reviewed-by: Eric Dumazet <edumazet@google.com>
Link: https://patch.msgid.link/20260602190436.139591-3-kuniyu@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
syzbot reported a weird reqsk->rsk_refcnt underflow in
__inet_csk_reqsk_queue_drop().
The captured reqsk_put() in __inet_csk_reqsk_queue_drop()
is called only when it successfully removes reqsk from ehash.
Moreover, reqsk_timer_handler() calls another reqsk_put()
after that.
This indicates that the reqsk was missing both refcnts for
ehash and the timer itself.
Since all the syzbot reports had PREEMPT_RT enabled, the only
possible scenario is that reqsk_queue_hash_req() is preempted
after mod_timer() and before refcount_set(), and then the timer
triggered after 1s aborts the reqsk due to its listener's close().
Let's wrap mod_timer() and refcount_set() with
preempt_disable_nested() and preempt_enable_nested().
Note that inet_ehash_insert() holds the normal spin_lock()
(mutex in PREEMPT_RT), so it must be called outside of
preempt_disable_nested(), but this is fine.
The lookup path just ignores 0 sk_refcnt entries in ehash
and tries to create another reqsk, but this will fail at
inet_ehash_insert().
[0]:
refcount_t: underflow; use-after-free.
WARNING: lib/refcount.c:28 at refcount_warn_saturate+0xb2/0x110 lib/refcount.c:28, CPU#0: ktimers/0/16
Modules linked in:
CPU: 0 UID: 0 PID: 16 Comm: ktimers/0 Tainted: G L syzkaller #0 PREEMPT_{RT,(full)}
Tainted: [L]=SOFTLOCKUP
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 04/18/2026
RIP: 0010:refcount_warn_saturate+0xb2/0x110 lib/refcount.c:28
Code: e4 7d d1 0a 67 48 0f b9 3a eb 4a e8 38 3d 23 fd 48 8d 3d e1 7d d1 0a 67 48 0f b9 3a eb 37 e8 25 3d 23 fd 48 8d 3d de 7d d1 0a <67> 48 0f b9 3a eb 24 e8 12 3d 23 fd 48 8d 3d db 7d d1 0a 67 48 0f
RSP: 0000:ffffc90000157948 EFLAGS: 00010246
RAX: ffffffff84a1301b RBX: 0000000000000003 RCX: ffff88801ca98000
RDX: 0000000000000100 RSI: 0000000000000000 RDI: ffffffff8f72ae00
RBP: ffffffff99ae3b01 R08: ffff88801ca98000 R09: 0000000000000005
R10: 0000000000000100 R11: 0000000000000004 R12: ffff8880425ef568
R13: ffff8880425ef4f8 R14: ffff8880425ef578 R15: 0000000000000000
FS: 0000000000000000(0000) GS:ffff888126386000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f7b46710e9c CR3: 000000000dbb6000 CR4: 00000000003526f0
Call Trace:
<TASK>
__refcount_sub_and_test include/linux/refcount.h:400 [inline]
__refcount_dec_and_test include/linux/refcount.h:432 [inline]
refcount_dec_and_test include/linux/refcount.h:450 [inline]
reqsk_put include/net/request_sock.h:136 [inline]
__inet_csk_reqsk_queue_drop+0x3ce/0x440 net/ipv4/inet_connection_sock.c:1007
reqsk_timer_handler+0x651/0xdf0 net/ipv4/inet_connection_sock.c:1137
call_timer_fn+0x192/0x5e0 kernel/time/timer.c:1748
expire_timers kernel/time/timer.c:1799 [inline]
__run_timers kernel/time/timer.c:2374 [inline]
__run_timer_base+0x6a3/0x9f0 kernel/time/timer.c:2386
run_timer_base kernel/time/timer.c:2395 [inline]
run_timer_softirq+0x67/0x170 kernel/time/timer.c:2403
handle_softirqs+0x1de/0x6d0 kernel/softirq.c:622
__do_softirq kernel/softirq.c:656 [inline]
run_ktimerd+0x69/0x100 kernel/softirq.c:1151
smpboot_thread_fn+0x541/0xa50 kernel/smpboot.c:160
kthread+0x388/0x470 kernel/kthread.c:436
ret_from_fork+0x514/0xb70 arch/x86/kernel/process.c:158
ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245
</TASK>
Fixes: d2d6422f8b ("x86: Allow to enable PREEMPT_RT.")
Reported-by: syzbot+e809069bc15f26300526@syzkaller.appspotmail.com
Closes: https://lore.kernel.org/all/6a1a7bcf.0a9e871e.332604.000b.GAE@google.com/
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Reviewed-by: Eric Dumazet <edumazet@google.com>
Reviewed-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Link: https://patch.msgid.link/20260601182101.3183993-1-kuniyu@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
UDP TX skb->destructor() is sock_wfree(), and UDP holds lock_sock()
only for UDP_CORK / MSG_MORE sendmsg().
Otherwise, sk->sk_write_space() may be read locklessly while SOCKMAP
rewrites sk->sk_write_space().
Let's use WRITE_ONCE() and READ_ONCE() for sk->sk_write_space().
Note that the write side is annotated by commit 2ef2b20cf4
("net: annotate data-races around sk->sk_{data_ready,write_space}").
Fixes: 7b98cd42b0 ("bpf: sockmap: Add UDP support")
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Reviewed-by: Jakub Sitnicki <jakub@cloudflare.com>
Link: https://patch.msgid.link/20260529193941.3897256-1-kuniyu@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Both struct socket and struct sock have a variable to
manage its state, sock->state and sk->sk_state.
When both are used, the former typically manages syscall
state and the latter manages the actual connection state.
AF_UNIX only uses sk->sk_state.
Let's remove unnecessary assignemnts for sock->state.
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://patch.msgid.link/20260529191829.3864438-1-kuniyu@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Since commit 21c05ca88a ("workqueue: Add warnings and ensure
one among WQ_PERCPU or WQ_UNBOUND is present"), we must explicitly
set WQ_PERCPU or WQ_UNBOUND when creating workqueue.
nvme_tcp_init_module() sets WQ_UNBOUND when the module param
wq_unbound is set, but otherwise, WQ_PERCPU is missing, triggering
the warning below:
workqueue: nvme_tcp_wq is using neither WQ_PERCPU or WQ_UNBOUND. Setting WQ_PERCPU.
WARNING: kernel/workqueue.c:5856 at __alloc_workqueue+0x1d02/0x2070 kernel/workqueue.c:5855, CPU#0: swapper/0/1
Let's set WQ_PERCPU if wq_unbound is false.
Reported-by: syzbot+d078cba4418e65f61984@syzkaller.appspotmail.com
Closes: https://lore.kernel.org/all/6a1a9a86.323e8352.141b09.0001.GAE@google.com/
Tested-by: Venkat Rao Bagalkote <venkat88@linux.ibm.com>
Reviewed-by: Nilay Shroff <nilay@linux.ibm.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Chaitanya Kulkarni <kch@nvidia.com>
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Signed-off-by: Keith Busch <kbusch@kernel.org>
ip netns add ns1
ip netns add ns2
ip -n ns1 link add vti6_test type vti6 remote ::1 local ::2 key 7
ip -n ns1 link set vti6_test netns ns2
ip -n ns2 link set vti6_test type vti6 remote ::3 local ::4 key 9
ip netns del ns2
ip netns del ns1
[ 132.495484] ------------[ cut here ]------------
[ 132.497609] kernel BUG at net/core/dev.c:12376!
Commit 61220ab349 ("vti6: Enable namespace changing") dropped
NETIF_F_NETNS_LOCAL from vti6 devices. A vti6 tunnel can then
move through IFLA_NET_NS_FD. After the move dev_net(dev) points
at the new netns while t->net stays at the creation netns.
vti6_changelink() and vti6_update() still use dev_net(dev) and
dev_net(t->dev). They unlink from one per netns hash and relink
into another. The creation netns is left with a stale entry.
cleanup_net() of that netns later walks freed memory.
Reachable from an unprivileged user namespace (unshare --user
--map-root-user --net). Cross tenant scope on container hosts.
Fixes: 61220ab349 ("vti6: Enable namespace changing")
Reported-by: Maoyi Xie <maoyi.xie@ntu.edu.sg>
Reviewed-by: Eric Dumazet <edumazet@google.com>
Cc: stable@vger.kernel.org # v5.15+
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://patch.msgid.link/20260521130555.3421684-2-maoyixie.tju@gmail.com
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
kernel test robot reported sparse warning in bareudp_sock_release():
drivers/net/bareudp.c:288:12: warning: incorrect type in assignment (different address spaces)
drivers/net/bareudp.c:288:12: expected struct sock *sk
drivers/net/bareudp.c:288:12: got struct sock [noderef] __rcu *sk
The warning is not new and exists since the initial bareudp commit
571912c69f ("net: UDP tunnel encapsulation module for tunnelling
different protocols like MPLS, IP, NSH etc.").
Let's use rtnl_dereference().
Note that bareudp_sock_release() is called from bareudp_stop()
under RTNL, so there is no real issue even without the helper.
Reported-by: kernel test robot <lkp@intel.com>
Closes: https://lore.kernel.org/oe-kbuild-all/202605062359.e3gOfZCr-lkp@intel.com/
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://patch.msgid.link/20260518050726.318824-6-kuniyu@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
synchronize_net() in bareudp_sock_release() has existed since
day 1, commit 571912c69f ("net: UDP tunnel encapsulation module
for tunnelling different protocols like MPLS, IP, NSH etc.").
It was most likely copied from a similar tunneling device like
vxlan or geneve.
bareudp_sock_release() is called from dev->netdev_ops->ndo_stop(),
and synchronize_net() in unregister_netdevice_many_notify() ensures
that inflight bareudp fast paths finish before bareudp_dev is freed.
Let's remove the redundant synchronize_net() in bareudp_sock_release().
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://patch.msgid.link/20260518050726.318824-5-kuniyu@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
When changing the geneve config, geneve_changelink() sandwiches
the config memcpy() between geneve_quiesce() and geneve_unquiesce().
geneve_quiesce() temporarily clears geneve->sock[46] and their
sk_user_data, and then calls synchronize_net() to wait for inflight
fast paths to finish.
geneve_unquiesce() then restores the cleared pointers, but it also
superfluously calls synchronize_net().
The latter synchronize_net() provides no benefit; with or without it,
inflight fast paths can see either the NULL pointers or the original
pointers alongside the new configuration.
Let's remove the redundant synchronize_net() in geneve_unquiesce().
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://patch.msgid.link/20260518050726.318824-4-kuniyu@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
vxlan previously had an issue where the fast path could access
stale pointers, which was fixed by commit c6fcc4fc5f ("vxlan:
avoid using stale vxlan socket.").
geneve later followed the same pattern, and commit fceb9c3e38
("geneve: avoid using stale geneve socket.") copied synchronize_net()
from vxlan_sock_release() into geneve_sock_release().
However, that change occurred after commit ca065d0cf8 ("udp: no
longer use SLAB_DESTROY_BY_RCU"), and geneve had already been
using kfree_rcu() to free geneve_sock.
Therefore, the synchronize_net() was never actually needed there.
Let's remove the redundant synchronize_net() in geneve_sock_release().
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://patch.msgid.link/20260518050726.318824-3-kuniyu@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Initially, a dedicated workqueue was used to defer calling
udp_tunnel_sock_release(vxlan_sock->sock) and kfree(vxlan_sock).
Later, commit 0412bd931f ("vxlan: synchronously and race-free
destruction of vxlan sockets") removed the workqueue and instead
invoked these two functions immediately after synchronize_net().
This was intended to prevent UAF of the UDP socket in the fast path.
( Note that the "nondeterministic behaviour" mentioned in that
commit was not addressed, as another thread not waiting RCU gp
still sees the same behaviour. )
However, a week prior to that change, commit ca065d0cf8 ("udp:
no longer use SLAB_DESTROY_BY_RCU") had already moved UDP socket
freeing to after the RCU grace period. This made the synchronize_net()
in vxlan_sock_release() completely redundant.
Since vxlan_sock now uses kfree_rcu() and is invoked after
udp_tunnel_sock_release(), vxlan_sock is guaranteed to be freed
either at the same time or after the UDP socket is released,
following the RCU grace period.
Let's remove the redundant synchronize_net() in vxlan_sock_release().
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://patch.msgid.link/20260518050726.318824-2-kuniyu@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
lockdep_sock_is_held() was added in tcp_ao_established_key()
by the cited commit.
It can be called from tcp_v[46]_timewait_ack() with twsk.
Since it does not have sk->sk_lock, the lockdep annotation
results in out-of-bound access.
$ pahole -C tcp_timewait_sock vmlinux | grep size
/* size: 288, cachelines: 5, members: 8 */
$ pahole -C sock vmlinux | grep sk_lock
socket_lock_t sk_lock; /* 440 192 */
Let's not use lockdep_sock_is_held() for TCP_TIME_WAIT.
Fixes: 6b2d11e2d8 ("net/tcp: Add missing lockdep annotations for TCP-AO hlist traversals")
Reported-by: Damiano Melotti <melotti@google.com>
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Reviewed-by: Eric Dumazet <edumazet@google.com>
Link: https://patch.msgid.link/20260508120853.4098365-1-kuniyu@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
When TCP socket migration happens in reqsk_timer_handler(),
@sk_listener will be updated with the new listener.
When we call __inet_csk_reqsk_queue_drop(), the listener must
be the one stored in req->rsk_listener.
The cited commit accidentally replaced oreq->rsk_listener with
sk_listener, leading to imbalanced icsk_accept_queue count.
Let's pass the correct listener to __inet_csk_reqsk_queue_drop().
Fixes: e8c526f2bd ("tcp/dccp: Don't use timer_pending() in reqsk_queue_unlink().")
Reported-by: Damiano Melotti <melotti@google.com>
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://patch.msgid.link/20260506035954.1563147-3-kuniyu@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
When TCP socket migration fails at inet_ehash_insert() in
reqsk_timer_handler(), we jump to the no_ownership: label
and free the new reqsk immediately with __reqsk_free().
Thus, we must stop the new reqsk's timer before jumping to the
label, but the timer might be missed since the cited commit,
resulting in UAF.
As we are in the original reqsk's timer context, we can safely
call timer_delete_sync() for the new reqsk.
Let's pass false to __inet_csk_reqsk_queue_drop() to stop
the new reqsk's timer.
Fixes: 83fccfc394 ("inet: fix potential deadlock in reqsk_queue_unlink()")
Reported-by: Damiano Melotti <melotti@google.com>
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://patch.msgid.link/20260506035954.1563147-2-kuniyu@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
sol_tcp_sockopt() only checks if sk->sk_protocol is IPPROTO_TCP,
but RAW socket can bypass it:
socket(AF_INET, SOCK_RAW, IPPROTO_TCP)
Let's use sk_is_tcp().
Note that initially sol_tcp_sockopt() checked sk->sk_prot->setsockopt.
Fixes: 2ab42c7b87 ("bpf: Check the protocol of a sock to agree the calls to bpf_setsockopt().")
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Signed-off-by: Martin KaFai Lau <martin.lau@kernel.org>
Link: https://patch.msgid.link/20260504210610.180150-7-kuniyu@google.com
If a socket is bound to a wildcard address, tcp_v[46]_connect()
updates it with a non-wildcard address based on the route lookup.
After bhash2 was introduced in the cited commit, we must call
inet_bhash2_update_saddr() to update the bhash2 entry as well.
If inet_bhash2_update_saddr() fails, we must release the refcount
for dst by ip_route_connect() or ip6_dst_lookup_flow().
While tcp_v4_connect() calls ip_rt_put() in the error path,
tcp_v6_connect() does not call dst_release().
Let's call dst_release() when inet_bhash2_update_saddr() fails
in tcp_v6_connect().
Fixes: 28044fc1d4 ("net: Add a bhash2 table hashed by port and address")
Reported-by: Damiano Melotti <melotti@google.com>
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Reviewed-by: Eric Dumazet <edumazet@google.com>
Link: https://patch.msgid.link/20260506070443.1699879-1-kuniyu@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Commit 3cf7203ca6 ("net/tunnel: wait until all sk_user_data
reader finish before releasing the sock") added synchronize_rcu()
in udp_tunnel_sock_release().
This was intended to protect the fast path of a dying vxlan device
from dereferencing vxlan_sock->sock->sk after sock_orphan() has set
sock->sk to NULL.
However, vxlan does not need to access struct socket itself
in the fast path; it only reads struct sock, and struct socket
is only used for tunnel setup and teardown.
This applies to all other UDP tunnel users, and they have been
converted to access struct sock directly.
In addition, each device-specific struct used in their fast paths
is freed after one RCU grace period. Since this occurs after
udp_tunnel_sock_release(), the struct is guaranteed to be freed
after struct udp_sock.
Therefore, synchronize_rcu() in udp_tunnel_sock_release() is
now redundant.
Let's remove it.
Tested:
A script creating/upping vxlan devices in 4000 netns runs 10x
faster with this change. We can see the same improvement with
other UDP tunnel devices as well.
$ cat vxlan.sh
for i in `seq 1 40`
do
(for j in `seq 1 100` ; do
unshare -n bash -c "ip link add vxlan0 type vxlan id 100 local 127.0.0.1 dstport 4789 && ip link set vxlan0 up";
done) &
done
wait
With bpftrace, we can see vxlan_stop() is significantly faster.
bpftrace -e '
kprobe:vxlan_stop {
@start[tid] = nsecs;
}
kretprobe:vxlan_stop /@start[tid]/ {
@duration_us = hist((nsecs - @start[tid]) / 1000);
delete(@start[tid]);
}
END {
printf("\nExecution time of vxlan_stop (us):\n");
}'
Before:
# time ./vxlan.sh // without bpftrace
real 0m50.615s
user 0m8.171s
sys 1m45.101s
@duration_us:
[4K, 8K) 1266 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ |
[8K, 16K) 1957 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@|
[16K, 32K) 764 |@@@@@@@@@@@@@@@@@@@@ |
[32K, 64K) 6 | |
[64K, 128K) 4 | |
[128K, 256K) 3 | |
After:
# time ./vxlan.sh // without bpftrace
real 0m5.247s
user 0m7.956s
sys 1m47.404s
@duration_us:
[16, 32) 3411 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@|
[32, 64) 383 |@@@@@ |
[64, 128) 107 |@ |
[128, 256) 79 |@ |
[256, 512) 16 | |
[512, 1K) 2 | |
[1K, 2K) 2 | |
Next step is to remove another synchronize_net() in vxlan_stop()
and variants in other devices.
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://patch.msgid.link/20260502031401.3557229-16-kuniyu@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Kuniyuki Iwashima