From: Zhi Wang <[email protected]> vGPU management queries need an owned response payload after the receive queue releases its slot. Sending and receiving must share one queue guard so another receiver cannot consume the reply.
Match the response flag, command ID and request sequence, then copy the payload and return the raw firmware status. Support the default receive timeout and a caller-selected timeout without converting firmware rejection into a transport error. Share the receive deadline and retry loop in CmdqInner with the existing GMC boot response wait. Preserve each caller's matching and status policy, consume every valid element once even when its callback fails, and keep a single deadline after sending. Document unmatched-message consumption and distinguish a zero response payload capacity from a command that sends no reply. Signed-off-by: Zhi Wang <[email protected]> Co-developed-by: Alok Kumar <[email protected]> Signed-off-by: Alok Kumar <[email protected]> --- Documentation/gpu/nova/core/interrupts.rst | 19 ++- drivers/gpu/nova-core/gsp/cmdq.rs | 28 +++- drivers/gpu/nova-core/gsp/cmdq/gmc.rs | 162 ++++++++++++++++++++- drivers/gpu/nova-core/gsp/fw.rs | 26 +++- 4 files changed, 215 insertions(+), 20 deletions(-) diff --git a/Documentation/gpu/nova/core/interrupts.rst b/Documentation/gpu/nova/core/interrupts.rst index 3fcc886e22c1..7f61e5d6334e 100644 --- a/Documentation/gpu/nova/core/interrupts.rst +++ b/Documentation/gpu/nova/core/interrupts.rst @@ -580,9 +580,12 @@ command reply or an unsolicited event, and the two differ in the function code. records, lifecycle notices) need no action and get no line of their own, because the receive trace at debug level already records every message's arrival with its sequence number, function code, and length. -* A GMC message carries a command id in place of a function code. Only the - ``GSP_INIT`` wait during boot claims GMC messages, so one that arrives - anywhere else is logged at warning level and dropped. +* A GMC message carries a command id in place of a function code. The + ``GSP_INIT`` wait and synchronous GMC transactions claim responses with the + expected command id and sequence number. These waits consume interleaved RPC + messages as events. Unmatched GMC messages are handled by the wait's callback + or logged and consumed; a queue drain with no waiting caller logs unclaimed + responses and unknown commands at warning level and drops them. A command's reply must carry the RPC sequence number that nova-core wrote into the command, as well as its function code. A message with the awaited function @@ -602,10 +605,12 @@ element stays at the queue head, and every later receive fails on it with The polling path and the IRQ thread both read the queue under the command-queue mutex. Replies and events share one queue and one read pointer, so one lock is held across the whole drain. A thread waiting for a reply logs each event that -arrives before the reply and keeps waiting. One deadline of 5 seconds applies to -the whole wait, rather than a fresh timeout after each message, and the thread -holds the mutex for the whole wait, so that no other caller consumes the message -that it waits for. +arrives before the reply and keeps waiting. One receive deadline applies to +all messages in the wait. The default timeout is 5 seconds; GMC transactions +can specify another timeout. For send-and-wait operations the receive deadline +starts after sending, so it does not bound waiting for the mutex or for +command-queue space. The thread holds the mutex from sending through receiving, +so no other caller consumes the message it waits for. With one lock, a drain waits for an in-flight command's receive to finish or time out. For log and error records that delay does not matter. diff --git a/drivers/gpu/nova-core/gsp/cmdq.rs b/drivers/gpu/nova-core/gsp/cmdq.rs index 4b25bd58ab93..70de84079d20 100644 --- a/drivers/gpu/nova-core/gsp/cmdq.rs +++ b/drivers/gpu/nova-core/gsp/cmdq.rs @@ -2,6 +2,7 @@ // SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. pub(super) mod gmc; + // No RPC command is sent since the switch to the r000 firmware: GSP_INIT and GSP_SUSPEND replaced // the RPC commands that the driver used during boot and shutdown. The RPC send and reply paths // stay for the commands that later series add, and are built as dead code until then. @@ -67,6 +68,9 @@ use super::regs; +#[expect(unused_imports)] +pub(crate) use gmc::GmcResponse; + /// Marker type representing the absence of a reply for a command. Commands using this as their /// reply type are sent using [`Cmdq::send_rpc_command_no_wait`]. pub(crate) struct NoReply; @@ -722,15 +726,29 @@ fn consume_element<R>( /// - Any error returned by `f` with the exception of `ENOMSG`. fn await_msg_with<R>(&mut self, f: impl Fn(GspParsedMessage<'_>) -> Result<R>) -> Result<R> { let deadline = Instant::<Monotonic>::now() + Cmdq::RECEIVE_TIMEOUT; + self.await_msg_with_deadline(deadline, |message| match f(message) { + Ok(value) => Ok(Some(value)), + Err(ENOMSG) => Ok(None), + Err(error) => Err(error), + }) + } + + /// Waits under the queue guard until `f` claims a message, using one receive deadline. + /// + /// Every valid element is consumed even if `f` fails. Unclaimed messages do not extend the + /// deadline. Framing errors stall the queue and are returned without consuming the element. + fn await_msg_with_deadline<R>( + &mut self, + deadline: Instant<Monotonic>, + mut f: impl FnMut(GspParsedMessage<'_>) -> Result<Option<R>>, + ) -> Result<R> { loop { let remaining = deadline - Instant::<Monotonic>::now(); if remaining.is_negative() { - break Err(ETIMEDOUT); + return Err(ETIMEDOUT); } - match self.consume_element(remaining, &f)? { - Ok(m) => break Ok(m), - Err(ENOMSG) => continue, - Err(e) => break Err(e), + if let Some(value) = self.consume_element(remaining, &mut f)?? { + return Ok(value); } } } diff --git a/drivers/gpu/nova-core/gsp/cmdq/gmc.rs b/drivers/gpu/nova-core/gsp/cmdq/gmc.rs index f3f4e265f023..392143e534f6 100644 --- a/drivers/gpu/nova-core/gsp/cmdq/gmc.rs +++ b/drivers/gpu/nova-core/gsp/cmdq/gmc.rs @@ -6,6 +6,11 @@ use kernel::{ device, prelude::*, + time::{ + Delta, + Instant, + Monotonic, // + }, transmute::{ AsBytes, FromBytes, // @@ -59,8 +64,8 @@ pub(crate) trait CommandToGsp { /// reply. type Reply; - /// Largest reply this command accepts, in bytes. Zero for commands that the GSP does not - /// answer. + /// Response payload capacity in bytes. Zero permits a status-only response; whether the + /// command sends a reply is determined by its protocol and [`Self::Reply`]. const MAX_REPLY_SIZE: u32; /// Error type returned by [`CommandToGsp::init`]. @@ -249,9 +254,8 @@ pub(crate) fn payload(&self) -> (&'a [u8], &'a [u8]) { /// Returns the status code of this message if it is the reply to `command` sent with sequence /// number `seq`, `None` otherwise. fn is_reply_to(&self, command: GmcCommandId, seq: u32) -> Option<u32> { - if matches!(self.header.command().id(), Ok(id) if id == command) - && self.header.sequence() == u64::from(seq) - { + // CAST: `GmcCommandId` is `repr(u32)`, so this cast is lossless. + if self.header.is_response_to(command as u32, seq) { self.header.status() } else { None @@ -295,7 +299,155 @@ pub(super) fn log(&self, dev: &device::Device) { } } +/// Response from a GMC API command. +pub(crate) struct GmcResponse { + /// Response status (`NV_STATUS` code). Zero means success. + #[expect(dead_code)] + pub(crate) status: u32, + /// Response payload copied out of the message queue. + #[expect(dead_code)] + pub(crate) payload: KVec<u8>, +} + +/// A byte payload sent with a runtime-selected GMC command ID. +struct RawGmcCommand<'a> { + command_id: u32, + payload: &'a [u8], + max_response_size: u32, +} + +impl CommandElement for RawGmcCommand<'_> { + const NVDM_TYPE: NvdmType = NvdmType::GmcApi; + + fn size(&self) -> usize { + size_of::<GmcApiHeader>() + self.payload.len() + } + + fn write(&self, dev: &device::Device, seq: u32, dst: &mut GspCommand<'_>) -> Result { + let (header, payload_0) = GmcApiHeader::from_bytes_mut_prefix(dst.contents.0).ok_or(EIO)?; + let init = GmcApiHeader::init_raw( + self.command_id, + u64::from(seq), + self.payload.len(), + self.max_response_size, + ); + // SAFETY: `header` is valid and is not touched if initialization fails. + unsafe { pin_init::raw_try_init(core::ptr::from_mut(header), init)? }; + SBufferIter::new_writer([payload_0, &mut dst.contents.1[..]]).write_all(self.payload)?; + dev_dbg!( + dev, + "GSP GMC: send: seq# {}, command={}, length=0x{:x}\n", + seq, + GmcCommand(self.command_id & 0x00ff_ffff), + self.payload.len(), + ); + Ok(()) + } +} + impl Cmdq<'_> { + /// Sends a GMC API command and waits for its matching response. + /// + /// Uses [`Self::RECEIVE_TIMEOUT`] as the receive timeout. Firmware rejection is returned in + /// [`GmcResponse::status`], not as an error. + /// + /// See [`Self::send_gmc_and_receive_timeout`] for message consumption and locking behavior. + /// + /// # Errors + /// + /// Returns the same errors as [`Self::send_gmc_and_receive_timeout`]. + #[expect(dead_code)] + pub(crate) fn send_gmc_and_receive( + &self, + command_id: u32, + payload: &[u8], + max_response_size: u32, + ) -> Result<GmcResponse> { + self.send_gmc_and_receive_timeout( + command_id, + payload, + max_response_size, + Self::RECEIVE_TIMEOUT, + ) + } + + /// Sends a GMC API command and waits up to `timeout` for its matching response. + /// + /// Matches both the command ID and the request's sequence number. A nonzero firmware status + /// is returned in [`GmcResponse::status`]; the caller decides how to handle rejection. + /// `max_response_size` advertises the response payload capacity in bytes, so zero still + /// permits a status-only response. + /// + /// Unmatched GMC responses and events are debug-logged and consumed. Interleaved RPC messages + /// are logged as events and consumed. The matching response is also consumed if copying its + /// payload fails. + /// + /// The queue stays locked from sending through receiving. One receive deadline starts after + /// sending and is not extended by other messages. It does not bound waiting for the mutex or + /// for space to send the request. + /// + /// # Errors + /// + /// - `EMSGSIZE` if the request exceeds the command queue's maximum element size. + /// - `ETIMEDOUT` if space does not become available to send the request, or if the matching + /// response does not arrive before the receive deadline. + /// - `EIO` if the command queue slot cannot hold the request headers, the receive queue is + /// stalled by an invalid element, or a received element fails framing validation. + /// - `ENOMEM` if the response payload cannot be allocated. + /// + /// Errors from initializing the request headers are propagated as-is. + pub(crate) fn send_gmc_and_receive_timeout( + &self, + command_id: u32, + payload: &[u8], + max_response_size: u32, + timeout: Delta, + ) -> Result<GmcResponse> { + let mut inner = self.inner.lock(); + let expected_sequence = inner.send_command_element(RawGmcCommand { + command_id, + payload, + max_response_size, + })?; + let dev = inner.dev; + + let deadline = Instant::<Monotonic>::now() + timeout; + inner.await_msg_with_deadline(deadline, |message| { + let GspParsedMessage::Gmc(message) = message else { + message.log(dev); + return Ok(None); + }; + let header = message.header(); + let (payload_0, payload_1) = message.payload(); + if !header.is_response_to(command_id, expected_sequence) { + let kind = if header.command().is_response() { + "response" + } else { + "event" + }; + dev_dbg!( + dev, + "GSP GMC: skip {} seq {} cmd {:#x}; want response seq {} cmd {:#x}\n", + kind, + header.sequence(), + GmcCommand::from(header.command()).0, + expected_sequence, + command_id, + ); + return Ok(None); + } + + // Each byte slice is at most `isize::MAX` bytes, so their sum fits in `usize`. + let mut payload = KVec::with_capacity(payload_0.len() + payload_1.len(), GFP_KERNEL)?; + payload.extend_from_slice(payload_0, GFP_KERNEL)?; + payload.extend_from_slice(payload_1, GFP_KERNEL)?; + Ok(Some(GmcResponse { + status: header.status().ok_or(EIO)?, + payload, + })) + }) + } + /// Sends `command` to the GSP and waits for the reply. /// /// GMC events that arrive before the reply are passed to `on_event`, and RPC messages are diff --git a/drivers/gpu/nova-core/gsp/fw.rs b/drivers/gpu/nova-core/gsp/fw.rs index dfe7aafcf31f..c75488e7a787 100644 --- a/drivers/gpu/nova-core/gsp/fw.rs +++ b/drivers/gpu/nova-core/gsp/fw.rs @@ -834,8 +834,8 @@ impl GmcApiHeader { /// Creates the header of a request for `command`, sent with sequence number `sequence` and /// followed by `payload_size` bytes of payload. /// - /// `max_reply_size` is the largest reply the sender accepts, and zero for a request that - /// GSP-RM does not answer. + /// `max_reply_size` is the response payload capacity in bytes. Zero permits a status-only + /// response; whether GSP-RM sends a reply is determined by the command protocol. /// /// # Errors /// @@ -845,9 +845,22 @@ pub(crate) fn init( sequence: u64, payload_size: usize, max_reply_size: u32, + ) -> impl Init<Self, Error> { + // CAST: `GmcCommandId` is `repr(u32)`, so this cast is lossless. + Self::init_raw(command as u32, sequence, payload_size, max_reply_size) + } + + /// Creates a request header for a runtime command ID, masking off command flags. + /// + /// Returns `EOVERFLOW` if `payload_size` does not fit its 32-bit field. + pub(super) fn init_raw( + command: u32, + sequence: u64, + payload_size: usize, + max_reply_size: u32, ) -> impl Init<Self, Error> { let init_inner = try_init!(bindings::GMCAPI_HEADER { - command: GmcCommandWord::zeroed().with_id(command).into(), + command: command & 0x00ff_ffff, size: payload_size.try_into().map_err(|_| EOVERFLOW)?, sequence, __bindgen_anon_1: bindings::GMCAPI_HEADER__bindgen_ty_1 { @@ -863,6 +876,13 @@ pub(crate) fn init( }) } + /// Matches the response flag, low 24 command-ID bits, and full request sequence number. + pub(crate) fn is_response_to(&self, command_id: u32, sequence: u32) -> bool { + self.command().is_response() + && GmcCommand::from(self.command()).0 == (command_id & 0x00ff_ffff) + && self.sequence() == u64::from(sequence) + } + /// Returns the `command` word of the header. pub(crate) fn command(&self) -> GmcCommandWord { GmcCommandWord::from(self.inner.command) -- 2.53.0
