Hello, I found that the issue I've been having with getting my custom RFNoC block to work is related to the CHDR packet size and MTU. I put in many ILA probes and found that If I create a gnuradio flowgraph that looks like (All blocks are dynamically connected):
RFNoC_RX_Radio -> RFNoC_DDC -> RX_Streamer -> QT_GUI_Time_Sink Everything works and the CHDR packet size is 7972. This CHDR packet size is consistent everywhere in the FPGA. This is the size reported in the CHDR header and I verified the total packet size via an ILA. If, on the other hand, I add my custom RFNoC block to the path (before, after, or in place of the DDC), then for some reason the CHDR packet size is 8000. This leaves no room for the Ethernet header, as my MTU size is set to 8000 per (https://kb.ettus.com/E320_Getting_Started_Guide). This is when I get errors like: gr::log :WARN: rfnoc_rx_streamer0 - RFNoC Streamer block received error ERROR_CODE_BAD_PACKET (Code: 0xf) [ERROR] [STREAMER] The receive transport caught a value exception. ValueError: Bad CHDR header or invalid packet length. I found that if in my gnuradio flowgraph I add an argument like "spp=1989" to RFNoC_RX_Radio, then everything works. The CHDR packet size is small enough. 1989 seems to be the upper limit. I'm wondering if I missed something and something should be set somewhere else to keep the packet size from being too large. I'm not sure why adding my block changes the packet size to 8000 (with no spp argument). Or, would it be acceptable/advisable to set the MTU on my interfaces to 9000 instead of 8000 to allow for Ethernet packets slightly larger than 8000? Thanks for any thoughts. And thanks Rob K for the help you've provided. Jim ________________________________ From: Rob Kossler <rkoss...@nd.edu> Sent: Thursday, October 1, 2020 11:19 AM To: Jim Palladino <j...@gardettoengineering.com> Cc: usrp-users@lists.ettus.com <usrp-users@lists.ettus.com> Subject: Re: [USRP-users] Trouble getting custom RFNoC block to work with gnuradio 3.8 / uhd 4.0 So, the only other thing I can think of for now relates to issues that I have had in connecting blocks that are not statically connected. For example, the newest default images for the X310 and N310 contain a "Replay" block where the Replay block is connected to/from an end-point in the same way as your example. In my UHD graph connection process, I never have an issue connecting "DUC=>Radio" because this is a static connection. But, I sometimes get a timeout error when connecting "Replay=>DUC" because this is a dynamic connection. I wonder if you are getting any connection error messages in the UHD log when you run from gnuradio. I'm guessing not, but wanted to mention it. In any case, if you run using "rfnoc_rx_to_file", you might be able to set the master clock rate low enough that you don't need a DDC to avoid overruns when using the graph "Radio30=>Block#0=>rx_streamer" (meaning that you might not need to modify the default example). I don't know a lot about the E320 so I'm not really sure what control you have over the master clock rate (and ultimately the radio output rate). On Thu, Oct 1, 2020 at 11:10 AM Jim Palladino via USRP-users <usrp-users@lists.ettus.com<mailto:usrp-users@lists.ettus.com>> wrote: Hi Rob, Thanks very much for the response. I checked and by default, in the rfnocmodtool generated verilog for my block, s_out_payload_tkeep is tied to '1'. Also, the autogenerated testbench runs just fine (all tests pass). I poked around at rfnoc_rx_to_file a while ago but didn't spend much time with it. I'll follow your suggestion and work with that some more. Also, the second flowgraph case I described in my original email works just fine now. This is the case where my flowgraph looks like: Constant_Source -> RFNoC_TX_Streamer -> RFNoC_Block -> RFNoC_RX_Streamer -> QT_GUI_Time_Sink Once I set the constant source to values between 0 and 1, I had no problems (it was a mistake on my part initially setting the values greater than 1). I also replaced the Constant_Source with a Signal_Source (cosine) and everything works just fine -- no errors and the plot looks good. So, this works, but the setup with RFNoC_RX_Radio as the source (followed by a DDC before my block) still does not work -- I get the CHDR errors with tready stuck low. Here is a copy of my Block_x310_rfnoc_image_core.yml. Even though the file name has x310 in it, I'm building for the e320 (and it is building for the e320). I wanted to change as little of what rfnocmodtool automatically generated as possible. ### Start of Block_x310_rfnoc_image_core.yml # General parameters # ----------------------------------------- schema: rfnoc_imagebuilder_args # Identifier for the schema used to validate this file copyright: '' # Copyright information used in file headers license: 'SPDX-License-Identifier: LGPL-3.0-or-later' # License information used in file headers version: 1.0 # File version rfnoc_version: 1.0 # RFNoC protocol version chdr_width: 64 # Bit width of the CHDR bus for this image device: 'e320' default_target: 'E320_XG' # A list of all stream endpoints in design # ---------------------------------------- stream_endpoints: ep0: # Stream endpoint name ctrl: True # Endpoint passes control traffic data: True # Endpoint passes data traffic buff_size: 32768 # Ingress buffer size for data ep1: # Stream endpoint name ctrl: False # Endpoint passes control traffic data: True # Endpoint passes data traffic buff_size: 32768 # Ingress buffer size for data ep2: # Stream endpoint name ctrl: False # Endpoint passes control traffic data: True # Endpoint passes data traffic buff_size: 8192 # Ingress buffer size for data ep3: # Stream endpoint name ctrl: False # Endpoint passes control traffic data: True # Endpoint passes data traffic buff_size: 8192 # Ingress buffer size for data ep4: # Stream endpoint name ctrl: False # Endpoint passes control traffic data: True # Endpoint passes data traffic buff_size: 32768 # Ingress buffer size for data # A list of all NoC blocks in design # ---------------------------------- noc_blocks: duc0: # NoC block name block_desc: 'duc.yml' # Block device descriptor file parameters: NUM_PORTS: 2 ddc0: block_desc: 'ddc.yml' parameters: NUM_PORTS: 2 radio0: block_desc: 'radio_2x64.yml' fifo0: block_desc: 'axi_ram_fifo_2x64.yml' parameters: # These parameters correspond to the memory interface on the E320 MEM_ADDR_W: 31 MEM_DATA_W: 64 MEM_CLK_RATE: "300e6" # Create two non-overlapping 32 MB buffers by default FIFO_ADDR_BASE: "{31'h02000000, 31'h00000000}" FIFO_ADDR_MASK: "{31'h01FFFFFF, 31'h01FFFFFF}" Block0: block_desc: 'Block.yml' # A list of all static connections in design # ------------------------------------------ # Format: A list of connection maps (list of key-value pairs) with the following keys # - srcblk = Source block to connect # - srcport = Port on the source block to connect # - dstblk = Destination block to connect # - dstport = Port on the destination block to connect connections: # ep0 to radio0(0) - RF0 TX - { srcblk: ep0, srcport: out0, dstblk: duc0, dstport: in_0 } - { srcblk: duc0, srcport: out_0, dstblk: radio0, dstport: in_0 } # radio0(0) to ep0 - RF0 RX - { srcblk: radio0, srcport: out_0, dstblk: ddc0, dstport: in_0 } - { srcblk: ddc0, srcport: out_0, dstblk: ep0, dstport: in0 } # ep1 to radio0(1) - RF1 TX - { srcblk: ep1, srcport: out0, dstblk: duc0, dstport: in_1 } - { srcblk: duc0, srcport: out_1, dstblk: radio0, dstport: in_1 } # radio0(1) to ep1 - RF1 RX - { srcblk: radio0, srcport: out_1, dstblk: ddc0, dstport: in_1 } - { srcblk: ddc0, srcport: out_1, dstblk: ep1, dstport: in0 } # ep2 to fifo0(0) - { srcblk: ep2, srcport: out0, dstblk: fifo0, dstport: in_0 } - { srcblk: fifo0, srcport: out_0, dstblk: ep2, dstport: in0 } # ep3 to fifo0(1) - { srcblk: ep3, srcport: out0, dstblk: fifo0, dstport: in_1 } - { srcblk: fifo0, srcport: out_1, dstblk: ep3, dstport: in0 } # Cust blk cnct ep4 to gain0 and gain0 to ep4 - { srcblk: ep4, srcport: out0, dstblk: Block0, dstport: in } - { srcblk: Block0, srcport: out, dstblk: ep4, dstport: in0 } # BSP Connections - { srcblk: radio0, srcport: ctrl_port, dstblk: _device_, dstport: ctrl_port } - { srcblk: _device_, srcport: x300_radio, dstblk: radio0, dstport: x300_radio } - { srcblk: _device_, srcport: time_keeper, dstblk: radio0, dstport: time_keeper } - { srcblk: fifo0, srcport: axi_ram, dstblk: _device_, dstport: dram } # A list of all clock domain connections in design # ------------------------------------------ # Format: A list of connection maps (list of key-value pairs) with the following keys # - srcblk = Source block to connect (Always "_device"_) # - srcport = Clock domain on the source block to connect # - dstblk = Destination block to connect # - dstport = Clock domain on the destination block to connect clk_domains: - { srcblk: _device_, srcport: radio, dstblk: radio0, dstport: radio } - { srcblk: _device_, srcport: rfnoc_chdr, dstblk: ddc0, dstport: ce } - { srcblk: _device_, srcport: rfnoc_chdr, dstblk: duc0, dstport: ce } - { srcblk: _device_, srcport: dram, dstblk: fifo0, dstport: mem } - { srcblk: _device_, srcport: rfnoc_chdr, dstblk: Block0, dstport: ce } ### End of Block_x310_rfnoc_image_core.yml Thanks again for any help. Jim ________________________________ From: Rob Kossler <rkoss...@nd.edu<mailto:rkoss...@nd.edu>> Sent: Thursday, October 1, 2020 10:19 AM To: Jim Palladino <j...@gardettoengineering.com<mailto:j...@gardettoengineering.com>> Cc: usrp-users@lists.ettus.com<mailto:usrp-users@lists.ettus.com> <usrp-users@lists.ettus.com<mailto:usrp-users@lists.ettus.com>> Subject: Re: [USRP-users] Trouble getting custom RFNoC block to work with gnuradio 3.8 / uhd 4.0 Hi Jim, I have also been playing around with UHD-4.0, but mostly in testbenches. I've only built one image (for the N310) with one of my custom blocks. The errors you mentioned seem very strange. A few questions/comments: - can you send your "XXX_x310_rfnoc_image_core.yml" contents? - would it be helpful to run directly with UHD examples (removing gnuradio and gr-ettus from the equation)? You could try "rfnoc_rx_to_file" as-is where you specify on the command line the desired "block-id" to insert between the Radio and the rx_streamer. With the X310, the Radio rate might be too high with your custom "thru" block so perhaps you could modify the example (in-tree would be easiest) to automatically include the DDC and then insert the command line "block-id" optionally after the DDC. - In my testbenches, I have occasionally seend CHDR error messages like you mentioned and it seemed to solve them if I set "s_out_payload_tkeep=1". I didn't think this was needed if there was only 1 output port, but I seem to recall that this fixed my CHDR error issue for my 1 port block. Rob On Wed, Sep 30, 2020 at 10:39 AM Jim Palladino via USRP-users <usrp-users@lists.ettus.com<mailto:usrp-users@lists.ettus.com>> wrote: Hello, Several weeks ago I went through the tutorial for producing the example "gain" block using rfnoc 3.8 and uhd 3.15. There were some bumps, but I did get that working fine. For the past couple weeks, I've been working with UHD 4.0 and the latest gr-ettus repo. I posted a question a week or two ago since I couldn't get UHD to recognize my custom block, as UHD doesn't look for the block yml file in the latest uhd 4.0 build. It just shows up as "0/Block#0" when probing. Thanks to Wade F. for the quick response to that question and for suggesting I either just continue OOT and use the "Block" name to interface with it or build in-tree for now. I chose to stick with OOT and just use the "Block" naming. I started with the gain example, but ended up simplifying it to literally just using what was generated by rfnocmodtool (just a pass through block in the FPGA) with no modifications. I made an rfnoc block, called "Block". I built this for an E320, so I did have to modify the "XXX_x310_rfnoc_image_core.yml" file accordingly. I built/installed everything, but this is what is happening. When I create a gnuradio-companion "waveform", it does run, but I get the following behavior: 1) If my setup is RFNoC_RX_Radio -> RFNoC_DDC_Block -> RFNoC_Block -> RFNoC_RX_Streamer -> QT_GUI_Freq_Sink: Everything runs, but the following repeats over and over and the output plot doesn't change: ---- gr::log :WARN: rfnoc_rx_streamer0 - RFNoC Streamer block received error ERROR_CODE_BAD_PACKET (Code: 0xf) [ERROR] [STREAMER] The receive transport caught a value exception. ValueError: Bad CHDR header or invalid packet length. gr::log :WARN: rfnoc_rx_streamer0 - RFNoC Streamer block received error ERROR_CODE_BAD_PACKET (Code: 0xf) [ERROR] [STREAMER] The receive transport caught a value exception. ValueError: Bad CHDR header or invalid packet length. ---- I put in some ILA probes and it seems that "ep4_to_xb_tready" is stuck low. ep4 is the endpoint tied to the in and out of my custom "Block." I'm guessing it didn't start low but a FIFO or something filled up somewhere. I should mention that if I use this same setup, but remove my custom RFNoC_Block and directly connect the DDC to the RX_Streamer, everything works fine. No errors, the spectrum looks fine, etc. 2) If my setup is Constant_Source(set to 4+9j) -> RFNoC_TX_Streamer -> RFNoC_Block -> RFNoC_RX_Streamer -> QT_GUI_Time_Sink: Everything runs and I do not have a problem with any gnuradio warnings. Also, TReady is high the entire time. But, the output plot for I and Q sit mostly constant stuck at "1", with non-periodic blips down to "0". I'm not seeing the complex constant I set in gnuradio. If I look at the payload_tdata in an ILA for my "Block" when tvalid is high and tready is high, I see that the data is sitting at 0x7fff7fff except when TLAST is high, tdata switches to 0xfffc7ff7. I'm at a bit of a loss trying to figure out what is happening. Could it be that UHD is not interfacing properly to my block (given that UHD doesn't look for my OOT yml file)? I did not change any block controller code or anything else. Oh, and the user_register that is included as part of the default design designated by rfnocmodtool seems to work fine. I can change the register value in gnuradio and I can see it change appropriately via and ILA. For reference, this is what I've been working with: 1) UHD (v4.0.0.0 tag) 2) gnuradio (3.8.2.0 tag) 3) gr-ettus (maint-3.8-uhd4.0 branch) RFNoC is new to me, so any thoughts on what could be wrong or how I could go about debugging this would be greatly appreciated. Hopefully, I'm just missing something simple. Thanks, Jim _______________________________________________ USRP-users mailing list USRP-users@lists.ettus.com<mailto:USRP-users@lists.ettus.com> http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com<https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.ettus.com_mailman_listinfo_usrp-2Dusers-5Flists.ettus.com&d=DwMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=XUEEtUEfpaAEGxRI-WGuqHauOvsPdD2NZkfwDnwpYx0&m=Fz9nkm5B61ZorZZVX1jSes3LSxtOxW2XFb-XA9r-7vI&s=HTsRqrOpxd15k2kgT7iXFYxZW4Xwm9wIh3gu8TDf4Pg&e=> _______________________________________________ USRP-users mailing list USRP-users@lists.ettus.com<mailto:USRP-users@lists.ettus.com> http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com<https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.ettus.com_mailman_listinfo_usrp-2Dusers-5Flists.ettus.com&d=DwMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=XUEEtUEfpaAEGxRI-WGuqHauOvsPdD2NZkfwDnwpYx0&m=b2AlTidElkmDJcJHy8NcfscGTp_HD7DhVUdl3Rfu4gI&s=lzHy3_c4sElRGuoM9AfUYyy5128FP1BXmHXnE3VCLzg&e=>
_______________________________________________ USRP-users mailing list USRP-users@lists.ettus.com http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com
