You could look at using the etapp for exported pdu and add tags for metadata then just pas the protocol data to a suitable dussector. /Anders
Den tis 6 maj 2025 04:13Michael Mann via Wireshark-dev < wireshark-dev@wireshark.org> skrev: > This is all very helpful (especially to search the mailing list archives > in the future). > > Some clarifications/follow ups: > 1. I was not looking to write the file back in the original (text based) > format. I was perfectly fine with "open as .trc file, save as .pcap (or > .pcapng)" and I thought if I didn't provide write function callbacks in the > wiretap, that's how Wireshark would behave. To me there isn't a (strong) > need to "write" back to the original file format, the reading conversion is > just to take advantage of Wireshark's superior dissection/filtering for > analysis. And having it being done "inline" in a single step within > Wireshark instead of two steps - text2pcap derivation then Wireshark > (especially when you may not need an actual pcap file, just the data loaded > in memory for analysis) > > 2. You are correct that I was conflating file formats and metadata as far > as the layers between wiretap and dissection. If WTAP_ENCAP_SOCKETCAN is > used, then its "metadata format" needs to be followed and that's what I > find a little clunky after looking at the dissection in Wireshark. Here, I > see wiretap as producing a "SocketCAN record", and I thought producing a > "pcap or pcapng record" would be better. I thought the wiretap > functionality I was writing was to "massage" into an existing "wtap_encap" > format, and I want to shoot for a pcap or pcapng record over SocketCAN to > take advantage of the LINKTYPE_LINUX_SLL link-layer format/dissection. > > 3. Apparently text2pcap was refactored a few times since I wrote my > "derived" applications. The most notable being in December 2021 when it was > converted to use wtap_dump, which replaced all of the pcapio functions my > applications were using. To me the "algorithm" text2pcap used to be (and > my applications stayed with because I didn't have a reason to change them): > a) write_file_header() that uses pcapng_write_section_header_block() and > pcapng_write_interface_description_block() > b) parse text data out of the file for packets and create them with > pcapng_write_enhanced_packet_block() > c) write_file_trailer that uses pcapng_write_interface_statistics_block() > to provides statistics of the packet data read > > What I was hoping for was a wiretap interface with a similar algorithm > where just "b" needed to be adjusted as needed for slight differences in > text formats (within the wth->subtype_read callback) and may not need a > lexical scanner like text2pcap has. Looking at the current > ui/text_import*.[chl], > it seems to focus on actual files and not "records" like I would want for > the PEAK wiretap interface. > > > > On Monday, May 5, 2025 at 01:43:19 PM EDT, Guy Harris <ghar...@sonic.net> > wrote: > > > On May 5, 2025, at 8:24 AM, Michael Mann via Wireshark-dev < > wireshark-dev@wireshark.org> wrote: > > > There have been several times where I've been given a simple, text-based > capture file for Serial or CAN communications. My (quick and dirty) > solution has been to write a text2pcap derived application to convert the > file to pcapng format and then view it Wireshark. The packet dissection > support is usually already there, but I have also supplemented with plugins > when needed. > > > > However, https://gitlab.com/wireshark/wireshark/-/merge_requests/18894 > has shown me the "right" way to handle it - and that's using wiretap. The > packet data comes from a CAN bus, so the original thought was to use the > SocketCAN file format, > > What is "the SocketCAN file format"? > > > but WTAP_ENCAP_SOCKETCAN is a little clunky and I think I'd prefer to > use the pcapng format > > WTAP_ENCAP_SOCKETCAN is a Wiretap link-layer encapsulation, and is used > for several file formats, including... > > ...pcap and pcapng format with the LINKTYPE_CAN_SOCKETCAN link-layer type > value: > > https://www.tcpdump.org/linktypes/LINKTYPE_CAN_SOCKETCAN.html > > and, in fact, WTAP_ENCAP_SOCKETCAN was *originally created* for > LINKTYPE_CAN_SOCKETCAN. The most common file types with > WTAP_ENCAP_SOCKETCAN are probably... pcap and pcapng. > > > (similar to my text2pcap applications) to pipe it through "better" > dissection tables (sll.ltype). > > OK, *that's the LINKTYPE_LINUX_SLL link-layer type value for pcap and > pcapng: > > https://www.tcpdump.org/linktypes/LINKTYPE_LINUX_SLL.html > > whose primary disadvantage relative to LINKTYPE_CAN_SOCKETCAN is that the > packet size is significantly larger, with is why it's *not* the default for > capturing on CAN bus devices on Linux: > > https://github.com/the-tcpdump-group/libpcap/issues/1052 > > > The file format also contains "non-packet data" that I would like to > eventually convert into other pcapng block types. > > ...and convert those block types *back* to the PEAK CAN file format when > *writing* the file. > > libwiretap provides a layer of abstraction above the file formats it > supports; it can handle reading and writing file formats that don't map to > pcap or pcapng in a simple fashion. > > > I looked around a little, but I didn't see any obvious examples in > wiretap of how to easily provide a pcapng record. > > It's not intended to provide pcap or pcapng or snoop or Microsoft Network > Monitor or BLF or ERF or AIX i-trace or... records. > > It *does* have a mechanism for producing "file-format-specific" record > types - REC_TYPE_FT_SPECIFIC_EVENT and REC_TYPE_FT_SPECIFIC_REPORT. > > It *also* has a mechanism for producing pcapng custom blocks - > REC_TYPE_CUSTOM_BLOCK. *Those* have the advantage that, if the user wants > to save a *non*-pcap/pcapng format into a pcapng file, it can be done more > easily. What it *does* require is assigning *some* Private Enterprise > Number for a given type: > > https://www.iana.org/assignments/enterprise-numbers/ > > If the organization that owns the file format has a PEN, *and* they're > willing to "own" pcapng custom blocks with that PEN, that would be one > way. The custom block should probably begin with a "custom block type for > this PEN" field, so that more than one type of custom block can be > supported. The organization would manage the space of custom block types. > > On the other hand, we could use *the Wireshark Foundation's* PEN: > > https://www.iana.org/assignments/enterprise-numbers/?q=Wireshark > > and manage the custom block types ourselves. > > > pcapng.c does its own processing and is a bit complex compared to the > APIs used in text2pcap. > > The "pcap" in "text2pcap" refers to the pcap file format: > > > https://ietf-opsawg-wg.github.io/draft-ietf-opsawg-pcap/draft-ietf-opsawg-pcap.html > > which is a bit less complex than the pcapng file format: > > > https://ietf-opsawg-wg.github.io/draft-ietf-opsawg-pcap/draft-ietf-opsawg-pcapng.html > > so, yes, APIs to write out pcapng files will be a bit complex compared to > APIs use to write pcap files. > > > Looking at other wiretap examples, they seem to have a "file dissection > layer" in epan/dissectors that corresponds to the wiretap handling (linking > to "wtap_encap" table) before data is passed to a different dissection > table for "packet dissection". > > That's for non-pcap/pcapng files that have their own stuff that count as > metadata; wiretap provides the metadata as a header for the packet data. > > This should perhaps be done by having the wiretap read APIs provide, and > the wiretap write APIs accept, separate blobs of "file type metadata" and > "packet data", so that, for example, an HP-UX nettl file would provide the > nettl metadata separately from the packet data, and the WTAP_ENCAP_ type > would be the type for the packet data. > > The "file type metadata" would be indicated by the file type. File types > are *not* identified by fixed assigned numbers (except for pcap and > pcapng), as code using wiretap is, in general, *not* supposed to do > specific things for specific file types - wiretap should provide general > indications such as "does this file support multiple link-layer types per > file?", so that the code using wiretap doesn't need to be updated to handle > new file types. The way to handle *that* is to have either built-in > modules or plugins for that file format, providing both wiretap code (which > would be handed the file type value when it registers its file type) and > file type metadata dissectors (which would register for that in a "file > type metadata" dissector. > > That would also be used to register to handle a particular type of > REC_TYPE_FT_SPECIFIC_EVENT or REC_TYPE_FT_SPECIFIC_REPORT. > > > Is pcapng.c the only source of what I'll have to look at as an > "example"? Can anyone provide more pointers on my desire to have > "text2pcap functionality in wiretap", > > "text2pcap functionality" is "converting a hex dump text file to a capture > file". In some sense, that functionality is already *in* several wiretap > modules for file format that are text files. > > This is independent of "handling file type specific information", which > could be done for text *or* binary formats. > > > > to make it easier to provide wiretaps for future (simple) text-based > packet data. > > > The "text2pcap functionality" is currently in ui/text_import*.[chl], > because it's used in *two* places - text2pcap and Wireshark's "import a > text file" facility. > > It could, for example, be moved into wsutil, or into its own text_import > library, to be more conveniently used by text2pcap and Wireshark *and* > wiretap. > > > _______________________________________________ > Wireshark-dev mailing list -- wireshark-dev@wireshark.org > To unsubscribe send an email to wireshark-dev-le...@wireshark.org >
_______________________________________________ Wireshark-dev mailing list -- wireshark-dev@wireshark.org To unsubscribe send an email to wireshark-dev-le...@wireshark.org
