Re: Raw Sockets: Two Questions
On 21.03.2018 10:55, Matt Joras wrote: > I'm going to be doing some stuff with raw sockets pretty soon, and > while scrounging around, looking for some nice coding examples, I > found the following very curious comment on one particular message > board: > > > https://stackoverflow.com/questions/7048448/raw-sockets-on-bsd-operating-systems > > "Using raw sockets isn't hard but it's not entirely portable. For > instance, both in BSD and in Linux you can send whatever you want, > but in BSD you can't receive anything that has a handler (like TCP > and UDP)." > > So, first question: Is the above comment actually true & accurate? Not for FreeBSD. >>> Are you saying that I can receive on a raw socket SCTP, TCP and UDP packets? >> >> No. I'm saying one can send/receive RAW IP packets no matter are they SCTP, >> TCP or UDP >> or something else by means of libdnet. It uses raw sockets and BPF internally >> but hides this complexity. nmap uses it just fine. >> > Saying "Not for FreeBSD" is needlessly confusing and not accurate. In > the common parlance "raw sockets" does not refer to libdnet, which is > not a part of the FreeBSD base system. You cannot use traditional raw > sockets on FreeBSD to receive traditional protocol packets. The goal is to send/receive RAW IP packets, not to use raw sockets, isn't it? > The only way to do that in the base system is to use a BPF handle directly. Not exactly. For example: if_ethersubr.c/ether_input_internal() performs some sanity checks then passes incoming frame to BPF and to NETGRAPH (ng_ether) before passing it to upper stack layers. One can do almost anything with the frame by means of NETGRAPH subsystem and pass it to userland too. One have to setup some NETGRAPH nodes before, though - just like ppp(8) does to process control PPPoE frames. ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: Raw Sockets: Two Questions
> On 21. Mar 2018, at 03:43, Eugene Grosbein wrote: > > On 21.03.2018 08:03, Michael Tuexen wrote: > >>> On 21. Mar 2018, at 00:39, Eugene Grosbein wrote: >>> >>> 21.03.2018 3:09, Ronald F. Guilmette wrote: >>> I'm going to be doing some stuff with raw sockets pretty soon, and while scrounging around, looking for some nice coding examples, I found the following very curious comment on one particular message board: https://stackoverflow.com/questions/7048448/raw-sockets-on-bsd-operating-systems "Using raw sockets isn't hard but it's not entirely portable. For instance, both in BSD and in Linux you can send whatever you want, but in BSD you can't receive anything that has a handler (like TCP and UDP)." So, first question: Is the above comment actually true & accurate? >>> >>> Not for FreeBSD. >> Are you saying that I can receive on a raw socket SCTP, TCP and UDP packets? > > No. I'm saying one can send/receive RAW IP packets no matter are they SCTP, > TCP or UDP > or something else by means of libdnet. It uses raw sockets and BPF internally > but hides this complexity. nmap uses it just fine. OK. Thanks for the clarification. Best regards Michael > > ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
netmap ixgbevf max frame size
I am unable to send frames larger than 9216 bytes (destination MAC through trailing CRC inclusive) using ixgbevf hardware with latest netmap code (LINUX). What is the source of this limitation? From the chip datasheet it appears that much larger frames are supported. There is mention of 9216 in some of the driver source files but as an MTU, the max frame size is larger. Joe Buehler ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: netmap ixgbevf max frame size
This turned out to be a limitation of the box we are using, there is apparently hardware between the ixgbevf chip and the fiber. Joe Buehler > I am unable to send frames larger than 9216 bytes (destination MAC through > trailing CRC inclusive) using ixgbevf hardware with latest netmap code > (LINUX). > > What is the source of this limitation? From the chip datasheet it appears > that much larger frames are supported. > > There is mention of 9216 in some of the driver source files but as an MTU, > the max frame size is larger. > > Joe Buehler > > ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: Raw Sockets: Two Questions
In message <5ab1a9c5.9050...@grosbein.net>, Eugene Grosbein wrote: >21.03.2018 3:09, Ronald F. Guilmette wrote: >> >> https://stackoverflow.com/questions/7048448/raw-sockets-on-bsd-operating-systems >> "Using raw sockets isn't hard but it's not entirely portable. For >> instance, both in BSD and in Linux you can send whatever you want, >> but in BSD you can't receive anything that has a handler (like TCP >> and UDP)." >> >> So, first question: Is the above comment actually true & accurate? > >Not for FreeBSD. Is it true for other *BSDs? >> Second question: If the above assertion is actually true, then how can >> nmap manage to work so well on FreeBSD, despite what would appear to be >> this insurmountable stumbling block (of not being able to receive replies)? > >nmap uses libdnet that provides some portability layer, including RAW socket >operations. >It uses bundled stripped-down version but we have "normal" one as net/libdnet >port/package. >You should consider using it too as convenience layer. Thank you. I will certainly look into this, however my needs are quite small and modest... probably so modest that a "convenience layer" wouldn't be a substantial help. ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: Raw Sockets: Two Questions
In message <5ab23fb9.7050...@grosbein.net>, Eugene Grosbein wrote: >On 21.03.2018 10:55, Matt Joras wrote: >> Saying "Not for FreeBSD" is needlessly confusing and not accurate. In >> the common parlance "raw sockets" does not refer to libdnet, which is >> not a part of the FreeBSD base system. You cannot use traditional raw >> sockets on FreeBSD to receive traditional protocol packets. > >The goal is to send/receive RAW IP packets, not to use raw sockets, isn't it? > >> The only way to do that in the base system is to use a BPF handle directly. > >Not exactly. For example: if_ethersubr.c/ether_input_internal() performs some >sanity checks >then passes incoming frame to BPF and to NETGRAPH (ng_ether) before passing it >to upper stack l >ayers. > >One can do almost anything with the frame by means of NETGRAPH subsystem >and pass it to userland too. One have to setup some NETGRAPH nodes before, >though - >just like ppp(8) does to process control PPPoE frames. OK, so, if I have understood all that has been said in this thread so far, then I would assert that, from the perspective of a simple-minded and naive end user (e.g. me), the assertion that I originally quoted -is- in fact correct, i.e. one -cannot- just simply do sendto/recvfrom (and expect to get back responses) if the raw packets that one sends out happen to be, for example, well formed TCP or UDP packets. If I have correctly understood Matt Joras, there -are- ways to get hold of such reply packets, under FreeBSD, but those require getting a bit more "under the hood" in order to actually get hold of them... more than just a simple recvfrom on the RAW socket. Have I understood correctly? ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: Raw Sockets: Two Questions
22.03.2018 0:38, Ronald F. Guilmette пишет: > > In message <5ab1a9c5.9050...@grosbein.net>, > Eugene Grosbein wrote: > >> 21.03.2018 3:09, Ronald F. Guilmette wrote: >>> >>> https://stackoverflow.com/questions/7048448/raw-sockets-on-bsd-operating-systems >>> "Using raw sockets isn't hard but it's not entirely portable. For >>> instance, both in BSD and in Linux you can send whatever you want, >>> but in BSD you can't receive anything that has a handler (like TCP >>> and UDP)." >>> >>> So, first question: Is the above comment actually true & accurate? >> >> Not for FreeBSD. > > Is it true for other *BSDs? Don't know about raw sockets. But I know that NETGRAPH that allows to send/receive RAW frames is FreeBSD specific. ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: Raw Sockets: Two Questions
22.03.2018 1:08, Ronald F. Guilmette wrote:
> OK, so, if I have understood all that has been said in this thread so
> far, then I would assert that, from the perspective of a simple-minded
> and naive end user (e.g. me), the assertion that I originally quoted
> -is- in fact correct, i.e. one -cannot- just simply do sendto/recvfrom
> (and expect to get back responses) if the raw packets that one sends out
> happen to be, for example, well formed TCP or UDP packets.
Not exactly. You can't use raw sockets to receive but that does not mean
you cannot use sendto/recvfrom (or similar calls) at all: there are
libpcap, libdnet and NETGRAPH allowing to send requests and receive answers.
> If I have correctly understood Matt Joras, there -are- ways to get hold
> of such reply packets, under FreeBSD, but those require getting a bit more
> "under the hood" in order to actually get hold of them... more than just
> a simple recvfrom on the RAW socket.
Why should you concentrate on RAW sockets?
I have small perl script that sends manually crafted PPPoE frames
and receives replies using simple libpcap interface:
use Net::Pcap qw(:DEFAULT :functions);
use constant V8021Q => 0x8100;
use constant ETHERTYPE_PPPOEDISC=> 0x8863;
use constant PPPOE_VER => 1;
use constant PPPOE_TYPE => 1;
use constant PADO_CODE => 7;
use constant PADI_CODE => 9;
use constant TAG_END_OF_LIST=> 0x;
use constant TAG_SERVICE_NAME => 0x0101;
use constant TAG_AC_NAME=> 0x0102;
use constant TAG_HOST_UNIQ => 0x0103;
$packet =
# Ethernet header: dst MAC, src MAC, TYPE
ether_aton('ff:ff:ff:ff:ff:ff') . $bmac . pack('n', ETHERTYPE_PPPOEDISC) .
# PPPoE PADI: VER, TYPE, CODE, SESSION_ID=0
pack('C', (PPPOE_VER<<4) + PPPOE_TYPE) . pack('C', PADI_CODE) . pack('n', 0)
# LENGTH, tags
pack('n', $tlen) . $tags;
# zero padding upto 60 bytes ethernet frame length (without checksum)
$packet .= pack('a' . (40-$tlen) , '') if $tlen < 40;
err("cannot open interface $interface: $err")
unless $pcap = pcap_open_live($interface, $snaplen, 0, 0, \$err);
err("could not send PADI")
if pcap_sendpacket($pcap, $packet) != 0;
$filter = "ether proto " . ETHERTYPE_PPPOEDISC . " and ether dst $mac";
err("cannot compile filter: $filter")
if pcap_compile($pcap, \$bpf, $filter, 1, 0) < 0;
pcap_setfilter($pcap, $bpf);
$ec = 0;
while($ec == 0) {
$ec = pcap_loop($pcap, -1, \&callback, undef);
}
pcap_close($pcap);
exit(0);
sub callback($$$) {
return if $_[1]->{'len'} < 20;# sanity check: short frame
my ($dst, $src, $ftype, $ftag, $fp) = unpack('a6a6na4a*' , $_[2]);
...
}
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: Raw Sockets: Two Questions
In message <5ab2ad9f.6040...@grosbein.net>, Eugene Grosbein wrote: >Why should you concentrate on RAW sockets? Well, for reasons that are completely legitimate, and that I'll explain in detail, if anyone is seriously interested, I'd like to check each IPv4 address within a set of about 90 or so modest sized CIDRs and find out which ones of those have a certain TCP port accepting connections. It would be (and is) trivial, of course, to just simply write a little program to perform an ordinary TCP connect, complete with all three parts of the TCP handshake, and then have the program do that, individually, for each IP in the specified CIDRs, but my opinion/impression is that this could be quite slow. Obviously, it would be rather faster to just send out the SYNs, full throttle, one per IP, and then see what SYN-ACKs come back. And actually, now that I've just reviewed the online man page for zmap, I see that (contrary to my earlier belief/suspicion) this has enough control options to me useful to me, so I'll probably just use that instead of rolling my own. (I first learned about zmap some long time ago, but only just today decided to actually take it out for a test drive.) Still, there may be some other reasons for me to educate myself about proper programming techniques utilizing raw sockets, but I'll talk about that in a separate post to follow this one. ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: Raw Sockets: Two Questions
22.03.2018 3:03, Ronald F. Guilmette wrote: >> Why should you concentrate on RAW sockets? > > Well, for reasons that are completely legitimate, and that I'll > explain in detail, if anyone is seriously interested, I'd like > to check each IPv4 address within a set of about 90 or so > modest sized CIDRs and find out which ones of those have a > certain TCP port accepting connections. > > It would be (and is) trivial, of course, to just simply write a > little program to perform an ordinary TCP connect, complete > with all three parts of the TCP handshake, and then have the > program do that, individually, for each IP in the specified > CIDRs, but my opinion/impression is that this could be quite > slow. > > Obviously, it would be rather faster to just send out the SYNs, > full throttle, one per IP, and then see what SYN-ACKs come back. It does not mean you need to stick with raw sockets API. libpcap can be used too, as I've shown in previous letter. In fact, I use that code for very similar task: I send out several thousands of PPPoE service discovery frames (PADI) at full throttle (one per vlan) and then wait for responses (PADO) to come back. No raw socket (ab)used in progress :-) ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: netmap ixgbevf mtu
I see. Unfortunately this breaks the API, so I don't think we can accept it. We should probably sum up the fragment lengths, remember which one was the first descriptor and write the olinfo field when we process the last descriptor ... I hope this does not slow down the simpler case where NS_MOREFRAG is not used. In any, case we should move this discussion to the github, if possible (so that the issue gets tracked). Cheers, Vincenzo 2018-03-20 20:54 GMT+01:00 Joe Buehler : > Attached is a patch that allows fragmented TX with the ixgbevf driver. > > For the first TX buffer set the slot length to the full length of the > frame and make sure that the slot buffer is fully filled. For succeeding > slots just set the length to the amount of the buffer filled. > > Not intended as the perfect solution but it works fine for my situation. > > Joe Buehler > > -- Vincenzo Maffione ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: Raw Sockets: Two Questions
In message <5ab2c0b1.3020...@grosbein.net>, Eugene Grosbein wrote: >It does not mean you need to stick with raw sockets API. >libpcap can be used too, as I've shown in previous letter. Thank you. If zmap ends up not suiting my needs, I will definitely look into libpcap. ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Same host or different? How can you tell "over the wire"?
This problem has been preplexing me for ages and ages. I looked at it again, just briefly, and re-read parts of some potentially relevant RFCs, just the other day, but frankly, I'm just too ignorant and/or too stupid to be able to think up a solution, so I'll just drop the problem description here and see if any of you more knowledgable people can devise or suggest a solution. The Problem: Suppose that there exist two IPv4 addresses, A and A'. Both addresses have the exact same set of ports open, and both respond in identical ways, at least at the application level, when sent identical inputs. In short, at the application layer level, at least, there appears to be no way to reliably differentiate between the case where the two IP addresses are being routed to a single common physical machine (or to a single common virtual OS instance) or to two separate physical machines (or two separate virtual OS instances). Is there any method which can be applied to A and A' over the Internet and which could reliably differentiate these two possible cases from one another (i.e. a single common host versus two separate hosts)? If any such method or mechanism exists, I would very much like to know all of the details thereof. Such a method, if one exists, would certainly have value in various types of forensic investigations. Regards, rfg P.S. It is my assumption that the kind of thing I'm looking for, if it exists at all, will be found somewhere below the application layer. I do not rule out however that there may be some way of differentiating the two cases described above by looking at application layer responses for some certain common applications. As far as I know however, it is not possible to make the desired differentiation on the basis of application layer responses for most typical network applications, e.g. various makes and model numbers of servers for HTTP, HTTPS, SMTP, SSH, DNS, etc. Of course, if I have simply missed something, and if there is in fact a way to differentiate the two cases on the basis of responses sent for any of these application protocols, then I sure would like to know about that too. ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: Same host or different? How can you tell "over the wire"?
22.03.2018 4:19, Ronald F. Guilmette wrote: > This problem has been preplexing me for ages and ages. I looked at it > again, just briefly, and re-read parts of some potentially relevant > RFCs, just the other day, but frankly, I'm just too ignorant and/or > too stupid to be able to think up a solution, so I'll just drop the > problem description here and see if any of you more knowledgable > people can devise or suggest a solution. > > The Problem: > > Suppose that there exist two IPv4 addresses, A and A'. Both addresses > have the exact same set of ports open, and both respond in identical > ways, at least at the application level, when sent identical inputs. > In short, at the application layer level, at least, there appears to > be no way to reliably differentiate between the case where the two > IP addresses are being routed to a single common physical machine > (or to a single common virtual OS instance) or to two separate physical > machines (or two separate virtual OS instances). > > Is there any method which can be applied to A and A' over the > Internet and which could reliably differentiate these two possible > cases from one another (i.e. a single common host versus two separate > hosts)? > > If any such method or mechanism exists, I would very much like to know > all of the details thereof. Such a method, if one exists, would > certainly have value in various types of forensic investigations. > > > Regards, > rfg > > > P.S. It is my assumption that the kind of thing I'm looking for, if > it exists at all, will be found somewhere below the application layer. > I do not rule out however that there may be some way of differentiating > the two cases described above by looking at application layer responses > for some certain common applications. As far as I know however, it is > not possible to make the desired differentiation on the basis of > application layer responses for most typical network applications, > e.g. various makes and model numbers of servers for HTTP, HTTPS, > SMTP, SSH, DNS, etc. Of course, if I have simply missed something, > and if there is in fact a way to differentiate the two cases on the > basis of responses sent for any of these application protocols, then > I sure would like to know about that too. If they respond truly identically, there are no reasons to treat them like distinct hosts despite of different IP addresses. And if you have such reason despite they respond truly identically, then such a reason steams from matters other than their response on requests to open ports. In this case you should differentiate them by other means too, not by open port's responses. ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: Same host or different? How can you tell "over the wire"?
Do you mean that the application banners for all applications are the same? A comprehensive scan with nmap shows no differences? I know you specified SSH as outside of the application layer, but I would think if it's even to the point that the same SSH key (or credentials) work for both machines, and upon login provide the same hostname in the prompt, you'd have to dig and see if the NIC configs show a difference, or perhaps that there are multiple NICs, or a single NIC aliased with the IP addresses you're reviewing. Kurt On Wed, Mar 21, 2018 at 2:19 PM, Ronald F. Guilmette wrote: > > This problem has been preplexing me for ages and ages. I looked at it > again, just briefly, and re-read parts of some potentially relevant > RFCs, just the other day, but frankly, I'm just too ignorant and/or > too stupid to be able to think up a solution, so I'll just drop the > problem description here and see if any of you more knowledgable > people can devise or suggest a solution. > > The Problem: > > Suppose that there exist two IPv4 addresses, A and A'. Both addresses > have the exact same set of ports open, and both respond in identical > ways, at least at the application level, when sent identical inputs. > In short, at the application layer level, at least, there appears to > be no way to reliably differentiate between the case where the two > IP addresses are being routed to a single common physical machine > (or to a single common virtual OS instance) or to two separate physical > machines (or two separate virtual OS instances). > > Is there any method which can be applied to A and A' over the > Internet and which could reliably differentiate these two possible > cases from one another (i.e. a single common host versus two separate > hosts)? > > If any such method or mechanism exists, I would very much like to know > all of the details thereof. Such a method, if one exists, would > certainly have value in various types of forensic investigations. > > > Regards, > rfg > > > P.S. It is my assumption that the kind of thing I'm looking for, if > it exists at all, will be found somewhere below the application layer. > I do not rule out however that there may be some way of differentiating > the two cases described above by looking at application layer responses > for some certain common applications. As far as I know however, it is > not possible to make the desired differentiation on the basis of > application layer responses for most typical network applications, > e.g. various makes and model numbers of servers for HTTP, HTTPS, > SMTP, SSH, DNS, etc. Of course, if I have simply missed something, > and if there is in fact a way to differentiate the two cases on the > basis of responses sent for any of these application protocols, then > I sure would like to know about that too. > ___ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org" ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: Same host or different? How can you tell "over the wire"?
> > This problem has been preplexing me for ages and ages. I looked at it > again, just briefly, and re-read parts of some potentially relevant > RFCs, just the other day, but frankly, I'm just too ignorant and/or > too stupid to be able to think up a solution, so I'll just drop the > problem description here and see if any of you more knowledgable > people can devise or suggest a solution. > > The Problem: > > Suppose that there exist two IPv4 addresses, A and A'. Both addresses > have the exact same set of ports open, and both respond in identical > ways, at least at the application level, when sent identical inputs. > In short, at the application layer level, at least, there appears to > be no way to reliably differentiate between the case where the two > IP addresses are being routed to a single common physical machine > (or to a single common virtual OS instance) or to two separate physical > machines (or two separate virtual OS instances). > > Is there any method which can be applied to A and A' over the > Internet and which could reliably differentiate these two possible > cases from one another (i.e. a single common host versus two separate > hosts)? > > If any such method or mechanism exists, I would very much like to know > all of the details thereof. Such a method, if one exists, would > certainly have value in various types of forensic investigations. > > > Regards, > rfg > > > P.S. It is my assumption that the kind of thing I'm looking for, if > it exists at all, will be found somewhere below the application layer. > I do not rule out however that there may be some way of differentiating > the two cases described above by looking at application layer responses > for some certain common applications. As far as I know however, it is > not possible to make the desired differentiation on the basis of > application layer responses for most typical network applications, > e.g. various makes and model numbers of servers for HTTP, HTTPS, > SMTP, SSH, DNS, etc. Of course, if I have simply missed something, > and if there is in fact a way to differentiate the two cases on the > basis of responses sent for any of these application protocols, then > I sure would like to know about that too. One thing you could look at is the OS finger printing of nmap, that could look for possible things to diffentiate the hosts. Depending on just what the host is there could be other tale tale signs picked up from "forensic" type of data captured with tcpdump while playing known packet sequences against each host at identical time. What you ask I believe could be done, but it non trivial and would require a very good understanding of both forensics and the differing ways that TCP/IP is implemented. One simple thing is a record route of a packet, it might show that the hosts are clearly at differing paths. If the hosts are very different a ssh connect could lead to an answer as it may give a differing answer string: telnet freefall.freebsd.org 22 Trying 2610:1c1:1:6074::16:84... Trying 96.47.72.132... Connected to freefall.freebsd.org. Escape character is '^]'. SSH-2.0-OpenSSH_7.5 FreeBSD-20170804 telnet localhost 22 Trying ::1... Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. SSH-2.0-OpenSSH_7.2 FreeBSD-20161230 Clearly these are 2 different machines It would also be possible to implemented controlled DOS techniques to cause "measureable" load on one IP, and then see if the other IP has a similiar measureable load factor. This does not work well if the DOS technique causes a commond mode issue, but that just fails in the "can not tell" mode. -- Rod Grimes rgri...@freebsd.org ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: Multicast/SSDP not working (on VLAN interface)
Thank you for bearing with me. On 21.03.18 01:44, Rodney W. Grimes wrote: ... Show me your full firewall rule set, without that I can only speculate as to where it is getting blocked, but given your symptoms I highly suspect the firewall is blocking the packets OUT of your SERVER back towards the client as they try to go out what ever interface your default route is on. OK, I see your point. But I obviously forgot to mention some important things. I am sorry for that. First of all, my rules file is 800+ lines, I am not sure how much it helps to throw that at anybody. I agree that this makes it somewhat more likely that the firewall IS the problem, but I do have reason to believe that it is not. I did check on the interface the default route is pointing through: there is no incorrect outgoing traffic. Obviously, if the firewall blocks the traffic, then I expect not to see any. But there are the following rules as well: allow ip4 from me to "table($internal)" out xmit re1\* keep-state allow ip4 from me to "table($broadcast)" out xmit re1\* table($internal) is basically RFC1918 and table($broadcast) contains 224.0.0.0/4 and ff00::/8. To test, I have also added the following rule allow ip4 from me to "table($broadcast)" out xmit but did not see any outgoing multicast traffic from MiniDLNA. Also, when MiniDLNA is running before I add the interface route for 244.0.0.0/4, the VLC test client does NOT discover the MiniDLNA server (despite the route). If, however, the MiniDLNA service is (re)started after the route is in place (but before the test client is running), it does detect the server immediately. I therefore suspect that the "MiniDLNA startup code" does something different when the route is there as compared to when it is not. But I don't know what... I assume it is not related to mrouted (which is NOT running)? Also, again, if setting the interface route for 224.0.0.0/4 absolutely IS required, I will not be able to make my intended setup work. I have multiple interfaces connected to potential clients for MiniDLNA. Best regards andreas ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: Same host or different? How can you tell "over the wire"?
On Wed, 21 Mar 2018 14:19:43 -0700 "Ronald F. Guilmette" wrote: > > This problem has been preplexing me for ages and ages. I looked at it > again, just briefly, and re-read parts of some potentially relevant > RFCs, just the other day, but frankly, I'm just too ignorant and/or > too stupid to be able to think up a solution, so I'll just drop the > problem description here and see if any of you more knowledgable > people can devise or suggest a solution. > > The Problem: > > Suppose that there exist two IPv4 addresses, A and A'. Both addresses > have the exact same set of ports open, and both respond in identical > ways, at least at the application level, when sent identical inputs. > In short, at the application layer level, at least, there appears to > be no way to reliably differentiate between the case where the two > IP addresses are being routed to a single common physical machine > (or to a single common virtual OS instance) or to two separate > physical machines (or two separate virtual OS instances). > > Is there any method which can be applied to A and A' over the > Internet and which could reliably differentiate these two possible > cases from one another (i.e. a single common host versus two separate > hosts)? > > If any such method or mechanism exists, I would very much like to know > all of the details thereof. Such a method, if one exists, would > certainly have value in various types of forensic investigations. > Perhaps I don't understand the question but: A ping should measure the "distance" to A and A', traceroute works too. If you disable protections (firewalls, ids, etc) you could inject tcp packets with fake ips. Or make a dDoS to one, the other should stay alive. If you go to layer 2 there are mac differences. Take the whois info from both, it will be the same, but you can ask the owner to switch off one of them. If SNMP is enabled and accesible it must show some differences (ethernet mac) > Regards, > rfg > > > P.S. It is my assumption that the kind of thing I'm looking for, if > it exists at all, will be found somewhere below the application layer. > I do not rule out however that there may be some way of > differentiating the two cases described above by looking at > application layer responses for some certain common applications. As > far as I know however, it is not possible to make the desired > differentiation on the basis of application layer responses for most > typical network applications, e.g. various makes and model numbers of > servers for HTTP, HTTPS, SMTP, SSH, DNS, etc. Of course, if I have > simply missed something, and if there is in fact a way to > differentiate the two cases on the basis of responses sent for any of > these application protocols, then I sure would like to know about > that too. ___ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org" --- --- Eduardo Morras ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: Same host or different? How can you tell "over the wire"?
In message <5ab2d11a.6060...@grosbein.net>, Eugene Grosbein wrote: >If they respond truly identically, there are no reasons to treat them like >distinct hosts >despite of different IP addresses. Well, for my purposes, it would be inapporpriate to make any such leap of faith. If address A is somehow established to be under the control of a given Bad Actor, then even if address A' is seen to yield essentially identical results at the level of the application layer, this is most certainly -not- an adequate justification for anyone (e.g. me, or anyone else for that matter) to affirmatively assert that A' is under the control of the exact same Bad Actor. Individual IPv4 addresses may often exhibit an identical set of open ports. And the responses provided when sending data to those ports may be "generic" and thus may be actually or virtually identical. This alone is not nearly enough to assert that A' is under the control of the exact same Bad Actor who is in control of A. >And if you have such reason despite they respond truly identically, >then such a reason steams from matters other than their response on requests >to open ports. >In this case you should differentiate them by other means too, not by open >port's responses. Yes... by other means -also-, e.g. DNS. Assume that this has already been done. Assume that two different (and somehow related) FQDNs point to two different IPv4 addreses, A and A'. As we all know, any fool on the Internet can point any FQDN for which he controls the DNS to any bloody address he wants. But any such "pointing", standing alone and by itself, does not -prove- a damn thing about the pointed-at addresses, or about who is -currently- controlling them. (I wish that I had a dollar for every FQDN I had ever come across that resolved to either 127/8 or 10/8, or that pointed to an address that is not currently routed, and which perhaps never has been.) If other data persuasively indicates that address A is under the control of a Bad Actor, and if there appears to be some connection between A and A' (such as some sort of association indicated by the DNS) then if there were a way to also establish that A and A' are both being routed to a single machine, then it could be reliably and persuasively asserted, without fear of contradiction, that A' is also under the control of the same Bad Actor. I would like to be able to make such logical inferences and assertions, which is what prompted my question. ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Same host or different? How can you tell "over the wire"?
"Kurt Buff" wrote: >Do you mean that the application banners for all applications are the >same? A comprehensive scan with nmap shows no differences? Correct. This is the case I was/am asking about. >I know you specified SSH as outside of the application layer, but I >would think if it's even to the point that the same SSH key (or >credentials) work for both machines, and upon login provide the same >hostname in the prompt In case it was not clear, none of the IPv4 addresses that are of interest, or that are relevant to my question, are ones for which *I* posses any type of SSH login credentials. But your question certainly raises an interesting possibility, and an interesting question... one that I myself am not at all equiped or qualified to answer (because I am almost totally ignorant about even the bare mechanics of the SSH protocol): How could one tickle an open SSH port and obtain from it not just its greeting banner (which may be, and often is, rather generic and non-specific) but also so as to get the host's host-specific public key? (Yes, I am indeed displaying an unforgivable level of laziness here. I can and most probably should, and most probably eventually -will- just go off now and read the relevant RFCs, but if anyone wants to save me the trouble, just for this one question, that would be appreciated.) >you'd have to dig and see if the NIC configs >show a difference, or perhaps that there are multiple NICs, or a >single NIC aliased with the IP addresses you're reviewing. Yes. This is yet a different way that the problem might be attacked. I am most interested in that last possibility you mentioned, and specifically I am interested in differentiating that case from all other possible cases. But I am far too ignorant of the relevant protocols to be able to work out a way to solve the problem this way, so if anyone might be willing to explain it to me, in detail, that also would be most appreciated. ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: Same host or different? How can you tell "over the wire"?
In message <201803212204.w2lm4g8h023...@pdx.rh.cn85.dnsmgr.net>, "Rodney W. Grimes" wrote: >One thing you could look at is the OS finger printing of nmap, >that could look for possible things to diffentiate the hosts. Yea, that idea occurred to me. But this solution has the same problem that I just mentioned in another one of my replies in this thread: Even if nmap says that two IP addresses have the exact same OS signature, that is far from enough to assert that they are both under the control of the exact same Bad Actor. You certainly wouldn't want to send someone to prison, or even to after-school detention, based on such limited circumstantial evidence. >Depending on just what the host is there could be other tale >tale signs picked up from "forensic" type of data captured >with tcpdump while playing known packet sequences against >each host at identical time. Such as? I'm all ears. >What you ask I believe could be done, but it non trivial and >would require a very good understanding of both forensics >and the differing ways that TCP/IP is implemented. I like to think that I am a quick learner. Please proceed with the lesson. >One simple thing is a record route of a packet, it might show >that the hosts are clearly at differing paths. Again, not adequate, I think. Not for my purposes. The addresses A and A' might very well be within the same /24, and the routing might thus be identical. Does that prove that both are under the control of the same single guy? I would say not. >If the hosts are very different a ssh connect could >lead to an answer as it may give a differing answer string: >... >SSH-2.0-OpenSSH_7.5 FreeBSD-20170804 >... >SSH-2.0-OpenSSH_7.2 FreeBSD-20161230 Believe me, if the problem were as easy to solve as in this example, I wouldn't even be here asking the questions I'm asking. The question is: What can be reliably deduced when we see this? Address A: SSH-2.0-OpenSSH_7.5 Address A': SSH-2.0-OpenSSH_7.5 i.e. the kind of identical outputs that I posited in my original statement of the problem. >It would also be possible to implemented controlled DOS techniques >to cause "measureable" load on one IP, and then see if the other >IP has a similiar measureable load factor. This also occured to me. Ignoring for the moment the various... and I hope obvious... ethical and legal questions, it isn't even clear to me how this might be made to work, in practice. I did make it a point to look over some online simple-minded tutorials relating to SYN cookies just the other day... not long before starting this thread. I am guessing that the use of those is now quite widespread, and it seemed to me that these (SYN cookies) would render any attempt at (for lack of a better term) "limit probing" non-useful, at least with respect to TCP. But maybe I'm wrong about that. In any case, the ideal solution to my original problem statement would quite certainly -not- involve anything as obtrusive (or as potentially illegal and/or unethical) as pushing any host hard enough to find its capacity limits. Rather, I am still vaguely hoping that there may be something burried deep down in the IP protocol or perhaps even lower (MAC addresses?) that will provide an elegant and minimally impactful solution. Alternatively, if it turns out that there is something anmything... that is host-specific and that can be teased out from some commonly used application layer protocol (e.g. an SSH server's public key) then that will probably do quite nicely also. ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: Multicast/SSDP not working (on VLAN interface)
> Thank you for bearing with me. > > On 21.03.18 01:44, Rodney W. Grimes wrote: > > ... > > > Show me your full firewall rule set, without that I can only speculate > > as to where it is getting blocked, but given your symptoms I highly > > suspect the firewall is blocking the packets OUT of your SERVER back > > towards the client as they try to go out what ever interface your > > default route is on. > > OK, I see your point. But I obviously forgot to mention some important > things. I am sorry for that. > > First of all, my rules file is 800+ lines, I am not sure how much it > helps to throw that at anybody. I agree that this makes it somewhat more > likely that the firewall IS the problem, but I do have reason to believe > that it is not. If your rule set is 800+ lines I highly suspect your doing something in a very non-optimal way. It also leads to be VERY suspeciious that it is infact in your firewall. Try as a very first rules: ipfw add 1 log allow ip from any to 224.0.0.0/4 ipfw add 2 log allow ip from 224.0.0.0/4 to any DO NOT put any restricting clauses on these.. if this makes things work simply move them down a few rules until you find the point at which things stop working. You may want to examine the log, as it would tell you what ipfw thinks the packet path is, and may shed some light on why other things are not happening the way you expect. Also do you have any nat, divert, or other things going on, they can all have an effect on mutlicast packets. > I did check on the interface the default route is pointing through: > there is no incorrect outgoing traffic. > > Obviously, if the firewall blocks the traffic, then I expect not to see any. > > But there are the following rules as well: > >allow ip4 from me to "table($internal)" out xmit re1\* keep-state >allow ip4 from me to "table($broadcast)" out xmit re1\* > > table($internal) is basically RFC1918 and table($broadcast) contains > 224.0.0.0/4 and ff00::/8. > > To test, I have also added the following rule You using an out xmit re1, and that is probably what is NOT happening. What does: route get default show? > >allow ip4 from me to "table($broadcast)" out xmit > > but did not see any outgoing multicast traffic from MiniDLNA. The packet may NOT match the specific out xmit $dgw clause. > Also, when MiniDLNA is running before I add the interface route for > 244.0.0.0/4, the VLC test client does NOT discover the MiniDLNA server > (despite the route). If, however, the MiniDLNA service is (re)started > after the route is in place (but before the test client is running), it > does detect the server immediately. > > I therefore suspect that the "MiniDLNA startup code" does something > different when the route is there as compared to when it is not. That could be, it could also be shutting down its response cause it gets an error from the firewall code saying "permission denied". > But I don't know what... I assume it is not related to mrouted (which is > NOT running)? You should not need mrouted, it does not sound as if your running as a multicast router. > Also, again, if setting the interface route for 224.0.0.0/4 absolutely > IS required, I will not be able to make my intended setup work. I have > multiple interfaces connected to potential clients for MiniDLNA. It should not be required. I run large amounts of OSPF which is a multicast protocol without any mrouted, and given the descriptiong of your topology your only trying to do mutlicast to directly attached machines, not routed multicast. -- Rod Grimes rgri...@freebsd.org ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: Same host or different? How can you tell "over the wire"?
On Wed, Mar 21, 2018 at 4:47 PM, Ronald F. Guilmette wrote: > > "Kurt Buff" wrote: > In case it was not clear, none of the IPv4 addresses that are of interest, > or that are relevant to my question, are ones for which *I* posses any type > of SSH login credentials. > > But your question certainly raises an interesting possibility, and an > interesting question... one that I myself am not at all equiped or > qualified to answer (because I am almost totally ignorant about even > the bare mechanics of the SSH protocol): How could one tickle an open > SSH port and obtain from it not just its greeting banner (which may be, > and often is, rather generic and non-specific) but also so as to get > the host's host-specific public key? > > (Yes, I am indeed displaying an unforgivable level of laziness here. > I can and most probably should, and most probably eventually -will- > just go off now and read the relevant RFCs, but if anyone wants to save > me the trouble, just for this one question, that would be appreciated.) Well, I'm not expert myself, but when I use putty from my Windows machine to talk with an ssh server that it's not seen before, I get a popup talking about the host ssh key which is new to putty., and that happens any time, e.g., the IP address of the machine changes. This query: https://www.google.com/search?q=scan+host+collect+ssh+key&ie=utf-8&oe=utf-8 reveals this tool: http://rc.quest.com/man.php?id=ssh-keyscan%281%29 which might be useful to you, and I do indeed see the man page for it on my box. Kurt ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: Same host or different? How can you tell "over the wire"?
> On Mar 21, 2018, at 4:47 PM, Ronald F. Guilmette > wrote: > > But your question certainly raises an interesting possibility, and an > interesting question... one that I myself am not at all equiped or > qualified to answer (because I am almost totally ignorant about even > the bare mechanics of the SSH protocol): How could one tickle an open > SSH port and obtain from it not just its greeting banner (which may be, > and often is, rather generic and non-specific) but also so as to get > the host's host-specific public key? Does the ssh-keyscan tool do what you want? # ssh-keyscan github.com # github.com:22 SSH-2.0-libssh_0.7.0 github.com ssh-rsa B3NzaC1yc2EBIwAAAQEAq2A7hRGmdnm9tUDbO9IDSwBK6TbQa+PXYPCPy6rbTrTtw7PHkccKrpp0yVhp5HdEIcKr6pLlVDBfOLX9QUsyCOV0wzfjIJNlGEYsdlLJizHhbn2mUjvSAHQqZETYP81eFzLQNnPHt4EVVUh7VfDESU84KezmD5QlWpXLmvU31/yMf+Se8xhHTvKSCZIFImWwoG6mbUoWf9nzpIoaSjB+weqqUUmpaaasXVal72J+UX2B+2RPW3RcT0eOzQgqlJL3RKrTJvdsjE3JEAvGq3lGHSZXy28G3skua2SmVi/w4yCE6gbODqnTWlg7+wC604ydGXA8VJiS5ap43JXiUFFAaQ== Unless you've copied the host ssh keys manually, this will be unique to the system. ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: Same host or different? How can you tell "over the wire"?
> > In message <201803212204.w2lm4g8h023...@pdx.rh.cn85.dnsmgr.net>, > "Rodney W. Grimes" wrote: > > >One thing you could look at is the OS finger printing of nmap, > >that could look for possible things to diffentiate the hosts. > > Yea, that idea occurred to me. But this solution has the same problem > that I just mentioned in another one of my replies in this thread: > Even if nmap says that two IP addresses have the exact same OS > signature, that is far from enough to assert that they are both > under the control of the exact same Bad Actor. You are not going to prove the "control of the exact same Bad Actor" without a warrant to search and seize. You might prove they are 2 different boxes if the nmap finger print shows a difference, but if they show identical you have proved nothing. > You certainly wouldn't want to send someone to prison, or even to > after-school detention, based on such limited circumstantial evidence. > > >Depending on just what the host is there could be other tale > >tale signs picked up from "forensic" type of data captured > >with tcpdump while playing known packet sequences against > >each host at identical time. > > Such as? > > I'm all ears. At this point I have to state I am not going to do your research work for free. I have given you plenty of free leads to persue. > >What you ask I believe could be done, but it non trivial and > >would require a very good understanding of both forensics > >and the differing ways that TCP/IP is implemented. > > I like to think that I am a quick learner. Please proceed with the > lesson. The rates for lessons in Forensics start at reasonable enough amounts, you can contact me off list if you wish to persue that. ... rest deleted ... -- Rod Grimes rgri...@freebsd.org ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
crash with ipfw nat on mips32
hi, I'm updating my freebsd wifi bits in preparation for more work and ... well: ath1: using multicast key search random: harvesting attach, 8 bytes (4 bits) from ath1 .. ipfw ipfw2 initialized, divert loadable, nat loadable, default to accept, logging disabled .. ipfw_nat *** Setting kern.random.harvest.mask=511 kern.random.harvest.mask: 2047 -> 511 *** bringing up loopback .. Trap cause = 2 (TLB miss (load or instr. fetch) - kernel mode) [ thread pid 11 tid 100010 ] Stopped at 0 db> bt Tracing pid 11 tid 100010 td 0x80673b40 dyn_expire_states+0x13c (?,?,?,?) ra c1d08f44 sp c1247c40 sz 144 dyn_tick+0x238 (0,?,?,?) ra 80214dfc sp c1247cd0 sz 120 itimer_fire+0x1440 (?,?,?,?) ra 802150c0 sp c1247d48 sz 88 softclock+0x9c (?,?,?,?) ra 0 sp c1247da0 sz 0 db> Has anyone seen this? FreeBSD 12.0-CURRENT #0 r331092M: Sun Mar 18 23:33:51 PDT 2018 adrian@test-2:/usr/home/adrian/work/freebsd/head-embedded/obj/mips_ap/usr/home/adrian/work/freebsd/head-embedded/src/mips.mips/sys/DIR-825C1 mips Thanks! -a ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: crash with ipfw nat on mips32
On 22.03.2018 09:23, Adrian Chadd wrote: > Trap cause = 2 (TLB miss (load or instr. fetch) - kernel mode) > [ thread pid 11 tid 100010 ] > Stopped at 0 > db> bt > Tracing pid 11 tid 100010 td 0x80673b40 > dyn_expire_states+0x13c (?,?,?,?) ra c1d08f44 sp c1247c40 sz 144 > dyn_tick+0x238 (0,?,?,?) ra 80214dfc sp c1247cd0 sz 120 > itimer_fire+0x1440 (?,?,?,?) ra 802150c0 sp c1247d48 sz 88 > softclock+0x9c (?,?,?,?) ra 0 sp c1247da0 sz 0 > db> Hi, this is not NAT related, it is ipfw's dynamic states. I'm not sure, but this is seems related to ConcurrencyKit. -- WBR, Andrey V. Elsukov signature.asc Description: OpenPGP digital signature
