Hi, wrote some ideas I had about it.
On 5/3/21 6:49 PM, Simon Kelley wrote:
> This is just to mark that I have spent the afternoon thinking about
> this, and I'm in the process of a rather larger re-factor of the code.
>
> The aim of this is to cover the points Petr makes below, but
> additionally, to:
>
> 1) Allow a fast matching between subdomains in the configuration and the
> query domain, s=to handle the ad-blocks that use thousands of
> --local=/porn.here/ configuration lines.
It seems to me comparison in something like wire format would be more
efficient. It might for example store each label separately, making it a
tree. If the name ends with .com, it does not need check filters with
.org, .net or .info. With some hierarchy, it could reduce number of
entries compared. Binary tree implementation still might be better, but
harder to implement well.
test.com example.com one.bad.net blocked.net becomes:
com net
/ | | \
test example bad blocked
|
one
Not sure how much pointers on 64 bit platforms would compare with
ability to store just non-overlaping labels. Just a simple hierarchy
would store complex domain names with a cascade of pointers. Should help
on hosts with a lot of domain filters, common parts would get reused.
Interesting especially for top level domains. Might fragment memory too
much on the other hand.
>
> 2) To make the storage of --local=/domain/ more efficient.
>
> 3) To tidy up the mess that is search servers.
>
>
> The first part of the plan is to remove the overloading of struct server
> to store --address, --local and --rebind-domain-ok
My design attempt just moved some flags to domain, making --local and
--rebind-domain-ok propery of domain, not the server.
>
>
> There will be a data-structure that just holds domains, around which a
> fast search algorithm can be put. Each domain will hold a pointer to a
> struct server and a pointer to a list of addresses, to handle the
> situation of
>
> --server=/example.com/1.2.3.4
> --address=/example.com/1.1.1.1
Is this scenario even documented? It it desirable to behave described
way? According to man page, --address should handle also all subdomains.
Unless we allow --server=/*.example.com/1.2.3.4, this seems like
configuration error, or warning --address would override --server
statement before. Should we mark .example.com with some flag, so its
meaning is different from example.com
I would say those instructions are in conflict and should result in
configuration error. Are they needed in specific cases?
>
> Which answers A queries for example.com with 1.1.1.1 but forwards other
> queries to 1.2.3.4
>
> The algorithm for forwarding is
>
> 1) look up query-domain for domain-structure above with longest match.
If those domains are sorted smart way, it can know no better match would
be found before walking over to the last domain record. Was that considered?
> 2) Is there a suitable address? If so answer with that.
> 3) Otherwise, the struct server pointer points to the one of the set of
> servers which cover this domain, all linked in a ring with a new
> group_next pointer.
>
> The last_server feature is implemented by setting a flag for whichever
> server in this ring is last used. That server has the forwarded counts
> and times as well.
When I was thinking about it, I considered also rotation of servers in
selected domain. When middle S2 becomes last_used server, just append
any servers before it to the end. s1-s2-s3 becomes s2-s3-s1 on setting
s2 as last successful server. If pointer to last item is stored, it
could be done without iteration to the end of list. I think it is more
CPU effective than searching the server with flag each time query should
be sent, trying the first one or potentially walking over all of them.
Send number count and last timestamp is property of domain IMO, not the
server record itself. But it won't be used on domain without forwarders.
Does it make sense to store query counts for other than non-last server,
multiple times per domain?
>
> This makes the forwarding code which cycles through all eligible servers
> very easy.
>
> This code replaces some really knarly and old code, so it's not a small
> job to re-write it, especially as the list of servers is dynamic, with
> DBUS etc.
Sure, it is quite a lot of changes. High risk of some regressions too.
But could improve speed and memory requirements in some situations
significantly. Any improvement would be great.
>
>
>
> Simon.
>
>
>
>
>
> On 19/04/2021 19:19, Petr Menšík wrote:
>> Hi,
>>
>> I managed to prepare minimal change required. I think there is plenty of
>> space to optimize daemon->servers handling, but added slight addition to
>> it only. It introduces structure for every domain, keeping its state.
>>
>> According to dig ch txt servers.bind, it does what I tried. Reduces
>> number of queries sent to alternative server, but queries it from time
>> to time. Just like servers without domain.
>>
>> It sometimes tries all servers, but usually just one of them. Of course
>> nothing should change with only single server=/example/1.1.1.1
>>
>> I tested it with --servers-file=servers.conf, it can survive adding,
>> commenting out and adding servers again.
>>
>> What do you think?
>>
>> On 4/16/21 6:26 PM, Petr Menšík wrote:
>>> Hi all,
>>>
>>> I am trying to improve dnsmasq to handle better redirections of some
>>> given domains to multiple servers. Current implementation allows
>>> specifying multiple domains in single --server= statement, but only one
>>> server. It can be specified by multiple --server statements.
>>>
>>> But some people would like domain specific forwards to choose outgoing
>>> forwarders in similar way to common forwarders. By common forwarders I
>>> mean multiple --server=<ip> without any domain specification, the same
>>> way /etc/resolv.conf parsing with at least two nameservers works.
>>> Current implementation chooses the best responding server (last_server)
>>> and only sometime tries also other servers.
>>>
>>> I would like to implement similar logic not only for global forwarders,
>>> but also for any domain specific forwarders, where multiple forwarders
>>> are configured. Current behavior is to send queries to all
>>> domain-specific forwarders used for the domain. Even if three forwarders
>>> are configured and all are perfectly working, requests would be
>>> forwarded to all of them for every single query. It is bombarding all of
>>> them, but just the first reply would be forwarded back to client and be
>>> used. Is there a good reason for such behaviour? Are multiple servers
>>> for a domain considered exceptional?
>>>
>>> --rev-server for example allows only single target IP specified, even if
>>> syntax would allow more easily. In DNS world a zone usually requires at
>>> least two servers handling it.
>>>
>>> Current code walks multiple times daemon->servers list. First it finds
>>> longest domain match in search_servers() function.
>>> All servers are checked, when FORWARD_TEST queries were sent or
>>> FORWARD_TIME elapsed since last all servers check, whichever is sooner.
>>> Also on error responses. Then it walks the same list again, starting on
>>> daemon->last_server, sending forwarded queries. Unless
>>> forward->forwardall is set, only last_server is used. For domain
>>> specific forwarders, forwardall is always set.
>>>
>>> It seems to me each domain should have at least a structure:
>>> struct server_domain {
>>> char *domain;
>>> struct server *last_server;
>>> time_t forwardtime;
>>> int forwardcount;
>>> struct server_domain *next;
>>> };
>>>
>>> Where it could store forwardcount, forwardtime and last_server for each
>>> different domain. I think it would make sense to add struct server
>>> *servers and int flags and iterate only servers for given domain on each
>>> forwarded query. But required changes for that seem to be huge, I failed
>>> to prepare working patch yet. Minimal change would use domain found by
>>> search_servers and just find server_domain structure with matching domain.
>>>
>>> What do you think? Do you use multiple servers for custom domains?
>>>
>>> Cheers,
>>> Petr
--
Petr Menšík
Software Engineer
Red Hat, http://www.redhat.com/
email: pemen...@redhat.com
PGP: DFCF908DB7C87E8E529925BC4931CA5B6C9FC5CB
OpenPGP_signature
Description: OpenPGP digital signature
_______________________________________________ Dnsmasq-discuss mailing list Dnsmasq-discuss@lists.thekelleys.org.uk https://lists.thekelleys.org.uk/cgi-bin/mailman/listinfo/dnsmasq-discuss
