To me, this is a lot more in line with many other systems connections, to have
the ability to have a single connection string / uri, is this really that left
field suggesting or wanting this?
If anything this bring kafka more standardised approach imo, to have a unified
resource identifier, protocol name and a set schema for that.
e.g.
Database connection strings like
oracle:
jdbc:oracle:thin:@(description=(address_list=
(address=(protocol=tcp)(port=1521)(host=prodHost)))
(connect_data=(INSTANCE_NAME=ORCL)))
or
postgres:
"jdbc:postgresql://localhost/test?user=fred&password=secret&ssl=true"
And then like wise on messaging front, systems like
rabbitmq
“amqp://myhost?heartbeat=10&connection_timeout=10000”
I personally like the suggestion +1.
Cheers
Mike
On 05/10/2017, 20:10, "Clebert Suconic" <clebert.suco...@gmail.com> wrote:
On Thu, Oct 5, 2017 at 2:20 PM, Colin McCabe <cmcc...@apache.org> wrote:
> We used URIs as file paths in Hadoop. I think it was a mistake, for a
> few different reasons.
>
> URIs are actually very complex. You probably know about scheme, host,
> and port, but did you know about authority, user-info, query, fragment,
> scheme-specific-part? Do you know what they do in Hadoop? The mapping
> isn't obvious (and it wouldn't be obvious in Kafka either).
URIs are just a hashmap of key=string.. just like Properties...
The Consumer and Producer is just having such hashMap.. and these
values are easy to translate to boolean, integer.. etc. We would just
need to add such mapping as part of this task when done. I don't see
anything difficult there.
>
> When you flip back and forth between URIs and strings (and you
> inevitably will do this, when serializing or sending things over the
> wire), you run into tons of really hard problems. Should you preserve
> the "fragment" (the thing after the hash mark) for your URI, or not? It
> may not do anything now, but maybe it will do something later. URIs
> also have complex string escaping rules. Parsing URIs is very messy,
> especially when you start talking about non-Java programming languages.
Why flip back and forth? URIs would generate the same HashMap that's
being generated today.. I don't see any mess here.
Besides... This would be an addition, not replacement...
And I'm talking only about the Java API now.
Again, All the properties on ProducerConfig and ConsumerConfig seems
easy to be mapped as primitive types (String, numbers.. booleans).
Serialization shouldn't be a problem there. it would generate the same
properties it's generated now.
>
> URIs are designed for a world where you talk to a single host over a
> single port. That isn't the world distributed systems live in. You
> don't want your clients to fail to bootstrap because the single server
> you specified is having a bad day, even when the other 8 servers are up.
I have seen a few projects using this style of URI: I would make it
doing the same here:
If you have multiple hosts:
KafkaConsumer consumer = new
KafkaConsumer("kafka:(kafka://host1:port,kafka://host2:port)?property1=value");
if you have a single host:
KafkaConsumer consumer = new
KafkaConsumer("kafka://host2:port?property1=value&property2=value2");
One example of an apache project using a similar approach is qpid-jms:
http://qpid.apache.org/releases/qpid-jms-0.25.0/docs/index.html#failover-configuration-options
> The bottom line is that URIs are the wrong abstraction for the job.
> They just don't express what we really want, and they introduce a lot of
> complexity and ambiguity.
I have seen the opposite to be honest. this has been simpler for me
and users I know than using a HashMap.. . users in my experience tend
to write this faster.
users can certainly put up with the HashMap.. but this is easier to
remember. I'm just proposing what I think it's a simpler API.
Perhaps we should move into the KIP discussion itself here.. I first
intended to start this thread to see if it would make sense or not...
But I don't have authorization to create the KIP page.. so again..
based on the contributing page.. can someone add me authorizations to
the WIKI space?
The information contained in this email is strictly confidential and for the
use of the addressee only, unless otherwise indicated. If you are not the
intended recipient, please do not read, copy, use or disclose to others this
message or any attachment. Please also notify the sender by replying to this
email or by telephone (+44(020 7896 0011) and then delete the email and any
copies of it. Opinions, conclusion (etc) that do not relate to the official
business of this company shall be understood as neither given nor endorsed by
it. IG is a trading name of IG Markets Limited (a company registered in England
and Wales, company number 04008957) and IG Index Limited (a company registered
in England and Wales, company number 01190902). Registered address at Cannon
Bridge House, 25 Dowgate Hill, London EC4R 2YA. Both IG Markets Limited
(register number 195355) and IG Index Limited (register number 114059) are
authorised and regulated by the Financial Conduct Authority.
