[
https://issues.apache.org/jira/browse/KAFKA-3692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15279351#comment-15279351
]
Geoff Anderson commented on KAFKA-3692:
---------------------------------------
I came across this bug, but it took me some time to understand what was going
on.
In summary, the problem manifests itself this way: if CLASSPATH is defined
externally, but contains multiple paths with wildcards (i.e. "*"), the
kafka-run-class.sh script will clobber CLASSPATH within the scope of the
script. This is problematic if you're for example running your own connector.
The cause seems to be the interaction of nullglob, and our lack of use of
quotes when using bash variables.
The issue came up when trying to run connectors in tests, and I had something
like this:
{code}
export CLASSPATH=$(echo
/opt/schema-registry/package-kafka-serde-tools/target/kafka-serde-tools-package-*-development/share/java/kafka-serde-tools/)*:$(echo
/opt/kafka-connect-jdbc/target/kafka-connect-jdbc-*-development/share/java/kafka-connect-jdbc/)*:/opt/mysql-jdbc/*:$(echo
/opt/kafka-connect-hdfs/target/kafka-connect-hdfs-*-development/share/java/kafka-connect-hdfs/)*;
export
KAFKA_LOG4J_OPTS="-Dlog4j.configuration=file:/mnt/connect/connect-log4j.properties";
export KAFKA_OPTS=; /opt/kafka-trunk/bin/connect-standalone.sh
/mnt/connect/connect.properties /mnt/connect/connect-connector-0.properties
{code}
The offending code is in kafka-run-class.sh is here:
{code}
shopt -s nullglob
for dir in "$base_dir"/core/build/dependant-libs-${SCALA_VERSION}*;
do
if [ -z $CLASSPATH ] ; then
CLASSPATH="$dir"/*
else
CLASSPATH="$CLASSPATH":"$dir"/*
fi
done
{code}
So what does bash do when nullglob is turned on?
If nullglob is turned on, a sequence of characters that does not expand to a
valid path, it will evaluate that string as empty.
E.g. p1/*:p2/* -> "" (p1/*:p2/* may be a valid CLASSPATH, but bash doesn't
know this and tries to match the entire expression as a path)
If I have nullglob turned off, this same sequence of characters just evaluates
as-is
E.g. p1/*:p2/* -> p1/*:p2/*
In the code block testing emptiness of CLASSPATH, we do if [ -z $CLASSPATH ]
Unfortunately, because the variable is not quoted, if CLASSPATH looks something
like p1/*:p2/*, then nullglob kicks in, $CLASSPATH evaluates as empty, and we
clobber CLASSPATH.
> Spaces in External CLASSPATH may cause it not be included in the CLASSPATH
> --------------------------------------------------------------------------
>
> Key: KAFKA-3692
> URL: https://issues.apache.org/jira/browse/KAFKA-3692
> Project: Kafka
> Issue Type: Bug
> Components: core
> Affects Versions: 0.10.0.0
> Reporter: Liquan Pei
> Assignee: Liquan Pei
> Fix For: 0.10.0.1
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> Currently, we doesn't use double quote when using CLASSPATH in
> kafka-run-class.sh. This could potentially cause issues as spaces in external
> CLASSPATH may result in the CLASSPATH to be incorrectly interpreted. As we
> perform a check on whether CLASSPATH is provided to determine the initial
> value of CLASSPATH, not using double quotes may cause the external CLASSPATH
> not be included in the final CLASSPATH. We can workaround this by double
> quoting the external CLASSPATH, but it is a good practice to use double
> quotes if we are unsure.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
