[
https://issues.apache.org/jira/browse/CASSANDRA-2799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13131525#comment-13131525
]
Steeve Morin edited comment on CASSANDRA-2799 at 10/20/11 11:41 AM:
--------------------------------------------------------------------
This is a version of the old Hadoop API basically just "wrapping" the new.
Please note however that the row key as a fixed size.
This is due to the old hadoop api wanting values by "writing" to them. This
however, can be changed in the job conf by setting:
cassandra.hadoop.max_key_size or
ColumnFamilyInputFormat2.CASSANDRA_HADOOP_MAX_KEY_SIZE.
Also, due to that, expect a small penalty hit. Albeit minimal.
Also, I'm not sure wether the stuff in ColumnFamilyInputFormat2.getSplits()
regarding the TaskAttemptID() is correct. Feedback welcomed!
Please also note that ColumnFamilyInputFormat2 only extends
ColumnFamilyInputFormat, which means you can use it with the new Hadoop API
also.
was (Author: steeve):
This is a version of the old Hadoop API basically just "wrapping" the new.
Please note however that the row key as a fixed size.
This is due to the old hadoop api wanting values by "writing" to them. This
however, can be changed in the job conf by setting:
cassandra.hadoop.max_key_size or
ColumnFamilyInputFormat2.CASSANDRA_HADOOP_MAX_KEY_SIZE.
Also, due to that, expect a small penalty hit. Albeit minimal.
Also, I'm not sure wether the stuff in ColumnFamilyInputFormat2.getSplits()
regarding the TaskAttemptID() is correct. Feedback welcomed!
> Implement old style api support for ColumnFamilyInputFormat and
> ColumnFamilyRecordReader
> ----------------------------------------------------------------------------------------
>
> Key: CASSANDRA-2799
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2799
> Project: Cassandra
> Issue Type: Improvement
> Affects Versions: 1.0.0
> Reporter: Jeremy Hanna
> Assignee: Jeremy Hanna
> Priority: Minor
> Labels: hadoop
> Attachments: ColumnFamilyInputFormat2.java,
> ColumnFamilyRecordReader2.java, ColumnFamilySplit2.java
>
>
> For better compatibility with hadoop, I would like to add old style hadoop
> support (mapred) to the ColumnFamilyInputFormat and ColumnFamilyRecordReader.
> We already have it in the output. Oozie in particular handles the old style
> api better. That is the motivation for us. I already did this as part of my
> patch for CASSANDRA-1497 so it should be trivial. We are just in a tight
> schedule right now and I'll come back to this once we have a bit of breathing
> room.
> I think it would help with compatibility with other systems that rely on
> hadoop as well.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
