[
https://issues.apache.org/jira/browse/AVRO-2888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vaibhav updated AVRO-2888:
--------------------------
Description:
Csharp specific reader fails when csharp language keyword is present in
namespace and datatype for on one of the element is array of complex type.
Steps to reproduce - Use following schema file to generate avro csharp classes
using Apache.Avro.Tools tool (v 1.10.0)
{code:java}
{
"name": "Parent",
"type": "record",
"namespace": "com.parent.event",
"fields": [
{
"name": "children",
"type": {
"type": "array",
"items": {
"name": "Child",
"type": "record",
"fields": [
{
"name": "name",
"type": "string"
}
]
}
}
}
]
}
{code}
Following classes will be generated. I have added link to files as content is
relatively large.
[https://github.com/vkhose/testcode/blob/master/com/parent/event/Parent.cs]
[https://github.com/vkhose/testcode/blob/master/com/parent/event/Child.cs]
Please note that Avro generator tool adds an escape character "@" in namespace
as event is a csharp keyword
e.g. namespace com.parent.@event
Next, I have created a test case which tries to deserialize the Parent class
object
[https://github.com/vkhose/testcode/blob/master/testCaseToReproduceError.cs]
{code:java}
[Test]
public void TestNamespaceWithCSharpKeyword()
{
var srcRecord = new Parent
{
children = new List<Child>
{
new Child
{
name = "test"
},
new Child
{
name = "test"
}
}
};
var stream = serialize(Parent._SCHEMA, srcRecord);
var dstRecord = deserialize<Parent>(stream,
Parent._SCHEMA, Parent._SCHEMA);
Assert.NotNull(dstRecord);
}
}{code}
This test case throws following exception
Unable to find type '[email protected]' in all loaded assemblies in field
children
----> Avro.AvroException : Unable to find type '[email protected]' in
all loaded assemblies
at Avro.Specific.SpecificDefaultReader.ReadRecord(Object reuse, RecordSchema
writerSchema, Schema readerSchema, Decoder dec)
When I debug the code it seems like same util methods are used to generate code
for specific record and read record during deserialization.
[https://github.com/apache/avro/blob/c0094b5bb1abb79304ce42a56cc115186370d407/lang/csharp/src/apache/main/CodeGen/CodeGenUtil.cs#L99]
{code:java}
if (ReservedKeywords.Contains(names[i]))
builder.Append(At);
{code}
Appending At(@) while generating code is correct but it should not appended
while deserializing. Deserialization fails as c# reflection does not expects
At(@) in string value provided to create class instance.
[https://github.com/apache/avro/blob/master/lang/csharp/src/apache/main/Specific/ObjectCreator.cs#L165]
Type.GetType(string) string parameter passed should be without "@" character in
namespace name.
It works if I modify code not to add "@" in CodeGenUtil.cs. I think we need a
fix where "@" is only added while auto generating classes and should not be
used during deserialization.
Thanks.
Vaibhav
was:
Csharp specific reader fails when csharp language keyword is present in
namespace and datatype for on one of the element is array of complex type.
Steps to reproduce - Use following schema file to generate avro csharp classes
using Apache.Avro.Tools tool (v 1.10.0)
{code:java}
{
"name": "Parent",
"type": "record",
"namespace": "com.parent.event",
"fields": [
{
"name": "children",
"type": {
"type": "array",
"items": {
"name": "Child",
"type": "record",
"fields": [
{
"name": "name",
"type": "string"
}
]
}
}
}
]
}
{code}
Following classes will be generated
[Parent.cs|[https://github.com/vkhose/testcode/blob/master/com/parent/event/Parent.cs]]
[Child.cs|[https://github.com/vkhose/testcode/blob/master/com/parent/event/Child.cs]]
Please note that Avro generator tool adds an escape character "@" in namespace
as event is a csharp keyword
e.g. namespace com.parent.@event
Next, I have created a test case which tries to deserialize the Parent class
object
[Test case
code|[https://github.com/vkhose/testcode/blob/master/testCaseToReproduceError.cs]]
{code:java}
[Test]
public void TestNamespaceWithCSharpKeyword()
{
var srcRecord = new Parent
{
children = new List<Child>
{
new Child
{
name = "test"
},
new Child
{
name = "test"
}
}
};
var stream = serialize(Parent._SCHEMA, srcRecord);
var dstRecord = deserialize<Parent>(stream,
Parent._SCHEMA, Parent._SCHEMA);
Assert.NotNull(dstRecord);
}
}{code}
This test case throws following exception
Unable to find type '[email protected]' in all loaded assemblies in field
children
----> Avro.AvroException : Unable to find type '[email protected]' in
all loaded assemblies
at Avro.Specific.SpecificDefaultReader.ReadRecord(Object reuse, RecordSchema
writerSchema, Schema readerSchema, Decoder dec)
When I debug the code it seems like same util methods are used to generate code
for specific record and read record during deserialization.
[https://github.com/apache/avro/blob/c0094b5bb1abb79304ce42a56cc115186370d407/lang/csharp/src/apache/main/CodeGen/CodeGenUtil.cs#L99]
{code:java}
if (ReservedKeywords.Contains(names[i]))
builder.Append(At);
{code}
Appending At(@) while generating code is correct but it should not appended
while deserializing. Deserialization fails as c# reflection does not expects
At(@) in string value provided to create class instance.
[https://github.com/apache/avro/blob/master/lang/csharp/src/apache/main/Specific/ObjectCreator.cs#L165]
Type.GetType(string) string parameter passed should be without "@" character in
namespace name.
It works if I modify code not to add "@" in CodeGenUtil.cs. I think we need a
fix where "@" is only added while auto generating classes and should not be
used during deserialization.
Thanks.
Vaibhav
> Csharp - Specific reader fails when using c# keyword in namespace name
> ----------------------------------------------------------------------
>
> Key: AVRO-2888
> URL: https://issues.apache.org/jira/browse/AVRO-2888
> Project: Apache Avro
> Issue Type: Bug
> Components: csharp
> Affects Versions: 1.9.2
> Reporter: Vaibhav
> Priority: Major
>
> Csharp specific reader fails when csharp language keyword is present in
> namespace and datatype for on one of the element is array of complex type.
> Steps to reproduce - Use following schema file to generate avro csharp
> classes using Apache.Avro.Tools tool (v 1.10.0)
> {code:java}
> {
> "name": "Parent",
> "type": "record",
> "namespace": "com.parent.event",
> "fields": [
> {
> "name": "children",
> "type": {
> "type": "array",
> "items": {
> "name": "Child",
> "type": "record",
> "fields": [
> {
> "name": "name",
> "type": "string"
> }
> ]
> }
> }
> }
> ]
> }
> {code}
> Following classes will be generated. I have added link to files as content is
> relatively large.
> [https://github.com/vkhose/testcode/blob/master/com/parent/event/Parent.cs]
> [https://github.com/vkhose/testcode/blob/master/com/parent/event/Child.cs]
> Please note that Avro generator tool adds an escape character "@" in
> namespace as event is a csharp keyword
> e.g. namespace com.parent.@event
> Next, I have created a test case which tries to deserialize the Parent class
> object
> [https://github.com/vkhose/testcode/blob/master/testCaseToReproduceError.cs]
>
> {code:java}
> [Test]
> public void TestNamespaceWithCSharpKeyword()
> {
> var srcRecord = new Parent
> {
> children = new List<Child>
> {
> new Child
> {
> name = "test"
> },
> new Child
> {
> name = "test"
> }
> }
> };
> var stream = serialize(Parent._SCHEMA, srcRecord);
> var dstRecord = deserialize<Parent>(stream,
> Parent._SCHEMA, Parent._SCHEMA);
> Assert.NotNull(dstRecord);
> }
> }{code}
> This test case throws following exception
> Unable to find type '[email protected]' in all loaded assemblies in
> field children
> ----> Avro.AvroException : Unable to find type '[email protected]' in
> all loaded assemblies
> at Avro.Specific.SpecificDefaultReader.ReadRecord(Object reuse, RecordSchema
> writerSchema, Schema readerSchema, Decoder dec)
>
> When I debug the code it seems like same util methods are used to generate
> code for specific record and read record during deserialization.
>
> [https://github.com/apache/avro/blob/c0094b5bb1abb79304ce42a56cc115186370d407/lang/csharp/src/apache/main/CodeGen/CodeGenUtil.cs#L99]
> {code:java}
> if (ReservedKeywords.Contains(names[i]))
> builder.Append(At);
> {code}
> Appending At(@) while generating code is correct but it should not appended
> while deserializing. Deserialization fails as c# reflection does not expects
> At(@) in string value provided to create class instance.
>
> [https://github.com/apache/avro/blob/master/lang/csharp/src/apache/main/Specific/ObjectCreator.cs#L165]
> Type.GetType(string) string parameter passed should be without "@" character
> in namespace name.
>
> It works if I modify code not to add "@" in CodeGenUtil.cs. I think we need a
> fix where "@" is only added while auto generating classes and should not be
> used during deserialization.
> Thanks.
> Vaibhav
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
