[
https://issues.apache.org/jira/browse/BEAM-13732?focusedWorklogId=722590&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-722590
]
ASF GitHub Bot logged work on BEAM-13732:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 08/Feb/22 05:14
Start Date: 08/Feb/22 05:14
Worklog Time Spent: 10m
Work Description: youngoli commented on a change in pull request #16598:
URL: https://github.com/apache/beam/pull/16598#discussion_r801273304
##########
File path: sdks/go/pkg/beam/io/xlang/schemaio/schemaio.go
##########
@@ -0,0 +1,81 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+// Package schemaio contains utilities for constructing cross-language IO
wrappers meant to
+// interface with the Java SDK's Schema IOs. Schema IO is an interface for any
IO that operates on
+// Beam schema supported elements. Various IOs are implemented via Schema IO,
and each
+// implementation requires its own IO wrapper in the Go SDK (for example, JDBC
IO or BigQuery IO),
+// and those IO wrappers can make use of these utilities.
+//
+// For implementation details of Schema IO see
https://s.apache.org/schemaio-development-guide.
+package schemaio
+
+import (
+ "bytes"
+ "reflect"
+
+ "github.com/apache/beam/sdks/v2/go/pkg/beam"
+ "github.com/apache/beam/sdks/v2/go/pkg/beam/core/graph/coder"
+ "github.com/apache/beam/sdks/v2/go/pkg/beam/internal/errors"
+)
+
+// Payload is a struct matching the expected cross-language payload of a
Schema IO.
+//
+// This documentation describes the expected usage of each field, but
individual IO implementations
+// are free to use this payload differently. For implementation details of
those IOs, refer to
+// SchemaIOProvider implementations in the Java SDK.
+type Payload struct {
+ // Location specifies the location to find the data (for example, a URL
to a database).
+ Location string `beam:"location"`
+
+ // Config is a Beam schema encoded struct containing configuration
details specific to the
+ // underlying IO implementation.
+ Config []byte `beam:"config"`
+
+ // DataSchema is an optional Beam schema encoded struct representing
the schema for data being
+ // read or written.
+ DataSchema *[]byte `beam:"dataSchema"`
+}
+
+// EncodeAsRow encodes a struct as a Beam schema Row, to embed within a cross
language payload.
+func EncodeAsRow(config interface{}) []byte {
+ rt := reflect.TypeOf(config)
+ enc, err := coder.RowEncoderForStruct(rt)
+ if err != nil {
+ err = errors.WithContextf(err, "getting Row encoder for type
%s", rt.Name())
+ panic(err)
+ }
+ var buf bytes.Buffer
+ if err := enc(config, &buf); err != nil {
+ err = errors.WithContextf(err, "encoding type %s as Row",
rt.Name())
+ panic(err)
+ }
+ return buf.Bytes()
+}
Review comment:
For now I made `EncodeAsRow` unexported and made the exported entry
point `EncodePayload`/`MustEncodePayload`.
I was strongly considering turning encodeAsRow into a function in
`core/graph/coder`, along with the other existing encoding and decoding
functions for the other coders, as well as adding a new decodeAsRow function
alongside it. Seems more user-friendly. If that sounds good I can do that as a
follow-up commit/PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@beam.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 722590)
Time Spent: 1h 40m (was: 1.5h)
> [Cross-Language] Implement Go SDK wrapper for xlang BigQuery IO
> ---------------------------------------------------------------
>
> Key: BEAM-13732
> URL: https://issues.apache.org/jira/browse/BEAM-13732
> Project: Beam
> Issue Type: New Feature
> Components: cross-language, sdk-go
> Reporter: Daniel Oliveira
> Assignee: Daniel Oliveira
> Priority: P2
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> Title says it all.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
