Apache spark does have a template for PR's https://github.com/apache/spark/blob/master/.github/PULL_REQUEST_TEMPLATE
søn. 25. aug. 2024 kl. 13:41 skrev Mich Talebzadeh < mich.talebza...@gmail.com>: > Unfortunately it is not that straight forward > > > 1. Committer Votes: The PR needs a sufficient number of "+1" votes > from *committers.* > 2. Review Process: Address feedback from the community and committers > to ensure the PR meets the necessary standards. > 3. Approval: Once approved by committers, the PR can be merged into > the main codebase. > > > HTH > > > > On Sun, 25 Aug 2024 at 08:17, Mark Andreev <mark.andr...@gmail.com> wrote: > >> Thank you for your review. >> >> Could you explain how to merge this commit into the upstream? I don't >> want this PR to be abandoned. >> >> Best regards, >> Mark Andreev >> >> >> On Wed, 21 Aug 2024 at 23:08, Mich Talebzadeh <mich.talebza...@gmail.com> >> wrote: >> >>> Hi Mark, >>> >>> You have already done that and have made the request for review. >>> >>> +1 for me >>> >>> Mich Talebzadeh, >>> >>> Architect | Data Engineer | Data Science | Financial Crime >>> PhD <https://en.wikipedia.org/wiki/Doctor_of_Philosophy> Imperial >>> College London <https://en.wikipedia.org/wiki/Imperial_College_London> >>> London, United Kingdom >>> >>> >>> view my Linkedin profile >>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> >>> >>> >>> https://en.everybodywiki.com/Mich_Talebzadeh >>> >>> >>> >>> *Disclaimer:* The information provided is correct to the best of my >>> knowledge but of course cannot be guaranteed . It is essential to note >>> that, as with any advice, quote "one test result is worth one-thousand >>> expert opinions (Werner >>> <https://en.wikipedia.org/wiki/Wernher_von_Braun>Von Braun >>> <https://en.wikipedia.org/wiki/Wernher_von_Braun>)". >>> >>> >>> On Wed, 21 Aug 2024 at 22:20, Mark Andreev <mark.andr...@gmail.com> >>> wrote: >>> >>>> Thank you, Mich. >>>> >>>> What is the correct procedure to request a review? >>>> >>>> On Tue, 20 Aug 2024 at 22:57, Mich Talebzadeh < >>>> mich.talebza...@gmail.com> wrote: >>>> >>>>> Hi Mark, >>>>> >>>>> Added a comment to Jira to provide more clarity to Description >>>>> >>>>> When encountering mixed schema rows, the current error message >>>>> "{actual} is not a valid external type for schema of {expected}" lacks >>>>> sufficient detail to identify the problematic column. This ambiguity >>>>> hinders troubleshooting and increases development time. >>>>> >>>>> To enhance error clarity, we propose incorporating the source column >>>>> name into the error message. For example: "Column 'my_column' has an >>>>> actual >>>>> type of {actual} which is not a valid external type for the expected >>>>> schema >>>>> of {expected}." >>>>> >>>>> By providing this additional context, developers can more efficiently >>>>> pinpoint and resolve schema mismatches. >>>>> >>>>> >>>>> HTH >>>>> >>>>> Mich Talebzadeh, >>>>> >>>>> Architect | Data Engineer | Data Science | Financial Crime >>>>> PhD <https://en.wikipedia.org/wiki/Doctor_of_Philosophy> Imperial >>>>> College London <https://en.wikipedia.org/wiki/Imperial_College_London> >>>>> >>>>> London, United Kingdom >>>>> >>>>> >>>>> view my Linkedin profile >>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> >>>>> >>>>> >>>>> https://en.everybodywiki.com/Mich_Talebzadeh >>>>> >>>>> >>>>> >>>>> *Disclaimer:* The information provided is correct to the best of my >>>>> knowledge but of course cannot be guaranteed . It is essential to note >>>>> that, as with any advice, quote "one test result is worth one-thousand >>>>> expert opinions (Werner >>>>> <https://en.wikipedia.org/wiki/Wernher_von_Braun>Von Braun >>>>> <https://en.wikipedia.org/wiki/Wernher_von_Braun>)". >>>>> >>>>> >>>>> On Tue, 20 Aug 2024 at 21:59, Mark Andreev <mark.andr...@gmail.com> >>>>> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> Could you review my small PR [SPARK-49044][SQL] ValidateExternalType >>>>>> should return a child in error ( >>>>>> https://github.com/apache/spark/pull/47522 )? Changes contain tests >>>>>> that verify results. >>>>>> >>>>>> TLDR: After fix error message will contain extra information: [B is >>>>>> not a valid external type for schema of string at >>>>>> getexternalrowfield(assertnotnull(input[0, org.apache.spark.sql.Row, >>>>>> true]), 1, f3) >>>>>> If you need more information, please let me know. If you're busy, >>>>>> please let me know the best time to reach you again. >>>>>> >>>>>> On Mon, 29 Jul 2024 at 18:15, Mark Andreev <mark.andr...@gmail.com> >>>>>> wrote: >>>>>> >>>>>>> Hi Spark Devs, >>>>>>> >>>>>>> Please review my PR [ https://github.com/apache/spark/pull/47522 ] >>>>>>> that relates to ticket [ >>>>>>> https://issues.apache.org/jira/browse/SPARK-49044 ]. >>>>>>> >>>>>>> Context: When we have mixed schema rows, the error message "{actual} >>>>>>> is not a valid external type for schema of {expected}" doesn't help to >>>>>>> understand the column with the problem. I suggest adding information >>>>>>> about >>>>>>> the source column. >>>>>>> >>>>>>> Example: >>>>>>> https://github.com/mrk-andreev/example-spark-schema/blob/main/spark_4.0.0/src/test/scala/ErrorMsgSuite.scala >>>>>>> >>>>>>> Before fix: [B is not a valid external type for schema of string >>>>>>> After fix: [B is not a valid external type for schema of string at >>>>>>> getexternalrowfield(assertnotnull(input[0, org.apache.spark.sql.Row, >>>>>>> true]), 1, f3) >>>>>>> >>>>>>> -- >>>>>>> Best regards, >>>>>>> Mark Andreev >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Best regards, >>>>>> Mark Andreev >>>>>> >>>>> >>>> >>>> -- >>>> Best regards, >>>> Mark Andreev >>>> >>> -- Bjørn Jørgensen Vestre Aspehaug 4, 6010 Ålesund Norge +47 480 94 297
