community bonding
Hi PostgreSQL community and mentors: Thanks for selecting my project as one of GSoC student projects! Pretty exciting and honor to join the development for PostgreSQL (the best database in the world :)). So for the first phase of this project (community bonding), I am planning to go ahead to set up my developing environment and familiar with the related code bases. And I have several questions regarding this. 1. What ide or command line tools do you guys used most for PostgreSQL development? 2. What version control do you use for postgres? Is github acceptable? 3. How do you look at code in PostgreSQL code base? do you have cross function reference where I can search function and definition online? Also let me know if you have requirements in this phase which are not covered by my plan. Thanks Charles.
GSOC 2018 ideas
Hi Aleksander, This is Yan from Columbia University. I saw PostgreSQL is selected in GSOC 2018 and pretty interested in the ideas of thrift data types support that proposed by you. So, I want to prepare for a proposal based on this idea. Can I have more detailed information of what documents or code that I need to understand? Also, if this idea is allocated to other student (or in other worlds, you prefer some student to work on it), do let me know, so that I can pick some other project in PostgreSQL. Any comments or suggestions are welcomed! Hope for your reply! Thanks Charles!
Re: GSOC 2018 ideas
Got it, Aleksander! Will study these documents carefully! 2018-02-26 4:21 GMT-08:00 Aleksander Alekseev : > Hello Charles, > > > I saw PostgreSQL is selected in GSOC 2018 and pretty interested in the > > ideas of thrift data types support that proposed by you. So, I want to > > prepare for a proposal based on this idea. > > Glad you are interested in this project! > > > Can I have more detailed information of what documents or code that I > > need to understand? > > I would recommend the following documents and code: > > * Source code of pg_protobuf > https://github.com/afiskon/pg_protobuf > * "Writing Postgres Extensions" tutorial series by Manuel Kniep > http://big-elephants.com/2015-10/writing-postgres-extensions-part-i/ > * "So you want to make an extension?" talk by Keith Fiske > http://slides.keithf4.com/extension_dev/#/ > * Apache Thrift official website > https://thrift.apache.org/ > * Also a great explanation of the Thrift format can be found in the > book "Designing Data-Intensive Applications" by Martin Kleppmann > http://dataintensive.net/ > > > Also, if this idea is allocated to other student (or in other worlds, > > you prefer some student to work on it), do let me know, so that I can > > pick some other project in PostgreSQL. Any comments or suggestions are > > welcomed! > > To my best knowledge currently there are no other students interested in > this particular work. > > -- > Best regards, > Aleksander Alekseev >
Re: GSOC 2018 ideas
Hi Aleksander, Went through the documents listed by you, and they are helpful! It seems the main purpose of extension pg_protobuf is to parse a protobuf struct and return the decoded field. May I ask how these kinds of extensions are used in postgreSQL (or in other words, the scenarios to use these plugins)? Thanks Charles! 2018-03-02 21:11 GMT-08:00 Charles Cui : > Got it, Aleksander! Will study these documents carefully! > > 2018-02-26 4:21 GMT-08:00 Aleksander Alekseev : > >> Hello Charles, >> >> > I saw PostgreSQL is selected in GSOC 2018 and pretty interested in the >> > ideas of thrift data types support that proposed by you. So, I want to >> > prepare for a proposal based on this idea. >> >> Glad you are interested in this project! >> >> > Can I have more detailed information of what documents or code that I >> > need to understand? >> >> I would recommend the following documents and code: >> >> * Source code of pg_protobuf >> https://github.com/afiskon/pg_protobuf >> * "Writing Postgres Extensions" tutorial series by Manuel Kniep >> http://big-elephants.com/2015-10/writing-postgres-extensions-part-i/ >> * "So you want to make an extension?" talk by Keith Fiske >> http://slides.keithf4.com/extension_dev/#/ >> * Apache Thrift official website >> https://thrift.apache.org/ >> * Also a great explanation of the Thrift format can be found in the >> book "Designing Data-Intensive Applications" by Martin Kleppmann >> http://dataintensive.net/ >> >> > Also, if this idea is allocated to other student (or in other worlds, >> > you prefer some student to work on it), do let me know, so that I can >> > pick some other project in PostgreSQL. Any comments or suggestions are >> > welcomed! >> >> To my best knowledge currently there are no other students interested in >> this particular work. >> >> -- >> Best regards, >> Aleksander Alekseev >> > >
Re: GSOC 2018 ideas
2018-03-05 1:42 GMT-08:00 Aleksander Alekseev : > Hello Charles, > > >Went through the documents listed by you, and they are helpful! > > It seems the main purpose of extension pg_protobuf is to parse > > a protobuf struct and return the decoded field. May I ask how these kinds > > of extensions are used in postgreSQL (or in other words, the scenarios to > > use these plugins)? > > There are a few ideas behind all of this. > > 1) Sometimes people are not quite happy with strict relational schema by > various reasons and prefer something more agile, like XML or JSON. These > formats are indeed more convenient under certain circumstances, for > instance in terms of ease of changing and migrating the schema. > > 2) One drawback of JSON is redundancy. For instance, you have to store > the names of all document fields. These names don't carry much > information but consume disk space and RAM thus affecting the overall > performance. ZSON extension [1] partially solved this issue. However I > wouldn't call it particularly convenient and the whole approach of > compressing JSON seems to me more like a dirty hack, not a solution. The > problem appeared because of using the wrong data format in the first > place. > > 3) Unlike JSON, formats like Protobuf or Thrift are binary formats and > most importantly don't store any field names. Thus they don't create a > problem described above. However, PostgreSQL is not capable to access > Protobuf fields out-of-the-box, for instance to index these fields. This > is what pg_protobuf is for. > > The idea of using flexible schema and build index on top of them is awesome! Will definitely submit a proposal and focus on this if get selected. Thanks for answering my questions. > Hopefully this answers you question. If you have other questions please > don't hesitate to ask! > > [1]: https://github.com/postgrespro/zson > > > -- > Best regards, > Aleksander Alekseev >
GSOC 2018 proposal
Hi Aleksander, I am currently preparing a proposal for pg_thrift project. I noticed that there are several protocols supported by thrift, which ones do we have higher priority? I mean which ones I need to implement during this project? Thanks, Charles.
Re: GSOC 2018 proposal
2018-03-12 1:25 GMT-07:00 Aleksander Alekseev : > Hello Charles, > > > I am currently preparing a proposal for pg_thrift project. I noticed > > that there are several protocols supported by thrift, which ones do we > > have higher priority? I mean which ones I need to implement during > > this project? > > Binary protocols, i.e. TBinaryProtocol and TCompactProtocol. The first > one is a bit faster but more redundant, the second one is slower but > more compact. It's your choice which one to implement first, but at > least one binary protocol should be fully supported (ideally - both). > > As far as I'm aware other protocols are rarely used and are not fully > implemented in most existing libraries. > Got it, thanks! it is helpful for the proposal. > > -- > Best regards, > Aleksander Alekseev >
[GSoC] thrift compact protocol byte interface get implemented
Hi mentors and hackers, The thrift compact protocol has been implemented. Right now there are two protocols supported in my plugin(binary and compact). I have added several tests and did some bug fixing. At the same time updated the documents in my repo ( https://github.com/charles-cui/pg_thrift/blob/master/README.md) Let me know if you have any questions. In the remaining time of GSoC I am going to focus on providing a user friendly interface for this plugin by providing custom Postgresql types. Also adding more detailed tests (by following the tools provided by Aleksander in the last email). Thanks Charles
[GSoC] working status
Hi mentors and hackers, The second review is coming. Here is my working status so far. 1. Complete the thrift compact protocol implementation using bytea interface. 2. Thrift type (binary protocol) is almost done, the only remaining part is struct encoding and decoding. With the thrift type, you can express your thrift struct using json, but stored using thrift bytes. 3. Set up travis CI. 4. better documents. Here is the repo with all recent changes (https://github.com/charles-cui/pg_thrift) Let me know if you have any questions.
Re: [GSoC] working status
yes, my CI test version is 9.4. will make it work on 10 by following your advices. Thanks. On Mon, Jul 9, 2018, 5:22 AM Aleksander Alekseeev wrote: > Hello Charles, > > > > ``` > > > pg_thrift.c:1313:26: error: too few arguments to function > > > ‘array_create_iterator’ ArrayIterator iter = > > > array_create_iterator(parray, 0); ^ > > > In file included from pg_thrift.c:5: > > > .../postgresql-install/include/server/utils/array.h:418:22: > > > note: declared here extern ArrayIterator > > > array_create_iterator(ArrayType *arr, int slice_ndim, ArrayMetaState > > > *mstate); ^ > > > ``` > > > > I should probably explain this in a little more detail. > > > > This was an attempt to build pg_thrift with PostgreSQL 10 (compiled > > from REL_10_STABLE branch). Same issue with upcoming 11 version. I > > believe you are testing the extension on some 9.x branch and don't > > see these errors. Corresponding change in the API was done quite a > > long time ago, in 2015 (see commit 13dbc7a8). > > > > To solve this issue you can either use #ifdef's and maintain two > > versions of the code for different versions of PostgreSQL or just > > support only 10+. Both options are fine, at least by me personally. > > And I should probably explain the "use #ifdef's" part. You can use > PG_VERSION_NUM macro to determine PostgreSQL version in this fashion: > > ``` > /* In version 11 these macros have been changed */ > #if PG_VERSION_NUM < 11 > #define PG_GETARG_JSONB_P(v) PG_GETARG_JSONB(v) > #define PG_RETURN_JSONB_P(v) PG_RETURN_JSONB(v) > #endif > ``` > > In your case you can declare a wrapper for the array_create_iterator > procedure to make it work on both 9.x and 10. > > -- > Best regards, > Aleksander Alekseev >
project updates
Hi mentors and hackers, Here is my current working status. 1. Complete the thrift_binary_in and thrift_binary_out functions, so that users can express their thrift struct using json. These two functions support both simple data struct and complex data structure like struct and map. 2. added test cases and document to cover thrift_binary_in and thrift_binary_out. 3. make the code compile-able from 9.4 to 11.0. One question though, for custom types, it seems rich operations are also very important besides storing and expressing using thrift type. What are the most important operators should I support? Any suggestions? Here is the updated repo https://github.com/charles-cui/pg_thrift Thanks Charles
Re: project updates
2018-07-20 2:18 GMT-07:00 Aleksander Alekseeev :
> Hello Charles,
>
> > Here is my current working status.
> > 1. Complete the thrift_binary_in and thrift_binary_out functions, so
> > that users can express their thrift struct using json. These two
> > functions support both simple data struct and complex data structure
> > like struct and map. 2. added test cases and document to cover
> > thrift_binary_in and thrift_binary_out. 3. make the code compile-able
> > from 9.4 to 11.0.
>
> I see multiple warnings during compilation with 11.0, e.g:
>
> ```
> pg_thrift.c:1120:72: warning: implicit declaration of function
> ‘JsonbGetDatum’; did you mean ‘JsonbPGetDatum’?
> [-Wimplicit-function-declaration]
>
> pg_thrift.c:1206:13: warning: the address of ‘is_big_endian’ will
> always evaluate as ‘true’ [-Waddress]
>
> pg_thrift.c:1227:18: warning: implicit declaration of function
> ‘PG_GETARG_JSONB’; did you mean ‘PG_GETARG_JSONB_P’?
> [-Wimplicit-function-declaration]
>
> pg_thrift.c:1227:18: warning: initialization of ‘Jsonb *’ {aka ‘struct
> *’} from ‘int’ makes pointer from integer without a cast
> [-Wint-conversion]
> ```
>
> Also tests (make clean && make && make install && make installcheck)
> don't pass.
>
I have tested compiled passed for 9.4, 9.5, 9.6, 10 and 11 (
https://travis-ci.org/charles-cui/pg_thrift/builds/404741899)
and make install && make installcheck passed for 9.4
The tests are not run for other versions because my mac is 9.4 postgresql.
Also for CI testing, the current travis scripts has some problems for
version other than 9.4.
I will try to upgrade my postgresql and see whether I can make it.
>
> > One question though, for custom types, it seems rich operations are
> > also very important besides storing and expressing using thrift type.
> > What are the most important operators should I support? Any
> > suggestions? Here is the updated repo
> > https://github.com/charles-cui/pg_thrift
>
> I believe get field / set field are most common ones. Naturally there
> are also enumerations, projections, you name it. You can check out list
> of operation supported by JSONB (or a list of operations on dict's in
> Python) to get a full picture.
>
Good idea, thanks
>
> --
> Best regards,
> Aleksander Alekseev
>
Re: project updates
yes, using home brew, will try that.
On Jul 21, 2018 12:33 PM, "David Fetter" wrote:
On Sat, Jul 21, 2018 at 12:00:48PM -0700, Charles Cui wrote:
> 2018-07-20 2:18 GMT-07:00 Aleksander Alekseeev :
>
> > Hello Charles,
> >
> > > Here is my current working status.
> > > 1. Complete the thrift_binary_in and thrift_binary_out functions, so
> > > that users can express their thrift struct using json. These two
> > > functions support both simple data struct and complex data structure
> > > like struct and map. 2. added test cases and document to cover
> > > thrift_binary_in and thrift_binary_out. 3. make the code compile-able
> > > from 9.4 to 11.0.
> >
> > I see multiple warnings during compilation with 11.0, e.g:
> >
> > ```
> > pg_thrift.c:1120:72: warning: implicit declaration of function
> > ‘JsonbGetDatum’; did you mean ‘JsonbPGetDatum’?
> > [-Wimplicit-function-declaration]
> >
> > pg_thrift.c:1206:13: warning: the address of ‘is_big_endian’ will
> > always evaluate as ‘true’ [-Waddress]
> >
> > pg_thrift.c:1227:18: warning: implicit declaration of function
> > ‘PG_GETARG_JSONB’; did you mean ‘PG_GETARG_JSONB_P’?
> > [-Wimplicit-function-declaration]
> >
> > pg_thrift.c:1227:18: warning: initialization of ‘Jsonb *’ {aka ‘struct
> > *’} from ‘int’ makes pointer from integer without a cast
> > [-Wint-conversion]
> > ```
> >
> > Also tests (make clean && make && make install && make installcheck)
> > don't pass.
> >
> I have tested compiled passed for 9.4, 9.5, 9.6, 10 and 11 (
> https://travis-ci.org/charles-cui/pg_thrift/builds/404741899)
> and make install && make installcheck passed for 9.4
> The tests are not run for other versions because my mac is 9.4 postgresql.
You can have several versions of PostgreSQL on your mac at once. Are
you using homebrew?
Best,
David.
--
David Fetter http://fetter.org/
Phone: +1 415 235 3778
Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate
Re: project updates
Having tried David's method to install 10.4 and 11 on my mac and turns out
worked for me. The compiling issue posted by Aleksander is because
some json helpers changed function name and is not backward compatible with
9.4 and 10. Using #if macro resolves the problem,
Here is the commit to solve this.
https://github.com/charles-cui/pg_thrift/commit/dd5b8ad17ab47e3c977943dd2d69e5abad34c6ad
Aleksander, do you want to test again?
2018-07-21 13:08 GMT-07:00 Charles Cui :
> yes, using home brew, will try that.
>
> On Jul 21, 2018 12:33 PM, "David Fetter" wrote:
>
> On Sat, Jul 21, 2018 at 12:00:48PM -0700, Charles Cui wrote:
> > 2018-07-20 2:18 GMT-07:00 Aleksander Alekseeev <
> a.aleks...@postgrespro.ru>:
> >
> > > Hello Charles,
> > >
> > > > Here is my current working status.
> > > > 1. Complete the thrift_binary_in and thrift_binary_out functions, so
> > > > that users can express their thrift struct using json. These two
> > > > functions support both simple data struct and complex data structure
> > > > like struct and map. 2. added test cases and document to cover
> > > > thrift_binary_in and thrift_binary_out. 3. make the code compile-able
> > > > from 9.4 to 11.0.
> > >
> > > I see multiple warnings during compilation with 11.0, e.g:
> > >
> > > ```
> > > pg_thrift.c:1120:72: warning: implicit declaration of function
> > > ‘JsonbGetDatum’; did you mean ‘JsonbPGetDatum’?
> > > [-Wimplicit-function-declaration]
> > >
> > > pg_thrift.c:1206:13: warning: the address of ‘is_big_endian’ will
> > > always evaluate as ‘true’ [-Waddress]
> > >
> > > pg_thrift.c:1227:18: warning: implicit declaration of function
> > > ‘PG_GETARG_JSONB’; did you mean ‘PG_GETARG_JSONB_P’?
> > > [-Wimplicit-function-declaration]
> > >
> > > pg_thrift.c:1227:18: warning: initialization of ‘Jsonb *’ {aka ‘struct
> > > *’} from ‘int’ makes pointer from integer without a cast
> > > [-Wint-conversion]
> > > ```
> > >
> > > Also tests (make clean && make && make install && make installcheck)
> > > don't pass.
> > >
> > I have tested compiled passed for 9.4, 9.5, 9.6, 10 and 11 (
> > https://travis-ci.org/charles-cui/pg_thrift/builds/404741899)
> > and make install && make installcheck passed for 9.4
> > The tests are not run for other versions because my mac is 9.4
> postgresql.
>
> You can have several versions of PostgreSQL on your mac at once. Are
> you using homebrew?
>
> Best,
> David.
>
> --
> David Fetter http://fetter.org/
> Phone: +1 415 235 3778
>
> Remember to vote!
> Consider donating to Postgres: http://www.postgresql.org/about/donate
>
>
>
[GSoC]The project summary
Hi mentors and hackers, The final review is coming. Here is the project summary for the thrift plugin work for Postgres database. Please let me know if there are anything missing for the final review. 1. Implement the thrift binary protocol for both simple data structures (e.g., int, double) and complex data structures (e.g., list, map and struct) in pg_thrift plugin. The interface is byte based which means user need to pass in a byte and can use rich apis to parse out required fields. 2. Implement the thrift compact protocol for both simple data structures and complex data structures. The interface is also byte based and user can use rich apis to parse out fields. 3. A set of APIs for both binary protocol and compact protocol to parse out fields with kinds of types. 4. A customized thrift type (thrift_binary) where user specifies json, but stores in the format of byte. This type makes the plugin more user friendly, currently we support simple getter on top of this type. There are some improvements that can be done in the future to make the type support more operations. 5. Set up CI to continuously compile for each commit. Currently the plugin works in 9.4, 10, and 11. 6. A set of unit tests to cover most important use cases( https://github.com/charles-cui/pg_thrift/blob/master/sql/pg_thrift.sql). 7. A detailed document to showcase how to use this plugin ( https://github.com/charles-cui/pg_thrift/blob/master/README.md). >From this document, user knows how to install pg_thrift, how to parse out required fields from byte using provided api, how to build index based on the thrift bytes by the use of the api, and how to use the customized thrift type.
Re: [GSoC]The project summary
got it, will do that asap. Thanks for the comments! On Mon, Aug 6, 2018, 7:06 AM Aleksander Alekseev wrote: > Hello Charles, > > Personally I believe the project is in a pretty good shape, well done! > However I would recommend to replace all sprintf calls with it's safer > version, snprintf. Also please consider to run the code through Clang > Static Analyzer and Valgrind one more time just to make sure there are no > stupid mistakes that will blow up someone's PostgreSQL instance :) > > > On Sun, Aug 5, 2018 at 11:04 PM, Aleksander Alekseev > wrote: > >> Hello Charles, >> >> Thanks for keeping us informed. As you probably already discovered the >> email I used previously doesn't work any longer. Please add >> afis...@gmail.com to CC instead. >> >> I will take a look tomorrow (it's pretty late in my timezone currently). >> >> On Sun, Aug 5, 2018 at 9:05 PM, Charles Cui >> wrote: >> >>> Hi mentors and hackers, >>> >>>The final review is coming. Here is the project summary for the >>> thrift plugin work for Postgres database. Please let me know if there are >>> anything missing for the final review. >>> 1. Implement the thrift binary protocol for both simple data structures >>> (e.g., int, double) and complex data structures (e.g., list, map and >>> struct) in pg_thrift plugin. The interface is byte based which means user >>> need to pass in a byte and can use rich apis to parse out required fields. >>> 2. Implement the thrift compact protocol for both simple data structures >>> and complex data structures. The interface is also byte based and user can >>> use rich apis to parse out fields. >>> 3. A set of APIs for both binary protocol and compact protocol to parse >>> out fields with kinds of types. >>> 4. A customized thrift type (thrift_binary) where user specifies json, >>> but stores in the format of byte. This type makes the plugin more user >>> friendly, currently we support simple getter on top of this type. There are >>> some improvements that can be done in the future to make the type support >>> more operations. >>> 5. Set up CI to continuously compile for each commit. Currently the >>> plugin works in 9.4, 10, and 11. >>> 6. A set of unit tests to cover most important use cases( >>> https://github.com/charles-cui/pg_thrift/blob/master/sql/pg_thrift.sql >>> ). >>> 7. A detailed document to showcase how to use this plugin ( >>> https://github.com/charles-cui/pg_thrift/blob/master/README.md). >>> From this document, user knows how to install pg_thrift, how to parse >>> out required fields from byte using provided api, how to build index based >>> on the thrift bytes by the use of the api, and how to use the customized >>> thrift type. >>> >> >> >> >> -- >> Best regards, >> Aleksander Alekseev >> > > > > -- > Best regards, > Aleksander Alekseev >
Re: [GSoC]The project summary
commit to change sprintf to snprintf https://github.com/charles-cui/pg_thrift/commit/1de265a661ab3c61aa593d7e99d3a313024170fc 2018-08-06 8:03 GMT-07:00 Charles Cui : > got it, will do that asap. Thanks for the comments! > > On Mon, Aug 6, 2018, 7:06 AM Aleksander Alekseev > wrote: > >> Hello Charles, >> >> Personally I believe the project is in a pretty good shape, well done! >> However I would recommend to replace all sprintf calls with it's safer >> version, snprintf. Also please consider to run the code through Clang >> Static Analyzer and Valgrind one more time just to make sure there are no >> stupid mistakes that will blow up someone's PostgreSQL instance :) >> >> >> On Sun, Aug 5, 2018 at 11:04 PM, Aleksander Alekseev >> wrote: >> >>> Hello Charles, >>> >>> Thanks for keeping us informed. As you probably already discovered the >>> email I used previously doesn't work any longer. Please add >>> afis...@gmail.com to CC instead. >>> >>> I will take a look tomorrow (it's pretty late in my timezone currently). >>> >>> On Sun, Aug 5, 2018 at 9:05 PM, Charles Cui >>> wrote: >>> >>>> Hi mentors and hackers, >>>> >>>>The final review is coming. Here is the project summary for the >>>> thrift plugin work for Postgres database. Please let me know if there are >>>> anything missing for the final review. >>>> 1. Implement the thrift binary protocol for both simple data structures >>>> (e.g., int, double) and complex data structures (e.g., list, map and >>>> struct) in pg_thrift plugin. The interface is byte based which means user >>>> need to pass in a byte and can use rich apis to parse out required fields. >>>> 2. Implement the thrift compact protocol for both simple data >>>> structures and complex data structures. The interface is also byte based >>>> and user can use rich apis to parse out fields. >>>> 3. A set of APIs for both binary protocol and compact protocol to parse >>>> out fields with kinds of types. >>>> 4. A customized thrift type (thrift_binary) where user specifies json, >>>> but stores in the format of byte. This type makes the plugin more user >>>> friendly, currently we support simple getter on top of this type. There are >>>> some improvements that can be done in the future to make the type support >>>> more operations. >>>> 5. Set up CI to continuously compile for each commit. Currently the >>>> plugin works in 9.4, 10, and 11. >>>> 6. A set of unit tests to cover most important use cases( >>>> https://github.com/charles-cui/pg_thrift/blob/master/sql/pg_thrift.sql >>>> ). >>>> 7. A detailed document to showcase how to use this plugin ( >>>> https://github.com/charles-cui/pg_thrift/blob/master/README.md). >>>> From this document, user knows how to install pg_thrift, how to parse >>>> out required fields from byte using provided api, how to build index based >>>> on the thrift bytes by the use of the api, and how to use the customized >>>> thrift type. >>>> >>> >>> >>> >>> -- >>> Best regards, >>> Aleksander Alekseev >>> >> >> >> >> -- >> Best regards, >> Aleksander Alekseev >> >
Re: [GSoC]The project summary
clang static analysis looks good ~/Desktop/checker-279/bin/scan-build make && make install && make installcheck scan-build: Using '/Users/wyx/Desktop/checker-279/bin/clang' for static analysis /Users/wyx/Desktop/checker-279/bin/../libexec/ccc-analyzer -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -Wno-unused-command-line-argument -O2 -g -O2 -Wall -std=c99 -I. -I./ -I/usr/local/Cellar/postgresql@11/11beta2/include/server -I/usr/local/Cellar/postgresql@11/11beta2/include/internal -I/usr/local/Cellar/icu4c/62.1/include -I/usr/local/opt/gettext/include -I/usr/local/opt/icu4c/include -I/usr/local/opt/openldap/include -I/usr/local/opt/openssl/include -I/usr/local/opt/readline/include -I/usr/local/opt/tcl-tk/include -c -o pg_thrift.o pg_thrift.c /Users/wyx/Desktop/checker-279/bin/../libexec/ccc-analyzer -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -Wno-unused-command-line-argument -O2 -bundle -multiply_defined suppress -o pg_thrift.so pg_thrift.o -L/usr/local/Cellar/postgresql@11/11beta2/lib-L/usr/local/opt/gettext/lib -L/usr/local/opt/icu4c/lib -L/usr/local/opt/openldap/lib -L/usr/local/opt/openssl/lib -L/usr/local/opt/readline/lib -L/usr/local/opt/tcl-tk/lib -Wl,-dead_strip_dylibs-bundle_loader /usr/local/Cellar/postgresql@11/11beta2/bin/postgres scan-build: Removing directory '/var/folders/z5/6q03lw915_v6b26zwmplw1m8gn/T/scan-build-2018-08-06-223529-18582-1' because it contains no reports. scan-build: No bugs found. /bin/sh /usr/local/Cellar/postgresql@11/11beta2/lib/pgxs/src/makefiles/../../config/install-sh -c -d '/usr/local/Cellar/postgresql@11/11beta2/lib' /bin/sh /usr/local/Cellar/postgresql@11/11beta2/lib/pgxs/src/makefiles/../../config/install-sh -c -d '/usr/local/Cellar/postgresql@11/11beta2/share/extension' /bin/sh /usr/local/Cellar/postgresql@11/11beta2/lib/pgxs/src/makefiles/../../config/install-sh -c -d '/usr/local/Cellar/postgresql@11/11beta2/share/extension' /usr/bin/install -c -m 755 pg_thrift.so '/usr/local/Cellar/postgresql@11 /11beta2/lib/pg_thrift.so' /usr/bin/install -c -m 644 .//pg_thrift.control '/usr/local/Cellar/postgresql@11/11beta2/share/extension/' /usr/bin/install -c -m 644 .//pg_thrift--1.0.sql '/usr/local/Cellar/postgresql@11/11beta2/share/extension/' /usr/local/Cellar/postgresql@11/11beta2/lib/pgxs/src/makefiles/../../src/test/regress/pg_regress --inputdir=./ --bindir='/usr/local/Cellar/postgresql@11/11beta2/bin' --dbname=contrib_regression pg_thrift (using postmaster on Unix socket, default port) == dropping database "contrib_regression" == DROP DATABASE == creating database "contrib_regression" == CREATE DATABASE ALTER DATABASE == running regression test queries== test pg_thrift ... ok = All 1 tests passed. ========= 2018-08-06 17:56 GMT-07:00 Charles Cui : > commit to change sprintf to snprintf > https://github.com/charles-cui/pg_thrift/commit/ > 1de265a661ab3c61aa593d7e99d3a313024170fc > > 2018-08-06 8:03 GMT-07:00 Charles Cui : > >> got it, will do that asap. Thanks for the comments! >> >> On Mon, Aug 6, 2018, 7:06 AM Aleksander Alekseev >> wrote: >> >>> Hello Charles, >>> >>> Personally I believe the project is in a pretty good shape, well done! >>> However I would recommend to replace all sprintf calls with it's safer >>> version, snprintf. Also please consider to run the code through Clang >>> Static Analyzer and Valgrind one more time just to make sure there are no >>> stupid mistakes that will blow up someone's PostgreSQL instance :) >>> >>> >>> On Sun, Aug 5, 2018 at 11:04 PM, Aleksander Alekseev >>> wrote: >>> >>>> Hello Charles, >>>> >>>> Thanks for keeping us informed. As you probably already discovered the >>>> email I used previously doesn't work any longer. Please add >>>> afis...@gmail.com to CC instead. >>>> >>>> I will take a look tomorrow (it's pretty late in my timezone currently). >>>> >>>> On Sun, Aug 5, 2018 at 9:05 PM, Charles Cui >>>> wrote: >>>> >>>>> Hi mentors and hackers, >>>>> >>>>>The final review is coming. Here is the project summary for the >>>>> thrift plugin work for Postgres database. Please let me know if there are >>>>> any
Re: community bonding
Thanks Aleksander for your comments! These are helpful! 2018-04-24 2:07 GMT-07:00 Aleksander Alekseev : > Hello Charles, > > > Thanks for selecting my project as one of GSoC student projects! Pretty > > exciting and honor to join the development for PostgreSQL (the best > > database in the world :)). > > Welcome! > > > 1. What ide or command line tools do you guys used most for PostgreSQL > > development? > > Personally I prefer Vim. Sublime Text and Visual Studio Code are fine too. > > > 2. What version control do you use for postgres? Is github acceptable? > > Github is OK. > > > 3. How do you look at code in PostgreSQL code base? do you have cross > > function reference where I can search function and definition online? > > Vim + Ctags. Editors like Sublime Text and Visual Studio Code have > features like "goto definition" out-of-the-box. > > If you have any other questions please don't hesitate to ask! > > -- > Best regards, > Aleksander Alekseev >
Re: community bonding
Thanks Gasper for the information! 2018-04-23 22:41 GMT-07:00 Gasper Zejn : > > On 24. 04. 2018 06:22, Charles Cui wrote: > > Hi PostgreSQL community and mentors: > > > > Thanks for selecting my project as one of GSoC student projects! > > Pretty exciting and honor to join the development for PostgreSQL (the > > best database in the world :)). So for the first phase of this project > > (community bonding), I am planning to go ahead to set up my developing > > environment and familiar with the related code bases. And I have > > several questions regarding this. > Welcome! > > I'm not a commiter, but I've been looking at Postgres as a side project > for a while now. You'll find that there's a lot to learn. > > 1. What ide or command line tools do you guys used most for PostgreSQL > > development? > I've had most success with Eclipse. Metin Döşlü presented a talk at > PGConf.EU on how to do that, but the slides[1] are not the most verbose, > as he did a live demo. My key takeaway was that you can get the backend > pid with pg_backend_pid() and then attach GDB in Eclipse. There's also a > page on Eclipse on wiki[2]. > > > > 2. What version control do you use for postgres? Is github acceptable? > Postgres uses GIT[3]. Github does host a mirror, however Postgres as a > project uses its own git and uses patch based workflow, which is > described on wiki[4]. > > > 3. How do you look at code in PostgreSQL code base? do you have cross > > function reference where I can search function and definition online? > > There's a doxygen[5] available, but most of that can also be done in > Eclipse. > > > > Also let me know if you have requirements in this phase which are not > > covered by my plan. > > > > Thanks Charles. > > A thing to note is that Postgres mailing lists are not top-post, so > please try not to do that. > Wiki is a great resource. > > Kind regards, > Gasper Zejn > > > [1] > https://www.postgresql.eu/events/pgconfeu2017/sessions/ > session/1605-hacking-postgresql-with-eclipse/ > [2] https://wiki.postgresql.org/wiki/Working_with_Eclipse > [3] https://www.postgresql.org/docs/current/static/git.html > [4] https://wiki.postgresql.org/wiki/Submitting_a_Patch > [5] https://doxygen.postgresql.org/ >
Re: community bonding
Thanks for your comments, Craig! 2018-04-26 21:03 GMT-07:00 Craig Ringer : > On 24 April 2018 at 12:22, Charles Cui wrote: > > Hi PostgreSQL community and mentors: > > > > Thanks for selecting my project as one of GSoC student projects! Pretty > > exciting and honor to join the development for PostgreSQL (the best > database > > in the world :)). So for the first phase of this project (community > > bonding), I am planning to go ahead to set up my developing environment > and > > familiar with the related code bases. And I have several questions > regarding > > this. > > 1. What ide or command line tools do you guys used most for PostgreSQL > > development? > > Varies. > > vim, emacs, whatever editor you want. Some people use VS Code. Some > use Eclipse. You could code on Windows in Visual Studio if you want, > or XCode on Mac OS X. > > I use vim with a bunch of plugins. Whatever floats your boat, really, > so long as you set it up to help you follow the coding conventions and > source structure rules re tab/spaces etc. > > > 2. What version control do you use for postgres? Is github acceptable? > > git. But patches are emailed to the list. Github is fine, but we don't > use its pull request facilities, and the PostgreSQL source on github > is a mirror of git.postgresql.org. > > The stuff about context diff on the wiki is pretty outdated, I think > most people use unified diff now. > > > 3. How do you look at code in PostgreSQL code base? do you have cross > > function reference where I can search function and definition online? > > Use whatever facilities your editor and tools provide. I use vim and > cscope mostly, via "vim -t" and "ctrl-]". Visual Studio's Intellisense > works, whatever you want really. > > There's doxygen generated documentation too, see > https://doxygen.postgresql.org/ . I don't find it that useful. > > -- > Craig Ringer http://www.2ndQuadrant.com/ > PostgreSQL Development, 24x7 Support, Training & Services >
GSoC 2018: thrift encoding format
Hi Aleksander, Start to study the format of thrift encoding format (binary protocol) and found this document( https://erikvanoosten.github.io/thrift-missing-specification/#_struct_encoding). Having one question. Can I assume the data in thrift is always send inside a struct? Otherwise, it needs other information to distinguish a number or a string (for example). I think this question also valid for protobuf? Thanks, Charles.
Re: GSoC 2018: thrift encoding format
Thanks for your confirm Aleksander! Also I am thinking of how to deal with complex data structure like map, list, or set. I guess one possible solution is to get raw data bytes for these data structure? Otherwise it could be hard to wrap into a Datum. 2018-05-02 8:38 GMT-07:00 Aleksander Alekseev : > Hello Charles, > > > Can I assume the data in thrift is always send inside a struct? > > Sure! > > > I think this question also valid for protobuf? > > Right, pg_protobuf assumes that data is always a message which is an > equivalent of Thrift's struct. > > -- > Best regards, > Aleksander Alekseev >
Re: GSoC 2018: thrift encoding format
Thanks guys for your ideas! I feel like it is easier to follow pg_protobuf 's method to design and implement pg_thrift for a postgres beginner like me. I can refer pg_protobuf's way of using functions, writing tests, etc. I will reconsider what's the returned format for list, sets, and struct, etc. when I touch that part. Right now, I assume inputs are a series of thrift bytes, and try to implement decoding logics for simple types. 2018-05-04 8:42 GMT-07:00 Stephen Frost : > Greetings, > > * Aleksander Alekseev (a.aleks...@postgrespro.ru) wrote: > > > I understand that you're open to having it as a new data type or as a > > > bytea, but I don't agree. This should be a new data type, just as json > > > is a distinct data type and so is jsonb. > > > > Could you please explain in a little more detail why you believe so? > > As mentioned elsewhere, there's multiple ways to encode thrift, no? We > should pick which one makes sense and make that the interface to the > data type and then we might actually store the data differently, not to > mention that we'll likely want to build on things like indexing > capabilities to this data type, as we have for jsonb, and that's much > cleaner to do with a proper data type than if everyone has to use bytea > to store the data and then functional indexes (if we could even make > that happen... I'm not thrilled with such an idea in any case). > > Data validation is another thing- if it's a thrift data type then we can > validate that it's correct on the way in, and depend on that correctness > on the way out (to some extent- obviously we have to be wary of > corruption possibilities and such). > > We could toss out all of our data types and store everything as bytea's > if we wanted to, but we don't, and for quite a few good reasons, these > are just a couple that I'm thinking of off-hand. > > > Also I wonder whether in your opinion the extension should provide > > implicit casts from/to bytea as well. > > I wouldn't make them implicit... > > Thanks! > > Stephen >
[GSoC] Question about returning bytea array
Hi Aleksander,
I have the requirements to return a bytea array for some functions in
pg_thrift plugin.
What I did was similar to the following,
bytea** ret = palloc(len * sizeof(bytea*));
for (int i = 0; i < len; i++) {
// allocate space
ret[i] = palloc(size);
// fill in ret[i]
...
}
PG_RETURN_POINTER(ret);
I can verify each ret[i] is correct by logging, but server crash when
executed PG_RETURN_POINTER(ret).
So, I am wondering what's the reason of the crash?
and what's the recommended way (best practice) to construct a return value
to be bytea array?
Re: [GSoC] Question about returning bytea array
construct_md_array works for me, thanks for inputs! 2018-05-17 13:42 GMT-07:00 Andrew Gierth : > >>>>> "Charles" == Charles Cui writes: > > Charles> I have the requirements to return a bytea array for some > Charles> functions in pg_thrift plugin. > > If you mean you want the return value to be of type bytea[], i.e. an SQL > array of bytea values, then you need to be using construct_array to > construct the result. > > -- > Andrew (irc:RhodiumToad) >
[GSoC] github repo and initial work
Hi mentors and hackers, I have set up a github repo for the pg thrift plugin work, and here is the address (https://github.com/charles-cui/pg_thrift). And I have first version of binary protocol implemented, although it is still a very early stage. The current interface is to return value for simple data structure(byte, string, int32, int16, int64, bool) and return raw bytes for complex data structure(list, map, set, struct). As suggested by Aleksander, I can add json interfaces to use this plugin easily. Let me know if you guys have any comments on this. Thanks, Charles.
Re: [GSoC] github repo and initial work
Hi guys, As mentioned in the last email that the current interface is based on bytea. To make users easily insert any thrift data structure and show them, we may want to have user friendly (user readable) format for this plugin. I can think about 2 ways to do this. One is to provide to_thrift(json object) and to_json(thrift object) interfaces in the plugin. And the format in database table is still bytea. So, when you want to insert data into the table, users provide json/jsonb object, then call to_thrift() to covert to thrift bytes. When one want to read table content, using to_json() to covert thrift bytes to json. The second is to provide thrift type just like json or jsonb. When you create a table, postgres knows ::thrift keywords. I think method one should be easier to implement because it only limits to this plugin. Method two needs modify postgres kernel to register a new type, which may time consuming, but more natural. Any ideas on this? Thanks, Charles 2018-05-21 21:27 GMT-07:00 Charles Cui : > Hi mentors and hackers, > >I have set up a github repo for the pg thrift plugin work, and here is > the address (https://github.com/charles-cui/pg_thrift). And I have first > version of binary protocol implemented, although it is still a very early > stage. The current interface is to return value for simple data > structure(byte, string, int32, int16, int64, bool) and return raw bytes for > complex data structure(list, map, set, struct). As suggested by Aleksander, > I can add json interfaces to use this plugin easily. Let me know if you > guys have any comments on this. > > Thanks, Charles. >
Re: [GSoC] github repo and initial work
Thanks for correcting me, will definitely study citext and see how a new type is registered in plugin. 2018-05-24 23:23 GMT-07:00 Aleksandr Parfenov : > On Thu, 24 May 2018 18:25:28 -0700 > Charles Cui wrote: > > The second is to provide thrift type just like json or jsonb. When you > > create a table, postgres knows ::thrift keywords. > > I think method one should be easier to implement because it only > > limits to this plugin. Method two needs modify postgres kernel to > > register a new type, which may time consuming, > > but more natural. Any ideas on this? > > > > Thanks, Charles > > Hi Charles, > > I prefer the second way with separate type. But I think it is good idea > to wait for an answer from your project mentor or someone other. > > I'm not an expert in PostgreSQL user-defined types, but AFAIK, it > doesn't require changes in PostgreSQL core, since types can be created > in extensions. It doesn't require changes to grammar or something. > You can look at citext contrib as an example. > > -- > Aleksandr Parfenov > Postgres Professional: http://www.postgrespro.com > Russian Postgres Company >
Re: [GSoC] github repo and initial work
Got it, will go for second method. Will let you guys know the progress. 2018-05-25 4:05 GMT-07:00 Aleksander Alekseev : > Hello Charles, > > I suggest to begin with the second approach and add to_jsonb/from_jsonb > later. Both approaches are fine but it seems to me that most users would > expect a separate type thus it's more important. > > -- > Best regards, > Aleksander Alekseev >
[GSoC] json helper functions
Hi mentors and hackers, I am working on providing a new plugin where creates new data type (thrift). What I am currently planning is to translate cstring to byte for thrift_in functions. The cstring should be json format. When I looking at json helper functions I find the header ./include/server/utils/json.h where all functions have the signature of Datum function(PG_FUNCTION_ARGS).This signature causes extra overhead when writing the code because I need to convert my intermediate variables to the required format. Wondering are there any json helpers that are used by postgresql and plugins can directly call? Thanks Charles
json results parsing
Hi mentors and hackers, Found the json API page here https://www.postgresql.org/docs/9.3/static/functions-json.html For some APIs, it returns set of results, for example, json_each(json) and json_object_keys(). Basically, I want to call these json API inside a plugin, and the return value for these APIs are Datum. I am wondering how to parse a set of json results from Datum in postgresql? Seems the helper functions I am aware of are only for simple types. Are there any examples to demonstrate the best practice of how to do that? Thanks, Charles!
Re: json results parsing
Thanks you guys for answering my questions, and sorry for confusion. To clarify, I think my question is functions like json_each or json_object_keys() are set returning functions ( https://www.postgresql.org/docs/9.5/static/functions-srf.html), which means it returns a set of results into a Datum. I am wondering how to parse the Datum? I know there are functions like DatumGetCString() which parse the Datum into a cstring. Are there any helpers for set returning functions? 2018-05-30 23:00 GMT-07:00 Andrew Gierth : > >>>>> "Charles" == Charles Cui writes: > > Charles> Hi mentors and hackers, > Charles>Found the json API page here > Charles> https://www.postgresql.org/docs/9.3/static/functions-json.html > > Charles> For some APIs, it returns set of results, for example, > Charles> json_each(json) and json_object_keys(). > > Charles> Basically, I want to call these json API inside a plugin, > > If you want to process the whole of a json (not jsonb) value inside a > plugin, then maybe pg_parse_json is what you should be looking at rather > than the SQL-callable functions. > > For jsonb, there's an iterator interface (JsonbIteratorInit/ > JsonbIteratorNext) > > -- > Andrew (irc:RhodiumToad) >
[GSoC] create type questions
Hi mentors and hackers, I have a new type defined like this CREATE TYPE thrift_binary ( INPUT = thrift_binary_in, OUTPUT = thrift_binary_out, LIKE = bytea ); in thrift_binary_in, it accepts cstring and returns thrift_binary. And in this function I returned a bytea because the create type tells the system thrift_binary and bytea are the same. However, the test passes only when I explicitly tell thrift_binary_in to return a bytea (change the return type from thrift_binary to bytea), and it does not work when returns thrift_binary. Any ideas why? Thanks, Charles.
Re: [GSoC] create type questions
Hi Tom, Thanks for your comments, I do forget create type shell. But even if I add this line still does not work. Here is the commit that can demo my bug ( https://github.com/charles-cui/pg_thrift/commit/8b43f3e2172f4a1b4e61211f7d76b061a90c38f7 ) To see it, download the repo and do make install && make installcheck. It cannot return a bytea as expected. Thanks Charles! 2018-06-01 18:28 GMT-07:00 Tom Lane : > Charles Cui writes: > >I have a new type defined like this > > CREATE TYPE thrift_binary ( > > INPUT = thrift_binary_in, > > OUTPUT = thrift_binary_out, > > LIKE = bytea > > ); > > > in thrift_binary_in, it accepts cstring and returns thrift_binary. > > OK, that's what it should do. > > > And in > > this function I returned a bytea because the create type tells the system > > thrift_binary and bytea are the same. > > Sure, PG_RETURN_BYTEA_P() would work in the C code. I think it might be > worth inventing a macro PG_RETURN_THRIFT_BINARY_P that's a thin wrapper > around that, just for clarity's sake, but it doesn't matter functionally. > > > However, the test passes only when I > > explicitly tell thrift_binary_in to return a bytea (change the return > type > > from thrift_binary to bytea), and it does not work when returns > > thrift_binary. > > What do you mean by "the test passes" (or doesn't pass)? What do you > mean by "tell thrift_binary_in to return a bytea"? You just said you > were already doing that. > > This works for me: > > regression=# create type thrift_binary; > CREATE TYPE > regression=# create function thrift_binary_in(cstring) returns > thrift_binary > regression-# strict immutable language internal as 'byteain'; > NOTICE: return type thrift_binary is only a shell > CREATE FUNCTION > regression=# create function thrift_binary_out(thrift_binary) returns > cstring > regression-# strict immutable language internal as 'byteaout'; > NOTICE: argument type thrift_binary is only a shell > CREATE FUNCTION > regression=# CREATE TYPE thrift_binary ( > regression(# INPUT = thrift_binary_in, > regression(# OUTPUT = thrift_binary_out, > regression(# LIKE = bytea > regression(# ); > CREATE TYPE > > I cheated here by pointing to byteain/byteaout instead of C functions > that'd actually do what you want, but CREATE TYPE doesn't know that. > > I suspect you may have forgotten the initial dummy creation of > thrift_binary as a "shell type". > > regards, tom lane >
Re: [GSoC] create type questions
Thanks Andrew, that should be the reason! Forget thrift_binary_in is not a simple function, it is always coupled with output function. 2018-06-02 10:44 GMT-07:00 Andrew Gierth : > >>>>> "Charles" == Charles Cui writes: > > Charles> Hi Tom, > Charles>Thanks for your comments, I do forget create type shell. > Charles> But even if I add this line still does not work. > Charles> Here is the commit that can demo my bug ( > Charles> https://github.com/charles-cui/pg_thrift/commit/ > 8b43f3e2172f4a1b4e61211f7d76b061a90c38f7 > Charles> ) > Charles> To see it, download the repo and do make install && make > installcheck. > Charles> It cannot return a bytea as expected. > > Your output function isn't doing anything, and in particular it's not > returning any value (just dropping off the end), so as soon as pg tries > to output a query result containing a thrift_binary value, it either > outputs garbage or crashes. > > -- > Andrew (irc:RhodiumToad) >
[GSoC] current working status
Hi mentors and hackers, The first evaluation is coming. Here is my progress so far. During the first stage of work, I have implemented the thrift binary protocol as the format of postgresql plugin. Currently, the main interface is byte. Users who use this plugin need to provide thrift bytes to the plugin, and there are helpers for each data type to parse out the value contained in the bytes. This method has been verified by the use of several unit tests. However, the current interface needs users understand thrift format very well to use this plugin. After discussing with my mentors, it will be more useful to implement the concept of "thrift type", which is a custom type where user provides user understandable format(e.g., json), then the plugin converts into byte. I am currently busy with implementing this feature and still need sometime to complete it. If this part is done, I will go ahead to implement thrift compact protocol. Let me know if you have comments. You can always track the progress from github ( https://github.com/charles-cui/pg_thrift) Thanks, Charles!
Re: [GSoC] current working status
Hi Aleksander, I saw the list of errors you posted. That's because I have some new function implemented and pushed without testing(in order forget my change in local machine). So you are saying it is best to keep each commit workable. Right? Thanks Charles. 2018-06-14 1:41 GMT-07:00 Aleksander Alekseeev : > Hello Charles, > > >The first evaluation is coming. Here is my progress so far. During > > the first stage of work, I have implemented the thrift binary > > protocol as the format of postgresql plugin. Currently, the main > > interface is byte. Users who use this plugin need to provide thrift > > bytes to the plugin, and there are helpers for each data type to > > parse out the value contained in the bytes. This method has been > > verified by the use of several unit tests. However, the current > > interface needs users understand thrift format very well to use this > > plugin. After discussing with my mentors, it will be more useful to > > implement the concept of "thrift type", which is a custom type where > > user provides user understandable format(e.g., json), then the plugin > > converts into byte. I am currently busy with implementing this > > feature and still need sometime to complete it. If this part is done, > > I will go ahead to implement thrift compact protocol. Let me know if > > you have comments. You can always track the progress from github > > ( https://github.com/charles-cui/pg_thrift) > > Thanks for keeping us informed! > > Unfortunately I'm having difficulties compiling your code on Linux with > PostgreSQL 10.4 or 11 and GCC 8.1.1. Here is a full list of warnings > and errors: https://afiskon.ru/s/6e/edbefe818e_pg-thrift-errors.txt > > I also tried a VM with Ubuntu 16.04, PostgreSQL 9.6 and GCC 5.4.0. No > luck either. Please make sure that your code compiles on Linux. Users > will most likely try to run it on this OS. > > It might be also a good idea to add your repository to travic-ci.org > service. > > -- > Best regards, > Aleksander Alekseev >
Re: [GSoC] current working status
good idea, will keep a feature branch, and set up ci. On Thu, Jun 14, 2018, 8:03 AM Aleksander Alekseeev < a.aleks...@postgrespro.ru> wrote: > Hello Charles, > > >I saw the list of errors you posted. That's because I have some new > > function implemented and pushed without testing(in order forget my > > change in local machine). > > So you are saying it is best to keep each commit workable. Right? > > Ideally, yes. You can always create a separate branch to experiment with > new code and merge it to master branch when it becomes stable. This is > not a strict GSoC requirement or something like this, but a good > practice. If something goes wrong you can just delete an experimental > branch and go back to working master branch. > > > -- > Best regards, > Aleksander Alekseev >
Re: [GSoC] current working status
Hi mentors, The repo currently can be build on my mac. You can check it out. Also I am working on the CI config to monitor each commit. Thanks 2018-06-14 8:29 GMT-07:00 Charles Cui : > good idea, will keep a feature branch, and set up ci. > > On Thu, Jun 14, 2018, 8:03 AM Aleksander Alekseeev < > a.aleks...@postgrespro.ru> wrote: > >> Hello Charles, >> >> >I saw the list of errors you posted. That's because I have some new >> > function implemented and pushed without testing(in order forget my >> > change in local machine). >> > So you are saying it is best to keep each commit workable. Right? >> >> Ideally, yes. You can always create a separate branch to experiment with >> new code and merge it to master branch when it becomes stable. This is >> not a strict GSoC requirement or something like this, but a good >> practice. If something goes wrong you can just delete an experimental >> branch and go back to working master branch. >> >> >> -- >> Best regards, >> Aleksander Alekseev >> >
Re: [GSoC] current working status
got it, i will resolve all these warnings. One wired thing when i set up CI is the docker image seems having a postgres version which conflicts with my test version. So, CI is broken now ,although tests pass at my local box. I will also figure this out. On Fri, Jun 15, 2018, 1:34 AM Aleksander Alekseeev < a.aleks...@postgrespro.ru> wrote: > Hello Charles, > > > The repo currently can be build on my mac. You can check it out. > > Also I am working on the CI config to monitor each commit. > > Tests pass on Arch Linux with PostgreSQL 11 and Ubuntu 16.04 with > PostgreSQL 9.6. However there is a name conflict in case of PostgreSQL > 11: https://afiskon.ru/s/7a/fe681b17f0_paste.txt I suggest to rename > the procedure to `convert_int8_to_char`. > > Also GCC still generates a lot of warnings (see my previous message). > I would advise to fix all these warnings, otherwise you can miss a > really important warning. Ideally, the code should be compiled with > -Wall flag. > > -- > Best regards, > Aleksander Alekseev >
[GSoC] array processing questions
Hi mentors and hackers, One question about array processing in postgresql. Assume my input is a Datum (which contains an array). Are there any examples to loop through the array and operates on each element? feel like it is not that straight-ford. Thanks Charles
[GSoC] working status
Hi mentors and hackers, Here is my current working status. Resolved all warnings found by Aleksander previously. Having two threads in parallel. One is the thrift binary type implementation, the other is thrift compact byte interface implementation. For these two threads, simple data type has been implemented and tested, but still need time to focus on complicated data type like struct, map or list. Let me know if you have any questions. Here is the repo with latest updates, https://github.com/charles-cui/pg_thrift Thanks, Charles.
Re: [GSoC] array processing questions
thanks Andrew, will take a look. On Sat, Jun 23, 2018, 5:44 PM Andrew Gierth wrote: > >>>>> "Charles" == Charles Cui writes: > > Charles> Hi mentors and hackers, > Charles> One question about array processing in postgresql. Assume > Charles> my input is a Datum (which contains an array). Are there any > Charles> examples to loop through the array and operates on each > Charles> element? feel like it is not that straight-ford. > > See array_iter_setup and array_iter_next > > -- > Andrew (irc:RhodiumToad) >
Re: [GSoC] working status
got it, will start to use these tools and add more comments. On Tue, Jun 26, 2018, 7:44 AM Aleksander Alekseeev < a.aleks...@postgrespro.ru> wrote: > Hello Charles, > > >Here is my current working status. Resolved all warnings found by > > Aleksander previously. Having two threads in parallel. One is the > > thrift binary type implementation, the other is thrift compact byte > > interface implementation. For these two threads, simple data type has > > been implemented and tested, but still need time to focus on > > complicated data type like struct, map or list. Let me know if you > > have any questions. Here is the repo with latest updates, > > https://github.com/charles-cui/pg_thrift > > Thanks for keeping us informed! > > I noticed that there are not many comments in your code. It's > considered good practice to write at least brief comments for every > procedure with 1) short description of what it does 2) what every > argument means 3) what does the procedure return 4) whether it changes > global state somehow (better avoid it). However, there is no strict > requirement to write these comments, especially if there are many > similar procedures and (1-4) are obvious. > > Also I advise to check your code with following tools unless you've > already done this: > > * lcov will show whether the code is covered with tests well. If some > part of code is not covered there is probably a test missing or maybe > the code is dead and should be removed. > * Clang Static Analyzer may find some errors that are difficult to > notice with a naked eye. > * Valgrind solves the same issue, but unlike Clang Static Analyzer > it checks your code not statically but in runtime. > > There are plenty of corresponding tutorials online. I wrote a few > myself [1][2][3]. However, although they should be readable through > Google Translate, I guess you may prefer tutorials in English. > > [1]: https://eax.me/c-code-coverage/ > [2]: https://eax.me/c-static-analysis/ > [3]: https://eax.me/valgrind/ > > -- > Best regards, > Aleksander Alekseev >
Seeking exciting PostgreSQL development opportunities
Hi mentors and hackers, Thanks again for supporting me to pass the GSoC summer project and I found that I have a learnt a lot during this project! Although the project is completed, I am thinking of contributing to PostgreSQL community in the long term. So, the purpose of this email is to seek the exciting project you are currently working on or you plan to work on and see whether I am a fit for these projects. I should be able to contribute in my part time and I can work remotely in China. Please feel free to share your project and let's start coding :) Thanks Charles.
