Re: Initial Review: JSON contrib modul was: Re: [HACKERS] Another swing at JSON

2011-07-04 Thread Joseph Adams
Thanks for reviewing my patch! On Mon, Jul 4, 2011 at 7:10 AM, Bernd Helmle wrote: > +comment = 'data type for storing and manipulating JSON content' > > I'm not sure, if "manipulating" is a correct description. Maybe i missed it, > but i didn't see functions to manipulate JSON strings directly,

Re: [HACKERS] Another swing at JSON

2011-06-17 Thread Joseph Adams
On Fri, Jun 17, 2011 at 2:29 AM, Bernd Helmle wrote: > Joseph, are you able to remove the compatibility code for this CF? Done. Note that this module builds, tests, and installs successfully with USE_PGXS=1. However, building without USE_PGXS=1 produces the following: CREATE EXTENSION json

Re: [HACKERS] VARIANT / ANYTYPE datatype

2011-05-11 Thread Joseph Adams
On Wed, May 11, 2011 at 7:53 AM, Robert Haas wrote: > That's likely to be how it gets implemented, but you seem to have > missed the point of some of the discussion upthread: the big problem > with that is that someone might type "DROP TYPE foo", and when they > do, you need an efficient way to fi

Re: [HACKERS] VARIANT / ANYTYPE datatype

2011-05-10 Thread Joseph Adams
On Tue, May 10, 2011 at 10:29 PM, Joseph Adams wrote: > The VARIANT type, or similar, would be useful for the JSON data type > I've been intermittently working on, as it would allow us to create a > function like this: > >    from_json(JSON) returns VARIANT This occurred

Re: [HACKERS] VARIANT / ANYTYPE datatype

2011-05-10 Thread Joseph Adams
It seems to me a reasonable way to implement VARIANT would be to have a data type called VARIANT that stores an OID of the inner type at the beginning, followed by the binary data. When you say pg_typeof(somevariant), you'll get 'variant'. Instead, you'd use a function like this: variant_typ

Re: [HACKERS] cast from integer to money

2011-04-04 Thread Joseph Adams
Attached is an updated version of the patch to allow conversion of int4/int8 directly to money. I added overflow checks, dropped int2->cash, and updated the documentation. - Joey diff --git a/doc/src/sgml/datatype.sgml b/doc/src/sgml/datatype.sgml index ecc79e2..13b888d 100644 --- a/doc/src/sgml

Re: [HACKERS] cast from integer to money

2011-04-03 Thread Joseph Adams
On Sun, Apr 3, 2011 at 11:23 PM, Robert Haas wrote: > Thanks for the patch, but I think you forgot to worry about overflow: > > rhaas=# select 9223372036854775807::money; >  money > >  -$1.00 > (1 row) cash_in doesn't test for overflow, either (tested on 8.4.0, 9.0.3, and HEAD): joey=#

Re: [HACKERS] psql 9.1 alpha5: connection pointer is NULL

2011-04-02 Thread Joseph Adams
2011/4/2 Devrim GÜNDÜZ : > > I'm getting the following message after upgrading to Alpha5 on my Fedora > 14 box: > > $ psql -p 5433 > psql: connection pointer is NULL > > which comes from libpq. Server is running, and I can connect it to via > 9.0's psql. > > This is a regular RPM build. Am I doing

Re: [HACKERS] cast from integer to money

2011-04-01 Thread Joseph Adams
On Thu, Mar 31, 2011 at 6:39 PM, Stephen Frost wrote: > Going just integer->money, with the "1" -> "$1.00", seems completely > reasonable to me.  As for being too late in the cycle..  if someone's > willing to do the work, I can't imagine it breaking anything, so I > wouldn't be against putting it

Re: [HACKERS] Another swing at JSON

2011-03-29 Thread Joseph Adams
On Tue, Mar 29, 2011 at 4:02 PM, Dimitri Fontaine wrote: > Here's the ugly trick from ip4r, that's used by more extension: > > PREFIX_PGVER = $(shell echo $(VERSION) | awk -F. '{ print $$1*100+$$2 }') Thanks. I applied a minor variation of this trick to the JSON module, so now it builds/installs

Re: [HACKERS] Another swing at JSON

2011-03-29 Thread Joseph Adams
On Tue, Mar 29, 2011 at 2:42 PM, Dimitri Fontaine wrote: > Joseph Adams writes: >> It would be nice if I could make a Makefile conditional that skips the >> relocatable test and loads init-pre9.1.sql if the new extension >> interface isn't available.  Is there a Make

Re: [HACKERS] Another swing at JSON

2011-03-29 Thread Joseph Adams
On Tue, Mar 29, 2011 at 2:42 PM, Dimitri Fontaine wrote: >> Also, should uninstall_json.sql be named something else, like >> json--uninstall--0.1.sql ? > > You don't need no uninstall script no more, try DROP EXTENSION json; and > DROP EXTENSION json CASCADE; It's there for pre-9.1, where DROP EX

Re: [HACKERS] Another swing at JSON

2011-03-28 Thread Joseph Adams
On Mon, Mar 28, 2011 at 1:48 PM, Robert Haas wrote: > On Mon, Mar 28, 2011 at 1:21 PM, Joseph Adams > wrote: >> Attached is a patch that adds a 'json' contrib module.  Although we >> may want a built-in JSON data type in the near future, making it a >> module

[HACKERS] patch: Add PGXS support to hstore's Makefile (trivial)

2011-01-23 Thread Joseph Adams
I discovered today that hstore's Makefile currently does not support PGXS; namely, the ability to install it like so: USE_PGXS=1 make sudo USE_PGXS=1 make install Moreover, hstore is the *only* contrib module whose Makefile does not contain "PGXS". The attached patch changes hstore's Mak

Re: [HACKERS] JSON data type status?

2011-01-21 Thread Joseph Adams
On Thu, Jan 20, 2011 at 8:13 PM, Itagaki Takahiro wrote: > On Fri, Jan 21, 2011 at 09:11, Bruce Momjian wrote: >> What happened to our work to add a JSON data type for PG 9.1? > > Nothing will happen in 9.1. > I assume we are in "competition" status: > > http://archives.postgresql.org/pgsql-hacke

Re: [HACKERS] patch: Add JSON datatype to PostgreSQL (GSoC, WIP)

2010-10-19 Thread Joseph Adams
On Tue, Oct 19, 2010 at 11:22 AM, Terry Laurenzo wrote: > Perhaps we should enumerate the attributes of what would make a good binary > encoding? Not sure if we're discussing the internal storage format or the binary send/recv format, but in my humble opinion, some attributes of a good internal f

Re: [HACKERS] Basic JSON support

2010-10-04 Thread Joseph Adams
On Mon, Oct 4, 2010 at 7:45 PM, Tom Lane wrote: > Yeah.  Joseph seems to be confusing copyrights with patents.  The idea > of "parse JSON with bison/flex" is not patentable by any stretch of the > imagination. What I meant is, anyone who sets out to write a JSON parser with bison/flex is probably

Re: [HACKERS] Basic JSON support

2010-10-04 Thread Joseph Adams
On Mon, Sep 20, 2010 at 12:38 AM, Joseph Adams wrote: > Here's one thing I'm worried about: the bison/flex code in your patch > looks rather similar to the code in > http://www.jsonlint.com/bin/jsonval.tgz , which is licensed under the > GPL.  In particular, the incorrect nu

Re: [HACKERS] patch: Add JSON datatype to PostgreSQL (GSoC, WIP)

2010-09-18 Thread Joseph Adams
On Sat, Sep 18, 2010 at 4:03 PM, Robert Haas wrote: > Hmm, yeah.  I'd be tempted to try to keep the user's original > whitespace as far as possible, but disregard it as far as equality > comparison goes.  However, I'm not quite sure what the right thing to > do about 0 vs 0.0 is.  Does the JSON sp

Re: [HACKERS] patch: Add JSON datatype to PostgreSQL (GSoC, WIP)

2010-09-17 Thread Joseph Adams
On Fri, Sep 17, 2010 at 8:32 AM, Itagaki Takahiro wrote: > On Fri, Aug 13, 2010 at 7:33 PM, Joseph Adams > wrote: >> Updated patch: the JSON code has all been moved into core, so this >> patch is now for a built-in data type. > > I have a question about the design of

Re: [HACKERS] WIP: extensible enums

2010-08-23 Thread Joseph Adams
On Mon, Aug 23, 2010 at 1:54 PM, Tom Lane wrote: > "Andrew Dunstan" writes: >> On Mon, August 23, 2010 11:49 am, Alvaro Herrera wrote: >>> What do you need AFTER for?  Seems to me that BEFORE should be enough. >>> (You already have the unadorned syntax for adding an item after the last >>> one, w

Re: [HACKERS] JSON Patch for PostgreSQL - BSON Support?

2010-08-16 Thread Joseph Adams
On Mon, Aug 16, 2010 at 7:24 PM, Tom Lane wrote: > > Well, if it's not just a binary encoding of JSON, I think we can forget > about it ... certainly it won't work in the form I was visualizing. > >                        regards, tom lane I just read the spec, and BSON has a lot of bells and whi

Re: [HACKERS] JSON Patch for PostgreSQL - BSON Support?

2010-08-16 Thread Joseph Adams
On Sun, Aug 15, 2010 at 11:47 PM, Andrew Dunstan wrote: > > If BSON is simply in effect an efficient encoding of JSON, then it's not > clear to me that we would want another type at all. Rather, we might want to > consider storing the data in this supposedly more efficient format, and > maybe also

Re: [HACKERS] patch: General purpose utility functions used by the JSON data type

2010-08-13 Thread Joseph Adams
On Fri, Aug 13, 2010 at 2:02 PM, David Fetter wrote: > On Fri, Aug 13, 2010 at 01:33:06PM -0400, Robert Haas wrote: >> Maybe so, but it's not clear the interface that Joseph implemented is >> the one everyone wants... > > Fair enough.  What's the interface now in a nutshell?  Lack of > nutshells m

Re: [HACKERS] patch: General purpose utility functions used by the JSON data type

2010-08-13 Thread Joseph Adams
On Fri, Aug 13, 2010 at 10:46 AM, Robert Haas wrote: > On Fri, Aug 13, 2010 at 5:45 AM, Joseph Adams > wrote: >> getEnumLabelOids >>  * Useful-ometer: ()---o >>  * Rationale: There is currently no streamlined way to return a custom >>

[HACKERS] patch: General purpose utility functions used by the JSON data type

2010-08-13 Thread Joseph Adams
I factored out the general-purpose utility functions in the JSON data type code into a patch against HEAD. I have made a few changes to them since I posted about them earlier ( http://archives.postgresql.org/pgsql-hackers/2010-08/msg00692.php ). A summary of the utility functions along with some

Re: [HACKERS] patch: utf8_to_unicode (trivial)

2010-08-13 Thread Joseph Adams
On Tue, Jul 27, 2010 at 1:31 PM, Robert Haas wrote: > On Sat, Jul 24, 2010 at 10:34 PM, Joseph Adams > wrote: >> In src/include/mb/pg_wchar.h , there is a function unicode_to_utf8 , >> but no corresponding utf8_to_unicode .  However, there is a static >> function calle

[HACKERS] patch: utf8_to_unicode (trivial)

2010-07-24 Thread Joseph Adams
In src/include/mb/pg_wchar.h , there is a function unicode_to_utf8 , but no corresponding utf8_to_unicode . However, there is a static function called utf2ucs that does what utf8_to_unicode would do. I'd like this function to be available because the JSON code needs to convert UTF-8 to and from U

Re: [HACKERS] patch: Add JSON datatype to PostgreSQL (GSoC, WIP)

2010-07-24 Thread Joseph Adams
Update: I'm in the middle of cleaning up the JSON code ( http://git.postgresql.org/gitweb?p=json-datatype.git;a=summary if you want to see the very latest ), so I haven't addressed all of the major problems with it yet. On Fri, Jul 23, 2010 at 2:34 PM, Robert Haas wrote: > - I was under the impre

Re: [HACKERS] extensible enum types

2010-06-18 Thread Joseph Adams
On Fri, Jun 18, 2010 at 1:59 PM, Andrew Dunstan wrote: > > > Robert Haas wrote: >> >> On Fri, Jun 18, 2010 at 12:59 PM, Andrew Dunstan >> wrote: >> >>> >>> You are just bumping up the storage cost. Part of the attraction of enums >>> is >>> their efficiency. >>> >> >> What's efficient about them?

[HACKERS] Should the JSON datatype be a specialization of text?

2010-06-16 Thread Joseph Adams
Currently, the JSON datatype (repository: http://git.postgresql.org/gitweb?p=json-datatype.git;a=summary ) is implemented somewhat like a specialization of TEXT, like XML is. I'm beginning to question if this is the right way to go. This doesn't concern whether the JSON datatype should retain the

Re: [HACKERS] hstore ==> and deprecate =>

2010-06-11 Thread Joseph Adams
On Fri, Jun 11, 2010 at 10:59 AM, Tom Lane wrote: > Peter Eisentraut writes: >> How about no operator at all?  It won't be as cool to read, but >> consider, the arguments are text and text, not involving any hstore type >> at all, so whatever operator you choose is in practice blocked from >> eve

Re: [HACKERS] Working with PostgreSQL enums in C code

2010-06-07 Thread Joseph Adams
On Fri, May 28, 2010 at 9:47 AM, Tom Lane wrote: > Robert Haas writes: >> On Fri, May 28, 2010 at 12:07 AM, Joseph Adams >> wrote: >>> I learned that to return an enum value from C, one needs to return the >>> OID of the right row of the pg_enum table.  I ev

Re: [HACKERS] functional call named notation clashes with SQL feature

2010-06-04 Thread Joseph Adams
On Fri, Jun 4, 2010 at 9:55 PM, Joseph Adams wrote: > If I had to choose between => and := for parameter naming, I'd go with > := because it seems more SQLish to me. On second thought, => might actually be a very intuitive syntax for defining dictionary types like hstore a

Re: [HACKERS] functional call named notation clashes with SQL feature

2010-06-04 Thread Joseph Adams
On Wed, May 26, 2010 at 9:28 PM, Tom Lane wrote: > Robert Haas writes: >> On Wed, May 26, 2010 at 8:21 PM, Tom Lane wrote: >>> If we go with the spec's syntax I think we'd have no realistic choice >>> except to forbid => altogether as an operator name.  (And no, I'm not >>> for that.) > >> I sup

[HACKERS] Working with PostgreSQL enums in C code

2010-05-27 Thread Joseph Adams
I encountered a situation while implementing JSON support where I needed to return an enum value from a C function. To clarify, here's the SQL: CREATE TYPE json_type_t AS ENUM ('null', 'string', 'number', 'bool', 'object', 'array'); CREATE OR REPLACE FUNCTION json_type(json) RETURNS json_type_t

Re: [HACKERS] JSON manipulation functions

2010-05-27 Thread Joseph Adams
I've started implementing the JSON datatype; the repo is at http://git.postgresql.org/gitweb?p=json-datatype.git . On Fri, May 14, 2010 at 1:15 PM, Robert Haas wrote: > On Thu, May 13, 2010 at 9:47 PM, Joseph Adams > wrote: >> Would it be a bad idea to give an enum and a funct

Re: [HACKERS] Fwd: Hiding data in postgresql

2010-05-25 Thread Joseph Adams
On Tue, May 25, 2010 at 3:39 PM, Hector Beyers wrote: > > Hi guys, > (I tried the question in another forum first) > Does someone have any ideas how I can hide data without the meta data > noticing? To explain further, I would like to save some collection of data > where the meta-data does not see

Re: [HACKERS] JSON manipulation functions

2010-05-25 Thread Joseph Adams
> Well, I think it's fine to use the wiki for brainstorming, but before > you change the design you probably need to talk about it here.  You > can't rely on everyone on -hackers to follow changes on a wiki page > somewhere.  It looks like the API has been overhauled pretty heavily > since the last

Re: [HACKERS] JSON manipulation functions

2010-05-25 Thread Joseph Adams
I started a wiki article for brainstorming the JSON API: http://wiki.postgresql.org/wiki/JSON_API_Brainstorm . I also made substantial changes to the draft of the API based on discussion here and on the #postgresql IRC channel. Is it alright to use the wiki for brainstorming, or should it stay on

Re: [HACKERS] JSON manipulation functions

2010-05-14 Thread Joseph Adams
On Fri, May 14, 2010 at 11:33 AM, Bruce Momjian wrote: > Joseph Adams wrote: >> == array/object conversion == >> >> The json_object function converts a tuple to a JSON object.  If there >> are duplicate column names, there will be duplicate keys in the >> resulti

[HACKERS] JSON manipulation functions

2010-05-13 Thread Joseph Adams
First off, thank you for allowing me to participate in Google Summer of Code 2010. I'm sorry I haven't been active for the past few weeks. Today, I added the wiki page for my project, but the project schedule is highly tentative: http://wiki.postgresql.org/wiki/JSON_datatype_GSoC_2010 . I'd like

[HACKERS] SELECT * in a CREATE VIEW statement doesn't update column set automatically

2010-05-06 Thread Joseph Adams
This isn't exactly a bug, but it could be considered unintuitive behavior. Consider this: CREATE VIEW foo AS SELECT * FROM a; CREATE VIEW foo_v AS SELECT * FROM foo; ALTER TABLE foo ADD COLUMN b INT; The ALTER TABLE statement affects VIEW foo, but the column addition does not propagate to VIEW f

Re: [HACKERS] Virtual Private Database

2010-04-10 Thread Joseph Adams
On Sat, Apr 10, 2010 at 10:00 AM, Jean-Gérard Pailloncy wrote: > Hello, > > 1) VPD: Virtual Private Database > I would appreciate to have a new feature in PostgreSQL. > This is an oracle-like feature that implement "Row Level Security". > This feature may be emulated by using VIEW/RULE but this is

Re: [HACKERS] GSOC PostgreSQL partitioning issue

2010-04-09 Thread Joseph Adams
On Fri, Apr 9, 2010 at 4:08 PM, Dimitri Fontaine wrote: > I guess a GSoC of reasonable size would be to define a spec for how to > implement partitioning in PostgreSQL with a sound and accepted proposal > on independent steps to contribute separately, in order to reach the > full implementation in

Re: [HACKERS] Gsoc XQuery

2010-04-09 Thread Joseph Adams
On Fri, Apr 9, 2010 at 5:02 PM, Alvaro Herrera wrote: > Necati Batur escribió: > >> *Delivarables on Timeline* ... >> *Implementation of a GUI for XQuery for the PostgreSql users (2 week) > > What, you will *also* implement a GUI?  Gimme a break.  There's no way > you can achieve most of these in

Re: [HACKERS] Proposal: Add JSON support

2010-04-06 Thread Joseph Adams
On Tue, Apr 6, 2010 at 12:03 PM, Tom Lane wrote: > Petr Jelinek writes: >> Dne 6.4.2010 7:57, Joseph Adams napsal(a): >>> To me, the most logical approach is to do the obvious thing: make >>> JSON's 'null' be SQL's NULL.  For instance, SELECTing on

Re: [HACKERS] Proposal: Add JSON support

2010-04-05 Thread Joseph Adams
Another JSON strictness issue: the JSON standard ( http://www.ietf.org/rfc/rfc4627.txt ) states that JSON text can only be an array or object. However, my implementation currently accepts any valid value. Thus, '3', '"hello"', 'true', 'false', and 'null' are all accepted by my implementation, bu

Re: [HACKERS] Proposal: Add JSON support

2010-04-03 Thread Joseph Adams
I've been wondering whether the JSON datatype should be strict or conservative. For one, there's strict JSON (following the exact specification). Then there's more conservative JSON variants. Some JSON parsers support comments, some support invalid number formats (e.g. '3.' or '+5'), etc.. The c

Re: [HACKERS] Proposal: Add JSON support

2010-03-31 Thread Joseph Adams
I ended up reinventing the wheel and writing another JSON library: http://constellationmedia.com/~funsite/static/json-0.0.1.tar.bz2 This is a first release, and it doesn't really have a name besides "json". It's very similar to cJSON, except it is (sans unknown bugs) more reliable, more correct,

Re: [HACKERS] Proposal: Add JSON support

2010-03-29 Thread Joseph Adams
On Mon, Mar 29, 2010 at 2:23 PM, David E. Wheeler wrote: > On Mar 29, 2010, at 9:02 AM, Tom Lane wrote: > >> If this is going to end up being one fairly small C file implementing >> a spec that is not a moving target, I'd vote against depending on an >> external library instead, no matter how spif

Re: [HACKERS] Proposal: Add JSON support

2010-03-28 Thread Joseph Adams
On Sun, Mar 28, 2010 at 5:19 PM, Robert Haas wrote: > On Sun, Mar 28, 2010 at 4:48 PM, Joseph Adams > wrote: >> Now my thoughts and opinions on the JSON parsing/unparsing itself: >> >> It should be built-in, rather than relying on an external library >> (like XML d

[HACKERS] Proposal: Add JSON support

2010-03-28 Thread Joseph Adams
I introduced myself in the thread "Proposal: access control jails (and introduction as aspiring GSoC student)", and we discussed jails and session-local variables. But, as Robert Haas suggested, implementing variable support in the backend would probably be way too ambitious a project for a newbie

Re: [HACKERS] Proposal: access control jails (and introduction as aspiring GSoC student)

2010-03-26 Thread Joseph Adams
On Fri, Mar 26, 2010 at 12:07 PM, Robert Haas wrote: > Hmm.  I'm not sure exactly what problem you're trying to solve here. > I don't think this is a particularly good design for supporting > variables inside the server, since, well, it doesn't actually support > variables inside the server.  If

Re: [HACKERS] Proposal: access control jails (and introduction as aspiring GSoC student)

2010-03-25 Thread Joseph Adams
I apologize for my silence, as I've been busy reading up more on the internals of PostgreSQL. >From what I can tell, a big problem with my jails idea (as well as the variables Robert described) is that there really isn't a way to store context in the backend specifically for the end client (e.g. a

[HACKERS] Proposal: access control jails (and introduction as aspiring GSoC student)

2010-03-21 Thread Joseph Adams
Hello, I'm Joey Adams, and I'm interested in applying for Google Summer of Code to work on PostgreSQL. I'm a former GSoC student (I worked on CCAN last year), and a strong C programmer, though I am still new to working with large, established communities. I apologize if this is the wrong place to