[HACKERS] 回复： [pgsql-zh-general] R: Re: [HACKERS] Chinese in Postgres
maybe your C++ program has something (such as charset or configuation) causing this strange thing mark 发件人： "ciifrance...@tiscali.it" 收件人： ha...@2ndquadrant.com 抄送： pgsql-hackers@postgresql.org; pgsql-zh-gene...@postgresql.org; pgsql-ru-gene...@postgresql.org 发送日期： 2013年8月16日, 星期五, 8:40 下午 主题: [pgsql-zh-general] R: Re: [HACKERS] Chinese in Postgres Thanks for your answer. Yes, the client is also UTF8: MyDB=# show client_encoding; client_encoding - UTF8 (1 row) Cheers Francesco Messaggio originale Da: ha...@2ndquadrant.com Data: 16/08/2013 14.16 A: "ciifrance...@tiscali.it" Cc: , , Ogg: Re: [HACKERS] Chinese in Postgres On 08/16/2013 01:25 PM, ciifrance...@tiscali.it wrote: > Hello all, > before writing this message, I wrote about this in other mailing lists > without solving my problem. > Maybe some of you can help me. > > I have problems with a DB in postgres, when i try to insert Chinese > strings in UTF-8 format. > If I insert the data using a C++ program I have empty squares, in this > format: ��� (3 empty squares for each chinese ideogram as that is the > length in UTF-8) > If the string contains chinese mixed with ASCII, the ASCII is OK but > the Chinese is broken: > 漢語1-3漢語 --> ��1- 3�� Can you cehck that your client encoding is also UTF8 hannu=# show client_encoding ; client_encoding - UTF8 (1 row) Cheers -- Hannu Krosing PostgreSQL Consultant Performance, Scalability and High Availability 2ndQuadrant Nordic OÜ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql- hackers Invita i tuoi amici e Tiscali ti premia! Il consiglio di un amico vale più di uno spot in TV. Per ogni nuovo abbonato 30 € di premio per te e per lui! Un amico al mese e parli e navighi sempre gratis: http://freelosophy.tiscali.it/ -- Sent via pgsql-zh-general mailing list (pgsql-zh-gene...@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-zh-general
Re: [HACKERS] need info about extensibility in other databases
Hi, When is the English version of "GiST programming tutorial" available? Regards, Song -- Sent from: http://www.postgresql-archive.org/PostgreSQL-hackers-f1928748.html -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hadoop backend?
1) Hadoop file system is very optimized for mostly read operation2) As of a few months ago, hdfs doesn't support file appending. There might be a bit of impedance to make them go together. However, I think it should a very good initiative to come up with ideas to be able to run postgres on distributed file system (doesn't have to be specific hadoop). Pi Song On Sun, Feb 22, 2009 at 7:17 AM, Paul Sheer wrote: > Hadoop backend for PostGreSQL > > A problem that my client has, and one that I come across often, > is that a database seems to always be associated with a particular > physical machine, a physical machine that has to be upgraded, > replaced, or otherwise maintained. > > Even if the database is replicated, it just means there are two or > more machines. Replication is also a difficult thing to properly > manage. > > With a distributed data store, the data would become a logical > object - no adding or removal of machines would affect the data. > This is an ideal that would remove a tremendous maintenance > burden from many sites well, at least the one's I have worked > at as far as I can see. > > Does anyone know of plans to implement PostGreSQL over Hadoop? > > Yahoo seems to be doing this: > > http://glinden.blogspot.com/2008/05/yahoo-builds-two-petabyte-postgresql.html > > But they store tables column-ways for their performance situation. > If one is doing a lot of inserts I don't think this is most efficient - ? > > Has Yahoo put the source code for their work online? > > Many thanks for any pointers. > > -paul > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-hackers >
Re: [HACKERS] Hadoop backend?
One more problem is that data placement on HDFS is inherent, meaning you have no explicit control. Thus, you cannot place two sets of data which are likely to be joined together on the same node = uncontrollable latency during query processing. Pi Song On Mon, Feb 23, 2009 at 7:47 AM, Robert Haas wrote: > On Sat, Feb 21, 2009 at 9:37 PM, pi song wrote: > > 1) Hadoop file system is very optimized for mostly read operation > > 2) As of a few months ago, hdfs doesn't support file appending. > > There might be a bit of impedance to make them go together. > > However, I think it should a very good initiative to come up with ideas > to > > be able to run postgres on distributed file system (doesn't have to be > > specific hadoop). > > In theory, I think you could make postgres work on any type of > underlying storage you like by writing a second smgr implementation > that would exist alongside md.c. The fly in the ointment is that > you'd need a more sophisticated implementation of this line of code, > from smgropen: > >reln->smgr_which = 0; /* we only have md.c at present */ > > Logically, it seems like the choice of smgr should track with the > notion of a tablespace. IOW, you might to have one tablespace that is > stored on a magnetic disk (md.c) and another that is stored on your > hypothetical distributed filesystem (hypodfs.c). I'm not sure how > hard this would be to implement, but I don't think smgropen() is in a > position to do syscache lookups, so probably not that easy. > > ...Robert >
Re: [HACKERS] Hadoop backend?
On Mon, Feb 23, 2009 at 3:56 PM, pi song wrote: > I think the point that you can access more system cache is right but that > doesn't mean it will be more efficient than accessing from your local disk. > Take Hadoop for example, your request for file content will have to go to > Namenode (file chunk indexing service) and then you go ask the data node > which then provides you data. Assuming that you're working on a large > dataset, the probability of the data chunk you need staying in system cache > is very low therefore most of the time you end up reading from a remote > disk. > > I've got a better idea. How about we make the buffer pool multilevel? The > first level is the current one. The second level represents memory from > remote machines. Things that are used less often should stay on the second > level. Has anyone ever thought about something like this before? > > Pi Song > > On Mon, Feb 23, 2009 at 1:09 PM, Robert Haas wrote: > >> On Sun, Feb 22, 2009 at 5:18 PM, pi song wrote: >> > One more problem is that data placement on HDFS is inherent, meaning you >> > have no explicit control. Thus, you cannot place two sets of data which >> are >> > likely to be joined together on the same node = uncontrollable latency >> > during query processing. >> > Pi Song >> >> It would only be possible to have the actual PostgreSQL backends >> running on a single node anyway, because they use shared memory to >> hold lock tables and things. The advantage of a distributed file >> system would be that you could access more storage (and more system >> buffer cache) than would be possible on a single system (or perhaps >> the same amount but at less cost). Assuming some sort of >> per-tablespace control over the storage manager, you could put your >> most frequently accessed data locally and the less frequently accessed >> data into the DFS. >> >> But you'd still have to pull all the data back to the master node to >> do anything with it. Being able to actually distribute the >> computation would be a much harder problem. Currently, we don't even >> have the ability to bring multiple CPUs to bear on (for example) a >> large sequential scan (even though all the data is on a single node). >> >> ...Robert >> > >
Re: [HACKERS] Hadoop backend?
| I believe there is more than that which would need to be done nowadays. I seem to recall that the storage manager| | abstraction has slowly been dedicated/optimized for md over the past 6 years or so. It may even be easier/preferred | to write a hadoop specific access method depending on what you're looking for from hadoop. I think you're very right. What Postgres needs is access method abstraction. One should be able to plug in access method for SSD or network file systems if appropriate. I don't talk about MapReduce bit in Hadoop because I think that's a different story. What you need for MapReduce are 1) data store which feeds you data and then 2) MapReduce does the query processing. This has nothing to share with Postgres query processor in common. If you just want data from Postgres then it should be easier to build postgres data feeder in Hadoop (which might even already exist). Pi Song On Tue, Feb 24, 2009 at 11:24 AM, Tom Lane wrote: > "Jonah H. Harris" writes: > > I believe there is more than that which would need to be done nowadays. > I > > seem to recall that the storage manager abstraction has slowly been > > dedicated/optimized for md over the past 6 years or so. > > As far as I can tell, the PG storage manager API is at the wrong level > of abstraction for pretty much everything. These days, everything we do > is atop the Unix filesystem API, and anything that smgr might have been > able to do for us is getting handled in kernel filesystem code or device > drivers. (Back in the eighties, when it was more plausible for PG to do > direct device access, maybe smgr was good for something; but no more.) > > It's interesting to speculate about where we could draw an abstraction > boundary that would be more useful. I don't think the MySQL guys got it > right either... > >regards, tom lane >
