Re: [GENERAL] PDF files: to store in database or not

2016-12-08 Thread Rich Shepard
On Thu, 8 Dec 2016, Chris Travers wrote: Assuming relatively small files, bytea makes much more sense than a large object. Hi Chris, Most of the documents are only a few pages in size. LOBs work best when you need a streaming interface (seek and friends) while bytea's are otherwise much m

Re: [GENERAL] PDF files: to store in database or not

2016-12-08 Thread John DeSoi
> On Dec 8, 2016, at 9:25 AM, Chris Travers wrote: > > Assuming relatively small files, bytea makes much more sense than a large > object. However note that encoding and decoding can be relatively memory > intensive depending on your environment. This is not a problem with small > files and

Re: [GENERAL] PDF files: to store in database or not

2016-12-08 Thread Rich Shepard
On Thu, 8 Dec 2016, Adrian Klaver wrote: http://initd.org/psycopg/docs/usage.html?highlight=binary#adapt-binary Thanks again, Adrian. Rich -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-g

Re: [GENERAL] PDF files: to store in database or not

2016-12-08 Thread Chris Travers
On Thu, Dec 8, 2016 at 7:16 AM, Rich Shepard wrote: > On Thu, 8 Dec 2016, John DeSoi wrote: > > I have been storing PDFs in Postgres for several years without any >> problems. Documents range in size from a few pages to 100+ pages. I'm >> using a bytea column, not large objects. I store the docum

Re: [GENERAL] PDF files: to store in database or not

2016-12-08 Thread Adrian Klaver
On 12/08/2016 07:16 AM, Rich Shepard wrote: On Thu, 8 Dec 2016, John DeSoi wrote: I have been storing PDFs in Postgres for several years without any problems. Documents range in size from a few pages to 100+ pages. I'm using a bytea column, not large objects. I store the documents in a separate

Re: [GENERAL] PDF files: to store in database or not

2016-12-08 Thread Rich Shepard
On Thu, 8 Dec 2016, John DeSoi wrote: I have been storing PDFs in Postgres for several years without any problems. Documents range in size from a few pages to 100+ pages. I'm using a bytea column, not large objects. I store the documents in a separate database from the rest of the application da

Re: [GENERAL] PDF files: to store in database or not

2016-12-08 Thread John DeSoi
> On Dec 6, 2016, at 1:09 PM, Eric Schwarzenbach > wrote: > > I've often wondered if we'd have been better off storing the files in the > database. This design decision was made some years ago, and our concerns > around this had to do with performance, but I don't know that we had any real >

Re: [GENERAL] PDF files: to store in database or not

2016-12-06 Thread John R Pierce
On 12/6/2016 1:02 PM, David Wall wrote: You can certainly put the files in a filesystem and point to them, but you'll likely need some access control or people will be able to download any/all PDFs in a given folder. In the DB, you surely will have access control as I presume you don't allow br

Re: [GENERAL] PDF files: to store in database or not

2016-12-06 Thread David Wall
On 12/6/16 12:33 PM, Tom Lane wrote: John R Pierce writes: On 12/6/2016 12:10 PM, Rich Shepard wrote: I did not realize that a BLOB is not the same as a bytea (page 217 of the 9.6 PDF manual), and I cannot find BLOB as a postgres data type. Please point me in the right direction to learn how t

Re: [GENERAL] PDF files: to store in database or not

2016-12-06 Thread Rich Shepard
On Tue, 6 Dec 2016, John R Pierce wrote: indeed BYTEA is postgres's type for storing arbitrary binary objects that are called BLOB in certain other databases. John, I thought so. Thanks, Rich -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your s

Re: [GENERAL] PDF files: to store in database or not

2016-12-06 Thread Tom Lane
John R Pierce writes: > On 12/6/2016 12:10 PM, Rich Shepard wrote: >> I did not realize that a BLOB is not the same as a bytea (page 217 >> of the >> 9.6 PDF manual), and I cannot find BLOB as a postgres data type. Please >> point me in the right direction to learn how to store PDFs as BLOBs. >

Re: [GENERAL] PDF files: to store in database or not

2016-12-06 Thread John R Pierce
On 12/6/2016 12:10 PM, Rich Shepard wrote: I did not realize that a BLOB is not the same as a bytea (page 217 of the 9.6 PDF manual), and I cannot find BLOB as a postgres data type. Please point me in the right direction to learn how to store PDFs as BLOBs. indeed BYTEA is postgres's type fo

Re: [GENERAL] PDF files: to store in database or not

2016-12-06 Thread Eric Schwarzenbach
On 12/06/2016 02:40 PM, Joshua D. Drake wrote: On 12/06/2016 11:12 AM, Eric Schwarzenbach wrote: On 12/06/2016 01:34 PM, Joshua D. Drake wrote: On 12/06/2016 10:30 AM, Rich Shepard wrote: My thinking is to not store these documents in the database, but to store them in subdirectories outsid

Re: [GENERAL] PDF files: to store in database or not

2016-12-06 Thread Rich Shepard
On Tue, 6 Dec 2016, David Wall wrote: The advantages of storing in the database is that a DB backup will have everything, instead of a DB backup and a file system backup. Using a BLOB, you can certainly keep track of variable length PDFs. David, I did not realize that a BLOB is not the same

Re: [GENERAL] PDF files: to store in database or not

2016-12-06 Thread Joshua D. Drake
On 12/06/2016 11:21 AM, David Wall wrote: The advantages of storing in the database is that a DB backup will have everything, instead of a DB backup and a file system backup. Using a BLOB, you can certainly keep track of variable length PDFs. This is true but also not necessarily an advantage.

Re: [GENERAL] PDF files: to store in database or not

2016-12-06 Thread Joshua D. Drake
On 12/06/2016 10:50 AM, Rich Shepard wrote: On Tue, 6 Dec 2016, Joshua D. Drake wrote: Due to the widely variable size of a PDF document, I would say no. I would store the metadata and file location. Joshua, I read your answer as "don't store them in the database, but store the location in

Re: [GENERAL] PDF files: to store in database or not

2016-12-06 Thread Joshua D. Drake
On 12/06/2016 11:12 AM, Eric Schwarzenbach wrote: On 12/06/2016 01:34 PM, Joshua D. Drake wrote: On 12/06/2016 10:30 AM, Rich Shepard wrote: My thinking is to not store these documents in the database, but to store them in subdirectories outside the database. Your thoughts? Due to the w

Re: [GENERAL] PDF files: to store in database or not

2016-12-06 Thread Joshua D. Drake
On 12/06/2016 11:09 AM, Moreno Andreo wrote: ... but what if database is replicated? Use a network mounted filesystem (or replicated filesystem). JD -- Command Prompt, Inc. http://the.postgres.company/ +1-503-667-4564 PostgreSQL Centered full stack s

Re: [GENERAL] PDF files: to store in database or not

2016-12-06 Thread John R Pierce
On 12/6/2016 11:21 AM, David Wall wrote: The advantages of storing in the database is that a DB backup will have everything, instead of a DB backup and a file system backup. Using a BLOB, you can certainly keep track of variable length PDFs. and one of the disadvantages of storing in the da

Re: [GENERAL] PDF files: to store in database or not

2016-12-06 Thread David Wall
On 12/6/16 11:12 AM, Eric Schwarzenbach wrote: On 12/06/2016 01:34 PM, Joshua D. Drake wrote: On 12/06/2016 10:30 AM, Rich Shepard wrote: My thinking is to not store these documents in the database, but to store them in subdirectories outside the database. Your thoughts? Due to the wid

Re: [GENERAL] PDF files: to store in database or not

2016-12-06 Thread Eric Schwarzenbach
On 12/06/2016 01:34 PM, Joshua D. Drake wrote: On 12/06/2016 10:30 AM, Rich Shepard wrote: My thinking is to not store these documents in the database, but to store them in subdirectories outside the database. Your thoughts? Due to the widely variable size of a PDF document, I would say

Re: [GENERAL] PDF files: to store in database or not

2016-12-06 Thread Eric Schwarzenbach
On 12/06/2016 01:30 PM, Rich Shepard wrote: With no experience of storing binary data in a bytea column I don't know when its use is appropriate. I suspect that for an application I'm developing it would be better to store row-related documents outside the database, and want to learn if that i

Re: [GENERAL] PDF files: to store in database or not

2016-12-06 Thread Moreno Andreo
... but what if database is replicated? Thanks Moreno. Il 06/12/2016 19:50, Rich Shepard ha scritto: On Tue, 6 Dec 2016, Joshua D. Drake wrote: Due to the widely variable size of a PDF document, I would say no. I would store the metadata and file location. Joshua, I read your answer as

Re: [GENERAL] PDF files: to store in database or not

2016-12-06 Thread Rich Shepard
On Tue, 6 Dec 2016, Joshua D. Drake wrote: Due to the widely variable size of a PDF document, I would say no. I would store the metadata and file location. Joshua, I read your answer as "don't store them in the database, but store the location in a column." Thanks for confirming, Rich -

Re: [GENERAL] PDF files: to store in database or not

2016-12-06 Thread Joshua D. Drake
On 12/06/2016 10:30 AM, Rich Shepard wrote: My thinking is to not store these documents in the database, but to store them in subdirectories outside the database. Your thoughts? Due to the widely variable size of a PDF document, I would say no. I would store the metadata and file locatio