PSPP and MDD files

2018-04-11 Thread Dave Trollope
Hi, I've been using PSPP for a while, but am relatively new to SPSS as a whole and I am wondering if it can generate a .mdd file from the .sav file? I know IBM's mrStudio has a script CreateMddFromSav.mrs and I'm wondering if PSPP has an equivalent? Cheers Dave ___

Re: PSPP and MDD files

2018-04-11 Thread Dave Trollope
-Collection-DDF-MDD-files I can't find a better reference. Cheers Dave On 4/11/18 2:58 PM, Ben Pfaff wrote: On Wed, Apr 11, 2018 at 02:15:27PM -0500, Dave Trollope wrote: I've been using PSPP for a while, but am relatively new to SPSS as a whole and I am wondering if it can generate a

CSV Generation Loading Entire SAV

2018-10-16 Thread Dave Trollope
Hi, We are using the following script to generate CSV's for a subset of variables in an SAV file and it looks like pspp is loading the entire SAV file in to memory. (Some of our SAV files are quite large - 4GB). I'm wondering if there is an option or a way we can reduce the memory usage when

Re: About Multiple response variables...

2019-10-21 Thread Dave Trollope
Can you expand on the known issues with MR sets in PSPP? We’ve been parsing SAV files and generating the metadata from them with the MDD Writer patch and recently noticed some variables not were not being included in the MR set. If there is a list of known issues with MR set support in parsing,

Re: About Multiple response variables...

2019-10-21 Thread Dave Trollope
Thanks folks - I’ll have to check if we did the upgrade to 1.2, if not we’ll make sure to rerun the test on it. And I’ll join the dev list. Cheers Dave ___ Pspp-users mailing list Pspp-users@gnu.org https://lists.gnu.org/mailman/listinfo/pspp-users

Excessive file system usage

2019-12-04 Thread Dave Trollope
We just moved Pspp to Kubernetes containers where we use it to extract csvs from sav files. The sav files are about 1gb and each csv is about 150mb. We’ve watched the file system as it does it and over 7gb of the file system is used while writing 150mb. I assume the SAVE command is doing lots of

Re: Excessive file system usage

2019-12-04 Thread Dave Trollope
tem-commands-explained-d778178f96f1 > https://softwareengineeringdaily.com/2019/01/11/why-is-storage-on-kubernetes-is-so-hard/ > > -Alan > > > On 12/4/2019 6:40 AM, Dave Trollope wrote: > > We just moved Pspp to Kubernetes containers where we use it to extract csvs > >

Re: Excessive file system usage

2019-12-04 Thread Dave Trollope
f you use the "pspp-convert" program? > It does not have the exact feature you want (in particular the > /CELLS=LABELS part) but it is better optimized in general for that > particular task. > > On Wed, Dec 4, 2019 at 4:42 AM Dave Trollope wrote: > > > > We jus

Re: Excessive file system usage

2019-12-04 Thread Dave Trollope
to directly and specifically address the behavior you're > > > seeing, but it looks like there could be these kinds of issues with > > > kubernetes and the PSPP devs wouldn't be able to help unless they knew > > > kubernetes: > > > > &

Re: Excessive file system usage

2019-12-04 Thread Dave Trollope
-convert as Ben suggested and report back. Cheers Dave On Dec 4, 2019, 2:06 PM -0600, Dave Trollope , wrote: > Once the conversion is complete the space is returned so its not a long term > problem - only during the conversion. This became an issue because in > kubernetes you con

Re: Excessive file system usage

2019-12-24 Thread Dave Trollope
space to write the csv. Unfortunately, we aren't ready to switch to it because its doesn't support labels or variable selection. That will be something to address next year. Cheers Dave On Wed, Dec 4, 2019 at 9:54 PM Dave Trollope wrote: > I can confirm this same behavior ex