Handling unpresented files with NSDocument

[maximilian.schm...@mto.io]

Hey everyone.

I’m writing a document-based audio-editor for macOS and have a problem with 
managing the user's audio files.

The app uses CoreAudio exclusively for everything audio related including 
reading and writing audio files to/from disk. When the user imports an audio 
file into the application (via drag and drop for example), it gets converted 
and written into the document’s filepackage by CoreAudio. The document doesn’t 
complain because I create file wrappers for each audio file when the my 
document’s fileWrapper(ofType:) function get’s called.

The audio files don’t get altered once they’re in the document-package. They're 
just there for CoreAudio to read. Also, combined they could get quite big. 500 
GB of audio files for the biggest projects users might create is not 
unreasonable. 

So these audio files are not managed by my NSDocument subclass (it only saves 
their url for CoreAudio). When duplicating the document or saving it to another 
location the audio files have to move, too. So I currently catch the related 
functions in my NSDocumentController subclass and move them manually. More wild 
problems and sketchy solutions occur and then there is the revert option and 
iCloud. I feel like I’m completely off-track here.

Is it possible for the NSDocument to know about the audio files in it’s package 
folder and handle them like they belong to it without wholly representing them 
in the model? Or can CoreData solve this? I think, however, that storing the 
audio data in anything other than actual audio files will massively complicate 
things with CoreAudio.

Can someone please point me in the right direction?

Thanks!
Max



___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

NSCollectionViewItem lifecycle

[Andrew Keller]

Hi all,

I’m experiencing what appears to be some sort of a memory leak with my use of 
NSCollectionView.  I’ve figured out that this problem has actually always been 
happening ever since I added this particular collection view to the project; I 
just never noticed because the memory footprint of the array items was so small 
that it didn’t negatively impact the computer (and I never looked at the memory 
usage).  Well, these days, the data that our users are shoving through the app 
has grown a lot, to the point where the app now uses about 10 GB of memory 
after only about 5 minutes.  As a temporary workaround, our users can restart 
the app — but you can imagine how that gets annoying after a while.

So first, some clerical details: this is Xcode 6, on Mac OS X 10.10, using the 
old-style (pre-10.11) NSCollectionView, with an ArrayController as the data 
source to the collection view, and using bindings to hook everything up.  
Nothing is subclassed.  The elements of the ArrayController are what I call 
“Media” objects.  The ArrayController’s data source is bound to an NSArray on 
my window’s controller, and I’m using [myController setMedia:newArray] to 
change the data.  The data is never reordered, and is usually completely 
swapped out (usually, when I change the content list, it’s a wholesale total 
rewrite, and none of the objects are ever repeated).  The collection view 
serves to simply display the items visually — not much more than that.

I initially noticed the leak when my computer ran out of memory.  Some quick 
debugging revealed that my Media objects (which have a much larger memory 
footprint than they used to) are not being deallocated — ever.

According to the memory debugger in Xcode 8 (which is really awesome, by the 
way), my Media objects are being kept alive by NSCollectionViewItem objects, 
which also appear to be piling up and never deallocated.  The memory debugger 
also showed me that my view prototype is being created fresh every time — a new 
one for every cell — every time I set the media array on my controller, and the 
old cell views are also never deallocated.   Not really knowing how 
NSCollectionView works under the hood, I’m unsure of the next step to 
troubleshoot this.

Any ideas?

On a related note, what is the expected lifecycle of an NSCollectionViewItem 
object?

Thanks,
 - Andrew Keller


___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Handling unpresented files with NSDocument

[Jerome Krinock]

> On 2016 Aug 03, at 07:56, maximilian.schm...@mto.io wrote:
> 
> can CoreData solve this?

If you are using Core Data for other reasons, then you should consider using 
BSManagedDocument, and you should read the discussion of saving and copying 
“additional content”, (which would be your audio files), which I had last year:

https://github.com/karelia/BSManagedDocument/issues/45

But if you are not already using Core Data in this app for other reasons, I 
don’t think it offers much more than NSDocument for your issue, and with the 
additional complexity, you probably don’t want Core Data.
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Handling unpresented files with NSDocument

[Quincey Morris]

On Aug 3, 2016, at 07:56 , maximilian.schm...@mto.io wrote:
> 
> Is it possible for the NSDocument to know about the audio files in it’s 
> package folder and handle them like they belong to it without wholly 
> representing them in the model?

I don’t believe there’s any good solution based on a unified document package. 
That’s because the NSDocument semantics are based on the reasonableness of 
reading-then-writing the entire contents of the document at save time, and it’s 
not reasonable in your scenario.

I think we tend to think, as developers, that if we use NSDocument for large 
asset-based editors, users will want to blithely copy and move their documents 
as if they were Pages documents. I’m not sure this is true. Instead, if there 
are a lot of bulky assets to collect, it feels quite natural for the bulk of 
the data to be in a fairly fixed location, especially it that data is more like 
a cache than like editable data.

I’ve tried a couple of alternatives over the last few years. (None has ended up 
in a released product yet, so I can’t vouch for user satisfaction.)

One approach is to ask the user via NSSavePanel, after creating a new document, 
for a folder location to use for “working storage”. You then create a file 
package at that location that holds the copies of your asset files. The 
document, now much smaller, has a URL reference to that package, so copying or 
moving the document doesn’t affect the asset package. If the user wishes to 
transfer the document to (say) a thumb disk or another Mac, then the user is 
responsible for copying the matching asset package, too, and your code is 
responsible for asking where to resolve the “broken” document URL reference at 
the other end.

Another approach is to create a free-standing asset repository (in, say, your 
Application Support subfolder), and have the user import assets there 
*independently of* any particular documents. These assets can be organized into 
“projects” and documents then created making use of the assets from one or more 
projects.

Or you can do something intermediate between these two extremes. (E.g. allow 
the user to create working storage folders at any desired location, which 
allows the big files to be on a separate volume.) Whatever you do, you might 
need an Export menu item that packages up everything a document needs, for 
cases where the entire project need to be moved to (say) another Mac.

Maintaining references between documents and assets is a fair amount of fiddly 
work (because you have to deal with various kinds of broken references), but 
it’s a lot less hinky than trying to mess with the NSDocument behavior directly.

FWIW

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Justification of collection cells in sections

[Doug Hill]

So, does anyone know if the current behavior I mentioned is a bug? Is the 
behavior of Flow Layout documented? Should I file a bug with Apple?

Also, what would be some quick ways to modify the flow layout behavior to 
handle this case with one item per section?

Doug Hill

> On Aug 2, 2016, at 6:11 PM, Peter Tomaselli  wrote:
> 
> I’ve been in the same situation too (wanting a “left-justified” flow layout) 
> and the only way I am aware of to tackle it is to subclass 
> UICollectionViewFlowLayout and tweak the frames yourself. Luckily this is not 
> nearly as complicated as handling the whole layout yourself!
> 
> Peter
> 
> On Aug 2, 2016, at 7:51 PM, Doug Hill  wrote:
> 
>> I was hoping to allow the Flow Layout to handle this rather than me handling 
>> the layout. Given how the layout works for multiple items in a section, I 
>> would presume there is some way to make this work the same for a single item 
>> in a section. Am I wrong?
>> 
>> Doug Hill
> 


___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Justification of collection cells in sections

[Peter Tomaselli]

I remain a non-expert on this topic, but my assumption has always been that
the current “look” is by design. Ugly, but by design. “Equally distribute
cells across the row” could be taken to mean that in the case of one cell,
it belongs in the middle… [shrug]

This is some of the first Cocoa code I wrote so please forgive… almost
everything about it, but here’s the FlowLayout subclass I mentioned:
https://github.com/Peterbing/CV-AutoLayout/blob/master/TemplateProject/TheFlowLayout.m#L31

This is a completely “left-justified” layout and it works by overriding the
`layoutAttributesFor*` methods to call super, and then squishes everything
over to the left before returning the attributes to the caller.

This is certainly not production-quality code but perhaps it’s a start!
IIRC there is Apple documentation (or perhaps it was a WWDC session) about
creating a simple “tweaked” UICollectionViewFlowLayout subclass — so my
impression is that this approach in general is kosher.

On Wed, Aug 3, 2016 at 3:38 PM, Doug Hill  wrote:

> So, does anyone know if the current behavior I mentioned is a bug? Is the
> behavior of Flow Layout documented? Should I file a bug with Apple?
>
> Also, what would be some quick ways to modify the flow layout behavior to
> handle this case with one item per section?
>
> Doug Hill
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Emailing from a daemon process

[Bruce Stephens]

Could use a simple Perl script… quick easy and nasty…
#!/usr/bin/perl
# 
…
# from an old online sales system that is still operational
...
{
open(MAIL2,"|/usr/sbin/sendmail -t -f nobody") ||
&mydie("Hmmm, a nasty horrible server error has 
occurred, please contact us...");
print MAIL2 "To: $emailAddress1\n";
print MAIL2 "Subject: The ORDER from $theName\n";
myResult(MAIL2);
close (MAIL2);
};


Works!!!
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Emailing from a daemon process

[Jens Alfke]

> On Jul 7, 2016, at 6:56 PM, Bruce Stephens  wrote:
> 
> Could use a simple Perl script… quick easy and nasty…

We already spent many messages dissecting why simply invoking sendmail doesn’t 
work.

> Works!!!

It might work on your machine, because you’ve configured Postfix to route mail 
properly. It’s not going to work on a stock Mac, because by default Postfix 
will try to connect directly to the SMTP server at the recipient’s domain, and 
that server is very unlikely to accept mail from some random computer on the 
Internet (because spammers.)

—Jens
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Handling unpresented files with NSDocument

[maximilian.schm...@mto.io]

> On Aug 3, 2016, at 11:51 PM, Quincey Morris 
>  wrote:
> 
> On Aug 3, 2016, at 07:56 , maximilian.schm...@mto.io 
>  wrote:
>> 
>> Is it possible for the NSDocument to know about the audio files in it’s 
>> package folder and handle them like they belong to it without wholly 
>> representing them in the model?
> 
> I don’t believe there’s any good solution based on a unified document 
> package. That’s because the NSDocument semantics are based on the 
> reasonableness of reading-then-writing the entire contents of the document at 
> save time, and it’s not reasonable in your scenario.

Hey Quincey! I thought his was the case but wasn’t a 100% sure. Thanks for 
clarifying. Makes total sense, but I thought there might be little features to 
help me.

> I think we tend to think, as developers, that if we use NSDocument for large 
> asset-based editors, users will want to blithely copy and move their 
> documents as if they were Pages documents. I’m not sure this is true. 
> Instead, if there are a lot of bulky assets to collect, it feels quite 
> natural for the bulk of the data to be in a fairly fixed location, especially 
> it that data is more like a cache than like editable data.

For a single user that might very well be the case. But when working in a team 
on a professional level (like I have for many years), when the document is 
copied around many times from Mac to Mac, to and from a shared server, the 
asset-management has to be as simple, transparent and bulletproof as possible.

Loosing audio files is the worst. It took hours for us to figure out what went 
wrong in Pro Tools and thankfully we always recovered them. It doesn’t happen 
often, but with a fixed deadline rushing towards you, you could potentially 
loose your job. That’s why I force the user to have a new copy of the imported 
audio file and not just a reference to where the original is located. I want an 
easy solution at the cost of storage. I’m also looking into completely locking 
the audio files to prevent the user from tinkering with them.

> I’ve tried a couple of alternatives over the last few years. (None has ended 
> up in a released product yet, so I can’t vouch for user satisfaction.)
> 
> One approach is to ask the user via NSSavePanel, after creating a new 
> document, for a folder location to use for “working storage”. You then create 
> a file package at that location that holds the copies of your asset files. 
> The document, now much smaller, has a URL reference to that package, so 
> copying or moving the document doesn’t affect the asset package. If the user 
> wishes to transfer the document to (say) a thumb disk or another Mac, then 
> the user is responsible for copying the matching asset package, too, and your 
> code is responsible for asking where to resolve the “broken” document URL 
> reference at the other end.

There are too many possibilities for the user to mess this up. And sometimes 
things have to move fast when working. So that option’s out. But a good idea to 
keep in mind for other projects :)

> Another approach is to create a free-standing asset repository (in, say, your 
> Application Support subfolder), and have the user import assets there 
> *independently of* any particular documents. These assets can be organized 
> into “projects” and documents then created making use of the assets from one 
> or more projects.

Yes, I thought about that. But it falls into the same category as the above and 
below.

> Or you can do something intermediate between these two extremes. (E.g. allow 
> the user to create working storage folders at any desired location, which 
> allows the big files to be on a separate volume.) Whatever you do, you might 
> need an Export menu item that packages up everything a document needs, for 
> cases where the entire project need to be moved to (say) another Mac.

That’s also a reasonable approach for many applications. But I want there to be 
no hidden obstacles when sharing your project with others. When he copy-pastes 
the project file, the user might very well think that he created a working 
project and I believe this absolutely should be the case.

> Maintaining references between documents and assets is a fair amount of 
> fiddly work (because you have to deal with various kinds of broken 
> references), but it’s a lot less hinky than trying to mess with the 
> NSDocument behavior directly.

Yes. I now believe that’s a rocky path I’m getting onto. But better it’s hard 
for me than for the user :/ 
Thanks for your input!

> FWIW
> 

Very worth!
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive

Re: Handling unpresented files with NSDocument

[maximilian.schm...@mto.io]

> On Aug 3, 2016, at 11:04 PM, Jerome Krinock  wrote:
> 
> 
>> On 2016 Aug 03, at 07:56, maximilian.schm...@mto.io wrote:
>> 
>> can CoreData solve this?
> 
> If you are using Core Data for other reasons, then you should consider using 
> BSManagedDocument, and you should read the discussion of saving and copying 
> “additional content”, (which would be your audio files), which I had last 
> year:
> 
> https://github.com/karelia/BSManagedDocument/issues/45
> 
> But if you are not already using Core Data in this app for other reasons, I 
> don’t think it offers much more than NSDocument for your issue, and with the 
> additional complexity, you probably don’t want Core Data.

Hey Jerome! Very similar issue you had there! I was thinking about dropping the 
duplicate feature but I’m close to solving it with my current approach (which 
is also using the FileManager to copy the files around). 

If your interested: I force an auto save onto the duplicated document (which is 
a copy of the old one without the audio files) and then copy all the audio 
files from the original into the auto save. When the user then saves the 
duplicate, I copy the files from the autosave to the new user-defined 
destination. The only problem right now is intercepting the function where the 
app automatically deletes the autosaves so can safely copy before that happens. 
Haven’t found that yet.

I don’t think using CoreData will be to any value to me when I look at my 
current design. The NSDocument can very well handle all the objects inside my 
app. So I don’t use CoreData. But I thought maybe it's able to manage the audio 
files for me when duplicating or reverting the document or moving it to iCloud. 
But reading through the CoreData getting started guide gave me no such clues.

Thanks for your input! I guess I’ll keep working on my current approach then.

> ___
> 
> Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)
> 
> Please do not post admin requests or moderator comments to the list.
> Contact the moderators at cocoa-dev-admins(at)lists.apple.com
> 
> Help/Unsubscribe/Update your Subscription:
> https://lists.apple.com/mailman/options/cocoa-dev/maximilian.schmidt%40mto.io
> 
> This email sent to maximilian.schm...@mto.io



___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: Handling unpresented files with NSDocument

[Quincey Morris]

On Aug 3, 2016, at 20:49 , maximilian.schm...@mto.io wrote:
> 
> But when working in a team on a professional level (like I have for many 
> years), when the document is copied around many times from Mac to Mac, to and 
> from a shared server, the asset-management has to be as simple, transparent 
> and bulletproof as possible.

> I force an auto save onto the duplicated document (which is a copy of the old 
> one without the audio files) and then copy all the audio files from the 
> original into the auto save. When the user then saves the duplicate, I copy 
> the files from the autosave to the new user-defined destination. The only 
> problem right now is intercepting the function where the app automatically 
> deletes the autosaves so can safely copy before that happens.

Taking the second part of this first, it’s likely that there’s NO safe 
mechanism for you to do this manually using the document architecture, because 
there are no atomic file system operations at the level you’re talking about. 
Still, you might find something that’s safe enough.

If this is for professional-level users who have sufficiently capable Macs 
(that is, at least a fusion drive if not a SSD, and fast networking everywhere) 
then I think you can safely ignore the file size and performance aspects of 
your problem, and your everything-in-the-document-package approach should work 
well without hacking up NSDocument and NSDocumentController.

> Is it possible for the NSDocument to know about the audio files in it’s 
> package folder and handle them like they belong to it without wholly 
> representing them in the model? Or can CoreData solve this? I think, however, 
> that storing the audio data in anything other than actual audio files will 
> massively complicate things with CoreAudio.

I went back to your original question, and read it a bit more carefully. The 
answer to this is “Yes!”, I think. When your NSDocument loading override is 
handed a file wrapper, you do not need to represent the audio files in your 
data model, nor do they need to be read into memory at all, unless you choose 
to do so. Just leave the audio file sub-wrappers alone, and keep the 
uber-wrapper around until save time. (What I usually do is delete the 
sub-wrappers for files that *are* changing, as edits are made. At save time, 
the absence of a file wrapper tells me I need to write out a new file and 
create its new sub-wrapper.) When you take that approach, the files with the 
untouched sub-wrappers should NOT need to be copied by the save**. Instead they 
are (atomically!) transferred to the new document package version representing 
the saved document.

Isn’t that what you’re trying to achieve?


** Provided you are saving to a local HFS+ file system. Network disks and other 
file systems may not support this and require re-copying of the unchanged files.

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com