-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Leigh,
On 4/30/14, 1:55 PM, Hayward, Leigh wrote:
> Hi Chris, Thank you so much for the speedy response . I have no
> idea what i'm doing formatting wise. Pretty much just wrapping and
> indenting all by hand so forgive me if it's a mess!
>
>> ________________________________________ From: Christopher
>> Schultz <ch...@christopherschultz.net> Sent: 30 April 2014 17:58
>> To: Tomcat Users List Subject: Re: Tomcat 8.0.3.0 getting never
>> before seen by google Illegal State Exception. Sevlets outputting
>> the >audio output from the previous runs of the program instead
>> of the current run.
>
> Leigh,
>
> On 4/30/14, 10:31 AM, Hayward, Leigh wrote:
>>>> My Java EE web application takes in multiple audio inputs
>>>> and outputs them as a single wav file via an
>>>> application/octet stream.
>
> Like a mixer?
>
>> Like a concatenator, adds songs one after the other into one long
>> song. It then overlays (mixes) another file into it containing
>> audio cues on top of the concatenated music. But the issue is
>> definitely arising at the concatenation phase.
Okay. Are you saying that you don't get the output file you are
expecting given a set of input files, even when under test conditions
(a single request)?
Do you accept all files at once for upload and return a single
concatenated file, or do you upload the files one at a time and then
request the completed file at the end of the workflow?
>> I have the audio writing to a file after each song is added in
>> the concatenation phase so that i can audibly debug and the
>> concatenator is definitely struggling to find the uploaded
>> files.
What do you mean "struggling to find the uploaded files"? Your file
upload library should be readily giving the files to you.
>>>> It seemingly randomly works correctly (i.e. outputting the
>>>> correctly manipulated audio file) but sometimes, the file
>>>> from a previous run of the program is output and I get one of
>>>> these errors:
>>>>
>>>> SEVERE [http-nio-8084-exec-30]
>>>> org.apache.catalina.loader.WebappClassLoader.clearReferencesThreads
>>>>
>>>>
The web application [/MyApp] is still processing a request that has
>>>> yet to finish. This is very likely to create a memory leak.
>>>> You can control the time allowed for requests to finish by
>>>> using the unloadDelay attribute of the standard Context
>>>> implementation.
>>>>
>>>> and to me seemingly random numbers of these errors:
>>>>
>>>> "SEVERE [http-nio-8084-exec-87]
>>>> org.apache.coyote.http11.AbstractHttp11Processor.process
>>>> Error processing request java.lang.IllegalStateException: The
>>>> resources may not be accessed if they are not currently
>>>> started?"
>
>
> These kinds of things are almost always due to storing of a request
> or response option in some kind of structure that survives past the
> end of a particular request.
>
> Can you explain how you build the response -- specifically
> involving any non-standard threading you may do?
>
>> The response is built in a method called finish which is called
>> in doPost like this :
>
>> finish(req, res); In finish I make a printwriter I set response
>> content type to "application/octet-stream", set the header
>> "Content-Disposition" to "attachment; filename=finished.wav" I
>> make a FileInputStream of the finished filepath Then I set the
>> header content length to the length of the finished file. then I
>> while loop the fileInputStream into the printwriter Then I close
>> the FIS.
That seems fairly straightforward. Why are you bothering with a
PrintWriter? The existing response OutputStream ought to be good
enough to pump bytes through. How are you pumping the bytes? Just a
simple, locally-declared byte array?
You should remember to flush and/or close the output stream as well. I
think Tomcat will flush and close this for you, but you might want to
check to see if it changes anything.
>> I have not touched threading since it looks terrifying. I did
>> experiment with making the servlet singlethreaded since I thought
>> it could have something to do with threads (bad practice I know)
>> but that did nothing to help.
Are you using any variables that have been declared at the class
level? Everything you do with a request should be declared inside of
the service method (or doPost, etc.) or a parameter passed-into a
utility method. Don't store references to request, response, etc.
outside of the thread of execution that is handling that request.
If single-threading fixes your problem (sounds like it doesn't), then
you have broken the above rule.
>>>> The files always upload correctly to my filesystem, but
>>>> something is going wrong when I try to access them in order
>>>> to process them.
>
> What mechanism do you use for upload?
>
>> My servlet has a location set in the @multipartconfig tag that is
>> where all files are saved to and retrieved from. I am getting the
>> Parts from the request then iterating through them, for each part
>> that is a file upload I use part.write("uploadedfile" + count +
>> ".wav");
In that case, the files should all have arrived on the disk before
your code gets a chance to read them. Remember that they aren't
required to be on the disk: you should be reading them from the
InputStream(s) that the container (Tomcat) gives you. Also remember
that the container (Tomcat) will probably delete those files after the
request has completed, so you can't rely on them being in any
particular place for any amount of time: if you want a copy of a file,
you'll have to make it yourself.
>>>> Also when it is downloading the file it appears to the user
>>>> to be several MB long despite the file that is output being
>>>> only a few thousand KB.
>
> NB a few thousand KiB is the definition of several MiB.
>
>> I am possibly just an idiot on that one. I think the deadline
>> sleep deprivation is getting to me
It happens to all of us.
> Is the response built before you stream any of it to the client?
> Are you setting a Content-Length before you send any data? Are you
> using chunked responses?
>
>> I call a finish(req, res) method in my doPost which is where all
>> my response building is done. I am setting a content length
>> before I send the data because I wanted to be able to see how
>> long it was going to take to download since the program is fairly
>> slow.
It doesn't seem like concatenating files would take that much time,
besides the upload time. I don't know much about WAV files, though...
perhaps you have to do the math to connect them together properly. If
you have to re-encode entire files, though, you are probably doing it
wrong.
>>>> Sometimes when it doesn't work it does not produce these
>>>> errors, but it never produces these errors when it works
>>>> correctly.
>
> Does (increased) load seem to make the situation worse?
>
>> Definitely. Always happens with long audio files and only
>> sometimes with shorter test ones.
So long files end up breaking things more than short files? When I
said "load", I really meant "numbers of simultaneous requests"
(increasing).
If you can't even handle one isolated request with long files, you
have a serious problem.
>>>> I have googled it but there's no reference to the second kind
>>>> of error anywhere on the web aside from svn commits by
>>>> tomcat developers, so while I am a total newbie to mailing
>>>> lists, after exhausting stackoverflow this seemed like the
>>>> only place to turn to. I'm developing a java EE web
>>>> application in Netbeans using Tomcat 8.0.3.0 on a windows 7
>>>> operating system.
>
> (You should upgrade to a later Tomcat 8 beta. Nothing should
> affect what you are seeing here, but it's not a bad idea to get the
> latest.)
>
>> I'm just a little nervous about moving or touching anything since
>> i'm not 100% confident it wont just break everything. Still new
>> at this. I wanted the most stable version, to balance out my
>> intellectual instability.
The most stable version of 7.0.53. The 8.0.x line hasn't yet been
voted stable, though it pretty much is. Latest 8.0.x is 8.0.5 and will
soon be 8.0.6, but it has only been voted beta at this point.
>>>> The web application is very basic and allows users to upload
>>>> files via a multipart html form. This then posts to a servlet
>>>> which first uploads these files to the programs file system
>>>> and then accesses them, concatenates them together and saves
>>>> them back on the file system. It's better explained by this
>>>> diagram http://imgur.com/Oacd4gq
>
> The real question is whether you are using non-request-processor
> threads to do any of this work. Also, where are the various data
> stored while you are working on them? Not the filesystem -- that
> seems self-evident -- but where in the program are the String[]
> objects that represent all the songs that have been uploaded?
>
>> C:\Users\Leigh\Documents\NetBeansProjects\MyApp\src\java\sound
>
>> All audio files get stored in and are accessed from there.
No, somewhere in your program you have a String[] declared where you
have a reference to those filenames. You shouldn't actually have
references to path names at all, because the container can buffer the
files in memory instead of ever writing them to the disk.
Anyhow, I was wondering if you were using a String[] declared at the
servlet level to track this stuff. That always results in disaster,
because multiple requests will fight over the same variable and step
on each other's toes.
Any chance you could post your code somewhere for inspection? Dropping
it right into a post to the list is just fine.
>> This is purely a dissertation project so I have no intention of
>> making it work for multiple users or in fact be functional
>> anywhere outside of my laptop. All files are being uploaded to
>> the file system of the project itself since that was the only way
>> I knew how to make it work.
Why bother with a web application interface, then?
> When you process a "transaction" (upload -> merge -> respond), are
> you taking care to ensure that the files are placed into different
> paths for each request? How are you choosing the filename of the
> merged audio clip?
>
>> all uploads are called uploadedfile1, uploadedfile2,
>> uploadedfile3 etc depending on how many files are uploaded.
If these files all go to the same place, then multiple requests will
overwrite each other. That's going to be a problem, except you said
you don't care. Honestly, you should either care about this and make
it work properly, or just drop the "web" requirement where it's okay
to be sloppy.
>> When these files don't exist already on my file system I
>> sometimes get null pointer errors and an empty output.
>> Overwriting works better than creating new.
>
>> The filename of the finished file is just called finished.wav.
>> Always. It just gets overwritten each time I run it. Or not as it
>> seems.
I would choose a different filename every time. Especially if it looks
like the old copy of the file hasn't been closed entirely, yet.
>>>> Could it be that the files are not being uploaded fully
>>>> before they are being accessed?
>
> I don't believe this is possible when using Tomcat's file-upload
> implementation. It is certainly possible to do if you are
> performing the multipart handling yourself. What tool are you using
> to fetch the multipart data from the request?
>
>> As I said above I just get the parts from the request, for loop
>> through them and write any audio files to the locations specified
>> in my MultipartConfig.
Looking at the code would be very helpful. Feel free to leave-out any
cutting-edge research-y stuff that you don't want to divulge ;)
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIcBAEBCAAGBQJTYUtlAAoJEBzwKT+lPKRYrtsQAKaQHBjvC8i4UJngL3mDtGMD
t3JxMET3Nb/AGAkeBFn2yJR+PTR4Vuj1gbEKrHN+SE3l1VlbVmjxfOQlPDAm3SPF
SE50zTnmYKKdcOMHpIuUeiMgJh/uKCym+bp3PNcuvORZq/FVJdJ3b8me0MQUO3Vh
juCqj/h/UV+eKZaFmLA+dcJAQoxmHgVoCidnwRN255LUve8/o/JQXthTjmCGuf4Z
rQ7ZarxVr8wbXZJ5wQZ/O1Z6K7IIZ07hNbDHCtj+3ZEeEcSzn5IEOMpskhSu48bC
5LTqn5gXgE7m+uFszjjpshRhoO2PPRf2sb39IYVkkExqVb5EOkdNpVVckRaBd371
H14S/CoWQcCvfq+QORl/i95vrMf+P7mMqD+wS3Bagyw3zD0spTu48oLcyG2Crmwd
zCFyezw3SIJdis8yQBsl7uGNMELF5gJ6n0oPvcUwjiY7tHbZ6f6eF5aEPAz/r1eS
uf3OHeqJswRW9qBUGPxEtvNOajs4q0o6BacfQpiQlHeLNzh6OzGsLmzUZJGoWiPV
zJR+52Z+5RFXDI38E+XSX/T9UFF7M1WjH0mfjpYz6TjIDZl7H+FH7Od5+tQ02M4C
tlr66gq2HqpQA1SlhnRpIpSsyF61PIJIteX1rcRW6qik6qizn4XJQ8GoweCwV8Jn
F2ASYC6UsDVoHpHtJfrg
=uGx4
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
