The method should be changed as follows:
private static final Logger log =
LoggerFactory.getLogger(CaptureScreen.class);
public void run() {
try {
while (active && !core.isReadyToRecord()) {
Thread.sleep(60);
}
Robot robot = new Robot();
BufferedImage image = null;
while (active) {
final long ctime = System.currentTimeMillis();
Rectangle screen = new Rectangle(spinnerX, spinnerY, spinnerWidth,
spinnerHeight);
long start = System.currentTimeMillis();
image = robot.createScreenCapture(screen);
log.debug(String.format("Image was captured in %s ms",
System.currentTimeMillis() - start));
try {
timestamp += timeBetweenFrames;
start = System.currentTimeMillis();
byte[] data = se.encode(screen, image, new Rectangle(resizeX, resizeY));
log.debug(String.format("Image was encoded in %s ms",
System.currentTimeMillis() - start));
pushVideo(data, timestamp);
} catch (Exception e) {
log.error("Error while encoding: ", e);
}
final int spent = (int) (System.currentTimeMillis() - ctime);
log.debug(String.format("Time spent %s ms; diff: %s ms", spent,
timeBetweenFrames - spent));
if (sendCursor) {
core.sendCursorStatus();
}
Thread.sleep(Math.max(0, timeBetweenFrames - spent));
}
} catch (Exception e) {
log.error("Error while running: ", e);
}
}
1280x1024 are captured encoded for too long
Will now try to speed up encoding.
On Thu, Oct 24, 2013 at 8:35 PM, Alexander Brovman <
alexander.brov...@gmail.com> wrote:
> Maxim,
>
> A) Interessting to say the very least, how did you manage to measure the
> encode and the screencapture? or what tools did you use in order to measure
> those, might come in handy for me when measuring performance.
>
> Right now i changed the sourcecode in the following way:
> 1) " timeBetweenFrames = (ScreenDimensions.quality ==
> ScreenQuality.VeryHigh) ? 30 : 100;"
>
>
> 2) public void run() {
> try {
> Robot robot = new Robot();
> BufferedImage image = null;
> while (active) {
> final long ctime =
> System.currentTimeMillis();
> Rectangle screen = new
> Rectangle(ScreenDimensions.spinnerX,
> ScreenDimensions.spinnerY,
>
> ScreenDimensions.spinnerWidth,
>
> ScreenDimensions.spinnerHeight);
>
> image = robot.createScreenCapture(screen);
>
> try {
> timestamp += timeBetweenFrames;
> byte[] data = se.encode(screen, image, new
> Rectangle(ScreenDimensions.resizeX,
>
> ScreenDimensions.resizeY));
>
> pushVideo(data, timestamp);
> } catch (Exception e) {
> e.printStackTrace();
> }
> final int spent = (int)
> (System.currentTimeMillis() - ctime);
>
> if (sendCursor) {
> core.sendCursorStatus();
> }
> Thread.sleep(Math.max(0, timeBetweenFrames
> - spent));
> }
> } catch (Exception e) {
> e.printStackTrace();
> }
> }
>
> I am definitely noticing a very strong improvement in terms of less video
> delay.
> using jnettop i can see that at a screen resolution of 1650x1080 it is
> approximately using between 4-8mbit for 1 user.
> So i guess the biggest issue here is the encoder, the encoder is
> definitely not efficient, i have been looking at solutions of other people
> on stackexchange in terms of algorithms.
>
> B) I have a question regarding the process, so if the screencapture takes
> place and is sent using the RTMP method/protocol to the server. Does the
> sever additionally encode the file again before sending it to the enduser?
> If so which javafiles contain those settings, because i would like to test
> if i can tweak them to reduce the delay even more by disabling the encode
> of the images/stream on the serverside and just send the files "as is".
> Although once again i fear the algo is the issue.
>
> Kind regards,
> Alexander
>
>
> On Thu, Oct 24, 2013 at 3:06 PM, Maxim Solodovnik <solomax...@gmail.com>wrote:
>
>> Here are my measurements:
>> Screen is captured in ~ 160-170ms
>> Encoded in ~120-200ms
>> Will try to speed up this somehow
>>
>>
>> On Thu, Oct 24, 2013 at 3:27 PM, Alexander Brovman <
>> alexander.brov...@gmail.com> wrote:
>>
>>> nevermind used the wrong ANT build.
>>>
>>> Got it to work now :)
>>>
>>>
>>> On Thu, Oct 24, 2013 at 9:11 AM, Alexander Brovman <
>>> alexander.brov...@gmail.com> wrote:
>>>
>>>> Maxim i made some changes to the code, i downloaded the source on the
>>>> server, changed the lines with nano. and then from the mainfolder simply
>>>> ran the ant command and it started compiling (albeit some errors appeared
>>>> since i only entered "ant" without any options/parameters.
>>>>
>>>> Openmeeting is working fine, i simply replaced the entire
>>>> screenshareing/ folder with the new one i just compiled... any suggestions
>>>> on what im doing wrong here?
>>>>
>>>> I am getting this error:
>>>> Unsigned Entry in resouce: slf4j-api-1.6.4.jar
>>>>
>>>>
>>>> On Thu, Oct 24, 2013 at 8:17 AM, Maxim Solodovnik <solomax...@gmail.com
>>>> > wrote:
>>>>
>>>>> I'm currently trying to modify the code to get 20+ fps (was requested
>>>>> many times, but I had no time on this :( currently I'm on vacation, will
>>>>> try to implement it)
>>>>>
>>>>>
>>>>> On Thu, Oct 24, 2013 at 10:02 AM, Alexander Brovman <
>>>>> alexander.brov...@gmail.com> wrote:
>>>>>
>>>>>> Maxim,
>>>>>>
>>>>>> Thank you for responding so quickly. The primary reason i asked is
>>>>>> because we sometimes need to output data like Financial Stock charts or
>>>>>> similar dynamic content and in a resolution size of 1650x1080. As you can
>>>>>> imagine right now the delay is.. manageable, perhaps somewhere along the
>>>>>> lines of 0.5-1 second delay id say.
>>>>>> We had an independant tested saying its an average of 130ms. But for
>>>>>> fast video/changes the framerate will sometimes drop or become laggy.
>>>>>>
>>>>>> Ill try setting the timebetweenFrames to 30ms which in "ideal"
>>>>>> conditions should give me 33.333 frames per second of video.
>>>>>> As for the Thread.Sleep(XXX) not working or functioning i will remove
>>>>>> the thread.sleep(60) from the top,
>>>>>> As for the Thread.Sleep(xxx) settings i will simply remove it; BUT i
>>>>>> will leave in the "Thread.sleep(Math.max(0, timeBetweenFrames - spent));"
>>>>>> since "worst" case scenario it will not be put to sleep at all since the
>>>>>> output will be 0, and "best case" scenario the thread will be put to
>>>>>> sleep
>>>>>> for 30 - 1 ms depending on how the calculation turns out.
>>>>>>
>>>>>> Perhaps it would be good to leave somekind of thread sleep in there
>>>>>> that is a static factor, from a development standpoint i would rather
>>>>>> prefer having a fixed constant in there. rather than putting the thread
>>>>>> to
>>>>>> sleep (potentially twice). Either that or im not understanding the reason
>>>>>> for the first thread.sleep (or the second one).
>>>>>> Either way i will make the changes now and compile, I will report my
>>>>>> findings here if you want me to :)
>>>>>>
>>>>>> @Sebastian,
>>>>>>
>>>>>> At the time of writing this i saw you respond, so i am writing this
>>>>>> short addendum:
>>>>>> 1) Could you please comment on whether or not Thread.Sleep(xxx) works?
>>>>>> 2) 2-4 frames per second and less may be great for static
>>>>>> presentations but not so much for dynamic. I understand the concerns
>>>>>> raised
>>>>>> on your behalf regarding bandwidth usage but since we have a dedicated
>>>>>> Gigabit connection we have no issues with dedicating anywhere between 2-5
>>>>>> Mbit per user. 20 Frames per second really is the minimum we are looking
>>>>>> at, i think it would be great if you presented people with an
>>>>>> option/alternative to gotowebinar. I understand that applications which
>>>>>> compile into native code will always be faster and outperform in certain
>>>>>> aspects, but when you have the motivation and perhaps the financial
>>>>>> resources to dedicate time and money into getting away from solutions
>>>>>> such
>>>>>> as GoToWebinar.
>>>>>>
>>>>>> Is there a way of "fixing" the position of the screen that the
>>>>>> viewers will see when they accept a desktop streaming broadcast from a
>>>>>> presenter? I ask this because i havent found a way to change the location
>>>>>> of the Chat Bar inside a conference room yet (neither have I found a way
>>>>>> to
>>>>>> change the colors/themes to be honest, need to do some more google
>>>>>> searches).
>>>>>>
>>>>>> Kind regards,
>>>>>> Alexander
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Thu, Oct 24, 2013 at 4:47 AM, seba.wag...@gmail.com <
>>>>>> seba.wag...@gmail.com> wrote:
>>>>>>
>>>>>>> Do we still have the super high quality setting in the sharing
>>>>>>> client?
>>>>>>> My problems in the past was more around the bandwidth issues then
>>>>>>> the FPS.
>>>>>>> If you do 2-4 FPS the bandwidth usage will go that up... there is
>>>>>>> probably a trick, maybe there is also something todo in the way the
>>>>>>> tiles
>>>>>>> are calculated that need to be resend. It normally should only try to
>>>>>>> resend the ones that really change. So if you show an hd movie you might
>>>>>>> see your bandwidth needs go up quite a bit.
>>>>>>>
>>>>>>> Seb
>>>>>>>
>>>>>>>
>>>>>>> 2013/10/24 Maxim Solodovnik <solomax...@gmail.com>
>>>>>>>
>>>>>>>> Hello Alexander,
>>>>>>>>
>>>>>>>> Your findings are correct :)
>>>>>>>> With the only exception: Screenshots are taken in single thread, so
>>>>>>>> there are some limitations in the speed :(
>>>>>>>> I believe Thread.sleep(xxx) doesn't work at all
>>>>>>>> I'm planning to rewrite this part And hopefully will be able to
>>>>>>>> increase FPS.
>>>>>>>>
>>>>>>>> Basic compilation steps are here:
>>>>>>>> http://openmeetings.apache.org/BuildInstructions.html
>>>>>>>> After compiling all jars in screensharing folders are being signed,
>>>>>>>> so you need to replace all of them, otherwise you will get security
>>>>>>>> error
>>>>>>>> due to different certificates
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Oct 24, 2013 at 3:07 AM, Alexander Brovman <
>>>>>>>> alexander.brov...@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> I have two quick questions, but first of all i just wanted to say
>>>>>>>>> that Openmeetings is fantastic. I had both OpenMeetings and BBB
>>>>>>>>> Installed
>>>>>>>>> and we have decided to go with OM.
>>>>>>>>>
>>>>>>>>> I am relatively apt at programming with Python and C# but I have
>>>>>>>>> never worked with a lot of Javaapplications like these ones, so i
>>>>>>>>> have the
>>>>>>>>> following questions:
>>>>>>>>>
>>>>>>>>> *1)* From what i understand the ScreenSharing is performed by
>>>>>>>>> taking snapshots every x milliseconds. Additionally there is an
>>>>>>>>> algorithm
>>>>>>>>> in place that divides the snapshot into tiles and measures the
>>>>>>>>> changes in
>>>>>>>>> tiles, if there is a change the tile is uploaded. If this is correct,
>>>>>>>>> and i
>>>>>>>>> want a more.. "fluid" video because there will be a lot of changes,
>>>>>>>>> cant i
>>>>>>>>> simply modify the "framerate" at which the images will be compared to
>>>>>>>>> improve the output? I am well aware that this would eat into the CPU
>>>>>>>>> and
>>>>>>>>> the bandwidth but i have a Full Duplex Gbit and 2x Quadcore Xeons
>>>>>>>>> available
>>>>>>>>> to me so this is not an issue. We are trying to test for an extreme
>>>>>>>>> case of
>>>>>>>>> 500-1500 users and how to optimize the output, i think the developers
>>>>>>>>> and
>>>>>>>>> other users will be interested in seeing the results.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> *2)* Where would these changes need to be made? So far i have
>>>>>>>>> isolated the CoreScreen.java , CaptureScreen.java and
>>>>>>>>> theScreenV1Encoder.java files that apparently contain the code which
>>>>>>>>> processes the desktop sharing.
>>>>>>>>>
>>>>>>>>> The CaptureScreen.Java has the:
>>>>>>>>> "timeBetweenFrames = (ScreenDimensions.quality ==
>>>>>>>>> ScreenQuality.VeryHigh) ? 100 : 500;" setting on line 54 which
>>>>>>>>> basically
>>>>>>>>> says that if Screequality is set to very high (condition is true) then
>>>>>>>>> select 100 as the value for timeBetweenFrames.
>>>>>>>>>
>>>>>>>>> This variable is apparently used lateron in the
>>>>>>>>> "Thread.sleep(Math.max(0, timeBetweenFrames - spent));" on line 96,
>>>>>>>>> which
>>>>>>>>> will pause the current loop for the amountof time that is the
>>>>>>>>> difference
>>>>>>>>> between timeBetweenFrames( in our case 100ms) minus the spent time
>>>>>>>>> (which
>>>>>>>>> is defined on line 91).
>>>>>>>>>
>>>>>>>>> So in Theory if i wanted to output the screen in realtime at 20 or
>>>>>>>>> 24 frames per second which would mean that a snapshot would need to be
>>>>>>>>> taken every 50 milliseconds or so, would i simply need to set a
>>>>>>>>> different
>>>>>>>>> frame setting for the time between frames, something like:
>>>>>>>>> "timeBetweenFrames = (ScreenDimensions.quality ==
>>>>>>>>> ScreenQuality.VeryHigh) ? *20* : 500;"
>>>>>>>>>
>>>>>>>>> Reduce the Thread.sleep(60); on line 60 to something like 20
>>>>>>>>> milliseconds so (Thread.sleep(20);)
>>>>>>>>> and then finally remove the the last part
>>>>>>>>> "Thread.sleep(Math.max(0, timeBetweenFrames - spent));" or perhaps
>>>>>>>>> leaving
>>>>>>>>> it in?
>>>>>>>>>
>>>>>>>>> I understand that this would be a .. "wrench and hammer" approach
>>>>>>>>> since this will probably eat up quite a bit of CPU usage. I mean the
>>>>>>>>> simpler solution here clearly would be to get rid of the algorithm
>>>>>>>>> itself
>>>>>>>>> and simply rewrite it to simply output a screenshot every x
>>>>>>>>> milliseconds or
>>>>>>>>> so, but i have no idea how deep i would be messing with openmeetings
>>>>>>>>> and if
>>>>>>>>> that would/could break other parts of openmeeting that access these
>>>>>>>>> files?
>>>>>>>>>
>>>>>>>>> So would this be a workable solution that i could at least try
>>>>>>>>> out? I would ofcourse be more than happy to supply the CPU Usage
>>>>>>>>> differences.
>>>>>>>>>
>>>>>>>>> *3)* My only question would be is how would i recompile these? I
>>>>>>>>> am asking because i read somewhere in the mailinglists (or the old
>>>>>>>>> google
>>>>>>>>> groups) that it is recommended to use ANT in order to recompile
>>>>>>>>> things.
>>>>>>>>> Also what will be the output name of the .jar that is essentially
>>>>>>>>> being
>>>>>>>>> loaded for people and located on the server?
>>>>>>>>> Because all i can find on my Debian box is
>>>>>>>>> the
>>>>>>>>> /usr/lib/red5/webapps/openmeetings/screensharing/openmeetings-screenshare-2.1.1-RELEASE.jar
>>>>>>>>> file.
>>>>>>>>>
>>>>>>>>> So yeah, if somebody could help me with recompiling i would
>>>>>>>>> appreciate it! :)
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> WBR
>>>>>>>> Maxim aka solomax
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Sebastian Wagner
>>>>>>> https://twitter.com/#!/dead_lock
>>>>>>> http://www.webbase-design.de
>>>>>>> http://www.wagner-sebastian.com
>>>>>>> seba.wag...@gmail.com
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> WBR
>>>>> Maxim aka solomax
>>>>>
>>>>
>>>>
>>>
>>
>>
>> --
>> WBR
>> Maxim aka solomax
>>
>
>
--
WBR
Maxim aka solomax
