Hello Alexander,
I seems to was able to enhance performance of screen sharing applet:
VeryHigh > 10 fps
High = 10fps
others = 2fps
Committed revision 1544855
On Fri, Oct 25, 2013 at 8:35 PM, Alexander Brovman <
alexander.brov...@gmail.com> wrote:
> Maxim thanks for the code, its interessting that the screenshot capture
> takes so long.. its surprising that the encode is faster than the actual
> capture, i am still researching methods on how to improve this, will share
> my findings with you.
>
>
> On Fri, Oct 25, 2013 at 9:57 AM, Maxim Solodovnik <solomax...@gmail.com>wrote:
>
>> The method should be changed as follows:
>>
>> private static final Logger log =
>> LoggerFactory.getLogger(CaptureScreen.class);
>> public void run() {
>> try {
>> while (active && !core.isReadyToRecord()) {
>> Thread.sleep(60);
>> }
>> Robot robot = new Robot();
>> BufferedImage image = null;
>> while (active) {
>> final long ctime = System.currentTimeMillis();
>> Rectangle screen = new Rectangle(spinnerX, spinnerY, spinnerWidth,
>> spinnerHeight);
>>
>> long start = System.currentTimeMillis();
>> image = robot.createScreenCapture(screen);
>> log.debug(String.format("Image was captured in %s ms",
>> System.currentTimeMillis() - start));
>> try {
>> timestamp += timeBetweenFrames;
>> start = System.currentTimeMillis();
>> byte[] data = se.encode(screen, image, new Rectangle(resizeX, resizeY));
>> log.debug(String.format("Image was encoded in %s ms",
>> System.currentTimeMillis() - start));
>>
>> pushVideo(data, timestamp);
>> } catch (Exception e) {
>> log.error("Error while encoding: ", e);
>> }
>> final int spent = (int) (System.currentTimeMillis() - ctime);
>>
>> log.debug(String.format("Time spent %s ms; diff: %s ms", spent,
>> timeBetweenFrames - spent));
>> if (sendCursor) {
>> core.sendCursorStatus();
>> }
>> Thread.sleep(Math.max(0, timeBetweenFrames - spent));
>> }
>> } catch (Exception e) {
>> log.error("Error while running: ", e);
>> }
>> }
>>
>> 1280x1024 are captured encoded for too long
>> Will now try to speed up encoding.
>>
>>
>> On Thu, Oct 24, 2013 at 8:35 PM, Alexander Brovman <
>> alexander.brov...@gmail.com> wrote:
>>
>>> Maxim,
>>>
>>> A) Interessting to say the very least, how did you manage to measure the
>>> encode and the screencapture? or what tools did you use in order to measure
>>> those, might come in handy for me when measuring performance.
>>>
>>> Right now i changed the sourcecode in the following way:
>>> 1) " timeBetweenFrames = (ScreenDimensions.quality ==
>>> ScreenQuality.VeryHigh) ? 30 : 100;"
>>>
>>>
>>> 2) public void run() {
>>> try {
>>> Robot robot = new Robot();
>>> BufferedImage image = null;
>>> while (active) {
>>> final long ctime =
>>> System.currentTimeMillis();
>>> Rectangle screen = new
>>> Rectangle(ScreenDimensions.spinnerX,
>>>
>>> ScreenDimensions.spinnerY,
>>>
>>> ScreenDimensions.spinnerWidth,
>>>
>>> ScreenDimensions.spinnerHeight);
>>>
>>> image =
>>> robot.createScreenCapture(screen);
>>>
>>> try {
>>> timestamp += timeBetweenFrames;
>>> byte[] data = se.encode(screen, image, new
>>> Rectangle(ScreenDimensions.resizeX,
>>>
>>> ScreenDimensions.resizeY));
>>>
>>> pushVideo(data, timestamp);
>>> } catch (Exception e) {
>>> e.printStackTrace();
>>> }
>>> final int spent = (int)
>>> (System.currentTimeMillis() - ctime);
>>>
>>> if (sendCursor) {
>>> core.sendCursorStatus();
>>> }
>>> Thread.sleep(Math.max(0,
>>> timeBetweenFrames - spent));
>>> }
>>> } catch (Exception e) {
>>> e.printStackTrace();
>>> }
>>> }
>>>
>>> I am definitely noticing a very strong improvement in terms of less
>>> video delay.
>>> using jnettop i can see that at a screen resolution of 1650x1080 it is
>>> approximately using between 4-8mbit for 1 user.
>>> So i guess the biggest issue here is the encoder, the encoder is
>>> definitely not efficient, i have been looking at solutions of other people
>>> on stackexchange in terms of algorithms.
>>>
>>> B) I have a question regarding the process, so if the screencapture
>>> takes place and is sent using the RTMP method/protocol to the server. Does
>>> the sever additionally encode the file again before sending it to the
>>> enduser? If so which javafiles contain those settings, because i would like
>>> to test if i can tweak them to reduce the delay even more by disabling the
>>> encode of the images/stream on the serverside and just send the files "as
>>> is". Although once again i fear the algo is the issue.
>>>
>>> Kind regards,
>>> Alexander
>>>
>>>
>>> On Thu, Oct 24, 2013 at 3:06 PM, Maxim Solodovnik
>>> <solomax...@gmail.com>wrote:
>>>
>>>> Here are my measurements:
>>>> Screen is captured in ~ 160-170ms
>>>> Encoded in ~120-200ms
>>>> Will try to speed up this somehow
>>>>
>>>>
>>>> On Thu, Oct 24, 2013 at 3:27 PM, Alexander Brovman <
>>>> alexander.brov...@gmail.com> wrote:
>>>>
>>>>> nevermind used the wrong ANT build.
>>>>>
>>>>> Got it to work now :)
>>>>>
>>>>>
>>>>> On Thu, Oct 24, 2013 at 9:11 AM, Alexander Brovman <
>>>>> alexander.brov...@gmail.com> wrote:
>>>>>
>>>>>> Maxim i made some changes to the code, i downloaded the source on the
>>>>>> server, changed the lines with nano. and then from the mainfolder simply
>>>>>> ran the ant command and it started compiling (albeit some errors appeared
>>>>>> since i only entered "ant" without any options/parameters.
>>>>>>
>>>>>> Openmeeting is working fine, i simply replaced the entire
>>>>>> screenshareing/ folder with the new one i just compiled... any
>>>>>> suggestions
>>>>>> on what im doing wrong here?
>>>>>>
>>>>>> I am getting this error:
>>>>>> Unsigned Entry in resouce: slf4j-api-1.6.4.jar
>>>>>>
>>>>>>
>>>>>> On Thu, Oct 24, 2013 at 8:17 AM, Maxim Solodovnik <
>>>>>> solomax...@gmail.com> wrote:
>>>>>>
>>>>>>> I'm currently trying to modify the code to get 20+ fps (was
>>>>>>> requested many times, but I had no time on this :( currently I'm on
>>>>>>> vacation, will try to implement it)
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Oct 24, 2013 at 10:02 AM, Alexander Brovman <
>>>>>>> alexander.brov...@gmail.com> wrote:
>>>>>>>
>>>>>>>> Maxim,
>>>>>>>>
>>>>>>>> Thank you for responding so quickly. The primary reason i asked is
>>>>>>>> because we sometimes need to output data like Financial Stock charts or
>>>>>>>> similar dynamic content and in a resolution size of 1650x1080. As you
>>>>>>>> can
>>>>>>>> imagine right now the delay is.. manageable, perhaps somewhere along
>>>>>>>> the
>>>>>>>> lines of 0.5-1 second delay id say.
>>>>>>>> We had an independant tested saying its an average of 130ms. But
>>>>>>>> for fast video/changes the framerate will sometimes drop or become
>>>>>>>> laggy.
>>>>>>>>
>>>>>>>> Ill try setting the timebetweenFrames to 30ms which in "ideal"
>>>>>>>> conditions should give me 33.333 frames per second of video.
>>>>>>>> As for the Thread.Sleep(XXX) not working or functioning i will
>>>>>>>> remove the thread.sleep(60) from the top,
>>>>>>>> As for the Thread.Sleep(xxx) settings i will simply remove it; BUT
>>>>>>>> i will leave in the "Thread.sleep(Math.max(0, timeBetweenFrames -
>>>>>>>> spent));"
>>>>>>>> since "worst" case scenario it will not be put to sleep at all since
>>>>>>>> the
>>>>>>>> output will be 0, and "best case" scenario the thread will be put to
>>>>>>>> sleep
>>>>>>>> for 30 - 1 ms depending on how the calculation turns out.
>>>>>>>>
>>>>>>>> Perhaps it would be good to leave somekind of thread sleep in there
>>>>>>>> that is a static factor, from a development standpoint i would rather
>>>>>>>> prefer having a fixed constant in there. rather than putting the
>>>>>>>> thread to
>>>>>>>> sleep (potentially twice). Either that or im not understanding the
>>>>>>>> reason
>>>>>>>> for the first thread.sleep (or the second one).
>>>>>>>> Either way i will make the changes now and compile, I will report
>>>>>>>> my findings here if you want me to :)
>>>>>>>>
>>>>>>>> @Sebastian,
>>>>>>>>
>>>>>>>> At the time of writing this i saw you respond, so i am writing this
>>>>>>>> short addendum:
>>>>>>>> 1) Could you please comment on whether or not Thread.Sleep(xxx)
>>>>>>>> works?
>>>>>>>> 2) 2-4 frames per second and less may be great for static
>>>>>>>> presentations but not so much for dynamic. I understand the concerns
>>>>>>>> raised
>>>>>>>> on your behalf regarding bandwidth usage but since we have a dedicated
>>>>>>>> Gigabit connection we have no issues with dedicating anywhere between
>>>>>>>> 2-5
>>>>>>>> Mbit per user. 20 Frames per second really is the minimum we are
>>>>>>>> looking
>>>>>>>> at, i think it would be great if you presented people with an
>>>>>>>> option/alternative to gotowebinar. I understand that applications which
>>>>>>>> compile into native code will always be faster and outperform in
>>>>>>>> certain
>>>>>>>> aspects, but when you have the motivation and perhaps the financial
>>>>>>>> resources to dedicate time and money into getting away from solutions
>>>>>>>> such
>>>>>>>> as GoToWebinar.
>>>>>>>>
>>>>>>>> Is there a way of "fixing" the position of the screen that the
>>>>>>>> viewers will see when they accept a desktop streaming broadcast from a
>>>>>>>> presenter? I ask this because i havent found a way to change the
>>>>>>>> location
>>>>>>>> of the Chat Bar inside a conference room yet (neither have I found a
>>>>>>>> way to
>>>>>>>> change the colors/themes to be honest, need to do some more google
>>>>>>>> searches).
>>>>>>>>
>>>>>>>> Kind regards,
>>>>>>>> Alexander
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Oct 24, 2013 at 4:47 AM, seba.wag...@gmail.com <
>>>>>>>> seba.wag...@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Do we still have the super high quality setting in the sharing
>>>>>>>>> client?
>>>>>>>>> My problems in the past was more around the bandwidth issues then
>>>>>>>>> the FPS.
>>>>>>>>> If you do 2-4 FPS the bandwidth usage will go that up... there is
>>>>>>>>> probably a trick, maybe there is also something todo in the way the
>>>>>>>>> tiles
>>>>>>>>> are calculated that need to be resend. It normally should only try to
>>>>>>>>> resend the ones that really change. So if you show an hd movie you
>>>>>>>>> might
>>>>>>>>> see your bandwidth needs go up quite a bit.
>>>>>>>>>
>>>>>>>>> Seb
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> 2013/10/24 Maxim Solodovnik <solomax...@gmail.com>
>>>>>>>>>
>>>>>>>>>> Hello Alexander,
>>>>>>>>>>
>>>>>>>>>> Your findings are correct :)
>>>>>>>>>> With the only exception: Screenshots are taken in single thread,
>>>>>>>>>> so there are some limitations in the speed :(
>>>>>>>>>> I believe Thread.sleep(xxx) doesn't work at all
>>>>>>>>>> I'm planning to rewrite this part And hopefully will be able to
>>>>>>>>>> increase FPS.
>>>>>>>>>>
>>>>>>>>>> Basic compilation steps are here:
>>>>>>>>>> http://openmeetings.apache.org/BuildInstructions.html
>>>>>>>>>> After compiling all jars in screensharing folders are being
>>>>>>>>>> signed, so you need to replace all of them, otherwise you will get
>>>>>>>>>> security
>>>>>>>>>> error due to different certificates
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Thu, Oct 24, 2013 at 3:07 AM, Alexander Brovman <
>>>>>>>>>> alexander.brov...@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> I have two quick questions, but first of all i just wanted to
>>>>>>>>>>> say that Openmeetings is fantastic. I had both OpenMeetings and BBB
>>>>>>>>>>> Installed and we have decided to go with OM.
>>>>>>>>>>>
>>>>>>>>>>> I am relatively apt at programming with Python and C# but I have
>>>>>>>>>>> never worked with a lot of Javaapplications like these ones, so i
>>>>>>>>>>> have the
>>>>>>>>>>> following questions:
>>>>>>>>>>>
>>>>>>>>>>> *1)* From what i understand the ScreenSharing is performed by
>>>>>>>>>>> taking snapshots every x milliseconds. Additionally there is an
>>>>>>>>>>> algorithm
>>>>>>>>>>> in place that divides the snapshot into tiles and measures the
>>>>>>>>>>> changes in
>>>>>>>>>>> tiles, if there is a change the tile is uploaded. If this is
>>>>>>>>>>> correct, and i
>>>>>>>>>>> want a more.. "fluid" video because there will be a lot of changes,
>>>>>>>>>>> cant i
>>>>>>>>>>> simply modify the "framerate" at which the images will be compared
>>>>>>>>>>> to
>>>>>>>>>>> improve the output? I am well aware that this would eat into the
>>>>>>>>>>> CPU and
>>>>>>>>>>> the bandwidth but i have a Full Duplex Gbit and 2x Quadcore Xeons
>>>>>>>>>>> available
>>>>>>>>>>> to me so this is not an issue. We are trying to test for an extreme
>>>>>>>>>>> case of
>>>>>>>>>>> 500-1500 users and how to optimize the output, i think the
>>>>>>>>>>> developers and
>>>>>>>>>>> other users will be interested in seeing the results.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> *2)* Where would these changes need to be made? So far i have
>>>>>>>>>>> isolated the CoreScreen.java , CaptureScreen.java and
>>>>>>>>>>> theScreenV1Encoder.java files that apparently contain the code which
>>>>>>>>>>> processes the desktop sharing.
>>>>>>>>>>>
>>>>>>>>>>> The CaptureScreen.Java has the:
>>>>>>>>>>> "timeBetweenFrames = (ScreenDimensions.quality ==
>>>>>>>>>>> ScreenQuality.VeryHigh) ? 100 : 500;" setting on line 54 which
>>>>>>>>>>> basically
>>>>>>>>>>> says that if Screequality is set to very high (condition is true)
>>>>>>>>>>> then
>>>>>>>>>>> select 100 as the value for timeBetweenFrames.
>>>>>>>>>>>
>>>>>>>>>>> This variable is apparently used lateron in the
>>>>>>>>>>> "Thread.sleep(Math.max(0, timeBetweenFrames - spent));" on line 96,
>>>>>>>>>>> which
>>>>>>>>>>> will pause the current loop for the amountof time that is the
>>>>>>>>>>> difference
>>>>>>>>>>> between timeBetweenFrames( in our case 100ms) minus the spent time
>>>>>>>>>>> (which
>>>>>>>>>>> is defined on line 91).
>>>>>>>>>>>
>>>>>>>>>>> So in Theory if i wanted to output the screen in realtime at 20
>>>>>>>>>>> or 24 frames per second which would mean that a snapshot would need
>>>>>>>>>>> to be
>>>>>>>>>>> taken every 50 milliseconds or so, would i simply need to set a
>>>>>>>>>>> different
>>>>>>>>>>> frame setting for the time between frames, something like:
>>>>>>>>>>> "timeBetweenFrames = (ScreenDimensions.quality ==
>>>>>>>>>>> ScreenQuality.VeryHigh) ? *20* : 500;"
>>>>>>>>>>>
>>>>>>>>>>> Reduce the Thread.sleep(60); on line 60 to something like 20
>>>>>>>>>>> milliseconds so (Thread.sleep(20);)
>>>>>>>>>>> and then finally remove the the last part
>>>>>>>>>>> "Thread.sleep(Math.max(0, timeBetweenFrames - spent));" or perhaps
>>>>>>>>>>> leaving
>>>>>>>>>>> it in?
>>>>>>>>>>>
>>>>>>>>>>> I understand that this would be a .. "wrench and hammer"
>>>>>>>>>>> approach since this will probably eat up quite a bit of CPU usage.
>>>>>>>>>>> I mean
>>>>>>>>>>> the simpler solution here clearly would be to get rid of the
>>>>>>>>>>> algorithm
>>>>>>>>>>> itself and simply rewrite it to simply output a screenshot every x
>>>>>>>>>>> milliseconds or so, but i have no idea how deep i would be messing
>>>>>>>>>>> with
>>>>>>>>>>> openmeetings and if that would/could break other parts of
>>>>>>>>>>> openmeeting that
>>>>>>>>>>> access these files?
>>>>>>>>>>>
>>>>>>>>>>> So would this be a workable solution that i could at least try
>>>>>>>>>>> out? I would ofcourse be more than happy to supply the CPU Usage
>>>>>>>>>>> differences.
>>>>>>>>>>>
>>>>>>>>>>> *3)* My only question would be is how would i recompile these?
>>>>>>>>>>> I am asking because i read somewhere in the mailinglists (or the
>>>>>>>>>>> old google
>>>>>>>>>>> groups) that it is recommended to use ANT in order to recompile
>>>>>>>>>>> things.
>>>>>>>>>>> Also what will be the output name of the .jar that is essentially
>>>>>>>>>>> being
>>>>>>>>>>> loaded for people and located on the server?
>>>>>>>>>>> Because all i can find on my Debian box is
>>>>>>>>>>> the
>>>>>>>>>>> /usr/lib/red5/webapps/openmeetings/screensharing/openmeetings-screenshare-2.1.1-RELEASE.jar
>>>>>>>>>>> file.
>>>>>>>>>>>
>>>>>>>>>>> So yeah, if somebody could help me with recompiling i would
>>>>>>>>>>> appreciate it! :)
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> WBR
>>>>>>>>>> Maxim aka solomax
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Sebastian Wagner
>>>>>>>>> https://twitter.com/#!/dead_lock
>>>>>>>>> http://www.webbase-design.de
>>>>>>>>> http://www.wagner-sebastian.com
>>>>>>>>> seba.wag...@gmail.com
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> WBR
>>>>>>> Maxim aka solomax
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> WBR
>>>> Maxim aka solomax
>>>>
>>>
>>>
>>
>>
>> --
>> WBR
>> Maxim aka solomax
>>
>
>
--
WBR
Maxim aka solomax
