That's great! Maxim you rule! :-) -- With best regards / с наилучшими пожеланиями, Alexei Fedotov / Алексей Федотов, http://dataved.ru/ +7 916 562 8095
[1] Start using Apache Openmeetings today, http://openmeetings.apache.org/ [2] Join Alexei Fedotov @linkedin, http://ru.linkedin.com/in/dataved/ [3] Join Alexei Fedotov @facebook, http://www.facebook.com/openmeetings On Sat, Nov 23, 2013 at 10:15 PM, Maxim Solodovnik <solomax...@gmail.com> wrote: > Hello Alexander, > > I seems to was able to enhance performance of screen sharing applet: > VeryHigh > 10 fps > High = 10fps > others = 2fps > > Committed revision 1544855 > > > On Fri, Oct 25, 2013 at 8:35 PM, Alexander Brovman > <alexander.brov...@gmail.com> wrote: >> >> Maxim thanks for the code, its interessting that the screenshot capture >> takes so long.. its surprising that the encode is faster than the actual >> capture, i am still researching methods on how to improve this, will share >> my findings with you. >> >> >> On Fri, Oct 25, 2013 at 9:57 AM, Maxim Solodovnik <solomax...@gmail.com> >> wrote: >>> >>> The method should be changed as follows: >>> >>> private static final Logger log = >>> LoggerFactory.getLogger(CaptureScreen.class); >>> public void run() { >>> try { >>> while (active && !core.isReadyToRecord()) { >>> Thread.sleep(60); >>> } >>> Robot robot = new Robot(); >>> BufferedImage image = null; >>> while (active) { >>> final long ctime = System.currentTimeMillis(); >>> Rectangle screen = new Rectangle(spinnerX, spinnerY, spinnerWidth, >>> spinnerHeight); >>> >>> long start = System.currentTimeMillis(); >>> image = robot.createScreenCapture(screen); >>> log.debug(String.format("Image was captured in %s ms", >>> System.currentTimeMillis() - start)); >>> try { >>> timestamp += timeBetweenFrames; >>> start = System.currentTimeMillis(); >>> byte[] data = se.encode(screen, image, new Rectangle(resizeX, resizeY)); >>> log.debug(String.format("Image was encoded in %s ms", >>> System.currentTimeMillis() - start)); >>> >>> pushVideo(data, timestamp); >>> } catch (Exception e) { >>> log.error("Error while encoding: ", e); >>> } >>> final int spent = (int) (System.currentTimeMillis() - ctime); >>> >>> log.debug(String.format("Time spent %s ms; diff: %s ms", spent, >>> timeBetweenFrames - spent)); >>> if (sendCursor) { >>> core.sendCursorStatus(); >>> } >>> Thread.sleep(Math.max(0, timeBetweenFrames - spent)); >>> } >>> } catch (Exception e) { >>> log.error("Error while running: ", e); >>> } >>> } >>> >>> 1280x1024 are captured encoded for too long >>> Will now try to speed up encoding. >>> >>> >>> On Thu, Oct 24, 2013 at 8:35 PM, Alexander Brovman >>> <alexander.brov...@gmail.com> wrote: >>>> >>>> Maxim, >>>> >>>> A) Interessting to say the very least, how did you manage to measure the >>>> encode and the screencapture? or what tools did you use in order to measure >>>> those, might come in handy for me when measuring performance. >>>> >>>> Right now i changed the sourcecode in the following way: >>>> 1) " timeBetweenFrames = (ScreenDimensions.quality == >>>> ScreenQuality.VeryHigh) ? 30 : 100;" >>>> >>>> >>>> 2) public void run() { >>>> try { >>>> Robot robot = new Robot(); >>>> BufferedImage image = null; >>>> while (active) { >>>> final long ctime = >>>> System.currentTimeMillis(); >>>> Rectangle screen = new >>>> Rectangle(ScreenDimensions.spinnerX, >>>> >>>> ScreenDimensions.spinnerY, >>>> >>>> ScreenDimensions.spinnerWidth, >>>> >>>> ScreenDimensions.spinnerHeight); >>>> >>>> image = >>>> robot.createScreenCapture(screen); >>>> >>>> try { >>>> timestamp += timeBetweenFrames; >>>> byte[] data = se.encode(screen, image, new >>>> Rectangle(ScreenDimensions.resizeX, >>>> >>>> ScreenDimensions.resizeY)); >>>> >>>> pushVideo(data, timestamp); >>>> } catch (Exception e) { >>>> e.printStackTrace(); >>>> } >>>> final int spent = (int) >>>> (System.currentTimeMillis() - ctime); >>>> >>>> if (sendCursor) { >>>> core.sendCursorStatus(); >>>> } >>>> Thread.sleep(Math.max(0, >>>> timeBetweenFrames - spent)); >>>> } >>>> } catch (Exception e) { >>>> e.printStackTrace(); >>>> } >>>> } >>>> >>>> I am definitely noticing a very strong improvement in terms of less >>>> video delay. >>>> using jnettop i can see that at a screen resolution of 1650x1080 it is >>>> approximately using between 4-8mbit for 1 user. >>>> So i guess the biggest issue here is the encoder, the encoder is >>>> definitely not efficient, i have been looking at solutions of other people >>>> on stackexchange in terms of algorithms. >>>> >>>> B) I have a question regarding the process, so if the screencapture >>>> takes place and is sent using the RTMP method/protocol to the server. Does >>>> the sever additionally encode the file again before sending it to the >>>> enduser? If so which javafiles contain those settings, because i would like >>>> to test if i can tweak them to reduce the delay even more by disabling the >>>> encode of the images/stream on the serverside and just send the files "as >>>> is". Although once again i fear the algo is the issue. >>>> >>>> Kind regards, >>>> Alexander >>>> >>>> >>>> On Thu, Oct 24, 2013 at 3:06 PM, Maxim Solodovnik <solomax...@gmail.com> >>>> wrote: >>>>> >>>>> Here are my measurements: >>>>> Screen is captured in ~ 160-170ms >>>>> Encoded in ~120-200ms >>>>> Will try to speed up this somehow >>>>> >>>>> >>>>> On Thu, Oct 24, 2013 at 3:27 PM, Alexander Brovman >>>>> <alexander.brov...@gmail.com> wrote: >>>>>> >>>>>> nevermind used the wrong ANT build. >>>>>> >>>>>> Got it to work now :) >>>>>> >>>>>> >>>>>> On Thu, Oct 24, 2013 at 9:11 AM, Alexander Brovman >>>>>> <alexander.brov...@gmail.com> wrote: >>>>>>> >>>>>>> Maxim i made some changes to the code, i downloaded the source on the >>>>>>> server, changed the lines with nano. and then from the mainfolder >>>>>>> simply ran >>>>>>> the ant command and it started compiling (albeit some errors appeared >>>>>>> since >>>>>>> i only entered "ant" without any options/parameters. >>>>>>> >>>>>>> Openmeeting is working fine, i simply replaced the entire >>>>>>> screenshareing/ folder with the new one i just compiled... any >>>>>>> suggestions >>>>>>> on what im doing wrong here? >>>>>>> >>>>>>> I am getting this error: >>>>>>> Unsigned Entry in resouce: slf4j-api-1.6.4.jar >>>>>>> >>>>>>> >>>>>>> On Thu, Oct 24, 2013 at 8:17 AM, Maxim Solodovnik >>>>>>> <solomax...@gmail.com> wrote: >>>>>>>> >>>>>>>> I'm currently trying to modify the code to get 20+ fps (was >>>>>>>> requested many times, but I had no time on this :( currently I'm on >>>>>>>> vacation, will try to implement it) >>>>>>>> >>>>>>>> >>>>>>>> On Thu, Oct 24, 2013 at 10:02 AM, Alexander Brovman >>>>>>>> <alexander.brov...@gmail.com> wrote: >>>>>>>>> >>>>>>>>> Maxim, >>>>>>>>> >>>>>>>>> Thank you for responding so quickly. The primary reason i asked is >>>>>>>>> because we sometimes need to output data like Financial Stock charts >>>>>>>>> or >>>>>>>>> similar dynamic content and in a resolution size of 1650x1080. As you >>>>>>>>> can >>>>>>>>> imagine right now the delay is.. manageable, perhaps somewhere along >>>>>>>>> the >>>>>>>>> lines of 0.5-1 second delay id say. >>>>>>>>> We had an independant tested saying its an average of 130ms. But >>>>>>>>> for fast video/changes the framerate will sometimes drop or become >>>>>>>>> laggy. >>>>>>>>> >>>>>>>>> Ill try setting the timebetweenFrames to 30ms which in "ideal" >>>>>>>>> conditions should give me 33.333 frames per second of video. >>>>>>>>> As for the Thread.Sleep(XXX) not working or functioning i will >>>>>>>>> remove the thread.sleep(60) from the top, >>>>>>>>> As for the Thread.Sleep(xxx) settings i will simply remove it; BUT >>>>>>>>> i will leave in the "Thread.sleep(Math.max(0, timeBetweenFrames - >>>>>>>>> spent));" >>>>>>>>> since "worst" case scenario it will not be put to sleep at all since >>>>>>>>> the >>>>>>>>> output will be 0, and "best case" scenario the thread will be put to >>>>>>>>> sleep >>>>>>>>> for 30 - 1 ms depending on how the calculation turns out. >>>>>>>>> >>>>>>>>> Perhaps it would be good to leave somekind of thread sleep in there >>>>>>>>> that is a static factor, from a development standpoint i would rather >>>>>>>>> prefer >>>>>>>>> having a fixed constant in there. rather than putting the thread to >>>>>>>>> sleep >>>>>>>>> (potentially twice). Either that or im not understanding the reason >>>>>>>>> for the >>>>>>>>> first thread.sleep (or the second one). >>>>>>>>> Either way i will make the changes now and compile, I will report >>>>>>>>> my findings here if you want me to :) >>>>>>>>> >>>>>>>>> @Sebastian, >>>>>>>>> >>>>>>>>> At the time of writing this i saw you respond, so i am writing this >>>>>>>>> short addendum: >>>>>>>>> 1) Could you please comment on whether or not Thread.Sleep(xxx) >>>>>>>>> works? >>>>>>>>> 2) 2-4 frames per second and less may be great for static >>>>>>>>> presentations but not so much for dynamic. I understand the concerns >>>>>>>>> raised >>>>>>>>> on your behalf regarding bandwidth usage but since we have a dedicated >>>>>>>>> Gigabit connection we have no issues with dedicating anywhere between >>>>>>>>> 2-5 >>>>>>>>> Mbit per user. 20 Frames per second really is the minimum we are >>>>>>>>> looking at, >>>>>>>>> i think it would be great if you presented people with an >>>>>>>>> option/alternative >>>>>>>>> to gotowebinar. I understand that applications which compile into >>>>>>>>> native >>>>>>>>> code will always be faster and outperform in certain aspects, but >>>>>>>>> when you >>>>>>>>> have the motivation and perhaps the financial resources to dedicate >>>>>>>>> time and >>>>>>>>> money into getting away from solutions such as GoToWebinar. >>>>>>>>> >>>>>>>>> Is there a way of "fixing" the position of the screen that the >>>>>>>>> viewers will see when they accept a desktop streaming broadcast from a >>>>>>>>> presenter? I ask this because i havent found a way to change the >>>>>>>>> location of >>>>>>>>> the Chat Bar inside a conference room yet (neither have I found a way >>>>>>>>> to >>>>>>>>> change the colors/themes to be honest, need to do some more google >>>>>>>>> searches). >>>>>>>>> >>>>>>>>> Kind regards, >>>>>>>>> Alexander >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Thu, Oct 24, 2013 at 4:47 AM, seba.wag...@gmail.com >>>>>>>>> <seba.wag...@gmail.com> wrote: >>>>>>>>>> >>>>>>>>>> Do we still have the super high quality setting in the sharing >>>>>>>>>> client? >>>>>>>>>> My problems in the past was more around the bandwidth issues then >>>>>>>>>> the FPS. >>>>>>>>>> If you do 2-4 FPS the bandwidth usage will go that up... there is >>>>>>>>>> probably a trick, maybe there is also something todo in the way the >>>>>>>>>> tiles >>>>>>>>>> are calculated that need to be resend. It normally should only try >>>>>>>>>> to resend >>>>>>>>>> the ones that really change. So if you show an hd movie you might >>>>>>>>>> see your >>>>>>>>>> bandwidth needs go up quite a bit. >>>>>>>>>> >>>>>>>>>> Seb >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> 2013/10/24 Maxim Solodovnik <solomax...@gmail.com> >>>>>>>>>>> >>>>>>>>>>> Hello Alexander, >>>>>>>>>>> >>>>>>>>>>> Your findings are correct :) >>>>>>>>>>> With the only exception: Screenshots are taken in single thread, >>>>>>>>>>> so there are some limitations in the speed :( >>>>>>>>>>> I believe Thread.sleep(xxx) doesn't work at all >>>>>>>>>>> I'm planning to rewrite this part And hopefully will be able to >>>>>>>>>>> increase FPS. >>>>>>>>>>> >>>>>>>>>>> Basic compilation steps are here: >>>>>>>>>>> http://openmeetings.apache.org/BuildInstructions.html >>>>>>>>>>> After compiling all jars in screensharing folders are being >>>>>>>>>>> signed, so you need to replace all of them, otherwise you will get >>>>>>>>>>> security >>>>>>>>>>> error due to different certificates >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Thu, Oct 24, 2013 at 3:07 AM, Alexander Brovman >>>>>>>>>>> <alexander.brov...@gmail.com> wrote: >>>>>>>>>>>> >>>>>>>>>>>> Hi, >>>>>>>>>>>> >>>>>>>>>>>> I have two quick questions, but first of all i just wanted to >>>>>>>>>>>> say that Openmeetings is fantastic. I had both OpenMeetings and BBB >>>>>>>>>>>> Installed and we have decided to go with OM. >>>>>>>>>>>> >>>>>>>>>>>> I am relatively apt at programming with Python and C# but I have >>>>>>>>>>>> never worked with a lot of Javaapplications like these ones, so i >>>>>>>>>>>> have the >>>>>>>>>>>> following questions: >>>>>>>>>>>> >>>>>>>>>>>> 1) From what i understand the ScreenSharing is performed by >>>>>>>>>>>> taking snapshots every x milliseconds. Additionally there is an >>>>>>>>>>>> algorithm in >>>>>>>>>>>> place that divides the snapshot into tiles and measures the >>>>>>>>>>>> changes in >>>>>>>>>>>> tiles, if there is a change the tile is uploaded. If this is >>>>>>>>>>>> correct, and i >>>>>>>>>>>> want a more.. "fluid" video because there will be a lot of >>>>>>>>>>>> changes, cant i >>>>>>>>>>>> simply modify the "framerate" at which the images will be compared >>>>>>>>>>>> to >>>>>>>>>>>> improve the output? I am well aware that this would eat into the >>>>>>>>>>>> CPU and the >>>>>>>>>>>> bandwidth but i have a Full Duplex Gbit and 2x Quadcore Xeons >>>>>>>>>>>> available to >>>>>>>>>>>> me so this is not an issue. We are trying to test for an extreme >>>>>>>>>>>> case of >>>>>>>>>>>> 500-1500 users and how to optimize the output, i think the >>>>>>>>>>>> developers and >>>>>>>>>>>> other users will be interested in seeing the results. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> 2) Where would these changes need to be made? So far i have >>>>>>>>>>>> isolated the CoreScreen.java , CaptureScreen.java and >>>>>>>>>>>> theScreenV1Encoder.java files that apparently contain the code >>>>>>>>>>>> which >>>>>>>>>>>> processes the desktop sharing. >>>>>>>>>>>> >>>>>>>>>>>> The CaptureScreen.Java has the: >>>>>>>>>>>> "timeBetweenFrames = (ScreenDimensions.quality == >>>>>>>>>>>> ScreenQuality.VeryHigh) ? 100 : 500;" setting on line 54 which >>>>>>>>>>>> basically >>>>>>>>>>>> says that if Screequality is set to very high (condition is true) >>>>>>>>>>>> then >>>>>>>>>>>> select 100 as the value for timeBetweenFrames. >>>>>>>>>>>> >>>>>>>>>>>> This variable is apparently used lateron in the >>>>>>>>>>>> "Thread.sleep(Math.max(0, timeBetweenFrames - spent));" on line >>>>>>>>>>>> 96, which >>>>>>>>>>>> will pause the current loop for the amountof time that is the >>>>>>>>>>>> difference >>>>>>>>>>>> between timeBetweenFrames( in our case 100ms) minus the spent time >>>>>>>>>>>> (which is >>>>>>>>>>>> defined on line 91). >>>>>>>>>>>> >>>>>>>>>>>> So in Theory if i wanted to output the screen in realtime at 20 >>>>>>>>>>>> or 24 frames per second which would mean that a snapshot would >>>>>>>>>>>> need to be >>>>>>>>>>>> taken every 50 milliseconds or so, would i simply need to set a >>>>>>>>>>>> different >>>>>>>>>>>> frame setting for the time between frames, something like: >>>>>>>>>>>> "timeBetweenFrames = (ScreenDimensions.quality == >>>>>>>>>>>> ScreenQuality.VeryHigh) ? 20 : 500;" >>>>>>>>>>>> >>>>>>>>>>>> Reduce the Thread.sleep(60); on line 60 to something like 20 >>>>>>>>>>>> milliseconds so (Thread.sleep(20);) >>>>>>>>>>>> and then finally remove the the last part >>>>>>>>>>>> "Thread.sleep(Math.max(0, timeBetweenFrames - spent));" or perhaps >>>>>>>>>>>> leaving >>>>>>>>>>>> it in? >>>>>>>>>>>> >>>>>>>>>>>> I understand that this would be a .. "wrench and hammer" >>>>>>>>>>>> approach since this will probably eat up quite a bit of CPU usage. >>>>>>>>>>>> I mean >>>>>>>>>>>> the simpler solution here clearly would be to get rid of the >>>>>>>>>>>> algorithm >>>>>>>>>>>> itself and simply rewrite it to simply output a screenshot every x >>>>>>>>>>>> milliseconds or so, but i have no idea how deep i would be messing >>>>>>>>>>>> with >>>>>>>>>>>> openmeetings and if that would/could break other parts of >>>>>>>>>>>> openmeeting that >>>>>>>>>>>> access these files? >>>>>>>>>>>> >>>>>>>>>>>> So would this be a workable solution that i could at least try >>>>>>>>>>>> out? I would ofcourse be more than happy to supply the CPU Usage >>>>>>>>>>>> differences. >>>>>>>>>>>> >>>>>>>>>>>> 3) My only question would be is how would i recompile these? I >>>>>>>>>>>> am asking because i read somewhere in the mailinglists (or the old >>>>>>>>>>>> google >>>>>>>>>>>> groups) that it is recommended to use ANT in order to recompile >>>>>>>>>>>> things. Also >>>>>>>>>>>> what will be the output name of the .jar that is essentially being >>>>>>>>>>>> loaded >>>>>>>>>>>> for people and located on the server? >>>>>>>>>>>> Because all i can find on my Debian box is the >>>>>>>>>>>> /usr/lib/red5/webapps/openmeetings/screensharing/openmeetings-screenshare-2.1.1-RELEASE.jar >>>>>>>>>>>> file. >>>>>>>>>>>> >>>>>>>>>>>> So yeah, if somebody could help me with recompiling i would >>>>>>>>>>>> appreciate it! :) >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> WBR >>>>>>>>>>> Maxim aka solomax >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Sebastian Wagner >>>>>>>>>> https://twitter.com/#!/dead_lock >>>>>>>>>> http://www.webbase-design.de >>>>>>>>>> http://www.wagner-sebastian.com >>>>>>>>>> seba.wag...@gmail.com >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> WBR >>>>>>>> Maxim aka solomax >>>>>>> >>>>>>> >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> WBR >>>>> Maxim aka solomax >>>> >>>> >>> >>> >>> >>> -- >>> WBR >>> Maxim aka solomax >> >> > > > > -- > WBR > Maxim aka solomax
