Maxim thanks for the code, its interessting that the screenshot capture
takes so long.. its surprising that the encode is faster than the actual
capture, i am still researching methods on how to improve this, will share
my findings with you.
On Fri, Oct 25, 2013 at 9:57 AM, Maxim Solodovnik <solomax...@gmail.com>wrote:
> The method should be changed as follows:
>
> private static final Logger log =
> LoggerFactory.getLogger(CaptureScreen.class);
> public void run() {
> try {
> while (active && !core.isReadyToRecord()) {
> Thread.sleep(60);
> }
> Robot robot = new Robot();
> BufferedImage image = null;
> while (active) {
> final long ctime = System.currentTimeMillis();
> Rectangle screen = new Rectangle(spinnerX, spinnerY, spinnerWidth,
> spinnerHeight);
>
> long start = System.currentTimeMillis();
> image = robot.createScreenCapture(screen);
> log.debug(String.format("Image was captured in %s ms",
> System.currentTimeMillis() - start));
> try {
> timestamp += timeBetweenFrames;
> start = System.currentTimeMillis();
> byte[] data = se.encode(screen, image, new Rectangle(resizeX, resizeY));
> log.debug(String.format("Image was encoded in %s ms",
> System.currentTimeMillis() - start));
>
> pushVideo(data, timestamp);
> } catch (Exception e) {
> log.error("Error while encoding: ", e);
> }
> final int spent = (int) (System.currentTimeMillis() - ctime);
>
> log.debug(String.format("Time spent %s ms; diff: %s ms", spent,
> timeBetweenFrames - spent));
> if (sendCursor) {
> core.sendCursorStatus();
> }
> Thread.sleep(Math.max(0, timeBetweenFrames - spent));
> }
> } catch (Exception e) {
> log.error("Error while running: ", e);
> }
> }
>
> 1280x1024 are captured encoded for too long
> Will now try to speed up encoding.
>
>
> On Thu, Oct 24, 2013 at 8:35 PM, Alexander Brovman <
> alexander.brov...@gmail.com> wrote:
>
>> Maxim,
>>
>> A) Interessting to say the very least, how did you manage to measure the
>> encode and the screencapture? or what tools did you use in order to measure
>> those, might come in handy for me when measuring performance.
>>
>> Right now i changed the sourcecode in the following way:
>> 1) " timeBetweenFrames = (ScreenDimensions.quality ==
>> ScreenQuality.VeryHigh) ? 30 : 100;"
>>
>>
>> 2) public void run() {
>> try {
>> Robot robot = new Robot();
>> BufferedImage image = null;
>> while (active) {
>> final long ctime =
>> System.currentTimeMillis();
>> Rectangle screen = new
>> Rectangle(ScreenDimensions.spinnerX,
>> ScreenDimensions.spinnerY,
>>
>> ScreenDimensions.spinnerWidth,
>>
>> ScreenDimensions.spinnerHeight);
>>
>> image = robot.createScreenCapture(screen);
>>
>> try {
>> timestamp += timeBetweenFrames;
>> byte[] data = se.encode(screen, image, new
>> Rectangle(ScreenDimensions.resizeX,
>>
>> ScreenDimensions.resizeY));
>>
>> pushVideo(data, timestamp);
>> } catch (Exception e) {
>> e.printStackTrace();
>> }
>> final int spent = (int)
>> (System.currentTimeMillis() - ctime);
>>
>> if (sendCursor) {
>> core.sendCursorStatus();
>> }
>> Thread.sleep(Math.max(0,
>> timeBetweenFrames - spent));
>> }
>> } catch (Exception e) {
>> e.printStackTrace();
>> }
>> }
>>
>> I am definitely noticing a very strong improvement in terms of less video
>> delay.
>> using jnettop i can see that at a screen resolution of 1650x1080 it is
>> approximately using between 4-8mbit for 1 user.
>> So i guess the biggest issue here is the encoder, the encoder is
>> definitely not efficient, i have been looking at solutions of other people
>> on stackexchange in terms of algorithms.
>>
>> B) I have a question regarding the process, so if the screencapture takes
>> place and is sent using the RTMP method/protocol to the server. Does the
>> sever additionally encode the file again before sending it to the enduser?
>> If so which javafiles contain those settings, because i would like to test
>> if i can tweak them to reduce the delay even more by disabling the encode
>> of the images/stream on the serverside and just send the files "as is".
>> Although once again i fear the algo is the issue.
>>
>> Kind regards,
>> Alexander
>>
>>
>> On Thu, Oct 24, 2013 at 3:06 PM, Maxim Solodovnik
>> <solomax...@gmail.com>wrote:
>>
>>> Here are my measurements:
>>> Screen is captured in ~ 160-170ms
>>> Encoded in ~120-200ms
>>> Will try to speed up this somehow
>>>
>>>
>>> On Thu, Oct 24, 2013 at 3:27 PM, Alexander Brovman <
>>> alexander.brov...@gmail.com> wrote:
>>>
>>>> nevermind used the wrong ANT build.
>>>>
>>>> Got it to work now :)
>>>>
>>>>
>>>> On Thu, Oct 24, 2013 at 9:11 AM, Alexander Brovman <
>>>> alexander.brov...@gmail.com> wrote:
>>>>
>>>>> Maxim i made some changes to the code, i downloaded the source on the
>>>>> server, changed the lines with nano. and then from the mainfolder simply
>>>>> ran the ant command and it started compiling (albeit some errors appeared
>>>>> since i only entered "ant" without any options/parameters.
>>>>>
>>>>> Openmeeting is working fine, i simply replaced the entire
>>>>> screenshareing/ folder with the new one i just compiled... any suggestions
>>>>> on what im doing wrong here?
>>>>>
>>>>> I am getting this error:
>>>>> Unsigned Entry in resouce: slf4j-api-1.6.4.jar
>>>>>
>>>>>
>>>>> On Thu, Oct 24, 2013 at 8:17 AM, Maxim Solodovnik <
>>>>> solomax...@gmail.com> wrote:
>>>>>
>>>>>> I'm currently trying to modify the code to get 20+ fps (was requested
>>>>>> many times, but I had no time on this :( currently I'm on vacation, will
>>>>>> try to implement it)
>>>>>>
>>>>>>
>>>>>> On Thu, Oct 24, 2013 at 10:02 AM, Alexander Brovman <
>>>>>> alexander.brov...@gmail.com> wrote:
>>>>>>
>>>>>>> Maxim,
>>>>>>>
>>>>>>> Thank you for responding so quickly. The primary reason i asked is
>>>>>>> because we sometimes need to output data like Financial Stock charts or
>>>>>>> similar dynamic content and in a resolution size of 1650x1080. As you
>>>>>>> can
>>>>>>> imagine right now the delay is.. manageable, perhaps somewhere along the
>>>>>>> lines of 0.5-1 second delay id say.
>>>>>>> We had an independant tested saying its an average of 130ms. But for
>>>>>>> fast video/changes the framerate will sometimes drop or become laggy.
>>>>>>>
>>>>>>> Ill try setting the timebetweenFrames to 30ms which in "ideal"
>>>>>>> conditions should give me 33.333 frames per second of video.
>>>>>>> As for the Thread.Sleep(XXX) not working or functioning i will
>>>>>>> remove the thread.sleep(60) from the top,
>>>>>>> As for the Thread.Sleep(xxx) settings i will simply remove it; BUT i
>>>>>>> will leave in the "Thread.sleep(Math.max(0, timeBetweenFrames -
>>>>>>> spent));"
>>>>>>> since "worst" case scenario it will not be put to sleep at all since the
>>>>>>> output will be 0, and "best case" scenario the thread will be put to
>>>>>>> sleep
>>>>>>> for 30 - 1 ms depending on how the calculation turns out.
>>>>>>>
>>>>>>> Perhaps it would be good to leave somekind of thread sleep in there
>>>>>>> that is a static factor, from a development standpoint i would rather
>>>>>>> prefer having a fixed constant in there. rather than putting the thread
>>>>>>> to
>>>>>>> sleep (potentially twice). Either that or im not understanding the
>>>>>>> reason
>>>>>>> for the first thread.sleep (or the second one).
>>>>>>> Either way i will make the changes now and compile, I will report my
>>>>>>> findings here if you want me to :)
>>>>>>>
>>>>>>> @Sebastian,
>>>>>>>
>>>>>>> At the time of writing this i saw you respond, so i am writing this
>>>>>>> short addendum:
>>>>>>> 1) Could you please comment on whether or not Thread.Sleep(xxx)
>>>>>>> works?
>>>>>>> 2) 2-4 frames per second and less may be great for static
>>>>>>> presentations but not so much for dynamic. I understand the concerns
>>>>>>> raised
>>>>>>> on your behalf regarding bandwidth usage but since we have a dedicated
>>>>>>> Gigabit connection we have no issues with dedicating anywhere between
>>>>>>> 2-5
>>>>>>> Mbit per user. 20 Frames per second really is the minimum we are looking
>>>>>>> at, i think it would be great if you presented people with an
>>>>>>> option/alternative to gotowebinar. I understand that applications which
>>>>>>> compile into native code will always be faster and outperform in certain
>>>>>>> aspects, but when you have the motivation and perhaps the financial
>>>>>>> resources to dedicate time and money into getting away from solutions
>>>>>>> such
>>>>>>> as GoToWebinar.
>>>>>>>
>>>>>>> Is there a way of "fixing" the position of the screen that the
>>>>>>> viewers will see when they accept a desktop streaming broadcast from a
>>>>>>> presenter? I ask this because i havent found a way to change the
>>>>>>> location
>>>>>>> of the Chat Bar inside a conference room yet (neither have I found a
>>>>>>> way to
>>>>>>> change the colors/themes to be honest, need to do some more google
>>>>>>> searches).
>>>>>>>
>>>>>>> Kind regards,
>>>>>>> Alexander
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Oct 24, 2013 at 4:47 AM, seba.wag...@gmail.com <
>>>>>>> seba.wag...@gmail.com> wrote:
>>>>>>>
>>>>>>>> Do we still have the super high quality setting in the sharing
>>>>>>>> client?
>>>>>>>> My problems in the past was more around the bandwidth issues then
>>>>>>>> the FPS.
>>>>>>>> If you do 2-4 FPS the bandwidth usage will go that up... there is
>>>>>>>> probably a trick, maybe there is also something todo in the way the
>>>>>>>> tiles
>>>>>>>> are calculated that need to be resend. It normally should only try to
>>>>>>>> resend the ones that really change. So if you show an hd movie you
>>>>>>>> might
>>>>>>>> see your bandwidth needs go up quite a bit.
>>>>>>>>
>>>>>>>> Seb
>>>>>>>>
>>>>>>>>
>>>>>>>> 2013/10/24 Maxim Solodovnik <solomax...@gmail.com>
>>>>>>>>
>>>>>>>>> Hello Alexander,
>>>>>>>>>
>>>>>>>>> Your findings are correct :)
>>>>>>>>> With the only exception: Screenshots are taken in single thread,
>>>>>>>>> so there are some limitations in the speed :(
>>>>>>>>> I believe Thread.sleep(xxx) doesn't work at all
>>>>>>>>> I'm planning to rewrite this part And hopefully will be able to
>>>>>>>>> increase FPS.
>>>>>>>>>
>>>>>>>>> Basic compilation steps are here:
>>>>>>>>> http://openmeetings.apache.org/BuildInstructions.html
>>>>>>>>> After compiling all jars in screensharing folders are being
>>>>>>>>> signed, so you need to replace all of them, otherwise you will get
>>>>>>>>> security
>>>>>>>>> error due to different certificates
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, Oct 24, 2013 at 3:07 AM, Alexander Brovman <
>>>>>>>>> alexander.brov...@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> I have two quick questions, but first of all i just wanted to say
>>>>>>>>>> that Openmeetings is fantastic. I had both OpenMeetings and BBB
>>>>>>>>>> Installed
>>>>>>>>>> and we have decided to go with OM.
>>>>>>>>>>
>>>>>>>>>> I am relatively apt at programming with Python and C# but I have
>>>>>>>>>> never worked with a lot of Javaapplications like these ones, so i
>>>>>>>>>> have the
>>>>>>>>>> following questions:
>>>>>>>>>>
>>>>>>>>>> *1)* From what i understand the ScreenSharing is performed by
>>>>>>>>>> taking snapshots every x milliseconds. Additionally there is an
>>>>>>>>>> algorithm
>>>>>>>>>> in place that divides the snapshot into tiles and measures the
>>>>>>>>>> changes in
>>>>>>>>>> tiles, if there is a change the tile is uploaded. If this is
>>>>>>>>>> correct, and i
>>>>>>>>>> want a more.. "fluid" video because there will be a lot of changes,
>>>>>>>>>> cant i
>>>>>>>>>> simply modify the "framerate" at which the images will be compared to
>>>>>>>>>> improve the output? I am well aware that this would eat into the CPU
>>>>>>>>>> and
>>>>>>>>>> the bandwidth but i have a Full Duplex Gbit and 2x Quadcore Xeons
>>>>>>>>>> available
>>>>>>>>>> to me so this is not an issue. We are trying to test for an extreme
>>>>>>>>>> case of
>>>>>>>>>> 500-1500 users and how to optimize the output, i think the
>>>>>>>>>> developers and
>>>>>>>>>> other users will be interested in seeing the results.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> *2)* Where would these changes need to be made? So far i have
>>>>>>>>>> isolated the CoreScreen.java , CaptureScreen.java and
>>>>>>>>>> theScreenV1Encoder.java files that apparently contain the code which
>>>>>>>>>> processes the desktop sharing.
>>>>>>>>>>
>>>>>>>>>> The CaptureScreen.Java has the:
>>>>>>>>>> "timeBetweenFrames = (ScreenDimensions.quality ==
>>>>>>>>>> ScreenQuality.VeryHigh) ? 100 : 500;" setting on line 54 which
>>>>>>>>>> basically
>>>>>>>>>> says that if Screequality is set to very high (condition is true)
>>>>>>>>>> then
>>>>>>>>>> select 100 as the value for timeBetweenFrames.
>>>>>>>>>>
>>>>>>>>>> This variable is apparently used lateron in the
>>>>>>>>>> "Thread.sleep(Math.max(0, timeBetweenFrames - spent));" on line 96,
>>>>>>>>>> which
>>>>>>>>>> will pause the current loop for the amountof time that is the
>>>>>>>>>> difference
>>>>>>>>>> between timeBetweenFrames( in our case 100ms) minus the spent time
>>>>>>>>>> (which
>>>>>>>>>> is defined on line 91).
>>>>>>>>>>
>>>>>>>>>> So in Theory if i wanted to output the screen in realtime at 20
>>>>>>>>>> or 24 frames per second which would mean that a snapshot would need
>>>>>>>>>> to be
>>>>>>>>>> taken every 50 milliseconds or so, would i simply need to set a
>>>>>>>>>> different
>>>>>>>>>> frame setting for the time between frames, something like:
>>>>>>>>>> "timeBetweenFrames = (ScreenDimensions.quality ==
>>>>>>>>>> ScreenQuality.VeryHigh) ? *20* : 500;"
>>>>>>>>>>
>>>>>>>>>> Reduce the Thread.sleep(60); on line 60 to something like 20
>>>>>>>>>> milliseconds so (Thread.sleep(20);)
>>>>>>>>>> and then finally remove the the last part
>>>>>>>>>> "Thread.sleep(Math.max(0, timeBetweenFrames - spent));" or perhaps
>>>>>>>>>> leaving
>>>>>>>>>> it in?
>>>>>>>>>>
>>>>>>>>>> I understand that this would be a .. "wrench and hammer" approach
>>>>>>>>>> since this will probably eat up quite a bit of CPU usage. I mean the
>>>>>>>>>> simpler solution here clearly would be to get rid of the algorithm
>>>>>>>>>> itself
>>>>>>>>>> and simply rewrite it to simply output a screenshot every x
>>>>>>>>>> milliseconds or
>>>>>>>>>> so, but i have no idea how deep i would be messing with openmeetings
>>>>>>>>>> and if
>>>>>>>>>> that would/could break other parts of openmeeting that access these
>>>>>>>>>> files?
>>>>>>>>>>
>>>>>>>>>> So would this be a workable solution that i could at least try
>>>>>>>>>> out? I would ofcourse be more than happy to supply the CPU Usage
>>>>>>>>>> differences.
>>>>>>>>>>
>>>>>>>>>> *3)* My only question would be is how would i recompile these? I
>>>>>>>>>> am asking because i read somewhere in the mailinglists (or the old
>>>>>>>>>> google
>>>>>>>>>> groups) that it is recommended to use ANT in order to recompile
>>>>>>>>>> things.
>>>>>>>>>> Also what will be the output name of the .jar that is essentially
>>>>>>>>>> being
>>>>>>>>>> loaded for people and located on the server?
>>>>>>>>>> Because all i can find on my Debian box is
>>>>>>>>>> the
>>>>>>>>>> /usr/lib/red5/webapps/openmeetings/screensharing/openmeetings-screenshare-2.1.1-RELEASE.jar
>>>>>>>>>> file.
>>>>>>>>>>
>>>>>>>>>> So yeah, if somebody could help me with recompiling i would
>>>>>>>>>> appreciate it! :)
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> WBR
>>>>>>>>> Maxim aka solomax
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Sebastian Wagner
>>>>>>>> https://twitter.com/#!/dead_lock
>>>>>>>> http://www.webbase-design.de
>>>>>>>> http://www.wagner-sebastian.com
>>>>>>>> seba.wag...@gmail.com
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> WBR
>>>>>> Maxim aka solomax
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>>
>>> --
>>> WBR
>>> Maxim aka solomax
>>>
>>
>>
>
>
> --
> WBR
> Maxim aka solomax
>
