Maxim,
A) Interessting to say the very least, how did you manage to measure the
encode and the screencapture? or what tools did you use in order to measure
those, might come in handy for me when measuring performance.
Right now i changed the sourcecode in the following way:
1) " timeBetweenFrames = (ScreenDimensions.quality ==
ScreenQuality.VeryHigh) ? 30 : 100;"
2) public void run() {
try {
Robot robot = new Robot();
BufferedImage image = null;
while (active) {
final long ctime =
System.currentTimeMillis();
Rectangle screen = new
Rectangle(ScreenDimensions.spinnerX,
ScreenDimensions.spinnerY,
ScreenDimensions.spinnerWidth,
ScreenDimensions.spinnerHeight);
image = robot.createScreenCapture(screen);
try {
timestamp += timeBetweenFrames;
byte[] data = se.encode(screen, image, new
Rectangle(ScreenDimensions.resizeX,
ScreenDimensions.resizeY));
pushVideo(data, timestamp);
} catch (Exception e) {
e.printStackTrace();
}
final int spent = (int)
(System.currentTimeMillis() - ctime);
if (sendCursor) {
core.sendCursorStatus();
}
Thread.sleep(Math.max(0, timeBetweenFrames
- spent));
}
} catch (Exception e) {
e.printStackTrace();
}
}
I am definitely noticing a very strong improvement in terms of less video
delay.
using jnettop i can see that at a screen resolution of 1650x1080 it is
approximately using between 4-8mbit for 1 user.
So i guess the biggest issue here is the encoder, the encoder is definitely
not efficient, i have been looking at solutions of other people on
stackexchange in terms of algorithms.
B) I have a question regarding the process, so if the screencapture takes
place and is sent using the RTMP method/protocol to the server. Does the
sever additionally encode the file again before sending it to the enduser?
If so which javafiles contain those settings, because i would like to test
if i can tweak them to reduce the delay even more by disabling the encode
of the images/stream on the serverside and just send the files "as is".
Although once again i fear the algo is the issue.
Kind regards,
Alexander
On Thu, Oct 24, 2013 at 3:06 PM, Maxim Solodovnik <solomax...@gmail.com>wrote:
> Here are my measurements:
> Screen is captured in ~ 160-170ms
> Encoded in ~120-200ms
> Will try to speed up this somehow
>
>
> On Thu, Oct 24, 2013 at 3:27 PM, Alexander Brovman <
> alexander.brov...@gmail.com> wrote:
>
>> nevermind used the wrong ANT build.
>>
>> Got it to work now :)
>>
>>
>> On Thu, Oct 24, 2013 at 9:11 AM, Alexander Brovman <
>> alexander.brov...@gmail.com> wrote:
>>
>>> Maxim i made some changes to the code, i downloaded the source on the
>>> server, changed the lines with nano. and then from the mainfolder simply
>>> ran the ant command and it started compiling (albeit some errors appeared
>>> since i only entered "ant" without any options/parameters.
>>>
>>> Openmeeting is working fine, i simply replaced the entire
>>> screenshareing/ folder with the new one i just compiled... any suggestions
>>> on what im doing wrong here?
>>>
>>> I am getting this error:
>>> Unsigned Entry in resouce: slf4j-api-1.6.4.jar
>>>
>>>
>>> On Thu, Oct 24, 2013 at 8:17 AM, Maxim Solodovnik
>>> <solomax...@gmail.com>wrote:
>>>
>>>> I'm currently trying to modify the code to get 20+ fps (was requested
>>>> many times, but I had no time on this :( currently I'm on vacation, will
>>>> try to implement it)
>>>>
>>>>
>>>> On Thu, Oct 24, 2013 at 10:02 AM, Alexander Brovman <
>>>> alexander.brov...@gmail.com> wrote:
>>>>
>>>>> Maxim,
>>>>>
>>>>> Thank you for responding so quickly. The primary reason i asked is
>>>>> because we sometimes need to output data like Financial Stock charts or
>>>>> similar dynamic content and in a resolution size of 1650x1080. As you can
>>>>> imagine right now the delay is.. manageable, perhaps somewhere along the
>>>>> lines of 0.5-1 second delay id say.
>>>>> We had an independant tested saying its an average of 130ms. But for
>>>>> fast video/changes the framerate will sometimes drop or become laggy.
>>>>>
>>>>> Ill try setting the timebetweenFrames to 30ms which in "ideal"
>>>>> conditions should give me 33.333 frames per second of video.
>>>>> As for the Thread.Sleep(XXX) not working or functioning i will remove
>>>>> the thread.sleep(60) from the top,
>>>>> As for the Thread.Sleep(xxx) settings i will simply remove it; BUT i
>>>>> will leave in the "Thread.sleep(Math.max(0, timeBetweenFrames - spent));"
>>>>> since "worst" case scenario it will not be put to sleep at all since the
>>>>> output will be 0, and "best case" scenario the thread will be put to sleep
>>>>> for 30 - 1 ms depending on how the calculation turns out.
>>>>>
>>>>> Perhaps it would be good to leave somekind of thread sleep in there
>>>>> that is a static factor, from a development standpoint i would rather
>>>>> prefer having a fixed constant in there. rather than putting the thread to
>>>>> sleep (potentially twice). Either that or im not understanding the reason
>>>>> for the first thread.sleep (or the second one).
>>>>> Either way i will make the changes now and compile, I will report my
>>>>> findings here if you want me to :)
>>>>>
>>>>> @Sebastian,
>>>>>
>>>>> At the time of writing this i saw you respond, so i am writing this
>>>>> short addendum:
>>>>> 1) Could you please comment on whether or not Thread.Sleep(xxx) works?
>>>>> 2) 2-4 frames per second and less may be great for static
>>>>> presentations but not so much for dynamic. I understand the concerns
>>>>> raised
>>>>> on your behalf regarding bandwidth usage but since we have a dedicated
>>>>> Gigabit connection we have no issues with dedicating anywhere between 2-5
>>>>> Mbit per user. 20 Frames per second really is the minimum we are looking
>>>>> at, i think it would be great if you presented people with an
>>>>> option/alternative to gotowebinar. I understand that applications which
>>>>> compile into native code will always be faster and outperform in certain
>>>>> aspects, but when you have the motivation and perhaps the financial
>>>>> resources to dedicate time and money into getting away from solutions such
>>>>> as GoToWebinar.
>>>>>
>>>>> Is there a way of "fixing" the position of the screen that the viewers
>>>>> will see when they accept a desktop streaming broadcast from a presenter?
>>>>> I
>>>>> ask this because i havent found a way to change the location of the Chat
>>>>> Bar inside a conference room yet (neither have I found a way to change the
>>>>> colors/themes to be honest, need to do some more google searches).
>>>>>
>>>>> Kind regards,
>>>>> Alexander
>>>>>
>>>>>
>>>>>
>>>>> On Thu, Oct 24, 2013 at 4:47 AM, seba.wag...@gmail.com <
>>>>> seba.wag...@gmail.com> wrote:
>>>>>
>>>>>> Do we still have the super high quality setting in the sharing client?
>>>>>> My problems in the past was more around the bandwidth issues then the
>>>>>> FPS.
>>>>>> If you do 2-4 FPS the bandwidth usage will go that up... there is
>>>>>> probably a trick, maybe there is also something todo in the way the tiles
>>>>>> are calculated that need to be resend. It normally should only try to
>>>>>> resend the ones that really change. So if you show an hd movie you might
>>>>>> see your bandwidth needs go up quite a bit.
>>>>>>
>>>>>> Seb
>>>>>>
>>>>>>
>>>>>> 2013/10/24 Maxim Solodovnik <solomax...@gmail.com>
>>>>>>
>>>>>>> Hello Alexander,
>>>>>>>
>>>>>>> Your findings are correct :)
>>>>>>> With the only exception: Screenshots are taken in single thread, so
>>>>>>> there are some limitations in the speed :(
>>>>>>> I believe Thread.sleep(xxx) doesn't work at all
>>>>>>> I'm planning to rewrite this part And hopefully will be able to
>>>>>>> increase FPS.
>>>>>>>
>>>>>>> Basic compilation steps are here:
>>>>>>> http://openmeetings.apache.org/BuildInstructions.html
>>>>>>> After compiling all jars in screensharing folders are being signed,
>>>>>>> so you need to replace all of them, otherwise you will get security
>>>>>>> error
>>>>>>> due to different certificates
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Oct 24, 2013 at 3:07 AM, Alexander Brovman <
>>>>>>> alexander.brov...@gmail.com> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> I have two quick questions, but first of all i just wanted to say
>>>>>>>> that Openmeetings is fantastic. I had both OpenMeetings and BBB
>>>>>>>> Installed
>>>>>>>> and we have decided to go with OM.
>>>>>>>>
>>>>>>>> I am relatively apt at programming with Python and C# but I have
>>>>>>>> never worked with a lot of Javaapplications like these ones, so i have
>>>>>>>> the
>>>>>>>> following questions:
>>>>>>>>
>>>>>>>> *1)* From what i understand the ScreenSharing is performed by
>>>>>>>> taking snapshots every x milliseconds. Additionally there is an
>>>>>>>> algorithm
>>>>>>>> in place that divides the snapshot into tiles and measures the changes
>>>>>>>> in
>>>>>>>> tiles, if there is a change the tile is uploaded. If this is correct,
>>>>>>>> and i
>>>>>>>> want a more.. "fluid" video because there will be a lot of changes,
>>>>>>>> cant i
>>>>>>>> simply modify the "framerate" at which the images will be compared to
>>>>>>>> improve the output? I am well aware that this would eat into the CPU
>>>>>>>> and
>>>>>>>> the bandwidth but i have a Full Duplex Gbit and 2x Quadcore Xeons
>>>>>>>> available
>>>>>>>> to me so this is not an issue. We are trying to test for an extreme
>>>>>>>> case of
>>>>>>>> 500-1500 users and how to optimize the output, i think the developers
>>>>>>>> and
>>>>>>>> other users will be interested in seeing the results.
>>>>>>>>
>>>>>>>>
>>>>>>>> *2)* Where would these changes need to be made? So far i have
>>>>>>>> isolated the CoreScreen.java , CaptureScreen.java and
>>>>>>>> theScreenV1Encoder.java files that apparently contain the code which
>>>>>>>> processes the desktop sharing.
>>>>>>>>
>>>>>>>> The CaptureScreen.Java has the:
>>>>>>>> "timeBetweenFrames = (ScreenDimensions.quality ==
>>>>>>>> ScreenQuality.VeryHigh) ? 100 : 500;" setting on line 54 which
>>>>>>>> basically
>>>>>>>> says that if Screequality is set to very high (condition is true) then
>>>>>>>> select 100 as the value for timeBetweenFrames.
>>>>>>>>
>>>>>>>> This variable is apparently used lateron in the
>>>>>>>> "Thread.sleep(Math.max(0, timeBetweenFrames - spent));" on line 96,
>>>>>>>> which
>>>>>>>> will pause the current loop for the amountof time that is the
>>>>>>>> difference
>>>>>>>> between timeBetweenFrames( in our case 100ms) minus the spent time
>>>>>>>> (which
>>>>>>>> is defined on line 91).
>>>>>>>>
>>>>>>>> So in Theory if i wanted to output the screen in realtime at 20 or
>>>>>>>> 24 frames per second which would mean that a snapshot would need to be
>>>>>>>> taken every 50 milliseconds or so, would i simply need to set a
>>>>>>>> different
>>>>>>>> frame setting for the time between frames, something like:
>>>>>>>> "timeBetweenFrames = (ScreenDimensions.quality ==
>>>>>>>> ScreenQuality.VeryHigh) ? *20* : 500;"
>>>>>>>>
>>>>>>>> Reduce the Thread.sleep(60); on line 60 to something like 20
>>>>>>>> milliseconds so (Thread.sleep(20);)
>>>>>>>> and then finally remove the the last part "Thread.sleep(Math.max(0,
>>>>>>>> timeBetweenFrames - spent));" or perhaps leaving it in?
>>>>>>>>
>>>>>>>> I understand that this would be a .. "wrench and hammer" approach
>>>>>>>> since this will probably eat up quite a bit of CPU usage. I mean the
>>>>>>>> simpler solution here clearly would be to get rid of the algorithm
>>>>>>>> itself
>>>>>>>> and simply rewrite it to simply output a screenshot every x
>>>>>>>> milliseconds or
>>>>>>>> so, but i have no idea how deep i would be messing with openmeetings
>>>>>>>> and if
>>>>>>>> that would/could break other parts of openmeeting that access these
>>>>>>>> files?
>>>>>>>>
>>>>>>>> So would this be a workable solution that i could at least try out?
>>>>>>>> I would ofcourse be more than happy to supply the CPU Usage
>>>>>>>> differences.
>>>>>>>>
>>>>>>>> *3)* My only question would be is how would i recompile these? I
>>>>>>>> am asking because i read somewhere in the mailinglists (or the old
>>>>>>>> google
>>>>>>>> groups) that it is recommended to use ANT in order to recompile things.
>>>>>>>> Also what will be the output name of the .jar that is essentially being
>>>>>>>> loaded for people and located on the server?
>>>>>>>> Because all i can find on my Debian box is
>>>>>>>> the
>>>>>>>> /usr/lib/red5/webapps/openmeetings/screensharing/openmeetings-screenshare-2.1.1-RELEASE.jar
>>>>>>>> file.
>>>>>>>>
>>>>>>>> So yeah, if somebody could help me with recompiling i would
>>>>>>>> appreciate it! :)
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> WBR
>>>>>>> Maxim aka solomax
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Sebastian Wagner
>>>>>> https://twitter.com/#!/dead_lock
>>>>>> http://www.webbase-design.de
>>>>>> http://www.wagner-sebastian.com
>>>>>> seba.wag...@gmail.com
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> WBR
>>>> Maxim aka solomax
>>>>
>>>
>>>
>>
>
>
> --
> WBR
> Maxim aka solomax
>
