On Mon, 2011-06-27 at 08:32 +0300, Yaniv Kaul wrote: > Licensing and patent concerns of x264 aside (see > http://mailman.videolan.org/pipermail/x264-devel/2010-July/007508.html), > the more I think about it, the more it makes sense to me to have H.264 > offload feature in QXL:
This sounds great (gaming on SPICE!) but that's a pretty big aside. How would we handle the patent issues? - John > 1. Many (Flash, HTML5, others) are moving to H.264 - and take advantage > of hardware acceleration if such is available, for complete decoding of > the stream. If QXL advertises itself as being able to HW accel. the > stream, we need to just pick the stream and move it to the client. On > the client, we can either download it to the local GPU which will HW > accelerate its decoding, or use software-based decoding (x264, for example). > 2. Then we can revisit x264 and see if it really provides a noticeable > advantage over mjpeg - clearly having one video solution is cleaner and > less hassle, and x264 seems to have some nice properties we can use (see > http://x264dev.multimedia.cx/archives/249 regarding low latency). > > Both are not trivial tasks, I'm afraid. > Y. > > > On 06/25/2011 06:33 PM, John A. Sullivan III wrote: > > Hello, all. Andrea Celestino was kind enough to email me off list about > > streaming video performance as reducing SPICE bandwidth consumption for > > streaming video is his project as a Computer Engineering student. He > > agreed that I could repost our conversation to invite input from others > > on the list. The (top posted - sorry) thread is below - John > > > > > > Hi, Andrea. No problem with your English. It is certainly better than > > my very rusty Italian! > > > > I know little about video other than as a network engineer and I am not > > a programmer but I do know what we need. From that ignorant > > perspective, I see several possibilities. > > > > Video is inherently brutal when it comes to network bandwidth and > > processing. There are also different usage scenarios. For example, it > > makes sense to stream when the video needs to be played only once and > > started immediately. However, if it is going to be viewed multiple > > times, the Citrix approach of downloading and playing locally makes more > > sense. I've mentioned on the list how we can do this with technology > > already in the X2Go project. It is probably the least desirable option > > and the least interesting for your project. Of course, we have to > > handle all this complexity in a way which does not confuse the end user > > who just wants to click on their file or web site and have it work! > > > > For streaming the video between the SPICE server and client, I think we > > also have a couple of options. I don't think there is any magic that > > can be done. It is a matter of using the most appropriate codec for the > > transmission. The challenge, as already mentioned on the list, is > > encoding on the fly. The clients need to decode no matter what but, in > > non-SPICE usage, the file is pre-encoded. So, in our selection of > > codecs, speed of encoding versus decoding becomes critical. We must > > also not only choose one which uses an open source license but one which > > is not patent encumbered. I believe that rules out X264. The most > > likely candidate is probably VP8 with Theora as an alternative. > > > > Once a codec (or a selection of codecs) has been chosen, the next > > challenge is how to switch. One option is to simply make it a setting > > in the SPICE parameters for qemu. The two advantages are that it is the > > simplest and it allows the system administrator (who hopefully knows his > > system well) to make the balance between CPU utilization and bandwidth. > > The disadvantage is that it does not adapt to the users environment and > > may require a reboot to change. > > > > The second option is to select the codec dynamically. Not only would we > > select whether to use a lossy or lossless codec based upon the pixel > > change rate, but we would then choose the codec based upon the bandwidth > > to the client balanced against the bitrate of the video. There is > > already precedent for this logic in that SPICE already chooses whether > > to use lossy or lossless codecs based upon detected bandwidth. From a > > previous post by Marian Krcmarik: > > > > 'Once client connects to a guest, "spice server" determines client's > > bandwidth (look at code for details, I believe It could be more > > appropriate and guys are working on it). "jpeg-wan-compression" and > > "zlib-glz-wan-compression" are enabled in case the client's bandwidth is > > <10Mbps, otherwise It's disabled. When jpeg compression and zlib over > > glz compression are enabled then photo-like bitmaps are compressed by > > lossy jpeg compression and textual/artificial bitmaps are compressed by > > lossy zlib on top of GLZ.' > > > > SPICE would compare the bandwidth to the bandwidth requirements of the > > video and, if the video bandwidth requirements are some percentage less > > than the available bandwidth (to leave some of other traffic - I do not > > know what that number should be - 80%?), it uses MJPEG for better > > quality and lower CPU utilization. If not, it uses VP8 or whatever we > > choose. This could be multiple choice depending on bitrate/bandwidth if > > that makes sense and not just MJPEG vs VP8. > > > > This would be the most interesting approach although we should sacrifice > > our programming interest to the best solution for the end users. It > > does have the advantage of not needing to be tweaked by the admin and > > adapting to whether the user is on the end of a 3G cell connection or a > > 100 Mbps WAN link. > > > > As I think about it, we already have precedent for making it the best of > > all worlds by using an auto or fixed setting like SPICE does for other > > parameters - auto activates the automated routine above and a fixed > > value (MJPEG or VP8) specifically locks it to always use the specified > > codec. This is probably a good idea in general but would be > > particularly helpful in the early stages while we tweak what that magic > > percentage is for allowing other traffic, i.e., does the higher > > utilization codec kick in at 80%, 60%, 95%. > > > > Would you mind if I posted this response to the list to see if others > > have helpful input? Thanks - John > > > > On Fri, 2011-06-24 at 11:18 +0200, Andrea Celestino wrote: > >> Hi, > >> > >> first of all I am studying the source coude of spice, because I need > >> to understand how spice detects video streaming, how it sends stream > >> data to client with socket and how it compress data with mjpeg. > >> I have not any idea at the moment, I need to do some tests. > >> I am very interested. > >> Do you have an idea? > >> > >> > >> Thanks. > >> > >> > >> p.s. sorry for my english :) > >> > >> > >> > >> > >> > >> 2011/6/23 John A. Sullivan III<jsulli...@opensourcedevel.com> > >> Certo! Hope I remembered that correctly. I would be extremely > >> interested. What did you have in mind? Thanks - John > >> > >> On Thu, 2011-06-23 at 10:29 +0200, Andrea Celestino wrote: > >> > Hi, > >> > I have read your message on Spice mailing lists and I am > >> interesting > >> > in streaming video performance with spice. I am an italian > >> student in > >> > computer engineering, during this period I am working on > >> reducing > >> > bandwidth usage with streaming video in Spice. I see that > >> you are > >> > interested in the same problem. I think that we can > >> cooperate. What do > >> > you think about it? > >> > >> > >> > >> > > > > > > _______________________________________________ > > Spice-devel mailing list > > Spice-devel@lists.freedesktop.org > > http://lists.freedesktop.org/mailman/listinfo/spice-devel > _______________________________________________ Spice-devel mailing list Spice-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/spice-devel
