[opensource-dev] Snowglobe as an mixed reality platform

[Kantonen Tuomas]

Hello,

I've been working on a project using Second Life as a platform to 
develop mixed reality teleconferencing/-collaboration system. A brief 
intro about the project can be found at
http://www.vtt.fi/multimedia/projects/mrconference.html
and a video demo of our work so far is at 
http://www.youtube.com/watch?v=DNB0_c-5TSk .

The project, continuing also in 2010, is split into two parts: 
overlaying a video image with avatars and virtual objects (augmented 
reality; AR) and using hand and head gestures to control avatars and 
interact with virtual objects (augmented virtuality; AV).

We have agreed to release our work as open source. However, lack of 
continuous "forward porting" would soon render the released code
unusable. It would therefore be better to have some parts of our
work incorporated into the Snowglobe sources.

I've tried to keep all our modifications as small as possible. The 
project is separated to our own ACME (Augmented Collaboration in Mixed
Environments) module, vanilla Snowglobe code and an interface between 
the two. 

I'd be ready to invest some time to implement a generic AR/AV support
for Snowglobe if we could come up with a design that could be accepted
by the Snowglobe developers. This would allow anyone to use Snowglobe
as AR/AV research platform, with or without our software.

--
- Tuomas Kantonen, VTT Technical Research Centre of Finland
___
Policies and (un)subscribe information available here:
http://wiki.secondlife.com/wiki/OpenSource-Dev
Please read the policies before posting to keep unmoderated posting privileges

[opensource-dev] Avatar animation control using gestures and augmented reality visualization.

[Kantonen Tuomas]

Hi.

Earlier  this year I presented our work on using Snowglobe as a platform for 
mixed reality meetings.
This includes controlling avatars by natural gestures and to be able to view 
avatars as if they would
be located in the same room as the user.

I opened two "new feature" issues into SL Jira (SNOW-820, SNOW-821) to discuss 
if some of this work could
be included into Snowglobe. The gesture recognition part might be easiest to 
integrate, since it does not require
changes in the viewer source, except to be able to initialize and execute the 
processing thread. However, since all
this image processing requires a lot of processing, and the implementation uses 
3rd party libraries, such as
OpenCV, I'm not sure what would be the method of integration that the Snowglobe 
community would like to
use. Augmented reality visualization needs to touch the rendering pipeline so 
I'd like to know how eager you
are to include this kind of work.

I have created a couple of web-pages describing how the system works:
 http://virtual.vtt.fi/virtual/proj2/multimedia/MWI/mwi.html.

Currently the system is implemented as a separate DLL + a small set of patches 
to initialize the DLL and transfer
calls between the viewer and the DLL. I can figure out at least two other 
possibilities: including all the DLL sources
(and adding all its dependencies) directly into the viewer source, or running 
the DLL as a separate executable using
some simple IPC interface to communicate with the viewer (not too unlike how 
the SLVoice works).

You can see all the sources and patches at the project web page. Since the 
project contains multiple different parts,
I tried to separate the patches into sensible sets. The patches are so small 
that I think that I could quite easily port
them to the latest viewer trunk but I'd rather not do it until I have a better 
idea what's the correct integration mechanism.

As a side note, we are currently working on face gestures that was proposed 
already on 2007
(https://jira.secondlife.com/browse/VWR-2122). If we can figure out a way to 
include all this client
side video processing, then this work would also be integrated.

--
- Tuomas Kantonen

___
Policies and (un)subscribe information available here:
http://wiki.secondlife.com/wiki/OpenSource-Dev
Please read the policies before posting to keep unmoderated posting privileges