Here is the overview of the work that I recently sent out internally.
I started out with a pure-functional scripting interface, but it very rapidly became
imperative, and it is in heavy flux. The sample scripts do capture the general
"flavor" that I want this to have -- I think it will be easy enough to get
interesting things happening that it will still tempt people who might be a little
hostile to working in a Lisp.
------------------------------------------
Here is the first preview of the general purpose VR scripting environment I
have been working on. There isn’t anything exciting to look at, but if you
squint real hard you can see where it might be going. After I get a bit
farther along, we can build a PC version as well for cross platform support.
Motivation:
Doing VR GUI development in the native apps is unpleasant – you can gut it out
in C++ with our current interfaces, but the iteration times and code structure
don’t make it a lot of fun.
In-editor prototyping is nice in Unity, but actual on-device testing cycle
times are terrible, and you wind up with all the bulk of Unity and poor access
to the native VR features like TimeWarp layers.
With app signing, we have no way for developers to casually share work with
each other or limited communities.
My solution is to support development of VR applications in a scripting
language with several modes:
“Web like”, where the script is downloaded from the net for each execution and
run by a single signed app that we provide (NetHMD). Fully specified by an
app-scheme URI, this allows VR experiences to be launched directly from web
pages or app-links on Facebook, which I think is VERY powerful – read about how
great something is while browsing on your phone, then just click the link and
get the “insert into HMD” dialog and jump right there. VR scripts can link to
other VR scripts without leaving VR. There is no direct monetization here, but
IAP can eventually be supported.
“App like”, where the script is bundled into a conventional apk to be
distributed through the store, and the scripting is simply used as a more
effective development environment for some classes of applications.
“Remote development”, where the script is actually executed in an IDE on a PC,
communicating with NetHMD over a TCP connection. This allows sub-second
code-change to VR change iteration cycles, and the use of a debugger. There
might also be a use for a related mode where a central server directly drives
one or more clients.
Where it will win first:
My “reference application” is the linking of multiple panoramic photos with
text and audio annotations. This is a completely obvious application that is
not performance sensitive, and even a non-programmer could copy-paste-modify
the simple script for this to make a high quality experience.
Once I hook up an ExoPlayer interface, this will completely subsume what I
wanted to accomplish with “experience files” – the addition of static prop
models to panoramic video, linking between different videos, merging of models
or panoramas with videos that aren’t fully panoramic (GoPro, etc), and so on.
Once I hook up the Social interfaces, this will make it much easier to build
trivial “activities” for multi-user experiences – board games, card games, etc.
It is already a big help for basic graphics development work – being able to
just position a few quads with a custom shader and iterate on the shader
programming remotely without rebuilding is a huge win.
Implementation:
This will be controversial, and I will have a longer discussion about it, but I
chose Scheme (a Lisp) for the scripting language. If this all works out, we
can eventually make C# and Javascript versions, but I have a bunch of reasons
for going this way first. I am using Racket http://racket-lang.org/ for the PC
development environment, and Chibi Scheme for the embedded interpreter on
Android. Download Racket and walk through their quick-intro-with-pictures to
get a feel for the language and you should be able to hack on the sample
scripts I have been writing pretty quickly. Note that Racket is a very
extended Scheme targeted at desktop apps, the embedded version is just the
basic standard.
All texture / model / sound resources are loaded and cached from network URI.
For testing, you can just paste in any URI from a Google image search, which is
convenient. For an actual project development, dropbox should work fine.
I am favoring ease of development over performance, and it definitely won’t be
suitable for a lot of tasks, but everything we have done with first party apps
so far should be at least a candidate for development in this style once it is
mature.
Everything is super-rough right now, and I keep thinking “one more day of work
will clean it up better for people to look at”, but it is past time I at least
get this on everyone’s radar.
<actual Android apk redacted -- it is too crappy to show the broader world right
now>
Running it will load a default home script, which has links to a few test
programs I have made. The back button returns to the home page.
The sample scripts are at:
http://s3.amazonaws.com/o.oculuscdn.com/netasset/home.rkt
http://s3.amazonaws.com/o.oculuscdn.com/netasset/office.rkt
http://s3.amazonaws.com/o.oculuscdn.com/netasset/shader.rkt
http://s3.amazonaws.com/o.oculuscdn.com/netasset/spin.rkt
http://s3.amazonaws.com/o.oculuscdn.com/netasset/text-panels.rkt
http://s3.amazonaws.com/o.oculuscdn.com/netasset/reversi.rkt
You could edit them dawn-of-the-web style by downloading, editing, uploading
someplace else, and sending a new intent to nethmd:
adb shell am start -d
nethmd://s3.amazonaws.com/o.oculuscdn.com/netasset/shader.rkt
However, the preferred development strategy is to use the DrRacket IDE. In
addition to the scripts you are working on, download the following two files
into the same directory:
https://s3.amazonaws.com/o.oculuscdn.com/netasset/vr.rkt
https://s3.amazonaws.com/o.oculuscdn.com/netasset/remote.rkt
Make sure you have your phone and PC on the same WiFi network, and they can
ping each other. Run the NetHmd app (which reports the IP address on the home
page for convenience). Open the script you want to work with in DrRacket and
put the ip address in the last line, which looks something like this:
(remote "172.22.52.94" #f tic)
Hit ctrl-R or select the Racket->Run menu option to start execution. NetHMD on
the phone should change from the home screen to the test scene. You can load and
run other scripts, or restart the current one, without restarting NetHmd on the
phone. You can move around with the joypad during development. Button Y / 4-dot
resets the view to the home position.
I am still figuring out exactly what the “standard library” should look like,
and I’m sure it will go through incompatible changes, but I would be happy to
support anyone interested in doing bleeding-edge experimentation with this.
