[vos-d] s5 scripting (design part 3)

[Peter Amstutz]

Third in a four part series on major design changes in s5
(What could also be called "VOS Apocalypse")

Scripting.

One of the things we have encountered over the course of our open source 
efforts was (obvious in retrospect) people don't like dealing with C++.  
While there remain compelling reasons to write the VOS "kernel" in C++, 
for most people the language is hard, setting up a compilation 
environment is difficult, and there are many things a statically 
compiled language simply can't do, such as evaluating new code on the 
fly.

With s3 and s4, we have tried to address scripting using the Simplified 
Wrapper and Interface Generator (SWIG), which is an excelent program 
which can read C and C++ header files and produce bindings into a 
variety of popular programming languages.  However, this approach 
suffers from a flaw: by design it is oriented towards calling C/C++ code 
from other languages.  It does not, however, provide much support for 
allowing your C/C++ code (or other languages hosting in the same system) 
to call back to that scripting code.  So, scripting languages are 
second-class citizens.

What I intend to do in s5 is see to it that all programming languages 
(scripting or otherwise) can be a first-class citizen in VOS, so that C, 
C++, Perl, Python, LISP, Java, C# etc code can all call each other, 
agreeing on the same APIs and data types.  The goal, is to allow users 
to write behaviors and extensions for VOS in their programming language 
of choice.

This is, in some ways, one of the holy grails of computing.  How many 
times has a project been rewritten, or required redundant work, because 
software A in one language needed to link with software B in another?  
Now, I don't want to overstate what we're doing, but I do believe that 
certain key aspects of VOS enable us to produce a cleaner solution than 
the existing state of the art.

There are three key design elements:

1. Actor model for Vobjects and message passing.

This is shakes out from the concurrency design discussed earlier.  
Because Vobjects are required to communicate via message passing, it is 
easy to add a translation step that converts message parameters from one 
language datatypes to another.  Datatypes will consists of either 
copyable primitives, or Vobjects, so object references that are 
communicated between languages are independent identifiers not tied to 
particular memory (or network) location.  Capabilities can be used to 
enforce security between runtimes (more on that later).  Finally, and 
most importantly, there is no inversion of control inherent when calling 
on a vobject that is handled by another language.  This means that the 
act of calling from one language to another does not result in the 
caller's thread being taken over to run the target scripting language 
runtime or virtual machine, thus avoiding various unintended side 
effects.

If a scripting language runtime or virtual machine is not threadsafe, it 
can be made to run in a separate, private thread and require that all 
inbound messages be placed in its message queue.  Also useful is in the 
case that VOS is be incorporated into an existing application or 
framework, all that is required is that the application's main event 
loop be able to periodically check for new incoming VOS events.


2. Object Type Definitions (OTDs) for Interfaces

Object type definitions describe the messages a vobject accepts and 
emits, and the child vobjects it might link to.  It is similar in spirit 
to an XML schema or a CORBA interface file, but merges ideas from both.  
It is independent of programming language, and I am working towards 
defining the OTD in itself.

The primary use of the OTD is input into a code generator that produces 
concrete classes that implement the marshaling of method parameters into 
messages, and from messages back to method parameters.  Since messages 
are language-neutral, they can now be passed to a handler written in any 
other language.  For each supported language, we will have either a code 
generator that produces the required classes, or for more dynamic 
languages, a module with that uses reflection and evaluation to generate 
the required classes.

The advantage of this is that interfaces can be added to any language 
runtime at any time without having to modify either the other languages 
or the C++ kernel.  The only essential interface to the C++ kernel will 
be a minimal C interface designed specifically for the FFI (foreign 
function interface) feature present used by many languages.  This allows 
most (or all) of the work to be done in the target language that the 
user is most comfortable in.


3. Runtime extensible object model and reflection

Vobjects arn't set in stone.  You are free to attach new types, new 
behaviors and new children onto existing vobjects at any time.  This is 
extremely useful for situations where you need to take several behaviors 
and smoosh them together into a single vobject (very, very common in VOS 
-- for example a typical avatar has at minimum types misc:avatar, 
a3dl:object3d.model, a3dl:actor and misc:talkative) which could now in 
s5 be potentially implemented is multiple languages.  This "mashup" 
capability is very powerful and fairly unique to VOS (I think you could 
probably do it in Croquet due to its Smalltalk basis, but that's about 
it).

This flexibilily also means applications are free to attach 
application-specific data to vobjects that will be ignored by tools that 
don't recognize it.  For example, a file format loader for some 3D scene 
could package up the parts it doesn't understand and include them along 
with the parts that have been converted to standard A3DL data.  When 
exporting that same scene back to the original file format, those parts 
can be retrieved and output again to yield the identical original file, 
rather than those parts having been stripped off and lost in the 
original loading.

VOS also provides reflection (OTDs will themselves be represented in the 
system as vobjects, essentially metaclasses).  For example, iterating 
over child vobjects is very similar to iterating over the fields of a 
class.  Vobject structures can be navigated and manipulated using 
generic tools, and scripts which process Vobjects can be simpler and 
require fewer assumptions about the underlying objects.


The big picture of what I'm trying to do with s5 is make VOS an 
object-oriented programming infrastructure that ties together all of 
computing, and happens to makes the end goal of an immersive, 
collaborative 3D world really easy to implement.  The reason is that I 
want to be able to leverage other people's work in programming 
languages, operating systems, databases and graphics in a way that is 
easy and effective.  This is the basic motivation for defining VOS as a 
platform: so in the long run, we don't have to do all the work.  So 
instead of stitching together a half-assed 3D engine, a half-assed 
database and a half-assed scripting language all created in-house, we 
can create a best-of-breed system that ties together the best of each 
category.

-- 
[   Peter Amstutz  ][ tetron at interreality.org ][ peter.amstutz at gdit.com ]
[Lead Programmer][Interreality Project][Virtual Reality for the Internet]
[ VOS: Next Generation Internet Communication][ http://interreality.org ]
[ http://interreality.org/~tetron ][ pgpkey:  pgpkeys.mit.edu  18C21DF7 ]

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url : http://www.interreality.org/pipermail/vos-d/attachments/20070420/f81dfc82/attachment.pgp