[vos-d] s5 progress and design

[Peter Amstutz]

Some notes on s5 development progress and design.

I have made enough progress on developing the s5 kernel that the next 
milestone (and release) will be to use the "real" s5 VOS in the 
Interreality 3D prototype.  My intention is to try to keep development 
grounded by incrementally developing features on the backend and then 
working to integrate those each feature and improve the frontend.

The main focus of development for the past couple of months has been on 
building the code generator which processes an interface description as 
input and produces C++ classes (interface, implementation, and wrappers) 
as output.  The generator works on a Vobject structure representing the 
abstract syntax tree, so the interface description is actually specified 
using a Vobject XML serialization format.  The first use of the code 
generator is to specify (and produce) the core VOS classes, which leads 
to a virtuous cycle where the practical requirements of being able to 
compile and run the code generator itself drives the first stage of 
development.

The general approach of the C++ binding is for each VOS interface to 
produce a pure virtual interface class, a concrete implementation stub 
(to be filled in by the developer), and a wrapper class.  The purpose of 
the wrapper class is to insulate the user from being able to call the 
implementation object directly, in order to provide a number of 
features:

 - Reference counting
   - Provide a standard smart pointer, where the count is decremented 
when the wrapper goes out of scope.  This will actually be safer than a 
typical smart pointer, as access to the underlying raw pointer will not 
be permitted at all.

 - Locking
   - To avoid locking and unlocking an object with every method call, 
the wrapper can lock it once, and release the lock when the wrapper goes 
out of scope.

 - Soft links
   - The wrapper can refer to an object by path, rather than by pointer, 
and perform late binding (or in the case of remote objects, no binding 
at all).

 - Marshaling
   - If the wrapper refers to an object owned by another thread or 
process (including remote objects), the wrapper is responsible for 
marshaling the method parameters a message which is queued for delivery.

 - Pipelining
   - Because all method calls will be potentially asynchronous, 
requiring every single call to a method be able to handle a result of 
"incomplete" would be excessively burdensome.  Piplining allows chaining 
together multiple calls where each call waits on the "promise" of a 
pending result.  When the entire chain is resolved (complete, or broken 
due to an error), the client code is called with the result so that it 
can continue.

For the last one (pipelining), I am still mulling over the possibility 
of using user-level threads instead of (or as an alternative to) 
requiring the user to save state and unwind the stack manually.

Also, although all the features above are planned, none of them are 
implemented, so if you look at the wrappers at the moment, they're just 
empty shells.

Here's a couple of samples from the code generator, to get a feeling for 
what the new API is shaping up like.  First, some code which creates an 
input stream, and loads a file as a site replica:

    FactoryWrapper factory = getLocalHost().getFactory();

    std::ifstream inp(inputFile);
    vosIO::CppStdInputStreamWrapper inpVobj(factory.create(getLocalHost(),
              vosIO::CppStdInputStreamImpl::getClassImplementation()));
    inpVobj.setStdStream(&inp);
    SiteWrapper root = importReplica(inpVobj);

Next, some code that looks for a child with the name "occurrence", tests 
to see if the return link is valid, and casts it to a string property 
vobject:

    ParentChildLinkWrapper occ = 
                       link.getChild().getChildByName("occurrence");
    StringWrapper sw;
    if (occ) {
	sw = StringWrapper(occ.getChild());
    }
    
    if (sw && sw.getValue() == "any") {
        // stuff
    }

Finally, here's a fragment of the interface specification, picked to 
show most of the features.

<?xml version="1.0" ?>
<site publicId="vos:0011223344556677889900aabbccddee" revision="4">

<vobject name="OTD" type="/OTD/vos/OTD">

  <vobject name="vos" type="/OTD/vos/Namespace">

    <property name="description" type="/OTD/vos/String">
      The core VOS namespace.
    </property>

    <vobject name="Component" type="/OTD/vos/Class">
      <vobject name="owner" type="/OTD/vos/Member">
	<child name="type" linkTo="/OTD/vos/Vobject" />
      </vobject>

      <vobject name="getEmbeddedChild" type="/OTD/vos/Method">
	<vobject name="formalParameters" type="/OTD/vos/StructDataType">
	  <vobject name="child" type="/OTD/vos/Member">
	    <child name="type" linkTo="/OTD/vos/StringDataType" />
	  </vobject>
	  <vobject name="offset" type="/OTD/vos/Member">
	    <child name="type" linkTo="/OTD/vos/Int32DataType" />
	  </vobject>
	</vobject>
	<child name="returnType" linkTo="/OTD/vos/ParentChildLink" />	
      </vobject>

      <vobject name="addToC++ImplClass" type="/OTD/vos/Pragma">
	<property name="code" type="/OTD/vos/String">
	  <![CDATA[
     friend class VobjectImpl;
	  ]]>
	</property>
    </vobject>

    <vobject name="DataType" type="/OTD/vos/Class">
      <child name="extends" linkTo="/OTD/vos/Component" />
      <property name="description" type="/OTD/vos/String">
	The root of the VOS data type system.
      </property>
      <vobject name="description" type="/OTD/vos/Member">
	<child name="type" linkTo="/OTD/vos/StringDataType" />
      </vobject>
    </vobject>

    <vobject name="KeyPair" type="/OTD/vos/StructDataType">
      <vobject name="publicKey" type="/OTD/vos/Member">
	<child name="type" linkTo="/OTD/vos/KeyDataType" />
      </vobject>
      <vobject name="privateKey" type="/OTD/vos/Member">
	<child name="type" linkTo="/OTD/vos/KeyDataType" />
      </vobject>
    </vobject>

    <vobject name="generateSiteKeyPair" type="/OTD/vos/Lambda">
      <vobject name="lambdaType" type="/OTD/vos/LambdaDataType">
	<vobject name="formalParameters" type="/OTD/vos/StructDataType"/>
	<child name="returnType" linkTo="/OTD/vos/KeyPair" />
      </vobject>
    </vobject>
  </vobject>
</vobject>
</site>

-- 
[   Peter Amstutz  ][ tetron at interreality.org ][ peter.amstutz at gdit.com ]
[Lead Programmer][Interreality Project][Virtual Reality for the Internet]
[ VOS: Next Generation Internet Communication][ http://interreality.org ]
[ http://interreality.org/~tetron ][ pgpkey:  pgpkeys.mit.edu  18C21DF7 ]

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url : http://www.interreality.org/pipermail/vos-d/attachments/20071204/18b10e51/attachment.pgp