[vos-d] thoughts and plans

[Peter Amstutz]

On Wed, Jun 27, 2007 at 12:29:43PM +0200, Karsten Otto wrote:

> >  - Scalability, clustering and recovery from failure.  The general  
> > ideas
> > is to use replication for load balancing and distributed locks for
> > consistancy control.
> >
> This sounds like a radical redesign... so far we had a single local  
> vobject acting as a sequencer for multiple remote vobjects.  
> Obviously, you have something new in mind. Please tell us more :-)

I'm shamelessly stealing from Croquet, actually :-)

The general idea is to roll up clustering, caching, persistance and 
remote objects under a single replication design.  After all, what is 
the purpose of a distributed system but for managing copies of objects?  
So, the focus of the design will be on locating, replicating and 
synchronizing objects, whether the copy comes from a cache, a site, or a 
member of a cluster.  This has a few implications:

 - The core vobject data model needs to include property data (a 
direction we were already going with "embeded children") so that the 
entire potential state of the vobject is known and explicit.

 - The vobject model needs to have a canonical serialization that can be 
used for checking hash codes.  However, this won't affect the ability to 
have multiple encodings used for data exchange.

 - Vobjects will need revision numbers and possibly last-modified times 
used in conjunction with the hash code to assist in determining if a 
copy is out of date.

Once the ability to read and copy vobjects is established, we still need 
to deal with how to write to them.  The design will attempt to move away 
from explicitly differentiating between "local" and "remote" objects, 
and instead look at whether the object's site is "local" or "remote".  
For remote objects, it is a simple matter of forwarding requests to the 
remote site [1]. In the case of clustering, the same site might owned by 
several processes, and a distributed locking protocol (or alternately a 
distributed transaction) will establish which node "owns" the vobject.

[1] I'm still working on what kind of speculative execution should be 
allowed on the "client" side.  It is easy to service a read request on a 
cached vobject, more difficult to execute code that will actually change 
this vobject and others, and could have a ripple effect on the system 
that would have to be rolled back if the request was denied.

A couple of key differences from Croquet: I'm concerned primarily with 
replicating end results, not computation.  Croquet works by requiring 
that all Croquet simulations accept the same input in the same order and 
produce the same deterministic results.  This is troublesome in a 
heterogenous environment where different versions of software (and even 
entirely different implementations) need to work together.

Second, I see distributed object ownership as being done primarily in a 
high performance or high load situation where a compute cluster makes 
sense.  There's no reason it couldn't be done between peers on the 
Internet, but it requires parties trust each other -- distributed 
ownership implies that there is no way to enforce access control between 
peers.  In most situations, which are more security conscious, you have 
a more typical centralized design (in terms of world ownership being 
located on a single site) accessed by clients which issue checked change 
requests.

I should mention that clustering isn't planned for 1.0, but I'm 
considering it now to ensure the design and implementation will be able 
to grow and support it later.

On a separate note, another goal is to support more "data-oriented" 
networking and routing, inspired by the Van Jacobson talk we discussed 
earlier.  Fully qualified vobject identifiers will be 
address-independent (unlike s4, which uses domain names and as a result 
suffers from a number of aliasing problems) which will allow parties 
other than the actual vobject owner to route messages and serve vobject 
replicas, through careful use of digital signatures to prevent forgery.

> >  * It should be designed to permit plugins or extensions to its
> > functionality.
> >
> I would go a bit further here - use plugins as the *basis* of the  
> software!
> The main client only needs to provide capabilities for displaying  
> (layered) 3D content and manage plugins. Everything else should be in  
> the plugins themselves. See below...

Yes, I agree.  The Emacs model is to provide a relatively small kernel 
of functionality, with virtually everything else built up as an 
extension.  I'm hoping that the extensions can be written in Python 
and/or Javascript.  The key insight for me is in considering, from a UI 
and application design perspective, what the "kernel" for the 3D VR 
browser actually needs to provide to have the right hooks that the 
extensions can grab on to.

I'll see about replying to some of your other points about UI later 
(right now it's late.)

-- 
[   Peter Amstutz  ][ tetron at interreality.org ][ peter.amstutz at gdit.com ]
[Lead Programmer][Interreality Project][Virtual Reality for the Internet]
[ VOS: Next Generation Internet Communication][ http://interreality.org ]
[ http://interreality.org/~tetron ][ pgpkey:  pgpkeys.mit.edu  18C21DF7 ]

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url : http://www.interreality.org/pipermail/vos-d/attachments/20070627/b71917f8/attachment.pgp