Core libs (libvos, libvip, libvutil):

Misc

Using core features, like listeners, need to be made as terse and easy as possible.
For COD and XOD: function to reload from file, either replacing all objects that were previously loaded from that file, or merging in new ones, or whatever.
VOSApp: log rotation
Make timeouts configurable. Env vars?
Would be great to have a simple function that can register almost any method as a message handler, with mapping between message fields and the method's parameters and from the method's return value and a reply message (optional)
Scheduler or cron object that does events every X secs/mins/hours/days or scheduled at a particular time. Singleton object (see below), or may need to be a special site feature.
Standard way of creating singleton objects on a site. Like a function that first checks if an object with a particular name (and type?) already exists and returns that, or creates one.
A way to globally disable all metaobject/component instances of a particular type from doing anything (if one is causing problems) (or a specific component instance of a specific object). Or, to pause (and unpause) processing so all objects of that type (or on one specific object).
Ensure that vobject persist storage writes are in background threads, perhaps buffered. Goal is to touch storage really lightly so that many diskless vos servers can easily share storage via something like nfs.
a way to map one message type to another, maybe through some kind of bridge object which relays the message. This way, any event, such as a listener event, can be mapped to some other trigger, e.g. send an email, for example. Also need a way to remotely set listeners passing through (?) that object.
Automatically distribute/move Vobjects among several Hosts (based on some user-supplied or user-chosen heuristic/rule set) on one computer (e.g. if a computer has more than one network interface but on the same network, bind each Host to a different interface) or cluster to distribute communications work. Especially useful on a "supercomputer" with lots of processing power and several network interfaces. Could easily be done with several Hosts in one process. Could also somewhat easily done between several processes with shared memory (tranferring a Vobjects from one process to another then is pretty quick and easy). More overhead to transfer objects between separate processes via VIP, but should also be implemented (allows it to run on a cluster).

Versioning

A way to categorize version branches (user-determined category strings, a centrally kept set that any user can add to)

Alternate message transports or VIP media (vs. the Internet Protocol)

HTTP tunnel (to get through firewalls on port 80)
Intrasite (between local objects): Put message data in shared memory. Use a fast local event with pointer to signal "transfer".
- Transactional Memory
- Or if property data access is generally threadsafe and versioned, don't need to copy; any particular version is always read only. (If you don't want to store versions for future use, you can just refcount them and delete them when no longer needed). Similarly, writes can be made by sending a reference to a "detached" property value that the receiver adds as the next version (keeping old versions safe for reads)
VIP over any Unix character stream device, but namely RS-232 serial port.
VIP over USB
VIP over bluetooth
VIP over WAP
VIP over ZigBee

Autoextend hook

Register an autoextender with the site-- whenever a metaobject with type X is created, automatically extend it with metaobject type Y. X does not have to be a real metaobject, it can be just a type string tag.

Extend logctrl to make messages available remotely

One option is to Collect errors and warnings as vobjects under a special "usererrors" vobject, visible from the UserInterface.

Another is just to have request to logctrl to get lists of channels, and requset to get recent messages in a channel above (below) a certain level.

Then users can use the UserInterface to see important errors and warnings, rather than simply seeing mysterious behavior, and having to go find the log file on the system.

Disconnect recovery, ghost objects

Tools for applications to recover remote child objects after local site quits and resumes (or the computer goes to sleep or is disconnected). (i.e. save remote child urls to disk in sigkill handler and reading them back, finding them, and re-inserting them at restart, could just be implemented by a pair of helpful functions; Or, "ghost object proxies" that can recover links to objects that vanish and come back (due to some failure on a server, or network disconnection, or remote site going to sleep, etc.)

Borg-site

Sites could form a single Borg-site by host-aliasing each other, and authenticating to each other, and talking to each other. (for example, lots of local sites talking with shared memory or domain sockets VOP sessions (VOS does not need to operate over network streams, remember, and also being authentic host aliases of each other). See next node about replication.

Object replication/relocation.

There is some of this in the "policy" metaobject. Tetron has also done some work on this (see the "renaldo" game in the CVS tree). One useful feature is to create objects locally, then relocate to another site when you quit. Or, when a site is heavily loaded or detects bandwidth saturation, to relocate objects to other sites. Need some protocol for negotiating with sites for relocation, and for recovering from the site you just sent your objects to suddenly dissapearing (so all "relocation" should really be "replication, then hide the original but keep it up to date?). This would only work for passive data objects, not "active" objects. But imagine a low bandwidth site on a modem which is the "originator" of the world, and as more people join, the objects start spreading out to the people with more bandwidth. Then as people leave, they merge back into earlier "authoritative" origination sites. (this can be represented by a hierarchical graph showing the replication history, followed back up to merge back to the prime state)

Related is the idea of simply doing redundent "gratuitous" replication and referring users to the duplicates randomly. (See previous Borg and replication notes)
Users with reliable broadband connections could run helper servers that accept replications from other servers (friend's servers, favorite worlds, anything on the interreality.org service list, etc.) to balance loads

Send via relays in response to network problems

Adapt to bandwidth and latency problems by indirect object addressing: if some user has less latency than others (WRT you), send all messages through them, and let them forward messages to others (though replies go back to you directly). Access control problems here? Not sure if this is a good idea or not. Would also need to be pretty smart about choosing a user to forward through (and deal with him ignoring your messages). Might require some infrastructure changes.

Recognize and disconnect from evil peers

We should also more completely implement a policy of disconnecting any peer sites that violate the protocol, by (for example) lying about their length fields.

API for making Message Blocks

["MessageBlockingAPI"]

Transactions and message ordering

["TransactionsAndMessageOrder"]

More Access control policies

MoreAccessControlPolicyClasses

Identities and Groups improvements

* More authentication schemes * Slow down repeated password attempts * Clobber buffers containing passwords and password hashes after use * Lock out after many failed authentication attempts * Combine with hypervos (see HyperVosIdeas)

Properties

List accessors to property data.

Property data is basically a chunk of text characters in byte units, but it can be formatted into a list of primitive types (list of floats, list of ints); or a string may be thought of as a list of words or substrings with some other delimiter. It would be useful to access and change an item in one of these lists without parsing the string each time. Alternatively, the data might be a packed sequence of fixed-width values or an n-dimensional array stored in sequence in the property data (e.g. 3D mesh object point coordinates). Addressing specific bytes or byte ranges in the data is already possible, but it would also be useful to access individual members of a list of stringified types, or to access a member of an array which has been stored in sequence in the property data (with the byte-width of values, and the grouping of subsequences into subarrays taken care of transparently). I'm not sure what the appropriate C++ API is for this. Here are three: * have a whole bunch of wordy accessor methods like

float Property::getFloatListMemberAt(int i); float Property::getPackedFloatArrayMember2D(int i, int j)

* operator:

template<T> T operator[size_t i];

* templatized adapter classes

We can support various ways of interpreting the data: if the return type T is int, float, double, bool, or std::string, and property is a list, return the list item at i; if T is a fixed size type -- char, int16_t, int32_t, etc. then return sizeof(T) bytes from the property data buffer at offset (i * sizeof(T)). If T is aconst pointer type, return a pointer into the property data. Any ideas?

For properties that are lists, accept messages to search the list-
- return whether a given value is in the list (and return first position
  - where it was found)
- return a certain item in the list (by position)
- return the number of items in the list
- insert an item into the list, by position
- sort the list?

Other property accessors

* Add << and >> operators for use with iostreams

Translations

Property subtype/subclass called property.trans that includes child properties that contain translations of the parent property in different languages (named using those languages' ISO codes). Only useful for text datatypes of course. The C++ MetaObject class for this type should have a static list of preferred languages, and try to read that one if it exists. Also would need a specific API to write to a translation, instead of the original. Each translated property would also know what it's original language was, and reflect that in a child automatically.

Optimizing Multi Core Access

* Option to align and pad property objects to cache lines to avoid cache line contention by multiple threads * Can write buffering reduce cache contention?

Peer-to-peer Property Sharing (Bittorrent style)

A new property metaobject subtype adds child vobjects to a property that store a list of alternate sources or mirrors of the property. These are really just remote objects on clients for this exact local property, but if you make a request through those clients, you should get their cached copy.

In Bittorrent terms, the original property has a "tracker" as a child.

If a message/request fails, then you have to try another one or fall back on the original.

But what about listening? Can a remote property listen to another remote property? Or would those remote properties alse have to be a special kind of property that specifically supported caching?

Also, if listening, you have to also listen to the tracker and choose a new cache if the one you're listening to goes away. (And be able to add a listener that does not result in an immediate notification send, as an optional flag in the add-listener message.)

Along with links to remote caches, the tracker could provide precomputed info about the caches like network segments they're on or ping times averaged from various clients.

Remote caches could also keep their own mirror of the tracker list, so if they insert their remote cache into another context, yet more remote clients can use that tracker cache rather than the central one-- also needs a recovery scheme.

The tracker, as an option, could also to verifications that clients claiming to share the file actually are (check purported checksums and lengths, and occasionally do a test download and compute a checksum and verify that).

We could also model it on the way CoralCache servers share cached data behind the scenes (Distributed Sloppy Hash Table algorithms). See http://www.coralcdn.org/pubs/ .

Misc MetaObject library (libmetaobject_misc)

Talkative Private Messaging improvements

Go to the ["Chat, Presence and Messaging Ideas"] page.

General purpose message relay

See RelayMessages. (This could also be a Site Extension.)

Computer Status

Changes to Metadata types

* See http://www.metadataworkinggroup.com/pdf/mwg_guidance.pdf for a (slightly IP-encumbered) guideline for common image file metadata features

Site Extensions, Persistence

* inspect-types: Pass it a type string, and it tells you what messages a type wants, what properties it has, etc. (Basically returns an OTD, but could be structured as a VOS message for easier parsing at the moment, since libotdutil is dead and was pretty limited anyway.) Then have a tool that makes a WSDL (web service description language) document for a type from this information (from an OTD document or from the message returned by inspect-types site extension.) * Web service gateway site extension (e.g. translate SOAP or whatever to VOS, if feasible.)

* Use metaobjects to enable fspersist publishing of a directory for one metaboject (e.g. set "filesystem" metattype and set the directory name in a property, and that whole directory is automatically available) and possibly its children as well? (Or use a UI tool to apply the metaobject type to each child). This would be useful for hypervos.

File import/export extensions

See ImportExportFileFormats

libvos language bindings and implementations

Python features

* Python Listener subclass that dynamically checks for existance of a function named for the property or object aspect that changed and calls it. E.g. if a property "foo" changes, call "propertyFooChanged" if it exists. (Or hook into a native Python Property notification system if such a thing exists)

* Override some operators to make it easy to set up objects and set properties.

Other bindings

Alternative libvos Implementation in ObjC

This is a somewhat ambitious idea-- to write a secondary/alternative implementation of all or part of libvos in Objective-C. (Could also only implement Vobject and MetaObject related classes in Objective-C, and still use utility classes from C++). Objective-C, while not well known and therefore not a good implementation language for VOS in general, offers many very interesting features that fit with the VOS design (especially concerning dynamic types, message delegation, external (Categorical) interception of messages (for remote invocation), etc.

Etoile is an interesting GUI framework that might fit well with VOS also.

C layer to facilitate language bindings

A light C wrapper around libvos for the core libvos operations,

and/or,

A C implementation of the most essential stuff --

sending/receiving messages.

Use SWIG to generate full bindings for libvos

This is in progress but is turning out to be difficult since libvos make extensive use of templates and other fancy C++ stuff... The future S5 series may instead use our own code generator to generate language bindings directly.

Visual Basic .NET

Could be done via SWIG, or the lightweight C wrapper, or a new implementation. Would be really useful for people who need to make quick applications, or interoperate with existing systems based on MS infrastructure. C# might be useful too.

R

http://www.r-project.org

Integrated into the framework as data sources, and also connecting to IRUI (see UserInterface) for control of parameters etc., for online code editing, and for visualization and report outputs.

Haskell

Protocol Adapters

To bridge to other message protocols, or just trigger preset VOS messages on events from other systems, including HTTP requests itself (See HyperVosIdeas), or HTTP-based protocols (XML-RPC etc.), OpenSoundControl, MIDI? ... dbus?

Misc ideas that could be in core, or new libraries

Adaptive access "barriers"

Step up access bariers as a function of activity. I.e. if nobody has posted on a blog, minimal hoops to jump through. If lots of people have posted in the past, require access control or capchas or moderation or other hoops.

Same for a virtual world. If there's nobody there, so what? If there are lots of people, add more restriction automatically.

This can be extended to act on patterns of activity as well that utilize that stored information about past activity. I.e. if more blog commentsinclude more keywords (from previous posts, from the article, from previus posters' names), then its more likely it's OK and we can give appropriate barriers.

SMS and Email Connectivity

Plugins or extensions that make it easy for users to interact with VOS using email and SMS. For example, get text rendered views of queries or searches (Gopher, basically!), or to send messages that create new objects (see the vos-process-email hack in s4 apps/tools). See http://www.frontlinesms.com/ for inspiration on a tool for managing SMS communication.

* Full email interface to VOS messages, with patterns to specify message

commands, extracting fields in a smart way, etc. Construct emails using triggering VOS message plus template stored in vobjects (same template mechanism as hypervos)

Implement Distributed Hash Table

http://en.wikipedia.org/wiki/Distributed_hash_table

Vobjects containig listeners implementing generic rules

Easy for users to manipulate and set up.

Components that handle housekeeping for online editing

Objects or properties under an object contain source code that implement listeners/message handlers. When saved (or given trigger message), make a new listener and swap it for the old one. (Depending on language (i.e. C++) could recompile and reload dynamic plugin object. Redirect compilation messages as vos log messages.) How about reloading a whole site (C++)? Need to save and restore all remote links though. (See above)

* Generic and configurable meta-listener that fires messages based on changes to

various properties or vobjects, with simple rule specification (cf email filtering rules, zendesk's rules, etc.). Should be somewhat easy to wrap in a web or native UI.

RandomLibraryIdeas (last edited 2009-09-29 15:07:39 by ReedHedges)