• HyperVosIdeas

Hypervos was the name of the code in S4 that used Vobjects to represent an XML document (or a template for an XML document), and a site extension turns them into text fetchable by HTTP. Those XML objects can be set up by hand, or generated by automatic proceses from some other vobjects (e.g. the hypervos_news plugin does this to contstruct HTML and RSS documents from a more abstract list of "news items") -- or any number of levels of processing.

See also HyperVos and http://interreality.org/projects/hypervos

Here are some ideas for improving it.

The vast majority of these ideas also apply to VosWeb in S5

TableOfContents

See also ApplicationToolbox for other things that are useful for hypervos, but not specific to it.

See also HyperVosDoc for documentation on existing hypervos features.

One general goal of Hypervos is to be a toolbox for you to construct your own site, and mix and match features. I.e. in user interface terms, there is no preexisting premade framework or application. You have to create it by constructing vobject structures.

Changes to currently implemented parts

• Do more processing of plain text properties to turn them into HTML??
  • (optional):
    • Insert paragraph elements when text has two line breaks

    • Turn URLs into <a> or <img> elements.

Meta or infrastructure stuff

The type and child prefix "hypervos" is sort of weird. It should be replaced by "www" maybe?

Instead of constructing the page document for each request, the first request for a page (or for any object) should create a listener which updates a cache whenever any part of the page changes. The web server then just returns the cache on each subsequent request. If a client wants updates pushed (WebSockets, AJAX polling, whatever), then these can easily be sent when or soon after the cache is updated. If no requests for the cached page come in for some period of time, the listener and cache are deleted and we start over.

In S4, most of HyperVOS was implemented in a site extension, specifically in HTTPServer::doGET(). In an S5-based hypervos, most of the page-construction logic should be in components (metaobjects) or a more widely usable library (e.g. could be included inside the UserInterface app), plus a simple standalone webserver or apache plugin (it doesn't really need to be a "site extension", and S5 doesn't have them I don't think).

Alternative Template Techniques and other Template Improvements

The current template system is not always usable. For example, in metaobjet where you need to use a certain child contextual name, and can't use a special templace substitution dummy object. Here are some additional ways that could be added:

• A "substitute value" property subtype. Instead of naming the property for the substitution, use this property subtype, and put the template parameter name in the property value or as part of the object type.
  • Perhaps allow any number of parameter tokens to be embedded at any point in the property value.
• "insertions" child object for template instances, contains directives that cause objects to seem to have been inserted at certain places (by name or position) within the template target. Then you can use any vobject structure as a template. An insertion object would have two children, one a property giving the relative path to where the object will appear to have been inserted, the other the object to pretend was inserted.
• Use listeners to update an output Vobject structure based on an inputs and templates? Only change output structure when an input or template changes. Makes returning pages faster.

• This could develop into a GeneralVosTemplate tool.

• Put substitutions in a seperate child (sibling to the template/prototype object tree)

• In the S5 UserInterface client, make a list of placeholders for possible substitutions, so users don't have to remember names, they can just drag and drop objects onto the placeholders, or click to create a new object there.

• Make possible substitutions be a specific metaobject type, and allow any name (and include title and description for UserInterface to use)

• automatic "page-wraper" mode: if you access an object that does not represent a known top-level XML element, then automatically wrap it in a predefined wrapper or template (defined in app. configuration, not attached to the object). Then you don't have to set up page header/footer stuff for each object you might want to reference. This would let you use any object as a full web page (or xml doc).

General ideas

• AntiAbuseFramework

• Automatically gather all xml:html (and other known "document" objects) into a group with "ui:workspace.webpages" type (Use a "gather" listener from ApplicationToolbox). Tag container object

used to serve HTTP requests with "ui:workspace.website" or "wwwroot" type.

• Only include property values if the property object also has type hypervos:text? And protect it with <!CDATA[...]> if it has type xml:protect-cdata? This would make it posssible to mix in non-hypertext vobjects in with hypertext vobjects and let hypervos ignore the non-hypertext objects, at the expense of making it a bit more difficult to author (though in the current design you still have to be aware that you use the "property:property" type to include text, this just adds a second one specific to hypervos)

• Listener that on every change, finds ancestor top-level elements (e.g. xml:html) and saves the whole output document as local files, or stores them in memory, or uploads them somewhere else via ftp/ssh/WEBDAV (uploaded or saved files can be efficiently served by apache, documents saved in memory can be efficiently served by the hypervos http server).
  • This also gives us the opportunity to be smart about Expires and E-Tag headers. (Even if the result page isn't cached, a listener that keeps track of changes helps.)
  • Or, optionally, instead of immediatly saving the files or uploading them, remember those top-level objects with modified children, and the site can accept a "save" or "upload" message to actually do the final saving or uploading.
  • could also call hooks that check for standards compliance and broken
  • optimally, the listeners would flag the page for regeneration after a very small timeout, and reset the timeout. Then a flurry of changes all at once (messageblock?) on one page would only result in one page write out.
  • May result in rewriting to always use a listener/property filter to propagate change notification from bottom up into bigger and bigger properties (even when not saving pages to disk, but resulting in a single property for each page for more efficient return). This may require optimizing property data copying and listeners a bit.
• Listener on page elements could check spelling and for broken links, setting flag properties on errors.
• General VOS thing: listeners could set an error state that prevent subsequent listeners from being called, and calls an error listener?

• Not part of hypervos but would be useful for editing sites: VobjectEditingGui

• Use top-level element (e.g. xml:html, which is <html>), to determine a DOCTYPE declration to return over HTTP. [DONE? partially?]

• Proper HTTP error codes (e.g. 404 for NoSuchObjectError, etc.) [DONE?]

• Add Last-Modified headers if document object has a misc:modified-time property. (Or ETag?). Respect conditional GETs and respond with 304 Not Modified.
• Flag on objects to require HTTP authentication; more advanced implementation could use an Identity object to verify username and attempt to log in to it. Might need to create a new site to represent their session?

• Special property filter tool that provides HTML output given another property as input that contains wiki-like syntax text. (See VosToolbox)

• Special property filter tool that escapes HTML special characters (greater than, less than, ampersand)
• Property filter that closes any HTML tags opened in the object.
• Some generic way of seperating large numbers of objects into pages (with first/next links, maybe an index)

• Configuration section that automatically sets up a template for any vobject type (including invented subtypes of hypervos/www:template), and also gives child names to use for template value substitutions (since they probably won't match the placeholders in the template). Could maybe use a generic listener from the ApplicationToolbox.

• Develop concept of "output formats" (currently have RDF and HTML).
  • Add graphviz output format?
  • Output formats that enumerate files in a zip or tgz archive
  • Translate PDF, Word, etc. into HTML?
• A factory that automatically creates a filter object using another object as input based on that other object's types or property datatypes. For example, in a forum or wiki, the CGI object creating the post would create a new property object containing the chunk of text posted by a user with metatypes of "userpost" and "datatype:text/plain". Then, in the app configuration section, specify that objcects with type "userpost" should have property filters applied to automatically create wiki links, and objects with both "userpost" and "datatype:text/plain" types should automatically detect URLs, and do other text-to-html translation, and escape HTML, while objects with both "userpost" and "datatype:text/html" should have the filter that closes tags. Or, another approach, would be to do all the filtering locally in the property; then the factory just has to apply the metaobject types to the property that add in hooks called during the local property's read() handler. This way is a bit simpler, though it

can't be remotely manipulated after the fact. Perhaps both of these approaches could be implemented, and share code.

• Automatically add a class to HTML objects based on their meta types. (Including having a mapping of types to classes in configuration section.)
• Automatically set element ID attributes based on site name if no ID attribute is given. (DONE)
• Object listener/metaobject/output filter that adds extra CSS classes to elements based on various criteria.

  • Check <a> elements, and add CSS classes for external links, broken links, etc. This can add style information and even little icons.

  • based on types

• Property filter or output filter that searches text for keywords (listed as children of some vobject) and inserts a reference to that keyword's target object. THis can be used to make wiki-like links, keyword links, etc. To investigate: what are the performance charactaristics of finding an object by cname in a huge child list? And generally working with an object that has a huge child list (hundreds of thousands of entries). (See also VosToolbox)

• Listener to automatically gather metadata properties from desncendent objects and insert as HTML meta objects in output document under its xml:head object.
• Build CSS stylesheet out of vobjects?
• Make information from HTTP request and user session available during handling of a request. (E.g. referer)

  • Record request info in an object when that object is requested (optionally). (Can then use a "gatherer" from the VosApplicationToolbox to collect them together to look at.)

  • Make request info available to templates used to post new objects in response to a request (e.g. comment on a blog can include referer)
  • Generally, request and session info can be used by custom handler code, maybe even listeners fired when a request handler creates objects or changes properties
• Generalize the news listener that generates HTML pages -- it should create a template instance for new objects. Listener would be configured as to what template to use and what parameters to set.
• When a GET request is recieved, check if requested object has some type like "Load-from-file", and if so, load the file (unless already loaded), process those objects, then set (or reset) a timer that will excise them after some time. (or excise them immediately.)
• Property filter that strips whitespace and comments from Javascript, CSS, etc.
• Object type that redirects to some other URL (use HTTP return code)
• Tool that generates a search-engine friendly HTML sitemap and engine-specific XML maps (google, yahoo have formats for files you can submit to them). Could be all pages, or selected pages, etc.
• Admin control panel (web page):
  • Use an omnivos plugin to monitor a directory for files and load or reload them as vobjects.
  • Button that copies one of these default configurations to a new subtree (i.e. creating a new blog for a new person). Or spawn a new omnivos to host it, and link its parent in to the central site (and also recover from central site restarting by finding them and linking them, or having the other sites keep checking for central site). For the newly created blogs or whatever, they can also use the directory monitor omnivos plugin to offer templates for users to copy or use directly.
  • "Development Overview" page-- contains lists of pages of different types, links to validation services for those pages, links to stats pages, links to those pages in edit/metainfo modes, shows broken links, sumarrizes errors, etc.
• Tool that imports existing website with same URLs. Or creates objects for those URLs that redirect.
• Top level object type that can be used for robots.txt based on links to forbidden or allowed pages.
• Optionally check local disk for files before returning 404.
• Generate alias object links based on recent search activity. Meatdata too. So
  • if people search for "foo", "bar" and "baz" a lot and click through to object "obj", make an alias child for "obj" named "foo", and (optionally) change the title attribute of "obj" to "Obj Title (foo, bar, baz)" or something. Autamtically add "foo", "bar" and "baz" to metadata keywords perhaps? Or have a seperate "auto-keywords" property? This is a SEO technique.
• Thing that automatically adds and removes a "New!" tag to elements (metaobject type? configurable time limit. configurable object to listen to (itself, or destination of a link, etc.)
• Insert some text in output on exception getting a child; configurable; configurable per-page; configurable based on site of the missing child.
• Store/retrieve cookies
• Objects that are simple to use but which "expand" into more complex DHTML widgets, see "Special HTML Tools" below.
• Mini scripts that can check cookies on request?.
• A mode that translates HTML vobjects into PDF (via TeX?). Or, a synchonization tool that creates hypervos vobjects named for TeX commands, with a mode in Hypervos to use TeX syntax instead of XML syntax when generating output for those vobjects.
• Conditionals - a type that affects HTML generation (or choosing a template), conditional on things like client hostname, referrer, cookies, user agent, referrer, (or any HTTP header value) authenticated/logged in, authentication group if logged in etc. This can be used to customize how any part of a site appears to different kinds of visitors. Editing UI should list useful conditional values, such as most recent and most popular referer/UA/etc.
• Pervasive support for keyboard controls. extend any object to have keyboard controls to jump to some other object, activate links, etc. keys are inherited from parent objects. (or they can be set to do nothing in child objects to cancel that behavior for certain keys). Include a widget that is a link that opens hidden tags next to various elements that are the destinations or targets of key actions showing what their keys are.
• Automatically detect things like addresses, currencies, times and dates, and
  • optionally add markup to them.
• Manually tag words to generate links to pages associated with the word, or to
  • an arbitrary URL incorporating the word (e.g. google search or google maps url or amazon search or wikipedia or whatever.)
• Adapt for different browsers. E.g. handhelds, mobile phones. cf.

  • http://ubik.com/ . See also http://en.wikipedia.org/wiki/WURFL , http://wurfl.sourceforge.net/ and http://deviceatlas.com/

• Crisis or slashdot mode. Quickly swap front page for version with only certain contents, perhaps filtered to remove embedded media like images, iframes, objects, etc.
• Designate an object that handles requests for nonexistant objects (404) and other errors. then it can return a custom page, including using any other hypervos feature, doing searches, etc.
• Easy to use built in OpenID provider. Use standard VOS user/auth system.

• Link user authentication to paypal payment/ subscription payment status. Either query Paypal for payment status during login (see https://cms.paypal.com/us/cgi-bin/?cmd=_render-content&content_ID=developer/e_howto_api_nvp_r_GetRecurringPaymentsProfileDetails), or do it asynchronously (based on the recurring payment cycle), and add a user to a group (or move from one group to another) if payment for current cycle has been received, and remove from group (or move to another group) if not yet received (and try again every day or every few days), suspend account if no payment received after trying a few times, and eventually delete account if suspended for too long.

User and group based customization

Generally customize user's experience based on his group

• Switch widget that chooses output based on user and/or group
• Customize which search indices are used for a search based on group

Workflow Infrastructures

• Component/customizable/extensible/flexible way to construct task oriented
  • interfaces for moving objects around or creating new ones.
  • An approval queue workflow-- take a group of objects as input, provide
    • user controls for approving, cancelling, prioritizing, moving to other queues, adding comments or metadata. Approval results in adding the object to an output group. Cancellation results in the object being removed from input group, perhaps moved into a trash can group. Input could be the output of another workflow, or be generated from an external RSS feed or other web syndication method. Could be results from a search.

Edit Mode

A way for people to edit stuff through the web page itself. This includes both property contents (text), and moving and linking objects, creating new ones, etc.

• Small span of editing controls under an object marked as web-editable, or if in webedit mode (url param).
• Do authentication
• Don't edit "core:autogen" objects.

• Button to open a view of the document <head> section, perhaps vobject-description mode with editing controls for the head items, particularly the stylesheet.

  • Eventually could have a more structured view of CSS than just a text field.

    • Show different stylesheets seperately, but merge in a dimmed out display of rules from previous stylesheets (even if overidden) so you can see them and maybe link to them. (Do the same in the UserInterface app)

  • This view could open in the firefox sidebar, in an iframe dynamically added to the document with javascript, or an overlay in the document, or in a popup window.

• Only put [Edit] or [Details] tags on <divs> or <p> and <body> when "editall" mode enabled. This tag toggles invisible controls within all of its contents that do editing things.

• Choosing another object to insert into the current page:

  • Maintain lists of objects by type (i.e. all <heads>s, all <img>s, all

    • objects with certain user-added tpe tags), by certain types or attributes like classes, etc.
  • When editing an object, you can add type tags to it for this purpose, or choose
    • from an existing one.
  • *OR*, when editing an object, you can just choose an existing category to
    • add it to, or create a new category.
• Different widgets and element types could have custom user-friendly editing
  • views.
• Apply a set of types and properties to new objects depending on authenticated user, plus default set for all users.

View Configuration for custom view and for Vobject editing and debugging

• Check cookies and GET (CGI) arguments for display parameters/flags
• Also check various other aspects of the request, like domain name. (E.g. check TLD country code to set default language-- .com or .au or .uk or .ca or .nz means english, .fr means french, .de means german etc. This may actually be a bad idea though. But domain name could be a signal to scripts doing more subtle modification of the page.)
• Check object types for special tags that enable certain parameters for that object only (i.e. a global setting for all viewecrs)
• "Macro" HTML types that generate HTML controls that let the user set these parameters.
• type tag that makes an object included only if a request parameter or cookie (or something else) is set. or a full object that only includes children if parameter or cookie is set.

Possible Parameters/Flags:

• Turn on extra HTML boxes to show what HTML elements correspond to Vobjects
• Turn on extra HTML that shows Vobject structures (all parents, non-HTML children, etc.)
• Turn on form elements that let you edit things (wiki "edit mode"). This also involves CGI and authentication integration (see above and below)
• Override the configured display mode (always-describe, description-format)
• Set language to use when reading properties with translations linked.

Stuff added to HTML output could be normally hidden except on mouseover of the an object in question (like tiddlywiki does) for more elegant use.

• Treat properties that have "alternatives" (e.g. translations or something) specially. Include small links to choose between the alternatives and replace the HTML element with the new one. Or, an "inline" variant where it puts all the alternatives on the page, if it's too long truncate it with a control to expand it with Javascript.

CGI

Some typical things to do with CGI would be to create some new objects using CGI arguments as property values etc.; change the value of a property or properties; ... what else?

A new type would be created to hold cgi info, "www:cgi-action".

One way to support CGI is to directly connect a CGI call to send a message to a vobject with the method and fields given in the CGI query. In the hypervos configuration, or by way of special type strings, or properties you would enable certain message methods for certain objects to be callable via CGI (attempts to use nonallowed methods would be rejected). The CGI parameters would contain the message name and fields, and the hypervos site would send the message to the target object. If specified to do so, it could then wait for a reply before returning the page (and use the reply message in the response somehow, see below).

Or only "www:cgi-action"-type objects could be targets, and be configred to have one target vobject and message name, and an optional mapping of cgi parameter names to vos message names (by default it would just send them along as is). It could also add stuff from cookies, http request headers, etc. Subtypes of the "www:cgi-action" type could be defined for common messages that need to be called from CGI, such as factories. If a target is missing the defaut target could be itself.

For example, an object /news-entries is a factory for news items and has available a message method "create-news-item" that takes fields "title", "author", "summary", "body", "keywords". In the hypervos configuration (or via a new type that /news-entries has), this method with those fields are defined as available through the CGI, and which method(s) to use to get the paremetrs. An object to return as the reply document could also be defined. Then you can create a form on an HTML page that has "/news-entries" as the target, and input fields with the names "title", "author", "summary", "body" and "keywords", and a hidden field named "method" with the value "create-news-item". When that form is submitted, then hypervos recieves the request URL ".../news-entries?method=create-news-item;title=...;author=...;summary=...;body=...;keywords=..." (or perhaps .../news-entries?vos-message;method=create-news-item;title=......), and sends that message to the /news-entries object, and then returns the previously defined result page. Or it waits for the message to be recieved, then returns the output or redirects to another object.

Or, using a dedicated "web:cgi-action" object with one message, you add a new web:cgi-action type object named something like "create-news-item" with a "target" child pointing to /news-entries and a "message" property containing the text "create-news-item".

Three CGI access modes can be supported-- ways to get the parameter values out of the HTTP request -- GET, POST, and XMLRPC (given as XML in the request body). To parse XMLRPC extra information is needed about the order of the parameters. An email gateway could also use a similar approach to initiate pre-designated message methods to target objects; this would be a useful alternative to forms that could easily be used without any extra effort by the user. For dynamic pages in response to a query, special CGI objects could do a vobject search query and only use those results to construct a page, or it could be a hypervos:template object but use a set of template substitution objects or strings returned in the reply to the CGI message.

The new metaobject type, "www:cgi-action", with children:

• www:cgi-target (if missing use self)
• www:cgi-method "get", "post", "get|post", or "xml-rpc"

If using the first approach, with many messages possible, specified in a parameter:

• www:cgi-messages
  • [method name]
    • add-fields
      • [additional field to add]: property containincg value of field to always add when sending message
      • ...
    • rename-fields
      • [http parameter]: property with name of vos message field to use instead
      • ...
    • wait-for-reply: boolean property (Or use a subtype of cgi-action?)
    • template-substitutions (only valid if www:cgi-target is a www:template, or we also have a www:template child to use) (or use a subtype of cgi-action typet o indicate this?)
      • [field name]: property with value substitution object name (e.g. "$val")
  • www:cgi-default-message (if given, use this message if no "method" parameter was given in the query). this child shoud link to one of the children of www:cgi-messages.

If using the second approach, with one message possible:

• message [property]
  • add-fields
    • ...
  • wait-for-reply
  • template-substs
  • rename-fields
    • ...

Behavior could also be different for GET and POST. GET should generally be used to set view flags, POST and XML-RPC to send a message to an object. This would align better with the purposes of GET and POST.

Anti-XSS tools

Some ideas from Jeff Atwood (http://www.codinghorror.com/blog/archives/001175.html):

The Felten and Zeller paper (pdf) recommends the "double-submitted cookie" method to prevent XSRF:

• When a user visits a site, the site should generate a (cryptographically strong) pseudorandom value and set it as a cookie on the user’s machine. The site should require every form submission to include this pseudorandom value as a form value and also as a cookie value. When a POST request is sent to the site, the request should only be considered valid if the form value and the cookie value are the same. When an attacker submits a form on behalf of a user, he can only modify the values of the form. An attacker cannot read any data sent from the server or modify cookie values, per the same-origin policy. This means that while an attacker can send any value he wants with the form, he will be unable to modify or read the value stored in the cookie. Since the cookie value and the form value must be the same, the attacker will be unable to successfully submit a form unless he is able to guess the pseudorandom value.

The advantage of this approach is that it requires no server state; you simply set the cookie value once, then every HTTP POST checks to ensure that one of the submitted <input> values contains the exact same cookie value. Any difference between the two means a possible XSRF attack.

An even stronger, albeit more complex, prevention method is to leverage server state -- to generate (and track, with timeout) a unique random key for every single HTML FORM you send down to the client. We use a variant of this method on Stack Overflow with great success. That's why with every <form> you'll see the following:

• <input id="fkey" name="fkey" type="hidden" value="df8652852f139" />

[From comments on Jeff's blog: The two keys don't need to be identical, but must have some 1:1 equivalence mapping of some kind (perhaps two public keys generated from the same private key that the server remembers as part of the user "session"?); This technique can be applied to any request sent back to the server, including AJAX requests; check all POSTs/AJAX requests for corret HTTP Referer. Also, you need to set cookies with the HttpOnly tag for these things to work.]

Application Ideas and User Stories

News/blog application

Can be embedded in any part of a web page.

• NOTE currently implemented a bit backwards. It's configuration is centralized and seperate from the pages, which would make it difficult to work with Edit Mode. Output objects ("hypervos:news-view" and "hypervos:rss-news-view" types?) should be metaobjects that point to their data sources and update themselves as the data source changes.

• add prev/next links (links, and also <link> HTML elments in document head) to news entry pages

  • need to extend template system to have minitemplates that are inserted under some target object? e.g.

    • $$nextlink_parent -> body/nav/nexlink-container-div $$nextlink_template {

      • xml:a href=$nextlink_url {
        • NEXT
        }
      }

  • maybe use a reusable widget that finds/listens-to the current document's <link>

    • elements in its head
• maintain link to most recent news entry's page
• accept new news entries by email

• add <link> objects to HTML linking to archives

• Accept messages using XML-RPC MetaWeblog or AtomPub API with new posts (requires first implementing CGI objects, and maybe a relay that parses XML-RPC stuff.) http://www.xmlrpc.com/metaWeblogApi , http://www.atomenabled.org/

  • Generally implement ATOM publishing apis
• Improved tagging/keywords -- link to keywords/tags as vobjects, not just things in a string. link related tag objects together (e.g. "tag" and "tag?" are related). Resolve minor othographic differences between what ought to be the same tag (e.g. tense, number, collapse punctuation and space, squash capitalization. learn synonyms?)
• Objects that output HTML view of some other object's tags. I.e. typical
  • tag-size-by-popularity, or a list of the X most popular, or the X least popular, or the X most unusual compared to total population of tags on the site, etc.
• Push posts and changes out to popular websites like myspace, facebook, flickr, picassa etc. to keep them in sync (or maybe an interco product?). Also pull profile change notifications? And bring in new items added to those other services?
• * Control behavior by post type (e.g. photos, status updates, news, link sharing, etc.)

• * See HyperVosForumApplication

• Configurable admin UI (and able to grab UI from a stored default, to revert changes if you screw it up)
• Widget with a form for adding comments (see below)
• Filter by any object type. So you could point it at any page on a site and automatically get one or more RSS feeds out of it. By default filter by "news-item".

• Different news item types. E.g. facebook has "status", "notes", "shared links", photos & video posts, "write" comments (not sure how that's supposed to be different from notes), events, favorite/shared/linked-to pages, etc.

Wiki Application

Meant to be combined with other applications. E.g. forum posts could be styled with wiki formatting, and links embedded with brackets. Forum posts could become potential wiki-like link targets, or a button could be added that adds a particular forum post/thread in to the list of potential wiki link targets.

• requires CGI (see above)
• method that creates a new object (with a template) for new page
• way to toggle editing mode on a page, where each property object in a page designated as wikiable have extra forms attached with editing fields, hidden under a DHTML/javascript expander button.
• apply a filter to each property that translates wiki formatting to html.

• filter properties to create links around words in brackets. when a word in brackets is found, look through "pages" grouping objects to find a vobject with that name to link to. could include hierarchies too, e.g. given text "[MyCategory/MyPage]", use vobject '/pages/MyCategory/MyPage'. If the vobject found is an indirect pointer to an HTTP url, link to that instead of to the hypervos page for the vobject

• to integrate with other wikis/websites, spider them and create hypercard-like indirect link vobjects containing the url, named for them with appropriate names in a wiki pages vobject so you can link to them.

Forum Application

See HyperVosForumApplication

"Silver" Application

(Info not public yet)

See SilverProject.

Product Ratings Website

HypervosRatingsStory

Storefront

HypervosStorefrontStory

HiCompApp

(not public yet)

See HiCompApp

To-do list

cf. Gootodo. Simple todo list with scheduled todos, email integration, nice UI (e.g. ajax submission of new posts with calendar popup when creating or moving items), ical publishing, outlook integration of some kind, able to have daily or other non-scheduled items, able to reschedule items to "1 week", "this weekend", "beginning of next week", "1 month", "beginning of next month", "1-4 weeks random", "5-8 months random", more. Good look on a phone.

Special HTML Widget Components (Widgets) and Filters

Widgets are Metaobjects/components/types. that encapsulate HTML/CSS/Javascript solutions to implement interesting "widgets" or behaviors, or just simplify the HTML structure. For inspiration or code to use. Could use: http://dojotoolkit.org/, http://www.jquery.org, etc.

Filters are things transparently added to pages, configured globally or by flags on the top-level element (i.e. page).

See also application ideas/stories above, which mention some of the features they will need

Ideas include:

• Option on all image and iframe objects to defer loading until after page is done loading (by loading the image with javascript that runs onload). This will prevent external resources (ads etc.) from holding up loading of the whole page. This option can be set per object, or automatically done for any external resource (i.e. not on the same exact host, or not in the same domain [e.g. .foo.com or .foo.co.uk] )
• Hide/show a div when you click a "button" image. With or without sliding animation. Contents may be fetched by ajax or just be a hidden div. The div to show is simply linked to the widget vobject as a child link. Very easy to do! (To fetch via ajax, just does an http request on that vobject, and hypervos processes it as normal. Or, might be neccesary to set a CGI parameter to tell hypervos to wrap it in a JSON structure or something.)
• "hypervos:webpage" widget for a whole HTML page, that can allow hypervos to

  • automatically insert stuff into the output page's <head> based on its contents (e.g. requires jquery or dojo for other widgets to work) and use misc:metadata info to add items to the <head>.

• Lay text out in columns using CSS! (Possibly by also modifying the HTML as well, e.g. inserting span elements as neccesary, or part of generalized framework for breaking object lists into page/column/grid/section chunks)
• widget for a google map, with markers created the location, title, description, etc. metadata properties of a list of vobjects (or an HTML snippet)
• Buttons which change contents of a div, image, or iframe on the same page

• [Done] object with a type "hypervos:link" OR "misc:hypercard" generates a hyperlink in the output language. For HTML this is <a href="...">...</a>.

Has properties for destination object (direct link), text, alt/title/mouseover text; plus check for xml:attributes to supply any other attributes you want (override if overlaps with hypervos:link properties)

• * Thing that automatically gives external links an additional "external link" CSS class so it can be formatted differently using CSS
• * Javascript thing that sends a message back to server when a link is clicked (for analytics, mainly)

• [Done, except for embedded images, cached images, and some other bits] objects with type "hypervos:image" and "hypervos:image.embed" generate either an <img src="..." .../> or <object

data="data:image/foo;base64, ...">...</object> tag, respectively. Properties can give alt text, dimensions, etc. Also check for xml:attributes. The type "hypervos:image" would choose a default method of including an image in the output (<img> element for HTML). Actually the actualy XML output would be determined by the output type (as determined by what the root is, "xml:html" results in HTML output). Other output formats (e.g. RDF, or theoretical future output formats) would result in different output text. Automatically add width= and height= XML attributes to an xml:img element when using "hypervos:image" and the target property has an "image-attributes" child containing properties such as height, width, etc.

• Subtype which given a URL, download that image and cache the image data. Also has a "refresh" action.
• Optional link to an element to swap with the image when clicked or when user clicks a switcher button. (Example use is to provide an interesting facsimile of a book or manuscript as an image, but let user swap for transcription.) Sounds and videos could do the same thing (show transcript). Search engines could be given the transcript instead of the image.

• object to create an HTML "object" or "embed" element for various embedded data types: flash, embedded streaming quicktime, java applet, that takes care of all the crap like classid.

• Use co-types to add "aspects" to things that normally require extra layers of HTML. E.g. map co-types to HTML element classes; if the co-type is on an object that's just cdata, then wrap it in a <span> in the output.

  • "badges" add images next to a paragraph (e.g. "NEW!", "Special Feature!", "Beta")
• Select a branch to output based on request domain name. (e.g. insert a little note or branch of different menu depending on whether the domain name is interreality.org or interreality.net or interreality-consulting.com)
• Select a branch to output based on HTTP login name (or Identity object that was in turn selected from HTTP login)
• Have a seperate output mode for CSS so you can tweak CSS rules with Vobjects.

• See TiddlyWiki for an example of many cool javascript UI tricks: http://www.tiddlywiki.com

• Make it simple to quickly create an object that redirects to an external URL (e.g. automatically by a link inserter)
• Digg-style AJAX vote box

• Types to create an element with a reference to another object: "refsrc" type, where the subtype indicates what kind of element, and a a "src" child is used as the target of the "src" or "href" attribute in the element. Optional "alt" child as well. "hypervos:refsrc.script" becomes <script src="..."></script>, "hypervos:refsrc.iframe" becomes <iframe src/"..."></iframe>, "hypervos:refsrc.link" becomes <link href="..." />, hypervos:refsrc.img becomes <img src="..." />. hypervos:refsrc.object becomes <object src="..."></object>. Perhaps also aliases (or rename to) "hypervos:script", "hypervos:iframe", "hypervos:object"/"hypervos:embed".

• Object that displays a little box containing the known referrers (incoming
  • links) to a page (using the built in "hypervos:referrers" thing)
• Object that displays a box showing a list of images, optionally with links to
  • alternative forms (e.g. full image if showing an icon/thumbnail). Can be used to drag and drop image URLs into post boxes, or just as a photo gallery or whatever.

• Object that displays a little box showing "related pages" based on a search or matchings between new and existing elements of some kind. for example, read discussion here: http://battellemedia.com/archives/003271.php

• Object that displays a little box showing page metadata, or union of all
  • metadata from objects included in the page.
• Widget or tool that abstracts the idea of tabs or a more generally, a menu (with layout options [horizontal, vertical, etc.]) showing the current page from the menu in a special style.
• Tool that does service discovery on its local network and finds other web sites (web services) to display in a menu, or finds hypervos objects (chosen by type tags or something) to link to or to include directly
• Widget that controls contents of a div on a page, AJAX style, or by toggling visibility.
• Widget that searches and places contents into a div on a page
• Generic box with header containing title and subtitles and optionally description and other metadata, a footer, different kinds of border, corners, colors and other visual options.
• Filter that automatically adds hyperlinks to elements. E.g. this Javascript function from Tim Bray's web site puts a little # link (I guess so you can copy/paste it):

// Found in http://www.tbray.org/ongoing/ongoing.js:
// stolen wholesale from Zeldman, Willison, and others
//  with thanks

function addpLinks() {
  var paras = document.getElementsByTagName('p');
  for (var i = 0; i < paras.length; i++) {
    var current = paras[i];
    if (/^p-/.test(current.id)) {
      // It's a purple link paragraph
      var plink = document.createElement('a');
      plink.href = document.location.href.split('#')[0] + 
        '#' + current.id;
      plink.className = 'plink';
      plink.appendChild(document.createTextNode(' ¶'));
      current.appendChild(plink);
    }
  }
}

• Type that wraps a UI element in an imagemap and transmits where the mouse
  • clicked on the UI if clicked, either async or along with a CGI post. Hook this in to user/UI analytics/logging in the server.
• A better popup menu than in HTML form. E.g. a grid instead of a list, with pictures or text for items. Targets a generic input, or a hidden field. If no JS, either fall back on a plain HTML popup menu, or inline the grid as a table of radio buttons.
• Widget that makes some objects more reactive to mouse movement (changing colors), and also adds controls for expanding/hiding/deleting chunks. Imagine giving a presentation using a web page as visual reference for points or agenda items.
• Widget that provides a poll or a quiz, storing both individual and cumulative results, optionally visible to users.
• iframe that lists children as "documents" with icons and thumbnail (if an image) and link or just link to document. include metadata under document name in list. user must provide icons but could come from an object that converts files on disk into vobject list (so could be provided by a debian package or standard apache or OS document icons folder).
• Widget that includes a form for users to write comments-- creates new objects
  • to hold the comment and inserts it somewhere (e.g. just prior to the comment form, or under another comment to which this is a reply)
• Floating elements. Given a list of objects, give them wrappers that position them on the page accourding to some properties and formulae, rather than a boring list, and affect other style items e.g. font size and color of whole object or select descendent or child elements using CSS selectors. This is a generalization of a "tag cloud" type display.
• Widget for google ads
• Widget or global filter for Amazon and other affiliate/referral/advertising links of various kinds.
  • tools for rolling your own referal/ad tracking and analysis

  • Support: Amazon (book isbn or item asin), BetterWorldBooks (ISBN), ... others? newegg?

• Widget for google visualization API (image generator service): http://code.google.com/apis/visualization/

• "Title" widget that is a mirror of the document's misc:title metadata property. Use for <title> and <h1> in pages.

• Optionally set form element defaults based on majority of previous inputs.
• Tools to help lay out helpful forms-- put annotations etc. in a side column,
  • put instructions in small print below for each form, get the tab order right, some helpful premade elements such as for addresses, phone numbers, countries, with good server side interpretation/parsing.
• Widget or global filter that embeds Javascript validation hints in forms
• Widget or global filter that adds a popup help box link to all or certain elements automatically (with a template, and getting help text from a hidden property)

• Widget or global filter that automatically adds <abbr> tags to certain strings.

• Widget or global filter that adds javascript that tracks and records peoples scrolling behavior (e.g. scrolled 1/4 down the page before closing or navigating away), how long they spent on a page, etc. clicks on external links, etc. Compare them to goal values stored with the pages in question. Produce reports. [could be a non-free add on.]
• Generate a unique but attractive icon based on IP address, to visually

differentiate posts on a blog (or whatever). Coud also use other inputs (and a combination of inputs). http://www.docuverse.com/blog/donpark/2007/01/18/visual-security-9-block-ip-identification

• Widget that creates other widgets, but fills in their properties first based on defaults provided by the creator widget, or what the parent of the creator is or other objects its linked to; specify which properties of the new widgets can be changed by what users/groups in what modes (edit/view). This facilitates edit mode, site administration/setup stuff, and expanding into customized user-modifiable websites.
• Generalized color scheme specifications that affects:
  • a filter that colorizes an input/"mask"/greyscale image (page icons, decorations, etc.)
  • CSS properties

• "section" objects, that group text with section headings (but result in simple "<hX>...</hX>...paragraphs..." HTML output), section metadata, user interface controls (editing controls, e.g.). Can be nested for subsections (and reduced HTML heading levels), etc.

• Load RSS feeds (RSS 1, RSS 2, Atom) and create/update list of hypervos news items
• Load OPML file and process each RSS feed

• Object that lists of news items (feeds), but attempts to remove duplicates by looking at contents (e.g. >90% match in words in the contents, same URL, certain parts of the content matches, etc.)

Identity, login, access control

• Reject/blacklist/temporarily blacklist users or sites that hammer hypervos with repeated requests for the same object (bots/spammers).
• Each requesting client has its own site that uses "remote" calls to access hypervos site??
• Or each client has a seperate requestor object used (instead of hypervos local site)??
• Limit number of downloads of an object, then block (unpublish) it either for
  • one user, one address pattern (i.e. domain), or all requests.

Performance

• benchmark with ab2:

http://httpd.apache.org/docs/2.0/programs/ab.html

Here are Drupal's results: http://buytaert.net/drupal-5-performance

Some quick initial tests show decent results, though some of the main pages with templates are a bit too slow. not sure if that's because they consist of lots of objects, or because they use templates.

Fix and update RDF output

Here's a message that Karsten Otto wrote to vos-d in December of 2006 (Message-Id: <73C375AA-5544-42E2-B080-B9B11EB39349@inf.fu-berlin.de>) about RDF:

<<<

• Am 05.12.2006 um 01:39 schrieb Reed Hedges:

  > > > > Hi Karsten, the HTTP "hypervos" extension in s4 has a mode where it > > can > > emit a description of the Vobjects in RDF (instead of a simple HTML > > listing) for objects that aren't actually meant to be served as an XML > > document by hypervos. > > > > I don't know if it's correct or not (probably not, I bascially guessed > > at stuff, and I think I see some XML sytax errors even), I added it > > as a > > starting point for someone who knows much more about RDF than I > > do. We > > should probebly improve it and perhaps make it compatible with your > > software (since your use of RDF is, I assume, much richer and > > generally > > better than mine!). > > Now that you mention it, I remember seeing this in the source code. I'll have another look at it, maybe I get the time to tinker with it a bit.

  > > The code is at libs/vos/extensions/http/httpserver.cc in the > > describeVobject_RDF() function. It only describes one Vobject, not a > > whole tree of them. Like this: > > > > <rdf:RDF xmlns:rdf="..." xmlns:rdfs="..." xmlns:vos="..fake schema..." > > xmlns:xsd="...." xml:base="http://interreality.org/otd/"> > > <rdf:Description rdf:about="vip://host/obj"> > > <rdf:type>http://interreality.org/otd/objectType</rdf:type> > > <!-- I guess the idea was that all object types would > > define > > their required children in the > > http://interreality.org/otd/objectType or the > > http://interreality.org/otd/ namespace --> > > <childContextualName1 rdf:resource="vip://host/childObj" /> > > <childContextualName2 rdfs:datatype="string">property object > > value</childContextualName2> > > </rdf:Description> > > </rdf:RDF> > > This basically looks good, though you should not use xml:base this way in RDF. Typically, you have qnames for all RDF properties, and only use the base to resolve rdf:ID names and relative URIs in rdf:about. Also, I don't use a single xmlns:vos, but several namespaces corresponding to the "package names" you have in VOS. For example,

  I'd use xmlns:a3dl="http://interreality.org/otd/a3dl/" as a sort of "include", and then write the a3dl:position contextual name as <a3dl:position rdf:resource="..."/> element. I like the concise way you handle VOS properties, but since property:Property is just another metatype, it could well be mixed with other metatypes that require children; you cannot do that with

  • RDF labels, unfortunately. So instead of

  <childContextualName2 rdfs:datatype="string">property object value</ childContextualName2> I model property Vobjects as RDF resources of their own, which also preserves their VOS URI. Serializing this in a nested way, you get for example:

  <rdf:Description rdf:about="vip://host/obj">

  • <rdf:type rdf:resource="http://interreality.org/otd/a3dl/Cone"/> <a3dl:position>

    • <rdf:Description rdf:about="vip://host/childProperty">

      • <rdf:value>1 2 3</rdf:value> <vos:datatype>list: float</vos:datatype> <vos:length>5</vos:length>

      </rdf:Description>

    <a3dl:position>

  </rdf:Description>

  • Of course, you can abbreviate this even more in RDF/XML if you want:

  <rdf:Description rdf:about="vip://host/obj">

  • <rdf:type rdf:resource="http://interreality.org/otd/a3dl/Cone"/> <a3dl:position>

    • <rdf:Description rdf:about="vip://host/childProperty"

      • rdf:value="1 2 3" vos:datatype="list: float" vos:length="5"/>

    </a3dl:position>

  </rdf:Description>

  Means the same Actually, to preserve ordering, I use two RDF properties with the same rdf:about target:

  <rdf:Description rdf:about="vip://host/obj">

  • <rdf:type rdf:resource="http://interreality.org/otd/a3dl/Cone"/> <a3dl:position rdf:resource="vip://host/childProperty"/> <rdf:_1 rdf:resource="vip://host/childProperty"/>

  </rdf:Description> and describe vip://host/childProperty separately, instead of embedding it as above. Regards, Karsten

}}}

Vague or Major Ideas

• Need to keep authors updated on who has access to private or semiprivate resources; what pages of theirs are still published and being read

(and how often), etc. "upward" links from "author" metadata to the author's identity object?

• Need to keep it easy to retain URLs forever

Live updates to pages from vos events

• Listen to some kind of persistant connection with Javascript, and recieve updates from hypervos on events (i.e. like a listener-- HTML element

inserted or removed or changed event created from Vobject or Property events). This requires that all elements have an id (By default it would be its vobject name).

• There is something called "Comet" (using a protocol called Bayeux) that is designed for this, and I think is supported by the Dojo javascript

toolkit. (Dojo has general purpose networking/binding stuff too, if we wanted to use our own protocol). See http://jotlive.com for an example. More info is linked from the dojo webpage.

Resources

• Xapian search engine library: http://www.xapian.org