WebXR DOM Overlays Module

1. Introduction

1.1. Overview

2. HTML API Integration

This module adds a new event type to the definition of GlobalEventHandlers.

2.1. onbeforexrselect

An XRSessionEvent of type beforexrselect is dispatched on the DOM overlay element before generating a WebXR selectstart input event if the input source’s targetRaySpace intersects the DOM overlay element at the time the input device’s primary action is triggered.

partial interface mixin GlobalEventHandlers {
  attribute EventHandler onbeforexrselect;
};

This event is an XRSessionEvent with type beforexrselect that bubbles, is cancelable, and is composed. Its target element is the topmost event target being intersected by the targetRaySpace and is either a descendant of the DOM overlay element or the DOM overlay element itself.

Cancelling this event by calling preventDefault() suppresses default WebXR input events that would normally be generated by the input source for this primary action. The selectstart, selectend, and select events will not be fired for this interaction sequence.

Note: Future WebXR modules MAY define additional events or WebXR input dependent data that are affected by cancelling this event, for example suppressing results from a transient input source’s hit test subscription.

This event and the actions taken by the event handler have no effect on DOM event processing, and are not synchronized with DOM event dispatch. The user’s action will separately generate appropriate DOM events such as `"pointerdown"`, and those DOM events can happen before or after the corresponding beforexrselect event. This happens regardless of whether the beforexrselect event was cancelled or not, and is independent of any further actions taken in XR input event handlers.

Note: This event provides a way for applications to suppress duplicate XR input events while the user is interacting with a DOM UI. Since this is a bubbling event, the application can register handlers on appropriate container elements, effectively marking regions of the DOM overlay as blocking XR input. This is independent of the visual opacity of DOM elements. It is possible to show noninteractive opaque or translucent DOM content such as text explanations that don’t block XR input events.

document.getElementById('button-container').addEventListener(
  'beforexrselect', ev => ev.preventDefault());

3. WebXR Device API Integration

This module expands the definitions of XRSessionInit and XRSession, and modifies the behavior of XRInputSource events.

3.1. XRSessionInit

This module introduces the string dom-overlay as a new valid feature descriptor for use in the requiredFeatures or optionalFeatures sequences for immersive sessions.

A device is capable of supporting the DOM overlay feature if it provides a way for the user to see and interact with DOM content for the duration of the immersive session.

NOTE: Implementation choices include a fullscreen overlay on a handheld AR device, or a floating rectangle in space for a VR or AR headset.

The DOM content MUST be composited as if it were the topmost content layer. It MUST NOT be occluded by content from the XRWebGLLayer or by images from a passthrough camera for an AR device. Applications can use normal CSS rules to control transparency and 2D placement of content within the DOM overlay itself.

The DOM overlay MUST be automatically visible to the user from the start of the session, without requiring the user to press buttons or take other manual actions to make it visible.

NOTE: A device should not claim to support a DOM overlay if the content element is only indirectly visible, for example if the user would need to take off their headset or manually enable a passthrough camera to view content on a separate 2D monitor that’s not normally visible during the session. However, an immersive CAVE system where a user is carrying a physical touchscreen device showing the DOM overlay content would be a valid implementation.

The XRSessionInit dictionary is expanded by adding a new domOverlay member. This is an optional member of XRSessionInit, but it MUST be specified when using the DOM overlay feature since there is no default overlay element.

partial dictionary XRSessionInit {
  XRDOMOverlayInit? domOverlay;
};

If the DOM overlay feature is a required feature but the application did not supply a domOverlay member, the UA MUST treat this as an unresolved required feature and reject the requestSession() promise with a NotSupportedError. If it was requested as an optional feature, the UA MUST ignore the feature request and not enable a DOM overlay.

NOTE: The UA MAY emit local warnings such as developer console messages explaining why the DOM overlay was not enabled.

3.2. XRSession

This module extends the XRSession interface to add a new readonly attribute which reflects the current state of the active DOM overlay element.

partial interface XRSession {
  readonly attribute XRDOMOverlayState? domOverlayState;
};

The domOverlayState attribute MUST be null if the dom-overlay feature is not supported or not enabled.

If the feature is enabled, the attribute value MUST be present.

NOTE: Applications can check the presence domOverlayState to verify that the DOM overlay feature is enabled and working, for example if it was requested as an optional feature.

NOTE: The DOM overlay may be temporarily invisible to the user, for example if the user agent places it at a fixed orientation or location where it may end up outside the user’s field of view after user movement. The domOverlayState attribute still remains set while this is happening.

While the session is active with a visible DOM overlay, the UA MUST treat this as rendering opportunity and execute Window requestAnimationFrame() callbacks at a rate suitable for animating DOM content. These MAY run at different times and frequencies than requestAnimationFrame() callbacks as used for drawing XRWebGLLayer content.

3.3. XRInputSource

When an XRInputSource begins the platform-specific action corresponding as its primary action the UA MUST run the following steps before starting input processing to decide if this is treated as a primary action:

NOTE: Effectively, cancelling the beforexrselect event suppresses XR input select events, none of selectstart, selectend, or select are generated for this action. For transient input sources, inputsourceschange events are still generated, but cancelling the beforexrselect event causes the action to be treated as an auxiliary action, similar to a secondary finger input.

4. Initialization

The application MUST provide configuration for the DOM overlay through the domOverlay dictionary.

dictionary XRDOMOverlayInit {
  required Element root;
};

The root attribute specifies the DOM element that will be displayed to the user as the content of the DOM overlay. This is a required attribute, there is no default.

let uiElement = document.getElementById('ui');
navigator.xr.requestSession('immersive-ar', {
    optionalFeatures: ['dom-overlay'],
    domOverlay: { root: uiElement } }).then((session) => {
    // session.domOverlayState.type is now set if supported,
    // or is null if the feature is not supported.
  }
}

While active, the DOM overlay element is automatically resized to fill the dimensions of the UA-provided DOM overlay rectangle. Its background color is automatically styled as transparent for the duration of the session.

NOTE: A UA MAY use the Fullscreen API Standard §user-agent-level-style-sheet-defaults to style the DOM overlay element, with an additional rule containing background-color: rgba(0,0,0,0) !important; to set the background transparent.

Once the session is active, the domOverlayState attribute provides information about the DOM overlay.

enum XRDOMOverlayType {
  "screen",
  "floating"
};

dictionary XRDOMOverlayState {
  XRDOMOverlayType type; 
};

• An overlay type of screen indicates that the DOM overlay element covers the entire physical screen for a screen-based device, for example handheld AR. Its visual extent MUST match the visual extent of the XRViewport(s) used for XRWebGLLayer rendering. For a monoscopic display, this is a single viewport. A steroscopic display screen would provide two viewports, in that case the DOM overlay MUST be rendered at the Z position matching the physical screen location, appearing identically in both eye views.

• An overlay type of floating indicates that the DOM overlay appears as a floating rectangle in space. The initial location of this rectangle in world space is up to the UA, and the UA MAY move it during the session to keep it in view, or support user-initiated manual placement.

NOTE: Future versions of this spec may add additional attributes to the overlay state, for example the current location in world space for a floating overlay, or additional modes such as head-locked.

5. Event handling for cross-origin content

The user agent MUST NOT provide poses or gamepad input state for user interactions with cross-origin content such as an HTMLIFrameElement nested within the DOM overlay element.

A user agent MAY meet this requirement by preventing user interactions with cross-origin content, for example by blocking DOM events that would normally be received by that content, or by not loading and displaying cross-origin content at all.

If the user agent supports interactions with cross-origin content in the DOM overlay, and if an input source’s targetRaySpace intersects cross-origin content as the topmost event target, the UA MUST enable WebXR Device API §limiting-header data adjustment, and populate the pose for XRSpaces associated with that input source accordingly treating the limit boolean as true. In addition, the UA MUST NOT update gamepad data for this input source while poses are limited.

NOTE: The application does not receive pose updates for the controller or its targeting ray while poses are limited in this way. The UA is responsible for drawing a pointer ray or other appropriate visualization as needed to enable interactions.

NOTE: the restriction on updating gamepad data is intended to avoid information leakage from interactions with the cross-origin content to the application. For example, if an input source’s primary action uses an analog trigger where the primary action happens at a certain trigger threshold, the application could infer when the user started and ended a primary action from the trigger value even if the corresponding events are blocked. Another example would be a device that uses a trackpad or joystick for text input in DOM content, where reading the axis values would allow the application to infer what text was being entered.

If a primary action ends inside cross-origin content, the UA MUST treat the primary action as cancelled, and MUST NOT send a select event. The UA MUST send the selectend event using the last available pose before entering the cross-origin content due to treating poses as limited.

If the input is a transient input source, and if the transient action begins inside cross-origin content, the user agent MUST delay adding the input source until the input location moves out of the cross-origin content. If the transient action ends while still inside the cross-origin content, the transient input source does not get added at all.

NOTE: On a handheld AR device using screen-mode input, this means that touches that stay inside cross-origin content don’t create an input source or associated XR input events. If a drag movement starts inside cross-origin content, the input source is created at the location where the touch location leaves the cross-origin content, emitting a cancelable beforexrselect event as usual.

6. Security, Privacy, and Comfort Considerations

6.1. Protected functionality

The DOM overlay does not in itself introduce any new sensitive information. However, since it combines existing technologies, it’s important to ensure that this combination does not lead to any unexpected interactions.

A primary design goal for this module was that the DOM overlay should follow existing semantics for 2D content where possible. Specifically, the information flows related to cross-origin embedded content should be similar to using iframes on a 2D page. For example, a 2D page can embed cross-origin content in an iframe, and then cover this iframe with a transparent element. In that case, the page will continue to receive mouse movement information, but the cross-origin content does not receive any input events in the covered area. For a DOM overlay, XR input event data is treated as similar to mouse movement data. Poses remain available to the outer page if there is no cross-origin content, or if the cross-origin content is not receiving input, but are limited (blocked) when interacting with cross-origin content.

Cross-origin content is potentially vulnerable to clickjacking threats. The UA MUST continue to apply mitigations such as Content Security Policy Level 3 §directive-frame-src when iframes are used in a DOM overlay. The UA MAY implement additional restrictions specifically for cross-origin content in a DOM overlay if necessary to address specific threats.

7. Acknowledgements

The following individuals have contributed to the design of the WebXR DOM Overlay specification:

• Brandon Jones (Google)

• Nell Waliczek (Amazon [Microsoft until 2018])