Capability Tiers

Overview

PCP does not require every application to support the same level of integration. Instead, it assigns each application a capability tier based on what that application can declare about its own interface. The result is a progressive-fidelity model: deeper integration yields richer, more precise capabilities, but every application on the system receives at least basic control through PCP.

The three tiers exist on a spectrum. Tier 1 applications ship a cryptographically signed capability manifest and expose typed, domain-specific operations directly. Tier 2 applications rely on the OS accessibility infrastructure, letting PCP derive semantic capabilities from structural patterns in their widget trees. Tier 3 applications expose nothing beyond surface metadata, so PCP falls back to window-level input simulation.

Tier assignment is fully automatic. PCP evaluates each application at install time and on every session start, populates the capability registry, and makes the results available to System Intelligence without any developer intervention. The developer can opt into a higher tier by providing more integration, but no application is excluded from the PCP ecosystem.

graph TD
    subgraph Tiers
        T1["Tier 1: SI-Native"]
        T2["Tier 2: Accessibility Bridging"]
        T3["Tier 3: Universal Fallback"]
    end

    T1 -->|"signed manifest, typed operations"| SI["System Intelligence"]
    T2 -->|"derived from accessibility trees"| SI
    T3 -->|"surface metadata, input simulation"| SI

    T1 -.->|"most precise"| Q1["Highest fidelity"]
    T2 -.->|"structural semantics"| Q2["Medium fidelity"]
    T3 -.->|"positional only"| Q3["Basic fidelity"]

Tier 1: SI-Native

Tier 1 is the highest integration level. An application at this tier ships a cryptographically signed CapabilityManifest alongside its binary. The manifest declares the semantic operations the application supports, the data types those operations accept and return, and the resource paths they act on. The daemon verifies the manifest’s signature at load time and registers the declared capabilities directly into the capability registry.

The manifest is not a configuration file that SI reads and interprets. It is a signed declaration that the daemon verifies and indexes. When SI invokes a capability on a Tier 1 application, the invocation travels over the application’s PCP socket to its server implementation. Parameters are validated against the manifest’s schemas, return values are structured, and failures propagate as typed protocol errors.

The PcpServer trait

Each Tier 1 application implements the PcpServer trait:

#[async_trait]
pub trait PcpServer: Send + Sync {
    async fn manifest(&self) -> CapabilityManifest;
    async fn invoke(&self, capability_id: &str, ctx: &InvocationContext,
                    params: Value) -> Result<CapabilityResult, PcpError>;
    async fn validate_params(&self, capability_id: &str, params: &Value)
        -> Result<(), PcpError>;
    async fn subscribe_events(&self, event_types: Vec<String>) -> Result<(), PcpError>;
    async fn query_dynamic_state(&self, capability_id: &str) -> Result<Value, PcpError>;
    async fn shutdown(&self);
}

The daemon maps capability identifiers to applications in the registry. When SI calls mail.compose, the daemon routes the invocation over the owning application’s socket to its PcpServer, which validates the parameters against the manifest schema and executes invoke. The structured result flows back over the same path.

Manifest example

A minimal manifest for an email client might declare:

{
  "app_id": "com.example.mailclient",
  "version": "1.0.0",
  "manifest_version": "1.0",
  "capabilities": [
    {
      "id": "mail.compose",
      "name": "Compose Message",
      "description": "Open a new compose window with pre-filled fields",
      "category": "COMMUNICATION",
      "parameters": {
        "type": "object",
        "properties": {
          "to": { "type": "string" },
          "subject": { "type": "string" }
        }
      },
      "returns": { "type": "object", "properties": {} },
      "side_effects": "write",
      "auth_level": "user"
    }
  ]
}

The parameters and returns fields are JSON schemas. The daemon validates every invocation against them; a call whose payload does not match the schema is rejected before it reaches the application.

Script injection is forbidden

Tier 1 explicitly prohibits script injection and DOM manipulation. An application’s capabilities must operate on its own data model through typed servers, not by injecting JavaScript, evaluating arbitrary code, or manipulating UI element trees from outside the application’s process. This restriction exists for three reasons.

First, auditability. Every capability invocation in PCP is logged with a timestamp, target, parameters, and result. Script injection breaks that guarantee because the injected code can perform arbitrary operations that the audit trail never captures.

Second, determinism. Typed capability servers have fixed, declared behavior. Script injection introduces dynamic, unpredictable execution paths. The daemon cannot reason about what an injected script will do, and neither can SI.

Third, trust. The manifest’s cryptographic signature binds the declared capabilities to a specific identity. If the application could inject arbitrary code, the signature would be meaningless. The signature guarantees that the declared operations are the only operations available through the PCP interface.

Developer effort

Tier 1 requires the most developer work: writing a manifest, implementing the server trait, and managing the signing pipeline. In return, it provides the richest possible integration. SI can read and write application data, invoke domain-specific operations with typed parameters, and receive structured results. No probing, no heuristics, no approximation.

Tier 2: Accessibility Bridging

Most desktop applications never ship a capability manifest. They expose their interfaces through standard toolkit widgets that participate in the OS accessibility infrastructure: AT-SPI2 on Linux, MSAA and UI Automation on Windows, and their equivalents on other platforms. Tier 2 targets these applications.

How it works

The accessibility adapter walks each application’s accessibility tree, extracts element roles, states, text content, and action sets, and maps them to the PCP Element model. The result is a semantic representation of the application’s interface that SI can query and act on without any developer cooperation.

The mapping from accessibility tree to PCP Element is largely structural. AT-SPI2 roles map to PCP roles (an ROLE_PUSH_BUTTON node becomes a Button element). States map directly (the STATE_FOCUSABLE flag becomes the focusable property). Text content, value ranges, selection boundaries, and caret positions transfer without transformation.

The harder problem is deriving capabilities from raw structural data. An accessibility tree tells you that a button exists, what it is labeled, and what actions it supports. It does not tell you that clicking that button composes a new email. That gap is where pattern recognition enters.

Pattern recognition

PCP maintains a set of application category templates. Each template defines the structural pattern expected for a known application type, the roles that must be present, and the approximate layout those roles should occupy relative to one another.

When the accessibility adapter finishes walking an application’s tree, PCP scores the result against each template. The scoring system is straightforward:

Condition	Score
Expected role present in expected location	+1.0
Expected role present in unexpected location	+0.3
Required role missing	-2.0

The sum produces a confidence value between 0 and the total expected roles. If that value exceeds the match threshold (0.6), the application is assigned the corresponding category and its derived capabilities are registered.

Application categories

Category	Detection Pattern	Derived Capabilities
Email Client	message_list + search_bar + compose_button + folder_tree	mail.search, mail.compose, mail.reply
Text Editor	document + toolbar + find_replace	editor.read, editor.edit, editor.save
Web Browser	address_bar + tabs + content_area	browser.navigate, browser.read_page
File Manager	directory_tree + file_list + path_bar	files.list, files.copy, files.rename
Terminal	text_buffer + shell_prompt	terminal.run_command, terminal.read_output

The “expected location” in the scoring system refers to the role’s position in the accessibility tree hierarchy, not its pixel coordinates. A compose button nested inside a toolbar under the main window is in the expected location. The same button floating in an unrelated dialog is in an unexpected location and receives a lower score.

Adapter lifecycle

The Tier 2 adapter follows a four-phase lifecycle for each application:

1. Detection. At session start, the adapter queries installed applications through their .desktop files (on Linux) or equivalent platform metadata. This static detection identifies the application’s category, executable, and toolkit before the application launches.

2. Matching. When the application starts, the adapter walks its accessibility tree and scores it against the category templates. If the match exceeds the confidence threshold, the category is confirmed and derived capabilities are registered.

3. Derivation. PCP maps the matched category to its capability set. Each derived capability is backed by the accessibility adapter’s element model. Invoking mail.compose activates the compose button in the application’s tree.

4. Execution. SI invokes derived capabilities through the daemon. The adapter resolves the capability to a specific element action, fires the action, and returns the result. No script injection, no input simulation. The action travels through the accessibility framework’s own activation mechanism.

If an application’s interface changes at runtime (a panel is closed, a layout is rearranged), the adapter re-scores the tree and updates the capability set accordingly.

Tier 3: Universal Fallback

Not every application exposes an accessibility tree. Games render their interfaces through custom graphics engines. Scientific visualization tools draw directly to hardware surfaces. Legacy applications built on frameworks that predate modern accessibility APIs may expose nothing beyond a raw window.

Tier 3 handles these cases. The adapter operates on compositor surface metadata alone: window position, size, title, z-order stacking, and state flags (minimized, maximized, fullscreen). No semantic element access is available at this tier.

What works

Window management is fully functional. SI can focus, minimize, maximize, move, resize, close, and tile Tier 3 windows. It can query which windows are visible, which workspace they occupy, and their relative z-order. These operations rely on compositor state, not application cooperation, so they work regardless of what the application renders.

What does not work

Element-level interaction is unavailable. SI cannot read text content, click specific buttons, or identify UI roles inside a Tier 3 window. Interaction collapses to synthetic input simulation: the adapter generates keystroke and pointer events targeted at the window’s surface coordinates. This is the same mechanism used by external automation frameworks, and it carries the same fragility. If the application moves a button, the coordinate target misses. If the application renders text as pixels rather than through a text API, the content is invisible to SI.

Why this is acceptable

Tier 3 is the fallback of last resort, and PCP makes no pretense that it provides rich interaction. What it provides is ecosystem inclusion. A game, a rendering tool, or a legacy application still appears in the desktop model. SI knows it exists, can manage its window, and can switch to it when the user requests. The application participates in the PCP ecosystem at a basic level, and the user retains control over it alongside Tier 1 and Tier 2 applications.

Tier Comparison

Dimension	Tier 1: SI-Native	Tier 2: Accessibility Bridging	Tier 3: Universal Fallback
Detection method	Signed manifest at load time	Static .desktop scan + runtime accessibility walk	Compositor surface enumeration
Capability source	Developer-declared servers	Pattern-matched derivation from accessibility tree	Window management only
Semantic depth	Full typed operations on application data model	Structural roles and actions from accessibility API	Window title, geometry, state flags
Interaction method	Typed socket invocation	Accessibility action activation	Synthetic input simulation
Developer effort	High (manifest + server + signing)	None	None
Example applications	Purpose-built PCP applications	GTK, Qt, Electron, Firefox apps	Games, custom renderers, legacy software

Adapter State Machine

Each adapter instance follows a defined lifecycle state machine. Understanding this lifecycle is relevant for implementing custom adapters and for debugging capability registration issues. The full specification of adapter states is covered in the capability registry documentation.

The states, in brief:

stateDiagram-v2
    [*] --> Idle
    Idle --> Detecting : app installed / session start
    Detecting --> Matched : confidence > 0.6
    Detecting --> Idle : confidence < 0.6
    Matched --> Active : capabilities registered
    Active --> Stale : interface change detected
    Stale --> Detecting : re-scan triggered
    Active --> Idle : app closed

Idle. The adapter has no work to do. The application is not running or has not been detected.

Detecting. The adapter is scanning the application’s accessibility tree (Tier 2) or surface metadata (Tier 3) and scoring it against category templates.

Matched. The application has been assigned a category with sufficient confidence, but capabilities have not yet been registered in the capability registry.

Active. Capabilities are registered and available for SI invocation. The adapter monitors for structural changes.

Stale. The application’s interface has changed enough that the previous match may no longer be valid. The adapter re-enters the detection phase without deregistering existing capabilities.

When an adapter transitions from Active back to Idle (application closed), all capabilities derived from that application are removed from the registry. SI receives a capability-removal event and can update its desktop model accordingly.