Execution Pipeline

Overview

Every action the System Intelligence layer takes flows through a strict, auditable six-step pipeline. No shortcuts, no bypass paths. When SI forms an intent, that intent enters the pipeline at target resolution and does not exit until it reaches audit logging. Each step has a defined responsibility, a defined failure mode, and a defined output. If any step fails, the pipeline aborts and the failure is recorded.

The pipeline exists for three reasons. First, safety. No action reaches an application without passing through permission checks and confirmation gates. Second, accountability. Every action, successful or failed, produces an audit entry. Third, reliability. State verification catches actions that appeared to succeed but did not actually change the intended state.

graph TD
    A["1. Target Resolution"] --> B["2. Permission Check"]
    B --> C["3. Action Resolution"]
    C --> D["4. Execution"]
    D --> E["5. State Verification"]
    E --> F["6. Audit Logging"]

The Six Steps

1. Target Resolution

Natural language references are ambiguous. “Open the email from John” does not specify which email client, which message, or which John. Target resolution converts these fuzzy references into concrete element IDs or application IDs that the rest of the pipeline can operate on.

The resolver works through three strategies, tried in order. Exact matching checks whether the reference maps directly to a known element or application by name or ID. Contextual matching uses the current desktop state (focus history, recent activity, spatial position) to narrow candidates. Semantic similarity scores remaining candidates against the reference intent and selects the best match.

When multiple applications share a capability, the resolver applies a deterministic priority chain: user-configured default, then contextual relevance, then last-used timestamp, then capability tier score, then a fallback prompt asking the user to disambiguate. This chain ensures that repeated requests resolve consistently unless the user’s context genuinely changes.

2. Permission Check

Once a target is resolved, the pipeline evaluates the requested action against the current trust level and action class. PCP classifies actions into six severity levels: none, read, write, destructive, network, and system. Each level carries its own permission requirements.

Destructive actions (delete, overwrite, format) and sensitive system actions (credential access, network configuration) trigger multi-modal confirmation gates. The system prompts the user through voice, visual overlay, or haptic feedback depending on the current input modality. The user must confirm through a different modality than the one that initiated the request. A voice command that deletes files, for example, requires visual or haptic confirmation.

The three-domain privilege model (app domain, compositor domain, system domain) enforces escalation boundaries. An action that starts in the app domain cannot escalate to system domain privileges without an explicit, modal confirmation step. Cross-domain escalation is blocked outright at the permission-check layer.

3. Action Resolution

With permissions granted, the pipeline determines the optimal execution method. The application’s capability tier drives this decision.

Tier 1 applications expose signed capability handlers. The pipeline invokes the handler directly with typed parameters and receives a typed result. This is the fastest and most precise path.

Tier 2 applications rely on the accessibility bridge. The pipeline translates the high-level intent into a sequence of AT-SPI2 operations: find the target element in the accessibility tree, perform the appropriate action (activate, set value, select text), and read back the result. This path is slower and depends on the accessibility tree accurately representing the application’s state.

Tier 3 applications expose nothing beyond surface metadata. The pipeline falls back to input simulation: synthetic keystrokes and pointer events targeted at window coordinates. This path is fragile and provides no structured feedback.

4. Execution

The resolved action is dispatched to the target through the appropriate execution path. Direct handler invocation for Tier 1, accessibility action activation for Tier 2, input event injection for Tier 3.

Each dispatch carries a timeout budget. If the target does not respond within the budget, the pipeline aborts the action, marks it as timed out, and logs the failure. Timeouts prevent hung operations from blocking the SI layer and from leaving the desktop in an indeterminate state where the user does not know whether the action ran.

5. State Verification

After execution, the pipeline checks whether the intended state change actually occurred. Verification method depends on the capability tier.

Tier 1 actions produce a typed result from the capability handler. The pipeline compares the result against the expected outcome. If the handler reports failure or returns an unexpected value, the pipeline flags the action as failed and reports the discrepancy to SI.

Tier 2 actions require a follow-up query. The pipeline re-reads the target element’s state through AT-SPI2 and compares it to the pre-action snapshot. If the state matches the expected post-action value, the action is confirmed. If the element cannot be found or its state has not changed, the action is flagged.

Tier 3 actions have no verification path. Input simulation provides no structured feedback, so the pipeline cannot determine whether the synthetic events produced the intended effect. These actions are marked as best-effort in the audit log.

6. Audit Logging

Every action that enters the pipeline produces an audit record, regardless of outcome. The record captures the timestamp, resolved target, action parameters, permission-check result, execution tier, timeout status, verification result, and total duration from entry to exit.

The audit log is an append-only, tamper-evident JSONL file. Records are written sequentially and cannot be modified or deleted after the fact. Each record includes a hash chain linking it to the previous entry, making retrospective tampering detectable.

Audit trails are separated by domain. App-domain actions log to the application audit trail. Compositor-domain actions log to the compositor trail. System-domain actions log to the system trail. This separation supports scoped auditing: a security review of application-level actions does not require parsing system-level entries, and vice versa.

Timeout Budgets

Each execution domain carries a default timeout. These values balance responsiveness against the reality that some operations legitimately take longer than others.

Domain	Default Timeout	Rationale
App domain	5s	Application operations should complete quickly. Longer waits degrade voice-command responsiveness.
Compositor domain	10s	Window management and layout operations may involve animation or deferred rendering.
System domain	15s	Network operations, package installs, and configuration changes legitimately require more time.

Timeouts serve two purposes. They protect the SI layer from blocking indefinitely on unresponsive targets, and they signal to the user that something has gone wrong. When a timeout fires, the pipeline reports the failure to SI, which can inform the user and suggest alternative actions.

State Verification by Tier

Verification strength tracks directly with capability tier. This is not a design tradeoff. It is a structural consequence of what each tier exposes to the pipeline.

Tier 1: Deterministic proof. The capability handler returns a typed result. The pipeline checks that result against the expected outcome. If the handler says the action succeeded and the result matches expectations, verification passes. If the handler reports an error, verification fails. This is the strongest guarantee the pipeline can offer because the handler operates on the application’s own data model.

Tier 2: State re-query. The pipeline reads the target element’s state through AT-SPI2 after the action completes. This works when the accessibility tree accurately reflects the application’s state, which it does for well-behaved toolkit applications. It fails when the tree is stale, incomplete, or does not update immediately after the action. The gap between action and tree update is the primary source of false negatives in Tier 2 verification.

Tier 3: Best-effort, no verification. Input simulation produces no structured feedback. The pipeline cannot distinguish between a successful action, a missed click, and an action that was intercepted by another window. All Tier 3 actions are logged as unverified. This limitation is why Tier 3 is the fallback of last resort, and why PCP encourages applications to integrate at a higher tier whenever possible.