PERF/RESILIENCE: instrument and isolate slow plugin extension hooks on critical UI/runtime paths #167

New issue

Open

opened 2026-09-09 15:05:20 +00:00 by nsaspy · 0 comments

nsaspy commented

2026-09-09 15:05:20 +00:00

Owner

Finding

Both backend and frontend extension dispatch are intentionally powerful but currently execute extensions serially with no latency budget or timeout/diagnostic boundary.

Backend (helpers/extension.py::call_extensions_async):

for cls in classes:
    result = cls(...).execute(...)
    if isinstance(result, Awaitable):
        await result

Frontend (webui/js/extensions.js::callJsExtensions):

for (const extension of extensions) {
  await extension.module.default(...data)
}

These dispatchers are used on latency-critical paths, including WebUI socket pushes/snapshot application, message/response streaming hooks, component behavior, and core agent loop extension points. A slow or never-resolving plugin can therefore make the entire core path appear hung even when Agent Zero itself is healthy.

Full plugin UI control must remain possible; the solution is not to delete extension hooks. The system needs observability, declared semantics, and failure containment appropriate to each hook.

Direction

Classify extension points by contract:

transform/blocking hooks where ordering/awaiting is intentionally required
notification/observer hooks that can be isolated or dispatched outside the critical path
high-frequency hooks with strict per-call budgets
startup hooks where a bounded timeout/failure UI is appropriate

Add per-extension timing and diagnostics so users can see which plugin is making the UI/agent slow.

For hooks where a timeout would break valid plugin semantics, surface a watchdog warning rather than silently canceling. For observer hooks, consider asynchronous isolation/concurrency with explicit ordering rules.

Acceptance

Extension dispatch records extension path/name, hook, elapsed time and failure for diagnostics without exposing secrets.
Slow-plugin warnings identify the actual extension rather than presenting a generic WebUI/agent hang.
Critical/high-frequency hook categories and ordering semantics are documented.
Non-mutating observer hooks cannot indefinitely block state synchronization/message rendering.
Mutating/transform hooks retain deterministic ordering and full plugin control.
A never-resolving frontend extension has a bounded/fail-visible startup/sync behavior appropriate to that hook class.
A never-resolving backend observer extension does not strand unrelated contexts indefinitely.
Add synthetic slow/hung plugin regression fixtures.
Modern WebUI work (#23) exposes plugin health/performance diagnostics rather than hiding them.

Mirrored from lost-rob0t/a0-symbolics#29 via tracker sync.

## Finding Both backend and frontend extension dispatch are intentionally powerful but currently execute extensions **serially with no latency budget or timeout/diagnostic boundary**. Backend (`helpers/extension.py::call_extensions_async`): ```python for cls in classes: result = cls(...).execute(...) if isinstance(result, Awaitable): await result ``` Frontend (`webui/js/extensions.js::callJsExtensions`): ```js for (const extension of extensions) { await extension.module.default(...data) } ``` These dispatchers are used on latency-critical paths, including WebUI socket pushes/snapshot application, message/response streaming hooks, component behavior, and core agent loop extension points. A slow or never-resolving plugin can therefore make the entire core path appear hung even when Agent Zero itself is healthy. Full plugin UI control must remain possible; the solution is **not** to delete extension hooks. The system needs observability, declared semantics, and failure containment appropriate to each hook. ## Direction Classify extension points by contract: - transform/blocking hooks where ordering/awaiting is intentionally required - notification/observer hooks that can be isolated or dispatched outside the critical path - high-frequency hooks with strict per-call budgets - startup hooks where a bounded timeout/failure UI is appropriate Add per-extension timing and diagnostics so users can see which plugin is making the UI/agent slow. For hooks where a timeout would break valid plugin semantics, surface a watchdog warning rather than silently canceling. For observer hooks, consider asynchronous isolation/concurrency with explicit ordering rules. ## Acceptance - [ ] Extension dispatch records extension path/name, hook, elapsed time and failure for diagnostics without exposing secrets. - [ ] Slow-plugin warnings identify the actual extension rather than presenting a generic WebUI/agent hang. - [ ] Critical/high-frequency hook categories and ordering semantics are documented. - [ ] Non-mutating observer hooks cannot indefinitely block state synchronization/message rendering. - [ ] Mutating/transform hooks retain deterministic ordering and full plugin control. - [ ] A never-resolving frontend extension has a bounded/fail-visible startup/sync behavior appropriate to that hook class. - [ ] A never-resolving backend observer extension does not strand unrelated contexts indefinitely. - [ ] Add synthetic slow/hung plugin regression fixtures. - [ ] Modern WebUI work (#23) exposes plugin health/performance diagnostics rather than hiding them. --- *Mirrored from [`lost-rob0t/a0-symbolics#29`](https://github.com/lost-rob0t/a0-symbolics/issues/29)* via tracker sync.