Asset registry: replace BBRF split-state with Hackmode/CouchDB-compatible asset management #3

New issue

Open

opened 2023-12-02 05:12:57 +00:00 by lost-rob0t · 0 comments

lost-rob0t commented

2023-12-02 05:12:57 +00:00

(Migrated from github.com)

Problem

Hackmode needs first-class asset management, but current state is split between Common Lisp/Tek9 objects and direct BBRF usage in the Emacs client. The old issue only said to extend/use BBRF as inspiration and did not define ownership or compatibility with StarIntel.

Current behavior

Hackmode core has typed domain, host, port, url, cert, and finding objects and persists operation data through Tek9.
emacs-hackmode uses the external bbrf CLI directly for domains/IPs/URLs/services.
starintel-server already has CouchDB storage/actors and a StarIntel document model.

Desired behavior

Hackmode owns a typed asset registry compatible with StarIntel documents and capable of using CouchDB without creating a second incompatible schema. BBRF behavior may be used as UX inspiration/compatibility input, not as the canonical data model.

Architecture

Asset discovery should follow:

discover -> normalize -> deterministic identity/dedupe -> local operation store -> publish asset event -> optional CouchDB/StarIntel synchronization

Initial asset types include domain/subdomain/hostname/url/ip-address/network/service/port/certificate/email/username/person/organization/document/finding.

Common metadata should cover ID, type, value/canonical value, operation, source/provenance, first/last seen, observed-at, confidence, tags, relationships, parent/child assets, and raw evidence where supported by the StarIntel model.

Implementation scope

build on the canonical operation/asset protocol from #6
define StarIntel-compatible serialization/mapping rather than a separate schema
add CouchDB-backed registry adapter using existing StarIntel/cl-couch abstractions where reusable
provide query/update operations needed by shell and Emacs
retain local-first operation behavior when CouchDB/StarIntel are offline
migrate direct BBRF callers behind the Hackmode asset protocol

Dependencies

#4 monorepo layout
#6 canonical operation/asset protocol
StarIntel document compatibility review

Acceptance criteria

asset identity/deduplication is deterministic
local operation store remains authoritative for offline work
CouchDB synchronization is retryable/idempotent
provenance survives serialization and synchronization
Emacs/shell do not need to know BBRF/CouchDB implementation details
no provider credentials or storage secrets leak to UI clients

Verification

discover/store/query same asset twice -> one canonical asset
operation isolation tests
CouchDB adapter round-trip tests
offline/local-only test
StarIntel document compatibility fixtures
migration compatibility test for BBRF-originated domain/IP/URL/service inputs

## Problem Hackmode needs first-class asset management, but current state is split between Common Lisp/Tek9 objects and direct BBRF usage in the Emacs client. The old issue only said to extend/use BBRF as inspiration and did not define ownership or compatibility with StarIntel. ## Current behavior - Hackmode core has typed `domain`, `host`, `port`, `url`, `cert`, and `finding` objects and persists operation data through Tek9. - `emacs-hackmode` uses the external `bbrf` CLI directly for domains/IPs/URLs/services. - `starintel-server` already has CouchDB storage/actors and a StarIntel document model. ## Desired behavior Hackmode owns a typed asset registry compatible with StarIntel documents and capable of using CouchDB without creating a second incompatible schema. BBRF behavior may be used as UX inspiration/compatibility input, not as the canonical data model. ## Architecture Asset discovery should follow: `discover -> normalize -> deterministic identity/dedupe -> local operation store -> publish asset event -> optional CouchDB/StarIntel synchronization` Initial asset types include domain/subdomain/hostname/url/ip-address/network/service/port/certificate/email/username/person/organization/document/finding. Common metadata should cover ID, type, value/canonical value, operation, source/provenance, first/last seen, observed-at, confidence, tags, relationships, parent/child assets, and raw evidence where supported by the StarIntel model. ## Implementation scope - build on the canonical operation/asset protocol from #6 - define StarIntel-compatible serialization/mapping rather than a separate schema - add CouchDB-backed registry adapter using existing StarIntel/cl-couch abstractions where reusable - provide query/update operations needed by shell and Emacs - retain local-first operation behavior when CouchDB/StarIntel are offline - migrate direct BBRF callers behind the Hackmode asset protocol ## Dependencies - #4 monorepo layout - #6 canonical operation/asset protocol - StarIntel document compatibility review ## Acceptance criteria - asset identity/deduplication is deterministic - local operation store remains authoritative for offline work - CouchDB synchronization is retryable/idempotent - provenance survives serialization and synchronization - Emacs/shell do not need to know BBRF/CouchDB implementation details - no provider credentials or storage secrets leak to UI clients ## Verification - discover/store/query same asset twice -> one canonical asset - operation isolation tests - CouchDB adapter round-trip tests - offline/local-only test - StarIntel document compatibility fixtures - migration compatibility test for BBRF-originated domain/IP/URL/service inputs