feat(pbs): resolve editorial chains and expose read-only external sources
Intrepid/Prometeu/Studio/pipeline/head Build started...
Test / Build skipped: 16, passed: 838
Intrepid/Prometeu/Studio/pipeline/pr-master This commit looks good

This commit is contained in:
2026-10-07 09:00:31 +01:00
parent d0737df0ac
commit 967b9ec598
38 changed files with 1856 additions and 42 deletions
@@ -8,6 +8,14 @@ tags: [studio, lsp, vscode, compiler-pbs, editor, definition]
# PBS go-to-definition uses editorial lookup and physical files
> Historical policy: DEC-0062 / PLN-0141 explicitly superseded the physical-only
> definition restriction in October 2026. SDK/core and dependencies now use
> revision-bound read-only external documents for supporting clients. Compiler
> Spec 23 and Studio Spec 11 define the current contract. The original lesson below
> preserves the earlier decision; it must not be read as prohibiting today's
> external navigation. References, symbols, links and hierarchies retain their
> separately scoped contracts. See LSN-0078 for the implementation and pitfalls.
## Original Problem
PBS editor assistance already had compiler-backed completion, hover, and signature help (`LSN-0051`). Users still could not jump from a use to a declaration. Wave 1 explicitly deferred `go to definition`.
@@ -0,0 +1,170 @@
---
id: LSN-0078
ticket: pbs-editorial-chains-sdk-navigation
title: PBS chains and revision-bound external sources
created: 2026-10-07
tags: [compiler, compiler-pbs, lsp, studio, sdk, navigation]
---
# PBS chains and revision-bound external sources
Domain owner: `compiler/pbs`. Propagation: `compiler/general` and `studio`.
DEC-0062 and PLN-0141 were implemented together. This lesson preserves the outcome
and validation after their workflow artifacts are removed.
## Original problem
Compilation supported expressions such as `Input.pad().left().down()`, but the
editorial receiver scanner only understood identifier/dot sequences. Hover,
definition, completion and signature help therefore lost the receiver after a call.
Separately, definition discarded SDK/core handles because their bundled resources
had virtual paths rather than regular files. SDK hover could succeed while F12
returned nothing. These were independent defects requiring integrated acceptance.
A third regression fixture confirmed that source discovery skipped a dependency
explicitly admitted under `.workspace`: the crawler tested every ancestor of the
absolute path against ignored directory names.
## Consolidated decision
Keep language resolution in the frontend. Share receiver typing across the four
project queries, using PBS parsing and flow semantics. Support known-return call
chains, nested arguments, typed/inferred variables, parentheses and recoverable
partial code. Unknown or ambiguous receivers stay unresolved. This changes no
language rule, compiler/linker pipeline, ABI or runtime behavior.
Expose SDK/core and admitted dependency sources as read-only in-memory documents.
SDK/core remain bundled, without editorial extraction. Libraries remain their
existing physical sources; a library inside the project directory is still an
external owner. Dependency installation, download, cache management and its future
Studio UI remain DSC-0072 / AGD-0074.
Each external document binds content, byte ranges and frontend semantic context to
the same analyzed revision. The host emits an opaque project-session URI; the
content digest is not an SDK version. VS Code and MCP share that source and
semantic authority. They do not implement another PBS resolver.
All proposals touching external sources are refused as a whole. Normal project
editing remains available. References, workspace symbols, links and hierarchies
retain their earlier physical scopes. External definition does not expand them.
## Final implementation
1. `PbsParser.parseExpression` exposes the existing expression grammar for complete
receiver fragments. `PbsEditorialSupportService` locates a fragment boundary and
delegates its type to `PbsFlowExpressionAnalyzer`. The same path serves all four
project queries, including trailing dots and unfinished arguments.
2. `SourceCrawler` tests ignored descendants relative to the explicitly admitted
traversal root. It does not admit incidental `.workspace` trees as project code.
3. `FrontendSourceDocument` carries exact text, origin, presentation metadata,
read-only ownership and opaque frontend context. Definition locations optionally
carry that source; invalid path/range combinations are rejected. PBS preserves
source ownership separately from `SourceKind.PROJECT`; external ownership wins
when paths overlap. Existing default-empty frontend capabilities remain valid.
4. `ExternalSourceRegistry` belongs to `TcpLspServerHost`, not the shared compiler
bridge. It retains source/context together, with separate consumer grants,
acknowledged uses and reconnect validation. Repeated uses in one semantic
snapshot coalesce. Another reader's release cannot invalidate a retained source.
5. LSP negotiates external support and emits `prometeu-source:` destinations only
to supporting clients. External read, retain, release and revalidate requests
share server-issued consumer identity. The bridge uses retained content for
positions, hover, definition and highlighting. Unsupported consumers keep local
physical navigation without receiving unusable external targets.
6. VS Code registers a `TextDocumentContentProvider`, assigns PBS language, routes
hover/F12/highlighting and prevents external edit commands. Disconnect preserves
displayed text but disables queries until validation; the status bar explains
unavailable external documents. Changed revisions require fresh navigation.
7. MCP adds `readExternalSource` to `studio_language_query`. Task-owned consumers
survive short-lived sockets, acknowledge returned definitions, revalidate before
reuse, and release on task return/cancellation. Cross-task/project replay is
refused. Failed navigation releases unacknowledged results; failed responses
roll back newly retained sources. No human editor is needed.
### Example
A project definition on `Gfx.clear` opens the bundled SDK source. Hover/F12 on its
`LowGfx.clear` call reaches the host declaration. F12 on `Color` reaches the core
resource. The same operations work through MCP by reading the returned external
URI and querying positions in that text. A dependency under `.workspace/repos/lib`
uses the same presentation and write protection.
If that library changes on disk while a document is open, the retained document's
hover/F12 still use its old AST and old text. After disconnect, revalidation against
the changed source fails; clients must navigate again for new content and ranges.
## Validation evidence
Failures were reproduced before production changes: all four chain assertions
failed, both crawler/admitted-FileTable fixtures failed, and the old bridge test
confirmed SDK hover with empty bundled definition alongside working local F12.
Final module results (2026-10-07):
| Suite | Executed successfully | Skipped |
| --- | ---: | ---: |
| PBS frontend | 245 | 0 |
| Frontend API | 27 | 0 |
| Build pipeline | 141 | 15 existing disabled cases |
| LSP | 38 | 1 opt-in development-host case in the ordinary run |
| App: AutomationLspQueries, McpContract, CompilerProjectKnowledge | 24 | 0 |
| Core: TaskLanguageQueries | 1 | 0 |
| Actual VS Code Development Host, run separately | 1 | 0 |
All 477 executed JUnit cases passed across these scoped runs. TypeScript compilation,
`discussion validate` and `git diff --check` also passed. The disabled main-project
pipeline integration test was not treated as evidence.
The actual VS Code host exercised typing refusal, declaration selection, semantic
highlighting, hover, continued F12, a physical `.workspace` library with Unicode/CRLF,
reconnection and normal project edits. Provider tests exercised cached disconnected
text, matching/changed revisions, close and canceled reads. MCP tests used the actual
TCP server independently of VS Code, including task return, aliases, mixed rename
refusal and changed revisions. Registry tests covered multiple readers, distinct
projects, last release, task/session lifetime and canceled/unopened navigation.
The source-snapshot regression changed a dependency on disk after navigation and
verified that old hover/F12 still used the original declaration and semantic context.
Dedicated write-boundary checks refuse external didChange/didSave, formatting,
rename, prepareRename and code actions, including unnegotiated identities.
Commands are preserved in the plan-derived development instructions and test owners:
```sh
./gradlew :prometeu-compiler:frontends:prometeu-frontend-pbs:test :prometeu-compiler:prometeu-frontend-api:test :prometeu-compiler:prometeu-build-pipeline:test :prometeu-lsp:prometeu-lsp-v1:test --console=plain
./gradlew :prometeu-app:test --tests '*AutomationLspQueriesTest' --tests '*McpContractTest' --tests '*CompilerProjectKnowledgeTest' :prometeu-studio-core:test --tests '*TaskLanguageQueriesTest' --console=plain
npm --prefix tools/vscode-extension run compile
PROMETEU_VSCODE_DEV_TEST=1 ./gradlew :prometeu-lsp:prometeu-lsp-v1:test --tests '*ExternalSourceDevelopmentHostTest' --console=plain
```
Packaged Studio, installed distributions and installed JAR behavior were explicitly
excluded by the user and were not validated. Development resource loading was tested.
## Pitfalls
- Compiler success does not prove editorial resolution. Query the four surfaces.
- A physical path under the project root is not proof of editable ownership.
- Retaining text alone is insufficient: fresh analysis can attach different meaning
to old positions. Keep the corresponding semantic context.
- MCP socket closure is not task completion. Retain consumer state independently.
- VS Code normalizes escaped URI components. The first actual-host run exposed
mismatched identities from escaped slashes; emitted URIs now use unreserved
characters and keep names separately in metadata.
- `TextEditor.edit` is an extension programming API, not user typing. The real UI
gate uses the `type` command; backend edit proposals are checked independently.
If another extension changes a virtual buffer, the provider refuses queries whose
text differs from the retained revision.
- VS Code MarkdownString properties need direct inspection; JSON serialization can
show `{}` even when hover contains valid content.
- Enriching a shared definition DTO can accidentally expand references. Keep each
surface's admission policy explicit; the compatibility suite caught this coupling.
- Do not silently drop external edits from a mixed proposal. Refuse the operation.
## References
- [Compiler Spec 23](../../../docs/specs/compiler/23.%20Compiler%20Pipeline%20Entry%20Points%20Specification.md).
- [PBS Spec 5](../../../docs/specs/compiler-languages/pbs/5.%20Manifest,%20Stdlib,%20and%20SDK%20Resolution%20Specification.md).
- [PBS Spec 8](../../../docs/specs/compiler-languages/pbs/8.%20Stdlib%20Environment%20Packaging%20and%20Loading%20Specification.md).
- [Studio Spec 11](../../../docs/specs/studio/11.%20MCP%20Tools%20and%20Project%20Knowledge%20Contract.md).
- [Extension guide](../../../tools/vscode-extension/README.md).
- [Historical physical-only policy, LSN-0068](../DSC-0039-pbs-lsp-go-to-definition/LSN-0068-pbs-go-to-definition-uses-editorial-lookup-and-physical-files.md).
- DEC-0031 / LSN-0049 still govern removal of the old embedded editor/VFS; none was restored.