Merge pull request 'feat(pbs): resolve editorial chains and expose read-only external sources' (#35) from dev/pbs-editorial-chains-sdk-navigation into master
Test / Build skipped: 16, passed: 942
JaCoCo Coverage #### Project Overview No changes detected, that affect the code coverage. * Line Coverage: 66.91% (23371/34928) * Branch Coverage: 56.47% (9948/17616) * Lines of Code: 34928 * Cyclomatic Complexity: 15041 #### Quality Gates Summary Output truncated.
Intrepid/Prometeu/Studio/pipeline/head This commit looks good

Reviewed-on: #35
This commit was merged in pull request #35.
This commit is contained in:
2026-10-07 08:36:18 +00:00
41 changed files with 1950 additions and 49 deletions
+14
View File
@@ -66,6 +66,20 @@ configuration; old task identities and operation receipts are not restored.
DEV displays connection separately from project authority. No project-local
launcher, Python transport script or embedded chat is required.
### Reading external source code
VS Code F12 opens SDK/core and admitted libraries as read-only documents, with
hover, continued F12 and syntax highlighting. SDK/core stay bundled; no editorial
copies are created. After a source revision changes or a session ends, navigate
again instead of reusing old positions.
For MCP, use `studio_language_query` with `operation: "definition"`, then pass a
returned `prometeu-source:` URI to `operation: "readExternalSource"`. The response
contains `descriptor` and `text`; hover and definition accept the same URI within
the owning task. Another task cannot reuse it. MCP does not require VS Code open.
External or mixed edit proposals are refused in full. Library installation/cache
management is outside this feature. See the [extension guide](tools/vscode-extension/README.md).
### Installation and IntelliJ development
Build and stage the complete distribution, including its Java runtime:
@@ -8,6 +8,14 @@ tags: [studio, lsp, vscode, compiler-pbs, editor, definition]
# PBS go-to-definition uses editorial lookup and physical files
> Historical policy: DEC-0062 / PLN-0141 explicitly superseded the physical-only
> definition restriction in October 2026. SDK/core and dependencies now use
> revision-bound read-only external documents for supporting clients. Compiler
> Spec 23 and Studio Spec 11 define the current contract. The original lesson below
> preserves the earlier decision; it must not be read as prohibiting today's
> external navigation. References, symbols, links and hierarchies retain their
> separately scoped contracts. See LSN-0078 for the implementation and pitfalls.
## Original Problem
PBS editor assistance already had compiler-backed completion, hover, and signature help (`LSN-0051`). Users still could not jump from a use to a declaration. Wave 1 explicitly deferred `go to definition`.
@@ -0,0 +1,190 @@
---
id: LSN-0078
ticket: pbs-editorial-chains-sdk-navigation
title: PBS chains and revision-bound external sources
created: 2026-10-07
tags: [compiler, compiler-pbs, lsp, studio, sdk, navigation]
---
# PBS chains and revision-bound external sources
Domain owner: `compiler/pbs`. Propagation: `compiler/general` and `studio`.
DEC-0062 and PLN-0141 were implemented together. This lesson preserves the outcome
and validation after their workflow artifacts are removed.
## Original problem
Compilation supported expressions such as `Input.pad().left().down()`, but the
editorial receiver scanner only understood identifier/dot sequences. Hover,
definition, completion and signature help therefore lost the receiver after a call.
Separately, definition discarded SDK/core handles because their bundled resources
had virtual paths rather than regular files. SDK hover could succeed while F12
returned nothing. These were independent defects requiring integrated acceptance.
A third regression fixture confirmed that source discovery skipped a dependency
explicitly admitted under `.workspace`: the crawler tested every ancestor of the
absolute path against ignored directory names.
## Consolidated decision
Keep language resolution in the frontend. Share receiver typing across the four
project queries, using PBS parsing and flow semantics. Support known-return call
chains, nested arguments, typed/inferred variables, parentheses and recoverable
partial code. Unknown or ambiguous receivers stay unresolved. This changes no
language rule, compiler/linker pipeline, ABI or runtime behavior.
Expose SDK/core and admitted dependency sources as read-only in-memory documents.
SDK/core remain bundled, without editorial extraction. Libraries remain their
existing physical sources; a library inside the project directory is still an
external owner. Dependency installation, download, cache management and its future
Studio UI remain DSC-0072 / AGD-0074.
Each external document binds content, byte ranges and frontend semantic context to
the same analyzed revision. The host emits an opaque project-session URI; the
content digest is not an SDK version. VS Code and MCP share that source and
semantic authority. They do not implement another PBS resolver.
All proposals touching external sources are refused as a whole. Normal project
editing remains available. References, workspace symbols, links and hierarchies
retain their earlier physical scopes. External definition does not expand them.
## Final implementation
1. `PbsParser.parseExpression` exposes the existing expression grammar for complete
receiver fragments. `PbsEditorialSupportService` locates a fragment boundary and
delegates its type to `PbsFlowExpressionAnalyzer`. The same path serves all four
project queries, including trailing dots and unfinished arguments.
2. `SourceCrawler` tests ignored descendants relative to the explicitly admitted
traversal root. It does not admit incidental `.workspace` trees as project code.
3. `FrontendSourceDocument` carries exact text, origin, presentation metadata,
read-only ownership and opaque frontend context. Definition locations optionally
carry that source; invalid path/range combinations are rejected. PBS preserves
source ownership separately from `SourceKind.PROJECT`; external ownership wins
when paths overlap. Existing default-empty frontend capabilities remain valid.
4. `ExternalSourceRegistry` belongs to `TcpLspServerHost`, not the shared compiler
bridge. It retains source/context together, with separate consumer grants,
acknowledged uses and reconnect validation. Repeated uses in one semantic
snapshot coalesce. Another reader's release cannot invalidate a retained source.
5. LSP negotiates external support and emits `prometeu-source:` destinations only
to supporting clients. External read, retain, release and revalidate requests
share server-issued consumer identity. The bridge uses retained content for
positions, hover, definition and highlighting. Unsupported consumers keep local
physical navigation without receiving unusable external targets.
6. VS Code registers a `TextDocumentContentProvider`, assigns PBS language, routes
hover/F12/highlighting and prevents external edit commands. Disconnect preserves
displayed text but disables queries until validation; the status bar explains
unavailable external documents. Changed revisions require fresh navigation.
7. MCP adds `readExternalSource` to `studio_language_query`. Task-owned consumers
survive short-lived sockets, acknowledge returned definitions, revalidate before
reuse, and release on task return/cancellation. Cross-task/project replay is
refused. Failed navigation releases unacknowledged results; failed responses
roll back newly retained sources. No human editor is needed.
### Example
A project definition on `Gfx.clear` opens the bundled SDK source. Hover/F12 on its
`LowGfx.clear` call reaches the host declaration. F12 on `Color` reaches the core
resource. The same operations work through MCP by reading the returned external
URI and querying positions in that text. A dependency under `.workspace/repos/lib`
uses the same presentation and write protection.
If that library changes on disk while a document is open, the retained document's
hover/F12 still use its old AST and old text. After disconnect, revalidation against
the changed source fails; clients must navigate again for new content and ranges.
## Validation evidence
Failures were reproduced before production changes: all four chain assertions
failed, both crawler/admitted-FileTable fixtures failed, and the old bridge test
confirmed SDK hover with empty bundled definition alongside working local F12.
Final module results (2026-10-07):
| Suite | Executed successfully | Skipped |
| --- | ---: | ---: |
| PBS frontend | 245 | 0 |
| Frontend API | 27 | 0 |
| Build pipeline | 141 | 15 existing disabled cases |
| LSP | 38 | 1 opt-in development-host case in the ordinary run |
| App: AutomationLspQueries, McpContract, CompilerProjectKnowledge | 24 | 0 |
| Core: TaskLanguageQueries | 1 | 0 |
| Actual VS Code Development Host, run separately | 1 | 0 |
All 477 executed JUnit cases passed across these scoped runs. TypeScript compilation,
`discussion validate` and `git diff --check` also passed. The disabled main-project
pipeline integration test was not treated as evidence.
The actual VS Code host exercised typing refusal, declaration selection, semantic
highlighting, hover, continued F12, a physical `.workspace` library with Unicode/CRLF,
reconnection and normal project edits. Provider tests exercised cached disconnected
text, matching/changed revisions, close and canceled reads. MCP tests used the actual
TCP server independently of VS Code, including task return, aliases, mixed rename
refusal and changed revisions. Registry tests covered multiple readers, distinct
projects, last release, task/session lifetime and canceled/unopened navigation.
The source-snapshot regression changed a dependency on disk after navigation and
verified that old hover/F12 still used the original declaration and semantic context.
Dedicated write-boundary checks refuse external didChange/didSave, formatting,
rename, prepareRename and code actions, including unnegotiated identities.
Commands are preserved in the plan-derived development instructions and test owners:
```sh
./gradlew :prometeu-compiler:frontends:prometeu-frontend-pbs:test :prometeu-compiler:prometeu-frontend-api:test :prometeu-compiler:prometeu-build-pipeline:test :prometeu-lsp:prometeu-lsp-v1:test --console=plain
./gradlew :prometeu-app:test --tests '*AutomationLspQueriesTest' --tests '*McpContractTest' --tests '*CompilerProjectKnowledgeTest' :prometeu-studio-core:test --tests '*TaskLanguageQueriesTest' --console=plain
npm --prefix tools/vscode-extension run compile
PROMETEU_VSCODE_DEV_TEST=1 ./gradlew :prometeu-lsp:prometeu-lsp-v1:test --tests '*ExternalSourceDevelopmentHostTest' --console=plain
```
Packaged Studio was excluded from the initial acceptance above. On 2026-10-07,
the user explicitly requested this additional validation. The existing
`InstalledKnowledgeTest` now launches its isolated probe with the Java runtime
from `installDist`, using distribution JARs plus test probe classes, with its working
directory and projects outside the checkout. It asserts that the frontend and LSP
bridge load from JARs and that SDK/core resources have `jar:` URLs.
The packaged gate passed:
```sh
./gradlew :prometeu-app:installedKnowledgeTest --console=plain
```
Besides reading the installed documentation and compiling its example, it verifies
project-to-SDK, SDK-to-host and SDK-to-core definitions, matching declaration lines,
exact SDK/core text against JAR bytes, read-only bundled descriptors, SDK hover and
semantic tokens, and source revalidation after reconnect. The project contains only
its original manifest and sources after navigation: no bundled source extraction.
The first run caught an incorrect cursor position in the test fixture; calculating
the position from the source fragment fixed the fixture. No production fix was needed.
This is a packaged resource/backend gate; it does not launch the Studio desktop or
repeat the VS Code UI checks against the installed process.
## Pitfalls
- Compiler success does not prove editorial resolution. Query the four surfaces.
- A physical path under the project root is not proof of editable ownership.
- Retaining text alone is insufficient: fresh analysis can attach different meaning
to old positions. Keep the corresponding semantic context.
- MCP socket closure is not task completion. Retain consumer state independently.
- VS Code normalizes escaped URI components. The first actual-host run exposed
mismatched identities from escaped slashes; emitted URIs now use unreserved
characters and keep names separately in metadata.
- `TextEditor.edit` is an extension programming API, not user typing. The real UI
gate uses the `type` command; backend edit proposals are checked independently.
If another extension changes a virtual buffer, the provider refuses queries whose
text differs from the retained revision.
- VS Code MarkdownString properties need direct inspection; JSON serialization can
show `{}` even when hover contains valid content.
- Enriching a shared definition DTO can accidentally expand references. Keep each
surface's admission policy explicit; the compatibility suite caught this coupling.
- Do not silently drop external edits from a mixed proposal. Refuse the operation.
## References
- [Compiler Spec 23](../../../docs/specs/compiler/23.%20Compiler%20Pipeline%20Entry%20Points%20Specification.md).
- [PBS Spec 5](../../../docs/specs/compiler-languages/pbs/5.%20Manifest,%20Stdlib,%20and%20SDK%20Resolution%20Specification.md).
- [PBS Spec 8](../../../docs/specs/compiler-languages/pbs/8.%20Stdlib%20Environment%20Packaging%20and%20Loading%20Specification.md).
- [Studio Spec 11](../../../docs/specs/studio/11.%20MCP%20Tools%20and%20Project%20Knowledge%20Contract.md).
- [Extension guide](../../../tools/vscode-extension/README.md).
- [Historical physical-only policy, LSN-0068](../DSC-0039-pbs-lsp-go-to-definition/LSN-0068-pbs-go-to-definition-uses-editorial-lookup-and-physical-files.md).
- DEC-0031 / LSN-0049 still govern removal of the old embedded editor/VFS; none was restored.
@@ -537,3 +537,22 @@ The current PBS design direction is:
10. `declare host` remains reserved to stdlib/toolchain interface modules.
11. Reserved attributes such as `[Host(...)]` are compile-time metadata, not runtime objects.
12. The compiler consumes `Host` metadata to produce runtime-facing host-binding artifacts defined elsewhere.
## Editorial exposure of external sources
SDK/core remain reserved bundled sources. Editorial navigation MUST NOT extract
or copy them into a project or disk cache. Compiler-known dependency sources,
including those under `.workspace`, and SDK/core are exposed as read-only external
documents with revision-consistent text, ranges and frontend semantic context.
Hover and continued definition use that retained context. Compilation, module
admission, ABI and linking rules are unchanged. See Compiler Spec 23 §8.3 and
Studio Spec 11 for the consumer protocol and atomic write restrictions (DEC-0062).
PBS project hover, definition, completion and signature help MUST share receiver
type resolution using PBS expression parsing and semantic overload rules. Coverage
includes function/method chains with known return types, nested call arguments,
explicitly typed and inferred variables, parentheses, completion after a trailing
dot and signature help during incomplete arguments. An unrelated parse error MUST
NOT invalidate a recoverable receiver. Unknown or ambiguous receivers MUST NOT
invent a type or overload. Indexing/operator-produced receivers are outside the
new coverage; existing behavior MUST NOT regress.
@@ -356,3 +356,13 @@ Content hashes identify the packaged texts, not a semantic language release.
Conformance MUST include reading all indexed references from installed JARs outside
the repository and compiling the shipped example against the installed SDK. No
network access or source checkout may be required by that reference-read path.
## Editorial exposure of external sources
SDK/core remain reserved bundled sources. Editorial navigation MUST NOT extract
or copy them into a project or disk cache. Compiler-known dependency sources,
including those under `.workspace`, and SDK/core are exposed as read-only external
documents with revision-consistent text, ranges and frontend semantic context.
Hover and continued definition use that retained context. Compilation, module
admission, ABI and linking rules are unchanged. See Compiler Spec 23 §8.3 and
Studio Spec 11 for the consumer protocol and atomic write restrictions (DEC-0062).
@@ -272,7 +272,17 @@ Absent editor capabilities MUST have deterministic fallback behavior at tooling
3. host protocols that can represent unsupported operations SHOULD do so explicitly;
4. and absent editor capability MUST NOT be reported as a compiler failure.
When a frontend exposes definition or navigation, each returned location MUST identify a compiler-known physical source file. A resolved symbol without a physical file MUST produce an empty definition response. Tooling MUST NOT require virtual documents, untitled buffers, or synthetic URIs for this capability.
When a frontend exposes definition or navigation, each returned location MUST identify a compiler-known source in the analyzed revision. Editable main-project locations use `file:`. Bundled SDK/core and ordinary dependency sources use server-issued `prometeu-source:` identities for consumers that negotiate external-source support. Ownership, not physical containment or `SourceKind.PROJECT`, determines editability. Clients without support receive only editable physical project destinations; absent frontend support retains the empty fallback. This explicitly supersedes the earlier physical-only definition policy (DEC-0062).
`FrontendLanguageService.supportsExternalSources` and `sourceDocument` are optional,
with false/empty defaults. `FrontendDefinitionLocation.source`, when present,
contains the known source's exact text, origin, name, language, read-only ownership
and opaque frontend context. Compiler contracts do not mint session URIs; that is
the project host's responsibility. The content digest is not an SDK version.
External source snapshots MUST retain the exact analyzed text, source origin and frontend semantic context together. Declaration offsets and UTF-16 protocol positions MUST be calculated from that text. The frontend owns hover and continued definition inside external documents; adapters MUST NOT resolve language semantics. SDK/core MUST remain bundled without editorial extraction. Unknown, expired, unreadable or unauthorized identities MUST report unavailable without arbitrary path fallback.
External URIs are scoped to a project-server session and a content revision, not an SDK semantic version. A project-owned registry retains snapshots per consumer, independently of transport connections. Last release or project shutdown releases text and semantic context together. Reconnection MUST revalidate before queries resume; changed revisions require fresh navigation, never old positions applied to new text. One consumer closing MUST NOT invalidate another. All edit proposals touching an external source, including physical aliases, MUST be refused atomically. References, workspace symbols, document links and hierarchies retain their existing physical scopes; external navigation does not expand them.
When a frontend exposes find-references, each returned usage or declaration location MUST identify a compiler-known physical source file. A cursor offset whose definition response is empty MUST produce an empty references response. Host `includeDeclaration` MAY add the canonical declaration location only when that declaration is a compiler-known physical source file; it MUST NOT invent a virtual, untitled, or synthetic location.
@@ -246,7 +246,7 @@ for one fixed port. The public LSP configuration default remains unchanged for
other hosts. A manifest language change requires reopening that project's server;
the adapter MUST NOT send queries to a server bound to another language/root.
Each query uses a short-lived client connection. For document requests, the
Each query uses a short-lived client connection. For physical document requests, the
adapter opens exactly the saved UTF-8 text and waits for that connection's
published diagnostics before issuing the semantic query. Other dependency files
are read from disk by the existing compiler bridge. A fresh connection prevents
@@ -266,14 +266,54 @@ Core bounds the request worker wait to 12 seconds. Documents are limited to 4 Mi
of UTF-8 and task-local hierarchy state to 1024 items. These are adapter resource
guards, unrelated to the runtime's deterministic frame/watchdog semantics.
Document URIs must resolve to known physical project/dependency source files.
Document URIs must resolve to known physical project/dependency source files or to an external identity owned by the current task and project.
Symlink targets are canonicalized before admission. External sources may be
queried, but edit-producing queries on read-only roots are rejected. Every URI in
returned workspace-edit proposals is checked against the same editable/read-only
scope; an unsafe proposal is rejected as a whole. File-operation edits and
executable commands are not offered; `workspace/applyEdit` is always refused.
No query writes a file. Virtual SDK declarations may not have physical definition
locations in the existing LSP; indexed SDK references remain available instead.
No query writes a file. `readExternalSource` reads the exact registered external
revision; hover and definition on its URI use the retained frontend semantic
context. Indexed SDK reference tools remain separate and available. MCP needs no
VS Code connection and MUST retain its consumer across short-lived query sockets.
Task return/cancellation releases that task's external uses; project shutdown
invalidates all identities. Cross-task/project replay is refused.
### External-source protocol (DEC-0062)
Initialize negotiates `capabilities.experimental.prometeuExternalSources: true`;
the server advertises the same flag only when supported. External definitions use
`prometeu-source:` URIs minted by the project host. A descriptor carries `uri`,
`revision` (content digest), `origin`, `name`, `languageId` and `readOnly: true`.
Locations carry ordinary LSP UTF-16 ranges over that exact source text.
The `prometeu/readExternalSource`, `prometeu/retainExternalSource`,
`prometeu/releaseExternalSource` and `prometeu/revalidateExternalSource` requests
carry `uri` and a server-issued `consumerId`. Initialize returns that identity in
`capabilities.experimental.prometeuExternalConsumerId`. Reconnecting clients pass
it as `initializationOptions.prometeuExternalConsumerId`; MCP also requests
`initializationOptions.prometeuExternalPersistentConsumer: true` on first use.
`releaseExternalSource` with `uri: null` releases the consumer and all its sources.
A read or explicit retain acknowledges a navigation result; unacknowledged results
are released when the socket closes. MCP acknowledges returned definitions before
closing its query connection and rolls back newly retained results on failure.
Read-only edit refusal uses LSP error `-32010`, mapped to MCP `READ_ONLY_SCOPE`.
Consumer identity is scoped to the
project session; the MCP adapter additionally binds it to its authorized task.
Read returns the descriptor plus `text`. Retain/release are idempotent per consumer.
Revalidation confirms the same admitted origin and revision before queries resume;
a changed revision reports unavailable and requires new navigation. Unknown,
expired, unauthorized and unreadable sources are explicit errors, never disk reads
chosen by client paths. Consumers are created by the server, not caller-selected.
VS Code uses a read-only text content provider, with the source language and no
save-back path. Disconnect may leave the immutable text visible, but blocks
semantic queries until revalidation. No content-change event may replace the old
revision under old positions. Closing the last consumer releases the snapshot;
one reader closing must not invalidate another reader. Edit operations and mixed
workspace edits are rejected in their entirety, including physical aliases.
Packaged Studio validation is excluded from this delivery; development tests must
exercise both VS Code and independent MCP navigation.
## 4. Result, Error and Operation Contracts
+11 -6
View File
@@ -27,17 +27,22 @@ pipeline {
--no-daemon
"""
}
junit testResults: '**/build/test-results/test/TEST-*.xml'
}
withChecks(name: 'JaCoCo Coverage', includeStage: false) {
recordCoverage(tools: [[parser: 'JACOCO', pattern: "build/reports/jacoco/jacocoTestReport/jacocoTestReport.xml"]],
id: 'jacoco',
name: 'JaCoCo Coverage',
checksName: 'JaCoCo Coverage',
sourceDirectories: [[path: 'glob:**/src/main/java']],
sourceCodeEncoding: 'UTF-8',
sourceCodeRetention: 'EVERY_BUILD',
qualityGates: [
[threshold: 50.0, metric: 'CLASS', baseline: 'PROJECT', unstable: true],
[threshold: 50.0, metric: 'METHOD', baseline: 'PROJECT', unstable: true],
[threshold: 50.0, metric: 'LINE', baseline: 'PROJECT', unstable: true],
[threshold: 50.0, metric: 'INSTRUCTION', baseline: 'PROJECT', unstable: true],
[threshold: 40.0, metric: 'BRANCH', baseline: 'PROJECT', unstable: true]])
junit testResults: '**/build/test-results/test/TEST-*.xml'
[threshold: 50.0, metric: 'CLASS', baseline: 'PROJECT', criticality: 'UNSTABLE'],
[threshold: 50.0, metric: 'METHOD', baseline: 'PROJECT', criticality: 'UNSTABLE'],
[threshold: 50.0, metric: 'LINE', baseline: 'PROJECT', criticality: 'UNSTABLE'],
[threshold: 50.0, metric: 'INSTRUCTION', baseline: 'PROJECT', criticality: 'UNSTABLE'],
[threshold: 40.0, metric: 'BRANCH', baseline: 'PROJECT', criticality: 'UNSTABLE']])
}
}
} // Test
@@ -10,6 +10,8 @@ import p.studio.api.automation.*;
import p.studio.api.project.ProjectCatalog;
import p.studio.core.language.LanguageQueryPort;
import p.studio.lsp.api.LspServerLifecycle;
import p.studio.lsp.services.protocol.ExternalSourceLanguageServer;
import org.eclipse.lsp4j.jsonrpc.Launcher;
import java.net.*;
import java.nio.file.*;
@@ -24,7 +26,7 @@ public final class AutomationLspQueries implements LanguageQueryPort {
private final ObjectMapper mapper;
private final com.google.gson.Gson gson = new MessageJsonHandler(Map.of()).getGson();
private final Map<TaskIdentity, Map<String, CallHierarchyItem>> items = new ConcurrentHashMap<>();
private static final Set<String> OPERATIONS = Set.of("diagnostics", "completion", "hover", "signatureHelp", "definition",
private static final Set<String> OPERATIONS = Set.of("readExternalSource", "diagnostics", "completion", "hover", "signatureHelp", "definition",
"references", "documentSymbols", "workspaceSymbols", "prepareRename", "rename", "codeActions", "formatting",
"documentLinks", "foldingRanges", "selectionRanges", "prepareCallHierarchy", "incomingCalls", "outgoingCalls", "semanticTokens");
private static final Set<String> EDITS = Set.of("completion", "prepareRename", "rename", "codeActions", "formatting");
@@ -33,7 +35,47 @@ public final class AutomationLspQueries implements LanguageQueryPort {
this.lifecycle = lifecycle; this.projects = projects; this.knowledge = knowledge; this.mapper = mapper;
}
@Override public void endTask(TaskIdentity task) { items.remove(task); }
private record ExternalTask(String consumer, p.studio.lsp.messages.LspServerEndpoint endpoint,
p.studio.lsp.messages.LspProjectContext binding, Set<String> uris) {}
private final Map<TaskIdentity, ExternalTask> externalTasks = new ConcurrentHashMap<>();
@Override public void endTask(TaskIdentity task) {
items.remove(task);
final var state = externalTasks.remove(task);
if (state == null) return;
final var workers = Executors.newCachedThreadPool();
try (var socket = new Socket()) {
socket.connect(new InetSocketAddress(state.endpoint().host(), state.endpoint().port()), 2000);
final var client = new QuietClient();
final var launcher = new Launcher.Builder<ExternalSourceLanguageServer>().setLocalService(client)
.setRemoteInterface(ExternalSourceLanguageServer.class).setInput(socket.getInputStream())
.setOutput(socket.getOutputStream()).setExecutorService(workers).create();
final var listening = launcher.startListening();
try {
final var server = launcher.getRemoteProxy();
final var init = new InitializeParams();
init.setRootUri(state.binding().projectRoot().toUri().toString());
init.setCapabilities(new ClientCapabilities());
init.getCapabilities().setExperimental(Map.of("prometeuExternalSources", true));
init.setInitializationOptions(Map.of("prometeuExternalConsumerId", state.consumer()));
server.initialize(init).get(2, TimeUnit.SECONDS);
server.releaseExternalSource(new ExternalSourceLanguageServer.Request(state.consumer(), null)).get(2, TimeUnit.SECONDS);
} finally { listening.cancel(true); }
} catch (Exception failure) {
// A closed project host has already released its registry. Other cleanup
// failures are surfaced instead of silently reporting a successful task end.
if (lifecycle.endpoint(state.binding()).isPresent())
throw new IllegalStateException("Could not release task external sources", failure);
} finally { workers.shutdownNow(); }
}
private static class QuietClient implements LanguageClient {
public void telemetryEvent(Object event) {}
public void publishDiagnostics(PublishDiagnosticsParams event) {}
public void showMessage(MessageParams message) {}
public CompletableFuture<MessageActionItem> showMessageRequest(ShowMessageRequestParams request) { return CompletableFuture.completedFuture(null); }
public void logMessage(MessageParams message) {}
}
@Override public Map<String, Object> query(TaskIdentity task, String operation, Map<String, Object> parameters) throws Exception {
if (!OPERATIONS.contains(operation)) throw failure(AutomationFailure.Code.UNSUPPORTED_CAPABILITY, "Unsupported language operation: " + operation);
@@ -55,7 +97,15 @@ public final class AutomationLspQueries implements LanguageQueryPort {
uri = params.path("uri").asText(null);
if (uri == null) throw new IllegalArgumentException("uri is required");
}
Path document = uri == null ? null : sourcePath(context, uri);
final boolean external = uri != null && uri.startsWith("prometeu-source:");
if (external) {
final var state = externalTasks.get(task);
if (state == null || !state.uris().contains(uri)) throw failure(AutomationFailure.Code.OUTSIDE_PROJECT_SCOPE, "Unknown task-owned external source");
if (EDITS.contains(operation)) throw failure(AutomationFailure.Code.READ_ONLY_SCOPE, "External sources are read-only");
if (!Set.of("readExternalSource", "hover", "definition").contains(operation))
throw failure(AutomationFailure.Code.UNSUPPORTED_CAPABILITY, "Operation is unavailable for external documents");
} else if (operation.equals("readExternalSource")) throw new IllegalArgumentException("An external source URI is required");
Path document = uri == null || external ? null : sourcePath(context, uri);
if (document != null && EDITS.contains(operation)) requireEditable(context, document.toUri().toString());
String source = null;
if (document != null) {
@@ -83,14 +133,28 @@ public final class AutomationLspQueries implements LanguageQueryPort {
return CompletableFuture.completedFuture(new ApplyWorkspaceEditResponse(false));
}
};
var launcher = LSPLauncher.createClientLauncher(client, socket.getInputStream(), socket.getOutputStream(), workers, consumer -> consumer);
var launcher = new Launcher.Builder<ExternalSourceLanguageServer>().setLocalService(client)
.setRemoteInterface(ExternalSourceLanguageServer.class).setInput(socket.getInputStream())
.setOutput(socket.getOutputStream()).setExecutorService(workers).create();
var listening = launcher.startListening();
final var server = launcher.getRemoteProxy();
final var acquired = new HashSet<String>();
boolean succeeded = false;
try {
var server = launcher.getRemoteProxy();
var initialize = new InitializeParams();
initialize.setRootUri(binding.projectRoot().toUri().toString());
initialize.setCapabilities(new ClientCapabilities());
initialize.getCapabilities().setExperimental(Map.of("prometeuExternalSources", true));
final var previous = externalTasks.get(task);
initialize.setInitializationOptions(previous == null
? Map.of("prometeuExternalPersistentConsumer", true)
: Map.of("prometeuExternalConsumerId", previous.consumer(), "prometeuExternalPersistentConsumer", true));
var capabilities = await(server.initialize(initialize), deadline).getCapabilities();
final var experimental = gson.toJsonTree(capabilities.getExperimental());
if (experimental.isJsonObject() && experimental.getAsJsonObject().has("prometeuExternalConsumerId")) {
final String consumer = experimental.getAsJsonObject().get("prometeuExternalConsumerId").getAsString();
externalTasks.putIfAbsent(task, new ExternalTask(consumer, endpoint, binding, ConcurrentHashMap.newKeySet()));
}
server.initialized(new InitializedParams());
requireCapability(operation, capabilities);
List<Diagnostic> currentDiagnostics = List.of();
@@ -99,9 +163,17 @@ public final class AutomationLspQueries implements LanguageQueryPort {
server.getTextDocumentService().didOpen(new DidOpenTextDocumentParams(new TextDocumentItem(uri, primary.id(), 1, source)));
currentDiagnostics = await(diagnostics, deadline);
}
if (external) {
final var request = new ExternalSourceLanguageServer.Request(externalTasks.get(task).consumer(), uri);
source = await(server.revalidateExternalSource(request), deadline).text();
if (source.getBytes(java.nio.charset.StandardCharsets.UTF_8).length > 4 * 1024 * 1024)
throw failure(AutomationFailure.Code.LIMIT_EXCEEDED, "External source exceeds 4 MiB");
}
var text = server.getTextDocumentService();
var id = uri == null ? null : new TextDocumentIdentifier(uri);
Object result = switch (operation) {
case "readExternalSource" -> await(server.readExternalSource(
new ExternalSourceLanguageServer.Request(externalTasks.get(task).consumer(), uri)), deadline);
case "diagnostics" -> currentDiagnostics;
case "hover" -> await(text.hover(new HoverParams(id, position(params.path("position"), source))), deadline);
case "completion" -> await(text.completion(new CompletionParams(id, position(params.path("position"), source))), deadline);
@@ -136,6 +208,17 @@ public final class AutomationLspQueries implements LanguageQueryPort {
};
var wire = mapper.readTree(gson.toJson(result));
validateEdits(context, wire);
if (operation.equals("definition") && externalTasks.containsKey(task)) {
final var returned = new HashSet<String>();
rememberExternalUris(returned, wire);
acquired.addAll(returned);
acquired.removeAll(externalTasks.get(task).uris());
for (String externalUri : returned) {
await(server.retainExternalSource(new ExternalSourceLanguageServer.Request(
externalTasks.get(task).consumer(), externalUri)), deadline);
externalTasks.get(task).uris().add(externalUri);
}
}
var limitations = new ArrayList<String>();
if (operation.equals("codeActions") && wire.isArray()) {
for (var action : wire) if (action.has("command")) {
@@ -147,11 +230,30 @@ public final class AutomationLspQueries implements LanguageQueryPort {
var output = new LinkedHashMap<String, Object>();
output.put("operation", operation); output.put("result", mapper.convertValue(wire, Object.class));
output.put("limitations", limitations);
succeeded = true;
return Collections.unmodifiableMap(output);
} finally { listening.cancel(true); }
} finally {
if (!succeeded && !acquired.isEmpty() && externalTasks.containsKey(task)) {
final var state = externalTasks.get(task);
final boolean interrupted = Thread.interrupted();
try {
for (String acquiredUri : acquired) {
state.uris().remove(acquiredUri);
server.releaseExternalSource(new ExternalSourceLanguageServer.Request(state.consumer(), acquiredUri))
.get(500, TimeUnit.MILLISECONDS);
}
} finally { if (interrupted) Thread.currentThread().interrupt(); }
}
listening.cancel(true);
}
} finally { workers.shutdownNow(); }
}
private void rememberExternalUris(Set<String> uris, com.fasterxml.jackson.databind.JsonNode node) {
if (node.isObject() && node.path("uri").asText().startsWith("prometeu-source:")) uris.add(node.path("uri").asText());
if (node.isContainerNode()) node.forEach(child -> rememberExternalUris(uris, child));
}
private void addItemIds(TaskIdentity task, com.fasterxml.jackson.databind.JsonNode node) {
if (node.isObject() && node.has("uri") && node.has("selectionRange") && node.has("name") && node.has("kind")) {
var stored = items.computeIfAbsent(task, ignored -> new ConcurrentHashMap<>());
@@ -163,6 +265,12 @@ public final class AutomationLspQueries implements LanguageQueryPort {
}
private static void requireCapability(String operation, ServerCapabilities caps) {
if (operation.equals("readExternalSource")) {
final var raw = new com.google.gson.Gson().toJsonTree(caps.getExperimental());
if (!raw.isJsonObject() || !raw.getAsJsonObject().has("prometeuExternalSources"))
throw failure(AutomationFailure.Code.UNSUPPORTED_CAPABILITY, "LSP does not advertise external sources");
return;
}
Object capability = switch (operation) {
case "diagnostics" -> caps.getTextDocumentSync();
case "hover" -> caps.getHoverProvider(); case "completion" -> caps.getCompletionProvider();
@@ -233,6 +341,7 @@ public final class AutomationLspQueries implements LanguageQueryPort {
catch (ExecutionException failure) {
if (failure.getCause() instanceof org.eclipse.lsp4j.jsonrpc.ResponseErrorException response) {
int code = response.getResponseError().getCode();
if (code == -32010) throw failure(AutomationFailure.Code.READ_ONLY_SCOPE, response.getResponseError().getMessage());
if (code == -32602) throw new IllegalArgumentException("LSP rejected query parameters");
if (code == -32601 || code == -32600)
throw failure(AutomationFailure.Code.UNSUPPORTED_CAPABILITY, response.getResponseError().getMessage());
@@ -367,7 +367,7 @@
{
"name": "studio_language_query",
"authority": "task",
"description": "Query supported project LSP capabilities after synchronizing saved files. Returns refactoring proposals, never applies edits.",
"description": "Query project LSP capabilities using saved project files or task-owned read-only external sources. readExternalSource reads an external URI returned by definition; hover and definition can continue inside it. Returns refactoring proposals, never applies edits.",
"inputSchema": {
"type": "object",
"properties": {
@@ -52,6 +52,7 @@ public final class InstalledKnowledgeProbe {
BuilderPipelineContext.fromConfig(new BuilderPipelineConfig(false, root.toString())), LogAggregator.empty());
check(result.diagnostics().stream().noneMatch(issue -> issue.isError()), "Example must compile without errors");
check(result.bytecodeBytes().length > 0, "Example must produce bytecode");
verifyExternalSources(provider.getClass().getClassLoader());
registration.unsubscribe();
System.out.println("Installed knowledge verified: " + docs.size() + " references; example compiled");
}
@@ -59,4 +60,64 @@ public final class InstalledKnowledgeProbe {
private static void check(boolean condition, String message) {
if (!condition) throw new AssertionError(message);
}
private static void verifyExternalSources(ClassLoader loader) throws Exception {
var resource = loader.getResource("game/stdlib/1/sdk/gfx/main.pbs");
check(resource != null && resource.getProtocol().equals("jar"), "SDK must be read from a JAR URL");
check(p.studio.lsp.services.compiler.CompilerLanguageServiceBridge.class.getProtectionDomain()
.getCodeSource().getLocation().getPath().endsWith(".jar"), "LSP bridge must come from installed JAR");
Path root = Files.createDirectory(Path.of("external-sources")).toRealPath();
Files.createDirectories(root.resolve("src"));
Files.writeString(root.resolve("prometeu.json"), """
{"name":"app","version":"1.0.0","language":"pbs","stdlib":"1","target":"Game","dependencies":[]}
""");
String source = "import { Gfx } from @sdk:gfx;\nfn main() -> void { Gfx.clear(0); }\n";
Path file = Files.writeString(root.resolve("src/main.pbs"), source);
Files.writeString(root.resolve("src/mod.barrel"), "pub fn main() -> void;\n");
var context = new p.studio.lsp.messages.LspProjectContext("installed", "pbs", root);
var bridge = new p.studio.lsp.services.compiler.CompilerLanguageServiceBridge();
try (var registry = new p.studio.lsp.services.compiler.ExternalSourceRegistry()) {
String consumer = registry.connect(null, false);
int[] sdkUse = position(source, "Gfx.clear", "Gfx.".length());
var targets = bridge.navigation(context, file.toUri().toString(), source, sdkUse[0], sdkUse[1], registry, consumer).locations();
check(targets.size() == 1, "Packaged SDK definition must resolve");
String uri = targets.getFirst().uri();
check(uri.startsWith("prometeu-source:"), "SDK must use a virtual URI");
var content = registry.read(consumer, uri);
check(content.descriptor().readOnly() && content.descriptor().origin().equals("bundled"), "Bundled SDK must be read-only");
try (var stream = resource.openStream()) {
check(content.text().equals(new String(stream.readAllBytes(), java.nio.charset.StandardCharsets.UTF_8)), "SDK text must exactly match JAR bytes");
}
int[] hostUse = position(content.text(), "LowGfx.clear", "LowGfx.".length());
check(bridge.externalHover(context, uri, hostUse[0], hostUse[1], registry, consumer).markdown().contains("clear"), "SDK hover must work");
var host = bridge.navigation(context, uri, null, hostUse[0], hostUse[1], registry, consumer).locations().getFirst();
check(registry.read(consumer, host.uri()).text().split("\n")[host.startLine()].contains("clear"), "SDK to host definition range must match");
check(!bridge.externalSemanticTokens(context, uri, registry, consumer).tokens().isEmpty(), "SDK semantic highlighting must work");
int[] colorUse = position(content.text(), "color: Color", 7);
var core = bridge.navigation(context, uri, null, colorUse[0], colorUse[1], registry, consumer).locations().getFirst();
var coreContent = registry.read(consumer, core.uri());
check(coreContent.descriptor().readOnly() && coreContent.descriptor().name().replace('\\', '/').contains("core/color"), "SDK to core definition must resolve");
var coreResource = loader.getResource("game/stdlib/1/core/color/main.pbs");
check(coreResource != null && coreResource.getProtocol().equals("jar"), "Core must be read from a JAR URL");
try (var stream = coreResource.openStream()) {
check(coreContent.text().equals(new String(stream.readAllBytes(), java.nio.charset.StandardCharsets.UTF_8)), "Core text must exactly match JAR bytes");
}
check(coreContent.text().split("\n")[core.startLine()].contains("Color"), "Core definition range must match");
registry.disconnect(consumer);
registry.connect(consumer, false);
check(registry.revalidate(consumer, uri, path -> bridge.currentSource(context, path)).text().equals(content.text()), "Bundled source must revalidate after reconnect");
}
try (var files = Files.walk(root)) {
check(files.filter(Files::isRegularFile).map(root::relativize).map(Path::toString).collect(java.util.stream.Collectors.toSet())
.equals(java.util.Set.of("prometeu.json", "src/main.pbs", "src/mod.barrel")), "Navigation must not extract bundled sources into the project");
}
System.out.println("Packaged external sources verified: JAR bytes, SDK/host/core navigation, hover, tokens, read-only, reconnect, no extraction");
}
private static int[] position(String text, String needle, int offset) {
int index = text.indexOf(needle);
check(index >= 0, "Missing source fragment: " + needle);
String prefix = text.substring(0, index + offset);
return new int[] {(int) prefix.chars().filter(c -> c == '\n').count(), prefix.length() - prefix.lastIndexOf('\n') - 1};
}
}
@@ -20,7 +20,7 @@ class InstalledKnowledgeTest {
}
String probeClasses = Path.of(InstalledKnowledgeProbe.class.getProtectionDomain().getCodeSource().getLocation().toURI()).toString();
Path output = outsideRepository.resolve("probe.log");
var process = new ProcessBuilder(Path.of(System.getProperty("java.home"), "bin", "java").toString(),
var process = new ProcessBuilder(distribution.resolve("runtime/bin/java").toString(),
"-cp", jars + java.io.File.pathSeparator + probeClasses, InstalledKnowledgeProbe.class.getName())
.directory(outsideRepository.toFile()).redirectErrorStream(true).redirectOutput(output.toFile()).start();
try {
@@ -28,6 +28,7 @@ class InstalledKnowledgeTest {
String log = Files.readString(output);
assertEquals(0, process.exitValue(), log);
assertTrue(log.contains("references; example compiled"), log);
assertTrue(log.contains("Packaged external sources verified"), log);
} finally { if (process.isAlive()) process.destroyForcibly().waitFor(); }
}
}
@@ -54,6 +54,36 @@ class AutomationLspQueriesTest {
Map<String, Object> at(int line, int character) {
return Map.of("uri", source.toUri().toString(), "position", Map.of("line", line, "character", character));
}
@Test void bundledSourcesAreReadableAcrossConnectionsAndOwnedByTheTask() throws Exception {
Files.writeString(source, "import { Gfx } from @sdk:gfx;\nfn main() -> void { Gfx.clear(0); }\n");
Files.writeString(source.resolveSibling("mod.barrel"), "pub fn main() -> void;\n");
acquire();
final var definition = query("definition", at(1, 24)).path("result");
assertFalse(definition.isEmpty(), definition.toString());
final String uri = definition.get(0).path("uri").asText();
assertTrue(uri.startsWith("prometeu-source:"), uri);
final var content = query("readExternalSource", Map.of("uri", uri)).path("result");
assertTrue(content.path("descriptor").path("readOnly").asBoolean());
assertEquals("bundled", content.path("descriptor").path("origin").asText());
final String text = content.path("text").asText();
final int offset = text.indexOf("LowGfx.clear") + "LowGfx.".length();
assertTrue(offset > "LowGfx.".length());
final String prefix = text.substring(0, offset);
final var position = Map.of("line", (int) prefix.chars().filter(c -> c == '\n').count(),
"character", offset - prefix.lastIndexOf('\n') - 1);
final var request = Map.<String, Object>of("uri", uri, "position", position);
assertTrue(query("hover", request).path("result").toString().contains("clear"));
final var host = query("definition", request).path("result");
assertFalse(host.isEmpty());
assertTrue(query("readExternalSource", Map.of("uri", host.get(0).path("uri").asText()))
.path("result").path("text").asText().contains("declare host LowGfx"));
assertEquals(AutomationFailure.Code.READ_ONLY_SCOPE, assertThrows(AutomationException.class,
() -> query("formatting", request)).failure().code());
session.automation().reclaim().toCompletableFuture().join();
acquire();
assertEquals(AutomationFailure.Code.OUTSIDE_PROJECT_SCOPE, assertThrows(AutomationException.class,
() -> query("readExternalSource", Map.of("uri", uri))).failure().code());
}
@Test void diagnosticsAndNavigationSeeSavedChangesOverTheActualServer() throws Exception {
acquire();
assertTrue(query("diagnostics", at(0, 0)).path("result").isArray());
@@ -186,26 +216,43 @@ class AutomationLspQueriesTest {
assertTrue(cleaned.get());
assertEquals(ControlState.MANUAL, session.automation().snapshot().state());
}
@Test void externalDependencyNavigationIsReadableButRefactoringIsReadOnly() throws Exception {
Path lib = Files.createDirectories(temporary.resolve("lib")).toRealPath();
@org.junit.jupiter.params.ParameterizedTest
@org.junit.jupiter.params.provider.ValueSource(booleans = {false, true})
void externalDependencyNavigationIsReadableButRefactoringIsReadOnly(boolean inWorkspace) throws Exception {
Path lib = Files.createDirectories(inWorkspace ? root.resolve(".workspace/repos/lib") : temporary.resolve("lib")).toRealPath();
manifest(lib, "lib", List.of());
Path module = Files.createDirectories(lib.resolve("src/shared"));
Path declaration = module.resolve("main.pbs");
Files.writeString(declaration, "fn answer() -> int { return 42; }\n");
Files.writeString(declaration, "// á 😀\r\nfn answer() -> int { return helper(); }\r\nfn helper() -> int { return 42; }\r\n");
Files.writeString(module.resolve("mod.barrel"), "pub fn answer() -> int;\n");
manifest(root, "app", List.of(Map.of("path", "../lib")));
manifest(root, "app", List.of(Map.of("path", inWorkspace ? ".workspace/repos/lib" : "../lib")));
Files.createDirectories(root.resolve(".workspace/repos")); Files.writeString(root.resolve(".workspace/repos/registry.json"), "{}");
Files.writeString(source, "import { answer } from @lib:shared;\nfn main() -> int { return answer(); }\n");
Files.writeString(source.resolveSibling("mod.barrel"), "pub fn main() -> int;\n");
acquire();
var locations = query("definition", at(1, 26)).path("result");
assertFalse(locations.isEmpty(), locations.toString());
assertEquals(declaration.toUri().toString(), locations.get(0).path("uri").asText());
assertTrue(locations.get(0).path("uri").asText().startsWith("prometeu-source:"));
assertEquals(Files.readString(declaration), query("readExternalSource", Map.of("uri", locations.get(0).path("uri").asText())).path("result").path("text").asText());
final String externalUri = locations.get(0).path("uri").asText();
assertEquals(1, locations.get(0).path("range").path("start").path("line").asInt());
final var nested = query("definition", Map.of("uri", externalUri, "position", Map.of("line", 1, "character", 28))).path("result");
assertFalse(nested.isEmpty(), nested.toString());
assertEquals(2, nested.get(0).path("range").path("start").path("line").asInt());
assertEquals("documentSymbols", query("documentSymbols", Map.of("uri", declaration.toUri().toString())).path("operation").asText());
assertEquals(AutomationFailure.Code.READ_ONLY_SCOPE, assertThrows(AutomationException.class,
() -> query("formatting", Map.of("uri", declaration.toUri().toString()))).failure().code());
final Path alias = Files.createSymbolicLink(root.resolve("src/dependency-alias.pbs"), declaration);
assertEquals(AutomationFailure.Code.READ_ONLY_SCOPE, assertThrows(AutomationException.class,
() -> query("formatting", Map.of("uri", alias.toUri().toString()))).failure().code());
var rename = new HashMap<>(at(1, 26)); rename.put("newName", "renamed");
assertEquals(AutomationFailure.Code.READ_ONLY_SCOPE, assertThrows(AutomationException.class, () -> query("rename", rename)).failure().code());
final String oldText = Files.readString(declaration);
Files.writeString(declaration, "// changed revision\n" + oldText);
assertThrows(AutomationException.class, () -> query("readExternalSource", Map.of("uri", externalUri)));
final var renewed = query("definition", at(1, 26)).path("result");
assertNotEquals(externalUri, renewed.get(0).path("uri").asText());
assertEquals(2, renewed.get(0).path("range").path("start").path("line").asInt());
}
private void manifest(Path path, String name, List<?> dependencies) throws Exception {
mapper.writeValue(path.resolve("prometeu.json").toFile(), Map.of("name", name, "version", "1.0.0", "language", "pbs", "stdlib", "1", "target", "Game", "dependencies", dependencies));
@@ -32,6 +32,7 @@ import p.studio.compiler.services.FrontendDocumentLink;
import p.studio.compiler.services.FrontendFoldingRange;
import p.studio.compiler.services.FrontendSelectionRange;
import p.studio.compiler.services.FrontendDefinitionLocation;
import p.studio.compiler.services.FrontendSourceDocument;
import p.studio.compiler.services.FrontendDocumentRequest;
import p.studio.compiler.services.FrontendDocumentSymbol;
import p.studio.compiler.services.FrontendRenameEdit;
@@ -63,6 +64,38 @@ import java.util.Objects;
import java.util.Optional;
public final class PBSFrontendLanguageService implements FrontendLanguageService {
@Override
public boolean supportsExternalSources() { return true; }
@Override
public Optional<FrontendSourceDocument> sourceDocument(
final FrontendDocumentRequest request,
final FrontendEditorialContext context,
final Path path) {
if (!(context.projectSurface() instanceof PbsSemanticReadSurface surface)
|| !(context.sourceFiles() instanceof FileTableReader files)) return Optional.empty();
FrontendSourceDocument projectSource = null;
for (final var entry : surface.astByFile().entrySet()) {
final var handle = files.get(entry.getKey());
if (handle == null || !handle.getCanonPath().toAbsolutePath().normalize().equals(path.toAbsolutePath().normalize())) continue;
final var text = handle.readUtf8();
if (text.isEmpty()) return Optional.empty();
final boolean bundled = surface.sourceKindByFile().get(entry.getKey()) == SourceKind.SDK_INTERFACE;
final Path owner = surface.ownerRootByFile().get(entry.getKey());
final boolean external = bundled || owner == null || !owner.toAbsolutePath().normalize().equals(request.projectRoot().toAbsolutePath().normalize());
final var source = new FrontendSourceDocument(path, text.orElseThrow(),
bundled ? "bundled" : external ? "dependency" : "project",
handle.getRelativePath().toString(), "pbs", external,
new FrontendEditorialContext(entry.getValue(),
surface.supplementalTopDeclsByFile().getOrDefault(entry.getKey(), ReadOnlyList.empty()),
context.semanticContext(), files, surface));
// A dependency may physically overlap a project source root. External
// ownership wins even if another file-table entry treats it as local.
if (external) return Optional.of(source);
projectSource = source;
}
return Optional.ofNullable(projectSource);
}
private final PbsEditorialSupportService editorialSupportService;
private final PBSSemanticTokenProvider semanticTokenProvider;
@@ -257,7 +290,12 @@ public final class PBSFrontendLanguageService implements FrontendLanguageService
typedSupplementalTopDecls,
feSurfaceContext,
offset)) {
toFrontendLocation(location, fileTable).ifPresent(locations::add);
final var handle = fileTable.get(location.span().getFileId());
if (handle != null) {
sourceDocument(request, editorialContext, handle.getCanonPath()).ifPresent(source ->
locations.add(new FrontendDefinitionLocation(source.path(),
toOffset(location.span().getStart()), toOffset(location.span().getEnd()), source)));
}
}
return List.copyOf(locations);
}
@@ -401,14 +439,16 @@ public final class PBSFrontendLanguageService implements FrontendLanguageService
}
final LinkedHashMap<String, FrontendRenameEdit> edits = new LinkedHashMap<>();
for (final PbsEditorialLocation site : planned.sites()) {
toFrontendLocation(site, fileTable).ifPresent(location ->
edits.putIfAbsent(
final var mapped = toFrontendLocation(site, fileTable);
if (mapped.isEmpty()) return FrontendRenameResult.refused("rename requires a non-editable source");
final var location = mapped.orElseThrow();
edits.putIfAbsent(
frontendLocationKey(location),
new FrontendRenameEdit(
location.path(),
location.startOffset(),
location.endOffset(),
newName)));
newName));
}
if (edits.isEmpty()) {
return FrontendRenameResult.refused("not renameable");
@@ -19,6 +19,7 @@ public final class PbsParser {
private final PbsTokenCursor cursor;
private final PbsTopLevelParser topLevelParser;
private final PbsBlockParser blockParser;
private final PbsExprParser expressionParser;
private PbsParser(
final ReadOnlyList<PbsToken> tokens,
@@ -29,6 +30,7 @@ public final class PbsParser {
this.cursor = context.cursor();
final var exprParser = new PbsExprParser(context, this::parseExpressionSurfaceBlock);
this.expressionParser = exprParser;
final var attributeParser = new PbsAttributeParser(context, this::isTopLevelRestartToken);
final var typeParser = new PbsTypeParser(context);
@@ -64,6 +66,16 @@ public final class PbsParser {
return topLevelParser.parseFile();
}
/** Parses a complete expression fragment using the same grammar as a source file. */
public static PbsAst.Expression parseExpression(
final ReadOnlyList<PbsToken> tokens,
final FileId fileId,
final p.studio.compiler.source.diagnostics.DiagnosticSink diagnostics) {
final var parser = new PbsParser(tokens, fileId, diagnostics, ParseMode.ORDINARY);
final var expression = parser.expressionParser.parseExpression();
return parser.cursor.isAtEnd() && !diagnostics.hasErrors() ? expression : null;
}
private PbsAst.Block parseExpressionSurfaceBlock(final String message) {
return blockParser.parseBlock(message);
}
@@ -5,6 +5,7 @@ import p.studio.compiler.pbs.ast.PbsAst;
import p.studio.compiler.pbs.lexer.PbsLexer;
import p.studio.compiler.pbs.lexer.PbsToken;
import p.studio.compiler.pbs.lexer.PbsTokenKind;
import p.studio.compiler.pbs.parser.PbsParser;
import p.studio.compiler.source.Span;
import p.studio.compiler.source.diagnostics.DiagnosticSink;
import p.studio.compiler.source.identifiers.FileId;
@@ -1396,6 +1397,11 @@ public final class PbsEditorialSupportService {
private TypeView resolveReceiverTypeBeforeDot(
final int dotTokenIndex,
final ScopeState scopeState) {
final PbsAst.Expression receiver = receiverExpressionBefore(dotTokenIndex);
if (receiver != null) {
return analyzeExpressionType(receiver, scopeState.scope(), null,
scopeState.returnType(), scopeState.resultErrorName(), scopeState.receiverType());
}
final List<String> segments = resolveChainSegmentsEndingAt(dotTokenIndex - 1);
if (segments.isEmpty()) {
return TypeView.unknown();
@@ -1407,6 +1413,34 @@ public final class PbsEditorialSupportService {
return current;
}
private PbsAst.Expression receiverExpressionBefore(final int dotTokenIndex) {
// Locate only the fragment boundary here; grammar and overload selection
// remain in the existing PBS parser and flow expression analyzer.
int start = dotTokenIndex;
int depth = 0;
for (int index = dotTokenIndex - 1; index >= 0; index--) {
final var kind = token(index).kind();
if (kind == PbsTokenKind.COMMENT) { start = index; continue; }
if (kind == PbsTokenKind.RIGHT_PAREN) {
depth++;
} else if (kind == PbsTokenKind.LEFT_PAREN) {
if (depth == 0) break;
depth--;
} else if (depth == 0 && kind != PbsTokenKind.IDENTIFIER
&& kind != PbsTokenKind.DOT && kind != PbsTokenKind.THIS
&& kind != PbsTokenKind.NEW) {
break;
}
start = index;
}
if (start == dotTokenIndex || depth != 0) return null;
final var fragment = new ArrayList<PbsToken>();
for (int index = start; index < dotTokenIndex; index++) fragment.add(token(index));
final int end = token(dotTokenIndex).start();
fragment.add(new PbsToken(PbsTokenKind.EOF, "", end, end));
return PbsParser.parseExpression(ReadOnlyList.wrap(fragment), ast.span().getFileId(), DiagnosticSink.empty());
}
private TypeView resolveBaseChainType(
final String baseName,
final ScopeState scopeState) {
@@ -132,11 +132,16 @@ public class PBSFrontendPhaseService implements FrontendPhaseService {
final Map<FileId, ReadOnlyList<p.studio.compiler.pbs.ast.PbsAst.TopDecl>> supplementalTopDeclsByFile = new LinkedHashMap<>();
final Map<FileId, ReadOnlyList<PbsInlineHintSurface>> inlineHintsByFile = new LinkedHashMap<>();
final Map<FileId, SourceKind> sourceKindByFile = new LinkedHashMap<>();
final Map<FileId, Path> ownerRootByFile = new LinkedHashMap<>();
for (final var entry : importedSemanticContexts.entrySet()) {
supplementalTopDeclsByFile.put(entry.getKey(), entry.getValue().supplementalTopDecls());
}
for (final var parsedSourceFile : assembly.parsedSourceFiles()) {
astByFile.put(parsedSourceFile.fileId(), parsedSourceFile.ast());
final var sourceHandle = ctx.fileTable.get(parsedSourceFile.fileId());
if (sourceHandle != null && parsedSourceFile.sourceKind() != SourceKind.SDK_INTERFACE) {
ownerRootByFile.put(parsedSourceFile.fileId(), ctx.projectTable.get(sourceHandle.getProjectId()).getRootPath());
}
sourceKindByFile.put(
parsedSourceFile.fileId(),
parsedSourceFile.sourceKind() == null ? SourceKind.PROJECT : parsedSourceFile.sourceKind());
@@ -158,6 +163,7 @@ public class PBSFrontendPhaseService implements FrontendPhaseService {
Map.copyOf(supplementalTopDeclsByFile),
Map.copyOf(inlineHintsByFile),
Map.copyOf(sourceKindByFile),
Map.copyOf(ownerRootByFile),
physicalModuleFiles,
moduleCompletionLabels(ctx, physicalModuleFiles));
}
@@ -225,6 +231,7 @@ public class PBSFrontendPhaseService implements FrontendPhaseService {
Map<FileId, ReadOnlyList<p.studio.compiler.pbs.ast.PbsAst.TopDecl>> supplementalTopDeclsByFile,
Map<FileId, ReadOnlyList<PbsInlineHintSurface>> inlineHintsByFile,
Map<FileId, SourceKind> sourceKindByFile,
Map<FileId, Path> ownerRootByFile,
Map<ModuleReference, Path> physicalModuleFileByReference,
List<String> moduleCompletionLabels) {
}
@@ -16,6 +16,79 @@ import static org.junit.jupiter.api.Assertions.*;
final class PbsEditorialSupportServiceTest {
private final PbsEditorialSupportService service = new PbsEditorialSupportService();
@Test
void receiverMatrixHandlesNestedCallsVariablesParenthesesAndRecovery() {
final String declarations = """
declare struct Button() { fn down(value: int) -> bool { return true; } }
declare struct Pad() { fn left() -> Button { return new Button(); } }
fn pad(value: int) -> Pad { return new Pad(); }
fn number() -> int { return 1; }
""";
for (final String receiver : List.of("pad(number()).left()", "(pad(1).left())", "typed", "inferred")) {
for (final String suffix : List.of("down(1);", "down(", "")) {
final String source = declarations + """
fn main() -> void {
let typed: Button = new Button();
let inferred = pad(1).left();
""" + receiver + "." + suffix + "\n}\ndeclare ;";
final var ast = parseAllowingErrors(source);
final int member = source.lastIndexOf(receiver + ".") + receiver.length() + 1;
assertTrue(service.completion(source, ast, ReadOnlyList.empty(), member).stream()
.anyMatch(item -> item.label().equals("down")), receiver + "/" + suffix);
if (!suffix.isEmpty()) {
assertTrue(service.hover(source, ast, ReadOnlyList.empty(), member).isPresent(), receiver);
assertFalse(service.definition(source, ast, ReadOnlyList.empty(), member).isEmpty(), receiver);
assertTrue(service.signatureHelp(source, ast, ReadOnlyList.empty(), member + 5).isPresent(), receiver);
}
}
}
}
@Test
void ambiguousAndUnknownReceiverDoNotInventMembers() {
final String source = """
declare struct A() { fn onlyA() -> int { return 1; } }
declare struct B() { fn onlyB() -> int { return 2; } }
fn choose(value: int) -> A { return new A(); }
fn choose(value: int) -> B { return new B(); }
fn main() -> void { choose(1).onlyA(); missing().onlyA(); }
""";
final var ast = parseOrdinary(source);
for (final String receiver : List.of("choose(1).", "missing().")) {
final int offset = source.indexOf(receiver) + receiver.length();
assertTrue(service.hover(source, ast, ReadOnlyList.empty(), offset).isEmpty());
assertTrue(service.definition(source, ast, ReadOnlyList.empty(), offset).isEmpty());
assertTrue(service.completion(source, ast, ReadOnlyList.empty(), offset).isEmpty());
assertTrue(service.signatureHelp(source, ast, ReadOnlyList.empty(), offset + 6).isEmpty());
}
}
@Test
void chainedReceiversShareTypeResolutionAcrossEditorialQueries() {
final String source = """
import { Input } from @sdk:input;
fn main() -> void {
Input.pad().left().down();
}
""";
final var ast = parseOrdinary(source);
final var supplemental = supplementalTopDecls("""
declare host Input { fn pad() -> Pad; }
declare builtin type Pad(pub id: int) { fn left() -> Button; }
declare builtin type Button(pub id: int) { fn down() -> bool; }
""");
final int offset = source.indexOf("down");
assertAll(
() -> assertEquals("Button", service.hover(source, ast, supplemental, offset)
.orElseThrow().origin()),
() -> assertEquals(100, service.definition(source, ast, supplemental, offset)
.getFirst().span().getFileId().getId()),
() -> assertTrue(service.completion(source, ast, supplemental, offset).stream()
.anyMatch(candidate -> candidate.label().equals("down"))),
() -> assertEquals("down() -> bool", service.signatureHelp(source, ast, supplemental, offset + 5)
.orElseThrow().signatures().getFirst().label()));
}
@Test
void shouldResolveGeneralCompletionAndHoverAcrossLocalAndImportedSymbols() {
final var source = """
@@ -26,6 +26,7 @@ final class SourceCrawler extends SimpleFileVisitor<Path> {
private final ReadOnlySet<String> allowedExtensions;
private final List<Path> paths;
private Path traversalRoot;
public SourceCrawler(
final ReadOnlySet<String> allowedExtensions,
@@ -36,7 +37,12 @@ final class SourceCrawler extends SimpleFileVisitor<Path> {
@Override
public FileVisitResult preVisitDirectory(final Path path, final BasicFileAttributes attrs) {
for (final var directory : path) {
if (traversalRoot == null) {
// The caller already admitted this source root. Ignore only descendants,
// not ancestors such as .workspace containing an explicitly resolved library.
traversalRoot = path;
}
for (final var directory : traversalRoot.relativize(path)) {
if (!isAllowedPath(directory)) {
return FileVisitResult.SKIP_SUBTREE;
}
@@ -83,4 +89,4 @@ final class SourceCrawler extends SimpleFileVisitor<Path> {
return allowedExtensions.contains(extension);
}
}
}
@@ -0,0 +1,64 @@
package p.studio.compiler.workspaces.stages;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.io.TempDir;
import p.studio.utilities.structures.ReadOnlySet;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.ArrayList;
import java.util.Set;
import static org.junit.jupiter.api.Assertions.assertEquals;
class SourceCrawlerTest {
@TempDir Path project;
@Test
void resolvedWorkspaceLibraryEntersFileTable() throws Exception {
final Path library = Files.createDirectories(project.resolve(".workspace/repos/lib/src")).getParent();
Files.createDirectories(project.resolve("src"));
Files.writeString(project.resolve("prometeu.json"), """
{"name":"app","version":"1.0.0","language":"pbs","stdlib":"1","target":"Game",
"dependencies":[{"path":".workspace/repos/lib"}]}
""");
Files.writeString(library.resolve("prometeu.json"), """
{"name":"lib","version":"1.0.0","language":"pbs","stdlib":"1","target":"Game",
"dependencies":[]}
""");
Files.writeString(project.resolve(".workspace/repos/registry.json"), "{}");
final Path source = Files.writeString(library.resolve("src/main.pbs"), "fn value() -> int { return 1; }");
Files.writeString(library.resolve("src/mod.barrel"), "export { value };\n");
final var context = p.studio.compiler.models.BuilderPipelineContext.fromConfig(
new p.studio.compiler.messages.BuilderPipelineConfig(false, project.toString()));
final var logs = p.studio.utilities.logs.LogAggregator.empty();
org.junit.jupiter.api.Assertions.assertFalse(new ResolveDepsPipelineStage().run(context, logs).hasErrors());
org.junit.jupiter.api.Assertions.assertFalse(new LoadSourcesPipelineStage().run(context, logs).hasErrors());
final var loaded = new ArrayList<Path>();
for (int i = 0; i < context.fileTable.size(); i++) {
loaded.add(context.fileTable.get(new p.studio.compiler.source.identifiers.FileId(i)).getCanonPath());
}
org.junit.jupiter.api.Assertions.assertTrue(loaded.contains(source.toRealPath()), loaded.toString());
}
@Test
void explicitlyAdmittedDependencyRootInsideWorkspaceIsTraversed() throws Exception {
final Path sourceRoot = Files.createDirectories(project.resolve(".workspace/repos/lib/src"));
final Path source = Files.writeString(sourceRoot.resolve("main.pbs"), "fn value() -> int { return 1; }");
final Path incidental = Files.createDirectories(sourceRoot.resolve(".workspace/incidental"));
Files.writeString(incidental.resolve("hidden.pbs"), "fn hidden() -> int { return 2; }");
final var paths = new ArrayList<Path>();
Files.walkFileTree(sourceRoot, new SourceCrawler(ReadOnlySet.wrap(Set.of("pbs")), paths));
assertEquals(java.util.List.of(source), paths);
}
@Test
void ordinaryRootDoesNotCollectIncidentalWorkspaceSources() throws Exception {
final Path source = Files.writeString(project.resolve("main.pbs"), "fn value() -> int { return 1; }");
final Path hidden = Files.createDirectories(project.resolve(".workspace/repos/lib"));
Files.writeString(hidden.resolve("hidden.pbs"), "fn hidden() -> int { return 2; }");
final var paths = new ArrayList<Path>();
Files.walkFileTree(project, new SourceCrawler(ReadOnlySet.wrap(Set.of("pbs")), paths));
assertEquals(java.util.List.of(source), paths);
}
}
@@ -6,7 +6,12 @@ import java.util.Objects;
public record FrontendDefinitionLocation(
Path path,
int startOffset,
int endOffset) {
int endOffset,
FrontendSourceDocument source) {
public FrontendDefinitionLocation(Path path, int startOffset, int endOffset) {
this(path, startOffset, endOffset, null);
}
public FrontendDefinitionLocation {
path = Objects.requireNonNull(path, "path").toAbsolutePath().normalize();
@@ -16,5 +21,9 @@ public record FrontendDefinitionLocation(
if (endOffset < startOffset) {
throw new IllegalArgumentException("endOffset must not be before startOffset");
}
if (source != null && (!path.equals(source.path())
|| endOffset > source.text().getBytes(java.nio.charset.StandardCharsets.UTF_8).length)) {
throw new IllegalArgumentException("location must fit its source revision");
}
}
}
@@ -11,6 +11,13 @@ import java.util.Optional;
* Optional frontend-owned editor services.
*/
public interface FrontendLanguageService {
default boolean supportsExternalSources() { return false; }
default Optional<FrontendSourceDocument> sourceDocument(
FrontendDocumentRequest request, FrontendEditorialContext context, java.nio.file.Path path) {
return Optional.empty();
}
default Optional<FrontendEditorialContext> prepareEditorialContext(
final FrontendPhaseContext phaseContext) {
return Optional.empty();
@@ -0,0 +1,31 @@
package p.studio.compiler.services;
import java.nio.file.Path;
import java.nio.charset.StandardCharsets;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.util.HexFormat;
import java.util.Objects;
/** A compiler-known source and the semantic context for exactly that content. */
public record FrontendSourceDocument(
Path path, String text, String origin, String name, String languageId,
boolean readOnly, FrontendEditorialContext context) {
public FrontendSourceDocument {
path = Objects.requireNonNull(path).toAbsolutePath().normalize();
Objects.requireNonNull(text);
Objects.requireNonNull(origin);
Objects.requireNonNull(name);
Objects.requireNonNull(languageId);
Objects.requireNonNull(context);
}
public String revision() {
try {
return HexFormat.of().formatHex(MessageDigest.getInstance("SHA-256")
.digest(text.getBytes(StandardCharsets.UTF_8)));
} catch (NoSuchAlgorithmException impossible) {
throw new IllegalStateException(impossible);
}
}
}
@@ -0,0 +1,21 @@
package p.studio.compiler.services;
import org.junit.jupiter.api.Test;
import java.nio.charset.StandardCharsets;
import java.nio.file.Path;
import static org.junit.jupiter.api.Assertions.*;
class FrontendSourceDocumentTest {
@Test void locationCannotMixPathsOrExceedTheExactUtf8SourceRevision() {
final Path path = Path.of("/virtual/stdlib/sdk/main.pbs");
final String text = "// á 😀\r\nfn f() {}\r\n";
final var context = new FrontendEditorialContext(null, null, null);
final var source = new FrontendSourceDocument(path, text, "bundled", "sdk/main.pbs", "pbs", true, context);
final int bytes = text.getBytes(StandardCharsets.UTF_8).length;
assertDoesNotThrow(() -> new FrontendDefinitionLocation(path, 0, bytes, source));
assertThrows(IllegalArgumentException.class, () -> new FrontendDefinitionLocation(path, 0, bytes + 1, source));
assertThrows(IllegalArgumentException.class, () -> new FrontendDefinitionLocation(Path.of("/other.pbs"), 0, 1, source));
assertNotEquals(source.revision(), new FrontendSourceDocument(path, text.replace("\r\n", "\n"),
"bundled", "sdk/main.pbs", "pbs", true, context).revision());
}
}
@@ -20,6 +20,7 @@ public final class TcpLspServerHost {
private final LspServerConfiguration serverConfiguration;
private final LspProjectContext context;
private final CompilerLanguageServiceBridge compilerBridge;
private final p.studio.lsp.services.compiler.ExternalSourceRegistry externalSources = new p.studio.lsp.services.compiler.ExternalSourceRegistry();
private final ExecutorService clientExecutor;
private volatile boolean closed;
private volatile ServerSocket serverSocket;
@@ -72,8 +73,8 @@ public final class TcpLspServerHost {
}
private void handleClient(final Socket socket) {
final var server = new PrometeuLanguageServer(context, compilerBridge, externalSources);
try (socket) {
final var server = new PrometeuLanguageServer(context, compilerBridge);
final Launcher<LanguageClient> launcher = LSPLauncher.createServerLauncher(
server,
socket.getInputStream(),
@@ -82,7 +83,7 @@ public final class TcpLspServerHost {
launcher.startListening().get();
} catch (Exception exception) {
if (!closed && !socket.isClosed()) throw new IllegalStateException("LSP client failed", exception);
} finally { clients.remove(socket); }
} finally { server.disconnected(); clients.remove(socket); }
}
public synchronized java.util.Optional<p.studio.lsp.messages.LspServerEndpoint> endpoint() {
@@ -97,6 +98,7 @@ public final class TcpLspServerHost {
return;
}
closed = true;
externalSources.close();
if (serverSocket != null) {
serverSocket.close();
}
@@ -42,6 +42,79 @@ import java.util.*;
import java.util.function.Function;
public final class CompilerLanguageServiceBridge implements LanguageServiceBridge {
public boolean supportsExternalSources(final LspProjectContext context) {
return frontendProvider(context).languageService()
.map(p.studio.compiler.services.FrontendLanguageService::supportsExternalSources).orElse(false);
}
public Optional<p.studio.compiler.services.FrontendSourceDocument> currentSource(
final LspProjectContext context, final Path path) {
return editorialDocument(context, path.toUri().toString(), null).flatMap(document ->
frontendProvider(context).languageService().orElseThrow().sourceDocument(
new FrontendDocumentRequest(context.projectRoot(), path, document.text()),
document.editorialContext(), path));
}
public BaselineHover externalHover(final LspProjectContext context, final String uri,
final int line, final int character, final ExternalSourceRegistry registry, final String consumer) {
final var source = registry.source(consumer, uri);
return frontendProvider(context).languageService().orElseThrow().hover(
new FrontendDocumentRequest(context.projectRoot(), source.path(), source.text()), source.context(),
new DocumentPositionMapper(source.text()).byteOffsetOf(line, character))
.map(symbol -> new BaselineHover(formatHoverMarkdown(symbol)))
.orElseGet(() -> new BaselineHover("No symbol information."));
}
public BaselineSemanticTokens externalSemanticTokens(final LspProjectContext context, final String uri,
final ExternalSourceRegistry registry, final String consumer) {
final var source = registry.source(consumer, uri);
final var provider = frontendProvider(context);
final var tokens = provider.languageService().orElseThrow().semanticTokens(
new FrontendDocumentRequest(context.projectRoot(), source.path(), source.text()), source.context());
final var mapper = new DocumentPositionMapper(source.text());
final var mapped = new ArrayList<BaselineSemanticToken>();
for (var token : tokens) {
final var start = mapper.positionOf(token.startOffset());
final var end = mapper.positionOf(token.endOffset());
if (start.line() == end.line()) mapped.add(new BaselineSemanticToken(start.line(), start.character(),
Math.max(1, end.character() - start.character()), token.semanticKey()));
}
return new BaselineSemanticTokens(provider.specification().getSemanticPresentation().semanticKeys(), mapped);
}
public BaselineDefinition navigation(final LspProjectContext context, final String uri, final String text,
final int line, final int character, final ExternalSourceRegistry registry, final String consumer) {
final var service = frontendProvider(context).languageService().orElseThrow();
final EditorialDocument document;
final Path path;
if (uri.startsWith("prometeu-source:")) {
final var source = registry.source(consumer, uri);
document = new EditorialDocument(source.text(), source.context());
path = source.path();
} else {
final var result = editorialDocument(context, uri, text);
if (result.isEmpty()) return BaselineDefinition.empty();
document = result.orElseThrow();
path = normalizeDocumentPath(uri);
}
final var request = new FrontendDocumentRequest(context.projectRoot(), path, document.text());
final var locations = service.definition(request, document.editorialContext(),
new DocumentPositionMapper(document.text()).byteOffsetOf(line, character));
final var mapped = new ArrayList<BaselineDefinitionLocation>();
for (var location : locations) {
final var source = location.source();
if (source == null) {
mapped.addAll(mapDefinitionLocations(List.of(location), path, document.text()));
continue;
}
final String targetUri = source.readOnly() ? registry.register(consumer, source) : source.path().toUri().toString();
final var mapper = new DocumentPositionMapper(source.text());
final var start = mapper.positionOf(location.startOffset());
final var end = mapper.positionOf(location.endOffset());
mapped.add(new BaselineDefinitionLocation(targetUri, start.line(), start.character(), end.line(), end.character()));
}
return new BaselineDefinition(List.copyOf(mapped));
}
private final AssetSurfaceContextLoader assetSurfaceContextLoader = new AssetSurfaceContextLoader();
private final Function<String, FrontendProvider> frontendProviderResolver;
@@ -203,7 +276,7 @@ public final class CompilerLanguageServiceBridge implements LanguageServiceBridg
document.editorialContext(),
offset);
return new BaselineDefinition(mapDefinitionLocations(
locations,
locations.stream().filter(location -> location.source() == null || !location.source().readOnly()).toList(),
request.documentPath(),
document.text()));
})
@@ -1197,7 +1270,11 @@ public final class CompilerLanguageServiceBridge implements LanguageServiceBridg
}
final ArrayList<BaselineDefinitionLocation> mapped = new ArrayList<>();
for (final FrontendDefinitionLocation location : locations) {
final String targetText = textForDefinitionTarget(location.path(), currentDocumentPath, currentDocumentText);
// Legacy physical-only surfaces (references/symbols) do not inherit
// external-document coverage from the enriched definition DTO.
if (location.source() != null && !Files.isRegularFile(location.path())) continue;
final String targetText = location.source() != null ? location.source().text()
: textForDefinitionTarget(location.path(), currentDocumentPath, currentDocumentText);
if (targetText == null) {
continue;
}
@@ -0,0 +1,165 @@
package p.studio.lsp.services.compiler;
import p.studio.compiler.services.FrontendSourceDocument;
import java.nio.file.Path;
import java.util.*;
import java.util.function.Function;
/** Owned by one project host, never by the shared compiler bridge. */
public final class ExternalSourceRegistry implements AutoCloseable {
public record Descriptor(String uri, String revision, String origin, String name,
String languageId, boolean readOnly) {}
public record Content(Descriptor descriptor, String text) {}
private static final class Entry {
final Descriptor descriptor;
final Path path;
FrontendSourceDocument source;
final Set<String> readers = new HashSet<>();
Entry(Descriptor descriptor, FrontendSourceDocument source) {
this.descriptor = descriptor;
this.path = source.path();
this.source = source;
}
}
private static final class Consumer {
final boolean persistent;
final Set<String> grants = new HashSet<>();
final Set<String> retained = new HashSet<>();
final Set<String> validated = new HashSet<>();
boolean connected = true;
Consumer(boolean persistent) { this.persistent = persistent; }
}
private final String session = UUID.randomUUID().toString();
private final Map<String, Consumer> consumers = new HashMap<>();
private final Map<String, Entry> entries = new HashMap<>();
private boolean closed;
public synchronized String connect(String existing, boolean persistent) {
if (closed) throw unavailable();
if (existing != null && !existing.isBlank()) {
consumer(existing).connected = true;
return existing;
}
final String id = UUID.randomUUID().toString();
consumers.put(id, new Consumer(persistent));
return id;
}
public synchronized String register(String consumerId, FrontendSourceDocument source) {
final var consumer = activeConsumer(consumerId);
if (!source.readOnly()) throw new IllegalArgumentException("external source must be read-only");
Entry entry = entries.values().stream().filter(e -> sameSnapshot(e.source, source)).findFirst().orElse(null);
if (entry == null) {
// Only unreserved URI characters: VS Code and LSP URI serializers
// must not change the opaque identity by normalizing escaped slashes.
final String uri = "prometeu-source://" + session + "/" + UUID.randomUUID() + "/"
+ source.revision() + "/" + source.name().replaceAll("[^A-Za-z0-9._-]", "-");
entry = new Entry(new Descriptor(uri, source.revision(), source.origin(), source.name(), source.languageId(), true), source);
entries.put(uri, entry);
}
consumer.grants.add(entry.descriptor.uri());
consumer.validated.add(entry.descriptor.uri());
entry.readers.add(consumerId);
return entry.descriptor.uri();
}
private static boolean sameSnapshot(FrontendSourceDocument left, FrontendSourceDocument right) {
return left == right || left != null && left.path().equals(right.path())
&& left.context().projectSurface() != null
&& left.context().projectSurface() == right.context().projectSurface()
&& left.text().equals(right.text());
}
public synchronized FrontendSourceDocument source(String consumerId, String uri) {
final Entry entry = admitted(consumerId, uri);
if (entry.source == null || !entry.readers.contains(consumerId)
|| !activeConsumer(consumerId).validated.contains(uri)) throw unavailable();
return entry.source;
}
public synchronized Content read(String consumerId, String uri) {
final var source = source(consumerId, uri);
activeConsumer(consumerId).retained.add(uri);
return new Content(entries.get(uri).descriptor, source.text());
}
public synchronized Content retain(String consumerId, String uri) {
return read(consumerId, uri); // Navigation already grants a coalesced consumer use.
}
public synchronized void release(String consumerId, String uri) {
final Consumer consumer = activeConsumer(consumerId);
if (uri == null) {
for (String granted : List.copyOf(consumer.grants)) release(consumerId, granted);
consumers.remove(consumerId);
return;
}
if (!consumer.grants.remove(uri)) return;
consumer.retained.remove(uri);
consumer.validated.remove(uri);
final Entry entry = entries.get(uri);
if (entry != null) {
entry.readers.remove(consumerId);
if (entry.readers.isEmpty()) entry.source = null;
if (consumers.values().stream().noneMatch(c -> c.grants.contains(uri))) entries.remove(uri);
}
}
public synchronized void disconnect(String consumerId) {
final Consumer consumer = consumers.get(consumerId);
if (consumer == null) return;
consumer.connected = false;
consumer.validated.clear();
for (String uri : List.copyOf(consumer.grants)) {
final Entry entry = entries.get(uri);
if (consumer.persistent && consumer.retained.contains(uri)) continue;
if (!consumer.retained.contains(uri)) consumer.grants.remove(uri);
if (entry != null) {
entry.readers.remove(consumerId);
if (entry.readers.isEmpty()) entry.source = null;
if (consumers.values().stream().noneMatch(c -> c.grants.contains(uri))) entries.remove(uri);
}
}
}
public synchronized Content revalidate(String consumerId, String uri,
Function<Path, Optional<FrontendSourceDocument>> resolve) {
final Entry entry = admitted(consumerId, uri);
activeConsumer(consumerId).validated.remove(uri);
final var current = resolve.apply(entry.path).orElseThrow(ExternalSourceRegistry::unavailable);
if (!current.readOnly() || !current.revision().equals(entry.descriptor.revision())
|| !current.origin().equals(entry.descriptor.origin())) throw unavailable();
// Existing readers keep their original semantic snapshot. Otherwise bind a
// newly validated snapshot of identical content, never replacement text.
if (entry.source == null) entry.source = current;
entry.readers.add(consumerId);
activeConsumer(consumerId).validated.add(uri);
return read(consumerId, uri);
}
private Entry admitted(String consumerId, String uri) {
if (!activeConsumer(consumerId).grants.contains(uri)) throw unavailable();
final Entry entry = entries.get(uri);
if (entry == null) throw unavailable();
return entry;
}
private Consumer consumer(String id) {
final Consumer result = consumers.get(id);
if (closed || result == null) throw unavailable();
return result;
}
private Consumer activeConsumer(String id) {
final Consumer result = consumer(id);
if (!result.connected) throw unavailable();
return result;
}
public static IllegalStateException unavailable() {
return new IllegalStateException("External source unavailable; navigate again after reconnecting.");
}
@Override public synchronized void close() {
closed = true;
entries.clear();
consumers.clear();
}
}
@@ -0,0 +1,14 @@
package p.studio.lsp.services.protocol;
import org.eclipse.lsp4j.jsonrpc.services.JsonRequest;
import org.eclipse.lsp4j.services.LanguageServer;
import p.studio.lsp.services.compiler.ExternalSourceRegistry.Content;
import java.util.concurrent.CompletableFuture;
public interface ExternalSourceLanguageServer extends LanguageServer {
record Request(String consumerId, String uri) {}
@JsonRequest("prometeu/readExternalSource") CompletableFuture<Content> readExternalSource(Request request);
@JsonRequest("prometeu/retainExternalSource") CompletableFuture<Content> retainExternalSource(Request request);
@JsonRequest("prometeu/releaseExternalSource") CompletableFuture<Void> releaseExternalSource(Request request);
@JsonRequest("prometeu/revalidateExternalSource") CompletableFuture<Content> revalidateExternalSource(Request request);
}
@@ -13,7 +13,9 @@ import p.studio.lsp.services.protocol.mapping.ProtocolMessageMapper;
import java.util.Objects;
import java.util.concurrent.CompletableFuture;
public final class PrometeuLanguageServer implements LanguageServer, LanguageClientAware {
public final class PrometeuLanguageServer implements ExternalSourceLanguageServer, LanguageClientAware {
private p.studio.lsp.services.compiler.ExternalSourceRegistry externalSources;
private String consumerId;
private final LspProjectContext project;
private final LanguageServiceBridge languageServiceBridge;
private final ProtocolMessageMapper protocolMessageMapper;
@@ -32,6 +34,47 @@ public final class PrometeuLanguageServer implements LanguageServer, LanguageCli
null);
}
public PrometeuLanguageServer(final LspProjectContext project,
final CompilerLanguageServiceBridge bridge,
final p.studio.lsp.services.compiler.ExternalSourceRegistry registry) {
this(project, bridge);
this.externalSources = registry;
}
public void disconnected() {
if (externalSources != null && consumerId != null) externalSources.disconnect(consumerId);
}
private void requireConsumer(Request request) {
if (consumerId == null || request == null || !consumerId.equals(request.consumerId()))
throw p.studio.lsp.services.compiler.ExternalSourceRegistry.unavailable();
}
private <T> CompletableFuture<T> external(Request request, java.util.function.Supplier<T> operation) {
try {
requireConsumer(request);
return CompletableFuture.completedFuture(operation.get());
} catch (RuntimeException failure) {
return CompletableFuture.failedFuture(new org.eclipse.lsp4j.jsonrpc.ResponseErrorException(
new org.eclipse.lsp4j.jsonrpc.messages.ResponseError(
org.eclipse.lsp4j.jsonrpc.messages.ResponseErrorCode.InvalidRequest,
failure.getMessage(), null)));
}
}
@Override public CompletableFuture<p.studio.lsp.services.compiler.ExternalSourceRegistry.Content> readExternalSource(Request request) {
return external(request, () -> externalSources.read(consumerId, request.uri()));
}
@Override public CompletableFuture<p.studio.lsp.services.compiler.ExternalSourceRegistry.Content> retainExternalSource(Request request) {
return external(request, () -> externalSources.retain(consumerId, request.uri()));
}
@Override public CompletableFuture<Void> releaseExternalSource(Request request) {
return external(request, () -> { externalSources.release(consumerId, request.uri()); return null; });
}
@Override public CompletableFuture<p.studio.lsp.services.compiler.ExternalSourceRegistry.Content> revalidateExternalSource(Request request) {
return external(request, () -> externalSources.revalidate(consumerId, request.uri(),
path -> ((CompilerLanguageServiceBridge) languageServiceBridge).currentSource(project, path)));
}
PrometeuLanguageServer(
final LspProjectContext project,
final LanguageServiceBridge languageServiceBridge,
@@ -69,8 +112,25 @@ public final class PrometeuLanguageServer implements LanguageServer, LanguageCli
}
return CompletableFuture.failedFuture(runtimeException);
}
return CompletableFuture.completedFuture(protocolMessageMapper
.mapInitializeResult(languageServiceBridge.describeServer(project)));
final var result = protocolMessageMapper.mapInitializeResult(languageServiceBridge.describeServer(project));
final var gson = new com.google.gson.Gson();
final var capability = gson.toJsonTree(params.getCapabilities() == null ? null : params.getCapabilities().getExperimental());
if (externalSources != null && languageServiceBridge instanceof CompilerLanguageServiceBridge bridge
&& bridge.supportsExternalSources(project) && capability.isJsonObject()
&& capability.getAsJsonObject().has("prometeuExternalSources")
&& capability.getAsJsonObject().get("prometeuExternalSources").getAsBoolean()) {
final var rawOptions = gson.toJsonTree(params.getInitializationOptions());
final var options = rawOptions.isJsonObject() ? rawOptions.getAsJsonObject() : new com.google.gson.JsonObject();
final String previous = options.has("prometeuExternalConsumerId") ? options.get("prometeuExternalConsumerId").getAsString() : null;
final boolean persistent = options.has("prometeuExternalPersistentConsumer") && options.get("prometeuExternalPersistentConsumer").getAsBoolean();
consumerId = externalSources.connect(previous, persistent);
final var experimental = gson.toJsonTree(result.getCapabilities().getExperimental()).getAsJsonObject();
experimental.addProperty("prometeuExternalSources", true);
experimental.addProperty("prometeuExternalConsumerId", consumerId);
result.getCapabilities().setExperimental(experimental);
((PrometeuTextDocumentService) textDocumentService).externalSources(externalSources, consumerId);
}
return CompletableFuture.completedFuture(result);
}
@Override
@@ -29,6 +29,43 @@ public final class PrometeuTextDocumentService implements TextDocumentService {
private final ProtocolMessageMapper protocolMessageMapper;
private final ConcurrentMap<String, String> documentTextByUri = new ConcurrentHashMap<>();
private volatile LanguageClient client;
private p.studio.lsp.services.compiler.ExternalSourceRegistry externalSources;
private String externalConsumer;
void externalSources(p.studio.lsp.services.compiler.ExternalSourceRegistry registry, String consumer) {
externalSources = registry;
externalConsumer = consumer;
}
private static boolean external(String uri) { return uri.startsWith("prometeu-source:"); }
private p.studio.lsp.services.compiler.CompilerLanguageServiceBridge compiler() {
return (p.studio.lsp.services.compiler.CompilerLanguageServiceBridge) languageServiceBridge;
}
private static ResponseErrorException refused(String message) {
return new ResponseErrorException(new ResponseError(ResponseErrorCode.InvalidRequest, message, null));
}
private void requireExternal(String uri) {
if (externalSources == null || externalConsumer == null) throw refused("External source support was not negotiated.");
try { externalSources.source(externalConsumer, uri); }
catch (IllegalStateException unavailable) { throw refused(unavailable.getMessage()); }
}
private void requireEditable(String uri) {
if (external(uri)) throw readOnly();
if (languageServiceBridge instanceof p.studio.lsp.services.compiler.CompilerLanguageServiceBridge bridge) {
final var source = bridge.currentSource(project, java.nio.file.Path.of(java.net.URI.create(uri)));
if (source.isEmpty() || source.orElseThrow().readOnly()) throw readOnly();
}
}
private static ResponseErrorException readOnly() {
return new ResponseErrorException(new ResponseError(-32010, "Source is not an editable project document.", null));
}
private void validateEdit(WorkspaceEdit edit) {
if (edit == null) return;
if (edit.getChanges() != null) edit.getChanges().keySet().forEach(this::requireEditable);
if (edit.getDocumentChanges() != null) for (var change : edit.getDocumentChanges()) {
if (!change.isLeft()) throw refused("File operations are not supported.");
requireEditable(change.getLeft().getTextDocument().getUri());
}
}
public PrometeuTextDocumentService(
final LspProjectContext project,
@@ -46,7 +83,12 @@ public final class PrometeuTextDocumentService implements TextDocumentService {
@Override
public void didOpen(final DidOpenTextDocumentParams params) {
final String uri = params.getTextDocument().getUri();
final String text = params.getTextDocument().getText();
if (external(uri)) { requireExternal(uri); externalSources.retain(externalConsumer, uri); return; }
String text = params.getTextDocument().getText();
if (languageServiceBridge instanceof p.studio.lsp.services.compiler.CompilerLanguageServiceBridge bridge) {
final var known = bridge.currentSource(project, java.nio.file.Path.of(java.net.URI.create(uri)));
if (known.isPresent() && known.orElseThrow().readOnly()) text = known.orElseThrow().text();
}
documentTextByUri.put(uri, text);
publishAnalysis(uri, text);
}
@@ -54,6 +96,7 @@ public final class PrometeuTextDocumentService implements TextDocumentService {
@Override
public void didChange(final DidChangeTextDocumentParams params) {
final String uri = params.getTextDocument().getUri();
requireEditable(uri);
final String text = params.getContentChanges().isEmpty()
? ""
: params.getContentChanges().getFirst().getText();
@@ -63,6 +106,9 @@ public final class PrometeuTextDocumentService implements TextDocumentService {
@Override
public void didClose(final DidCloseTextDocumentParams params) {
if (external(params.getTextDocument().getUri())) {
if (externalSources != null) externalSources.release(externalConsumer, params.getTextDocument().getUri()); return;
}
documentTextByUri.remove(params.getTextDocument().getUri());
final LanguageClient currentClient = client;
if (currentClient == null) {
@@ -73,6 +119,7 @@ public final class PrometeuTextDocumentService implements TextDocumentService {
@Override
public void didSave(final DidSaveTextDocumentParams params) {
requireEditable(params.getTextDocument().getUri());
final LanguageClient currentClient = client;
if (currentClient == null) {
return;
@@ -84,6 +131,7 @@ public final class PrometeuTextDocumentService implements TextDocumentService {
@Override
public CompletableFuture<Either<List<CompletionItem>, CompletionList>> completion(final CompletionParams params) {
final String uri = params.getTextDocument().getUri();
if (external(uri)) return CompletableFuture.completedFuture(Either.forLeft(List.of()));
final String text = documentTextByUri.get(uri);
return CompletableFuture.completedFuture(Either.forRight(protocolMessageMapper.mapCompletion(
languageServiceBridge.completion(
@@ -97,6 +145,9 @@ public final class PrometeuTextDocumentService implements TextDocumentService {
@Override
public CompletableFuture<Hover> hover(final HoverParams params) {
final String uri = params.getTextDocument().getUri();
if (external(uri)) requireExternal(uri);
if (external(uri)) return CompletableFuture.completedFuture(protocolMessageMapper.mapHover(
compiler().externalHover(project, uri, params.getPosition().getLine(), params.getPosition().getCharacter(), externalSources, externalConsumer)));
final String text = documentTextByUri.get(uri);
return CompletableFuture.completedFuture(protocolMessageMapper.mapHover(languageServiceBridge.hover(
project,
@@ -110,6 +161,9 @@ public final class PrometeuTextDocumentService implements TextDocumentService {
public CompletableFuture<Either<List<? extends Location>, List<? extends LocationLink>>> definition(
final DefinitionParams params) {
final String uri = params.getTextDocument().getUri();
if (external(uri)) requireExternal(uri);
if (externalSources != null) return CompletableFuture.completedFuture(Either.forLeft(protocolMessageMapper.mapDefinition(
compiler().navigation(project, uri, documentTextByUri.get(uri), params.getPosition().getLine(), params.getPosition().getCharacter(), externalSources, externalConsumer))));
final String text = documentTextByUri.get(uri);
final List<Location> locations = protocolMessageMapper.mapDefinition(languageServiceBridge.definition(
project,
@@ -123,6 +177,7 @@ public final class PrometeuTextDocumentService implements TextDocumentService {
@Override
public CompletableFuture<List<? extends Location>> references(final ReferenceParams params) {
final String uri = params.getTextDocument().getUri();
if (external(uri)) return CompletableFuture.completedFuture(List.of());
final String text = documentTextByUri.get(uri);
final boolean includeDeclaration = params.getContext() != null
&& Boolean.TRUE.equals(params.getContext().isIncludeDeclaration());
@@ -140,6 +195,7 @@ public final class PrometeuTextDocumentService implements TextDocumentService {
public CompletableFuture<Either3<Range, PrepareRenameResult, PrepareRenameDefaultBehavior>> prepareRename(
final PrepareRenameParams params) {
final String uri = params.getTextDocument().getUri();
requireEditable(uri);
final String text = documentTextByUri.get(uri);
return CompletableFuture.completedFuture(protocolMessageMapper.mapPrepareRename(
languageServiceBridge.prepareRename(
@@ -153,6 +209,7 @@ public final class PrometeuTextDocumentService implements TextDocumentService {
@Override
public CompletableFuture<WorkspaceEdit> rename(final RenameParams params) {
final String uri = params.getTextDocument().getUri();
requireEditable(uri);
final String text = documentTextByUri.get(uri);
final var result = languageServiceBridge.rename(
project,
@@ -168,12 +225,15 @@ public final class PrometeuTextDocumentService implements TextDocumentService {
null);
return CompletableFuture.failedFuture(new ResponseErrorException(error));
}
return CompletableFuture.completedFuture(protocolMessageMapper.mapRename(result));
final var edit = protocolMessageMapper.mapRename(result);
validateEdit(edit);
return CompletableFuture.completedFuture(edit);
}
@Override
public CompletableFuture<List<CallHierarchyItem>> prepareCallHierarchy(final CallHierarchyPrepareParams params) {
final String uri = params.getTextDocument().getUri();
if (external(uri)) return CompletableFuture.completedFuture(List.of());
final String text = documentTextByUri.get(uri);
return CompletableFuture.completedFuture(protocolMessageMapper.mapCallHierarchyItems(
languageServiceBridge.prepareCallHierarchy(
@@ -199,7 +259,7 @@ public final class PrometeuTextDocumentService implements TextDocumentService {
}
private List<BaselineCallSite> directedCallSites(final CallHierarchyItem item, final boolean incoming) {
if (item == null || item.getUri() == null || item.getSelectionRange() == null) {
if (item == null || item.getUri() == null || external(item.getUri()) || item.getSelectionRange() == null) {
return List.of();
}
final String text = documentTextByUri.get(item.getUri());
@@ -219,6 +279,7 @@ public final class PrometeuTextDocumentService implements TextDocumentService {
@Override
public CompletableFuture<List<? extends TextEdit>> formatting(final DocumentFormattingParams params) {
final String uri = params.getTextDocument().getUri();
requireEditable(uri);
final String text = documentTextByUri.get(uri);
return CompletableFuture.completedFuture(protocolMessageMapper.mapFormatting(
languageServiceBridge.formatDocument(project, uri, text)));
@@ -227,6 +288,7 @@ public final class PrometeuTextDocumentService implements TextDocumentService {
@Override
public CompletableFuture<List<FoldingRange>> foldingRange(final FoldingRangeRequestParams params) {
final String uri = params.getTextDocument().getUri();
if (external(uri)) return CompletableFuture.completedFuture(List.of());
final String text = documentTextByUri.get(uri);
return CompletableFuture.completedFuture(protocolMessageMapper.mapFoldingRanges(
languageServiceBridge.foldingRanges(project, uri, text)));
@@ -235,6 +297,7 @@ public final class PrometeuTextDocumentService implements TextDocumentService {
@Override
public CompletableFuture<List<SelectionRange>> selectionRange(final SelectionRangeParams params) {
final String uri = params.getTextDocument().getUri();
if (external(uri)) return CompletableFuture.completedFuture(List.of());
final String text = documentTextByUri.get(uri);
final ArrayList<DocumentPosition> positions = new ArrayList<>();
if (params.getPositions() != null) {
@@ -252,6 +315,7 @@ public final class PrometeuTextDocumentService implements TextDocumentService {
@Override
public CompletableFuture<List<DocumentLink>> documentLink(final DocumentLinkParams params) {
final String uri = params.getTextDocument().getUri();
if (external(uri)) return CompletableFuture.completedFuture(List.of());
final String text = documentTextByUri.get(uri);
return CompletableFuture.completedFuture(protocolMessageMapper.mapDocumentLinks(
languageServiceBridge.documentLinks(project, uri, text)));
@@ -259,6 +323,7 @@ public final class PrometeuTextDocumentService implements TextDocumentService {
@Override
public CompletableFuture<List<Either<Command, CodeAction>>> codeAction(final CodeActionParams params) {
requireEditable(params.getTextDocument().getUri());
if (!quickFixRequested(params.getContext())) {
return CompletableFuture.completedFuture(List.of());
}
@@ -268,15 +333,21 @@ public final class PrometeuTextDocumentService implements TextDocumentService {
|| params.getContext().getDiagnostics() == null
? List.of()
: params.getContext().getDiagnostics();
return CompletableFuture.completedFuture(protocolMessageMapper.mapCodeActions(
final var actions = protocolMessageMapper.mapCodeActions(
languageServiceBridge.codeActions(project, uri, text, baselineDiagnostics(contextDiagnostics)),
contextDiagnostics));
contextDiagnostics);
for (var action : actions) {
if (action.isLeft()) throw refused("Executable actions are not supported.");
validateEdit(action.getRight().getEdit());
}
return CompletableFuture.completedFuture(actions);
}
@Override
public CompletableFuture<List<Either<SymbolInformation, DocumentSymbol>>> documentSymbol(
final DocumentSymbolParams params) {
final String uri = params.getTextDocument().getUri();
if (external(uri)) return CompletableFuture.completedFuture(List.of());
final String text = documentTextByUri.get(uri);
final List<Either<SymbolInformation, DocumentSymbol>> symbols = new ArrayList<>();
for (final DocumentSymbol symbol : protocolMessageMapper.mapDocumentSymbols(
@@ -289,6 +360,7 @@ public final class PrometeuTextDocumentService implements TextDocumentService {
@Override
public CompletableFuture<SignatureHelp> signatureHelp(final SignatureHelpParams params) {
final String uri = params.getTextDocument().getUri();
if (external(uri)) return CompletableFuture.completedFuture(null);
final String text = documentTextByUri.get(uri);
return CompletableFuture.completedFuture(protocolMessageMapper.mapSignatureHelp(
languageServiceBridge.signatureHelp(
@@ -302,6 +374,9 @@ public final class PrometeuTextDocumentService implements TextDocumentService {
@Override
public CompletableFuture<SemanticTokens> semanticTokensFull(final SemanticTokensParams params) {
final String uri = params.getTextDocument().getUri();
if (external(uri)) requireExternal(uri);
if (external(uri)) return CompletableFuture.completedFuture(protocolMessageMapper.mapSemanticTokens(
compiler().externalSemanticTokens(project, uri, externalSources, externalConsumer)));
final String text = documentTextByUri.getOrDefault(uri, "");
return CompletableFuture.completedFuture(protocolMessageMapper.mapSemanticTokens(
languageServiceBridge.semanticTokens(project, uri, text)));
@@ -0,0 +1,63 @@
package p.studio.lsp.services;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.condition.EnabledIfEnvironmentVariable;
import org.junit.jupiter.api.io.TempDir;
import p.studio.lsp.messages.*;
import p.studio.lsp.services.compiler.CompilerLanguageServiceBridge;
import java.nio.file.*;
import java.util.concurrent.TimeUnit;
import static org.junit.jupiter.api.Assertions.*;
/** Opt-in actual VS Code test against the development classpath (never installDist). */
@EnabledIfEnvironmentVariable(named = "PROMETEU_VSCODE_DEV_TEST", matches = "1")
class ExternalSourceDevelopmentHostTest {
@TempDir Path temporary;
@Test void realEditorReadsBundledSourceWithoutMaterializingIt() throws Exception {
Path repository = Path.of("").toAbsolutePath();
while (!Files.isDirectory(repository.resolve("tools/vscode-extension"))) repository = repository.getParent();
final Path root = Files.createDirectories(temporary.resolve("project")).toRealPath();
Files.createDirectories(root.resolve("src"));
Files.createDirectories(root.resolve(".vscode"));
Files.writeString(root.resolve("prometeu.json"), """
{"name":"app","version":"1.0.0","language":"pbs","stdlib":"1","target":"Game","dependencies":[{"path":".workspace/repos/lib"}]}
""");
Files.writeString(root.resolve("src/main.pbs"), "import { Gfx } from @sdk:gfx;\nimport { answer } from @lib:shared;\nfn main() -> void { Gfx.clear(0); answer(); }\n");
Files.writeString(root.resolve("src/mod.barrel"), "pub fn main() -> void;\n");
final Path library = Files.createDirectories(root.resolve(".workspace/repos/lib/src")).getParent();
Files.createDirectories(library.resolve("src/shared"));
Files.writeString(library.resolve("prometeu.json"), """
{"name":"lib","version":"1.0.0","language":"pbs","stdlib":"1","target":"Game","dependencies":[]}
""");
Files.writeString(root.resolve(".workspace/repos/registry.json"), "{}");
final String libraryText = "// á 😀\r\nfn answer() -> int { return helper(); }\r\nfn helper() -> int { return 42; }\r\n";
Files.writeString(library.resolve("src/shared/main.pbs"), libraryText);
Files.writeString(library.resolve("src/shared/mod.barrel"), "pub fn answer() -> int;\n");
final var host = new TcpLspServerHost(LspServerConfiguration.builder()
.endpoint(new LspServerEndpoint("127.0.0.1", 0)).build(), new LspProjectContext("dev", "pbs", root),
new CompilerLanguageServiceBridge());
host.start();
try {
Files.writeString(root.resolve(".vscode/settings.json"), "{\"prometeu.studio.port\":" + host.endpoint().orElseThrow().port() + "}");
final String executable = System.getenv().getOrDefault("PROMETEU_VSCODE_EXECUTABLE", "/Applications/Visual Studio Code.app/Contents/MacOS/Code");
final Path extension = repository.resolve("tools/vscode-extension");
final var builder = new ProcessBuilder(executable, "--no-sandbox", "--disable-extensions", "--disable-workspace-trust",
"--skip-welcome", "--skip-release-notes", "--user-data-dir=" + temporary.resolve("vscode-user"),
"--extensions-dir=" + temporary.resolve("vscode-extensions"), "--extensionDevelopmentPath=" + extension,
"--extensionTestsPath=" + extension.resolve("out/test/externalSources.js"), root.toString());
builder.environment().remove("ELECTRON_RUN_AS_NODE");
final Path log = Path.of("/tmp/pln141-vscode-development.log");
final var process = builder.redirectErrorStream(true).redirectOutput(log.toFile()).start();
try {
assertTrue(process.waitFor(90, TimeUnit.SECONDS), "VS Code test timed out: " + log);
assertEquals(0, process.exitValue(), () -> read(log));
} finally { process.destroyForcibly(); }
assertEquals(libraryText, Files.readString(library.resolve("src/shared/main.pbs")));
try (var directories = Files.list(root.resolve(".workspace"))) {
assertEquals(java.util.List.of("repos"), directories.map(p -> p.getFileName().toString()).toList(), "No editorial SDK cache");
}
} finally { host.close(); }
}
private static String read(Path path) { try { return Files.readString(path); } catch (Exception e) { return e.toString(); } }
}
@@ -28,6 +28,82 @@ import static org.junit.jupiter.api.Assertions.assertTrue;
class CompilerLanguageServiceBridgeTest {
@Test
void retainedDependencyUsesOriginalSemanticSnapshotAfterDiskChanges(@TempDir Path temporary) throws Exception {
final Path root = temporary.toRealPath();
final Path library = Files.createDirectories(root.resolve(".workspace/repos/lib/src")).getParent();
Files.createDirectories(root.resolve("src"));
Files.writeString(root.resolve("prometeu.json"), """
{"name":"app","version":"1.0.0","language":"pbs","stdlib":"1","target":"Game",
"dependencies":[{"path":".workspace/repos/lib"}]}
""");
Files.createDirectories(library.resolve("src/shared"));
Files.writeString(library.resolve("prometeu.json"), """
{"name":"lib","version":"1.0.0","language":"pbs","stdlib":"1","target":"Game","dependencies":[]}
""");
Files.writeString(root.resolve(".workspace/repos/registry.json"), "{}");
final String projectText = "import { answer } from @lib:shared;\nfn main() -> int { return answer(); }\n";
final Path projectFile = Files.writeString(root.resolve("src/main.pbs"), projectText);
final String old = "// á 😀\r\nfn answer() -> int { return original(); }\r\nfn original() -> int { return 1; }\r\n";
final Path libraryFile = Files.writeString(library.resolve("src/shared/main.pbs"), old);
Files.writeString(library.resolve("src/shared/mod.barrel"), "pub fn answer() -> int;\n");
final var bridge = new CompilerLanguageServiceBridge();
final var context = new LspProjectContext("snapshot", "pbs", root);
try (var registry = new ExternalSourceRegistry()) {
final String consumer = registry.connect(null, false);
final String uri = bridge.navigation(context, projectFile.toUri().toString(), projectText, 1, 26, registry, consumer)
.locations().getFirst().uri();
final var retained = registry.source(consumer, uri);
assertEquals(old, registry.read(consumer, uri).text());
Files.writeString(libraryFile, "// different revision\n" + old.replace("original", "replacement"));
final var target = bridge.navigation(context, uri, null, 1, 28, registry, consumer).locations().getFirst();
assertEquals(2, target.startLine());
assertEquals(old, registry.read(consumer, target.uri()).text());
assertTrue(bridge.externalHover(context, uri, 1, 28, registry, consumer).markdown().contains("original"));
assertTrue(retained.context() == registry.source(consumer, uri).context());
registry.disconnect(consumer);
registry.connect(consumer, false);
org.junit.jupiter.api.Assertions.assertThrows(IllegalStateException.class,
() -> registry.revalidate(consumer, uri, path -> bridge.currentSource(context, path)));
org.junit.jupiter.api.Assertions.assertThrows(IllegalStateException.class, () -> registry.source(consumer, uri));
}
}
@Test
void externalNavigationRetainsBundledTextAndContinuesIntoHostDeclarations() {
final Path root = findRepoRoot(Path.of("").toAbsolutePath()).resolve("test-projects/main");
final var context = new LspProjectContext("main", "pbs", root);
final var bridge = new CompilerLanguageServiceBridge();
final String source = "import { Gfx } from @sdk:gfx;\nfn frame() -> void { Gfx.clear(0); }\n";
final var position = new DocumentPositionMapper(source).positionOf(source.indexOf("clear"));
try (var registry = new ExternalSourceRegistry()) {
final String consumer = registry.connect(null, false);
final var locations = bridge.navigation(context, root.resolve("src/main.pbs").toUri().toString(), source,
position.line(), position.character(), registry, consumer).locations();
assertEquals(1, locations.size());
final String uri = locations.getFirst().uri();
assertTrue(uri.startsWith("prometeu-source:"));
final var content = registry.read(consumer, uri);
assertEquals("bundled", content.descriptor().origin());
assertTrue(content.descriptor().readOnly());
assertTrue(content.text().contains("LowGfx.clear"));
final int call = content.text().indexOf("LowGfx.clear") + "LowGfx.".length();
final var hostUse = new DocumentPositionMapper(content.text()).positionOf(call);
assertTrue(bridge.externalHover(context, uri, hostUse.line(), hostUse.character(), registry, consumer)
.markdown().contains("clear"));
final var host = bridge.navigation(context, uri, null, hostUse.line(), hostUse.character(), registry, consumer)
.locations().getFirst();
final var hostContent = registry.read(consumer, host.uri());
final String[] lines = hostContent.text().split("\n");
assertTrue(lines[host.startLine()].contains("clear"), lines[host.startLine()]);
assertFalse(bridge.externalSemanticTokens(context, uri, registry, consumer).tokens().isEmpty());
final var colorUse = new DocumentPositionMapper(content.text()).positionOf(content.text().indexOf("color: Color") + 7);
final String coreUri = bridge.navigation(context, uri, null, colorUse.line(), colorUse.character(), registry, consumer)
.locations().getFirst().uri();
assertTrue(registry.read(consumer, coreUri).descriptor().name().replace('\\', '/').contains("core/color"));
}
}
@Test
void editorialEndpointsReturnNeutralResponsesWhenLanguageServiceIsAbsent() {
final var provider = new CompileOnlyFrontendProvider();
@@ -0,0 +1,77 @@
package p.studio.lsp.services.compiler;
import org.junit.jupiter.api.Test;
import p.studio.compiler.services.FrontendEditorialContext;
import p.studio.compiler.services.FrontendSourceDocument;
import java.nio.file.Path;
import java.util.Optional;
import static org.junit.jupiter.api.Assertions.*;
class ExternalSourceRegistryTest {
private FrontendSourceDocument source(String text) {
return new FrontendSourceDocument(Path.of("/virtual/stdlib/sdk/main.pbs"), text, "bundled", "sdk/main.pbs", "pbs", true,
new FrontendEditorialContext(new Object(), null, new Object()));
}
@Test void readersShareRevisionWithoutSharingReleaseAndUnknownIdentitiesAreRejected() {
try (var registry = new ExternalSourceRegistry(); var otherProject = new ExternalSourceRegistry()) {
final var source = source("// á 😀\r\nfn f() {}\r\n");
final String one = registry.connect(null, false), two = registry.connect(null, false);
final String uri = registry.register(one, source);
assertEquals(uri, registry.register(two, source));
registry.retain(one, uri);
registry.release(one, uri);
assertThrows(IllegalStateException.class, () -> registry.read(one, uri));
assertSame(source.context(), registry.source(two, uri).context());
assertEquals(source.text(), registry.read(two, uri).text());
assertThrows(IllegalStateException.class, () -> otherProject.read(otherProject.connect(null, false), uri));
assertThrows(IllegalStateException.class, () -> registry.read(two, "file:///etc/passwd"));
registry.release(two, uri);
assertThrows(IllegalStateException.class, () -> registry.read(two, uri));
}
}
@Test void disconnectedDocumentsRequireRevalidationAndChangedRevisionCannotReplaceText() {
try (var registry = new ExternalSourceRegistry()) {
final String consumer = registry.connect(null, false);
final var old = source("old");
final String uri = registry.register(consumer, old);
registry.retain(consumer, uri);
registry.disconnect(consumer);
assertThrows(IllegalStateException.class, () -> registry.read(consumer, uri));
registry.connect(consumer, false);
assertThrows(IllegalStateException.class, () -> registry.read(consumer, uri));
assertThrows(IllegalStateException.class, () -> registry.revalidate(consumer, uri, p -> Optional.of(source("new"))));
assertEquals("old", registry.revalidate(consumer, uri, p -> Optional.of(source("old"))).text());
}
}
@Test void taskConsumerOutlivesTransportButNotTaskOrProject() {
final var registry = new ExternalSourceRegistry();
final String consumer = registry.connect(null, true);
final var snapshot = source("source");
final String uri = registry.register(consumer, snapshot);
registry.retain(consumer, uri);
registry.disconnect(consumer);
registry.connect(consumer, true);
assertThrows(IllegalStateException.class, () -> registry.source(consumer, uri));
registry.revalidate(consumer, uri, p -> Optional.of(source("source")));
assertSame(snapshot, registry.source(consumer, uri));
registry.release(consumer, null);
assertThrows(IllegalStateException.class, () -> registry.connect(consumer, true));
final String next = registry.connect(null, true);
registry.register(next, snapshot);
registry.close();
assertThrows(IllegalStateException.class, () -> registry.read(next, uri));
}
@Test void cancelledOrUnopenedNavigationDoesNotRetainASnapshot() {
try (var registry = new ExternalSourceRegistry()) {
final String consumer = registry.connect(null, true);
final String uri = registry.register(consumer, source("unused"));
registry.disconnect(consumer);
registry.connect(consumer, true);
assertThrows(IllegalStateException.class, () -> registry.revalidate(consumer, uri, p -> Optional.of(source("unused"))));
}
}
}
@@ -0,0 +1,33 @@
package p.studio.lsp.services.protocol;
import org.eclipse.lsp4j.*;
import org.eclipse.lsp4j.jsonrpc.ResponseErrorException;
import org.junit.jupiter.api.Test;
import p.studio.lsp.messages.LspProjectContext;
import p.studio.lsp.services.compiler.CompilerLanguageServiceBridge;
import p.studio.lsp.services.protocol.mapping.Lsp4jProtocolMessageMapper;
import java.nio.file.Path;
import java.util.List;
import static org.junit.jupiter.api.Assertions.*;
class ExternalWriteBoundaryTest {
@Test void externalEditsAreRefusedEvenWithoutARegisteredConsumer() {
final var service = new PrometeuTextDocumentService(new LspProjectContext("test", "pbs", Path.of(".")),
new CompilerLanguageServiceBridge(), new Lsp4jProtocolMessageMapper());
final String uri = "prometeu-source://session/revision/main.pbs";
final var id = new TextDocumentIdentifier(uri);
final var versioned = new VersionedTextDocumentIdentifier(uri, 2);
final var position = new Position(0, 0);
final List<org.junit.jupiter.api.function.Executable> attempts = List.of(
() -> service.didChange(new DidChangeTextDocumentParams(versioned, List.of(new TextDocumentContentChangeEvent("changed")))),
() -> service.didSave(new DidSaveTextDocumentParams(id)),
() -> service.formatting(new DocumentFormattingParams(id, new FormattingOptions(4, true))),
() -> service.rename(new RenameParams(id, position, "changed")),
() -> service.prepareRename(new PrepareRenameParams(id, position)),
() -> service.codeAction(new CodeActionParams(id, new Range(position, position), new CodeActionContext(List.of()))));
for (var attempt : attempts) assertEquals(-32010, assertThrows(ResponseErrorException.class, attempt).getResponseError().getCode());
assertTrue(service.references(new ReferenceParams(id, position, new ReferenceContext(true))).join().isEmpty());
assertTrue(service.documentLink(new DocumentLinkParams(id)).join().isEmpty());
assertTrue(service.prepareCallHierarchy(new CallHierarchyPrepareParams(id, position)).join().isEmpty());
}
}
+4 -2
View File
@@ -2,7 +2,9 @@
node_modules/
# Build output
out/
out/*
!out/extension.js
!out/externalSources.js
dist/
# VS Code package output
@@ -35,4 +37,4 @@ Thumbs.db
.vscode/*.log
# AI
debug
debug
+28
View File
@@ -153,3 +153,31 @@ The extension currently provides only the minimal LSP client bridge:
* receives LSP messages from the Java/LSP4J server
The Studio remains the source of trut
## Reading SDK/core and libraries
F12 opens bundled SDK/core and admitted dependency declarations as read-only
`prometeu-source:` documents. Hover, continued F12 and semantic highlighting work
inside them. Files in the editable project keep normal `file:` editing. A library
under `.workspace` is still external. SDK/core are never copied to an editorial
folder or cache. Formatting, rename and code actions cannot change external sources;
a proposal touching both project and dependency files is refused in full.
An open external document keeps its source revision. Disconnect leaves its text
visible, with queries unavailable until reconnection and revalidation. If the
source changed, navigate again from the project to obtain new text and positions.
The status bar indicates when an active external source is unavailable.
Compile with `npm run compile` before running the extension. Development acceptance
can launch an isolated real VS Code host against the test LSP classpath:
```sh
PROMETEU_VSCODE_DEV_TEST=1 ./gradlew :prometeu-lsp:prometeu-lsp-v1:test --tests '*ExternalSourceDevelopmentHostTest' --console=plain
```
Run that Gradle command from the repository root. Set `PROMETEU_VSCODE_EXECUTABLE`
when VS Code is not installed at the default macOS application path. The test covers
read-only typing, declaration ranges, hover/F12, highlighting, dependency sources
with Unicode/CRLF, reconnection and normal project editing. It does not validate a
packaged Studio distribution or installed JAR. Dependencies must already exist on
disk; dependency management remains a separate product task.
+29 -2
View File
@@ -37,6 +37,7 @@ exports.activate = activate;
exports.deactivate = deactivate;
const net = __importStar(require("node:net"));
const vscode = __importStar(require("vscode"));
const externalSources_1 = require("./externalSources");
const node_1 = require("vscode-languageclient/node");
let client;
let output;
@@ -44,12 +45,14 @@ let reconnectTimer;
let reconnectContext;
let stoppingClient = false;
let lastAppliedThemeSignature;
const externalSources = new externalSources_1.ExternalSources();
const RECONNECT_DELAY_MS = 2000;
const VSCODE_HOST_ID = "vscode";
function activate(context) {
reconnectContext = context;
output = vscode.window.createOutputChannel("Prometeu LSP");
output.appendLine("Prometeu VS Code extension activated.");
context.subscriptions.push(externalSources, vscode.workspace.registerTextDocumentContentProvider(externalSources_1.externalScheme, externalSources), vscode.window.onDidChangeActiveTextEditor(() => externalSources.refreshStatus()), vscode.workspace.onDidOpenTextDocument(document => void externalSources.opened(document)), vscode.workspace.onDidCloseTextDocument(document => void externalSources.closeDocument(document)));
const connectCommand = vscode.commands.registerCommand("prometeu.connectStudioLsp", async () => {
await startClient(context);
});
@@ -61,7 +64,12 @@ function activate(context) {
void startClient(context);
}
async function deactivate() {
await stopClient();
try {
await externalSources.shutdown();
}
finally {
await stopClient();
}
}
async function startClient(context) {
clearReconnectTimer();
@@ -113,7 +121,7 @@ async function startClient(context) {
{
scheme: "file",
language: "pbs"
}
}, { scheme: externalSources_1.externalScheme, language: "pbs" }
],
synchronize: {
fileEvents: [
@@ -121,10 +129,20 @@ async function startClient(context) {
]
},
initializationOptions: {
...externalSources.initializationOptions,
clientKind: "vscode",
prometeuClientVersion: context.extension.packageJSON.version
},
outputChannel: output,
middleware: {
prepareRename: (document, position, token, next) => document.uri.scheme === externalSources_1.externalScheme ? null : next(document, position, token),
provideRenameEdits: (document, position, name, token, next) => document.uri.scheme === externalSources_1.externalScheme ? null : next(document, position, name, token),
provideDocumentFormattingEdits: (document, options, token, next) => document.uri.scheme === externalSources_1.externalScheme ? [] : next(document, options, token),
provideCodeActions: (document, range, context, token, next) => document.uri.scheme === externalSources_1.externalScheme ? [] : next(document, range, context, token),
provideHover: (document, position, token, next) => externalSources.canQuery(document) ? next(document, position, token) : null,
provideDefinition: (document, position, token, next) => externalSources.canQuery(document) ? next(document, position, token) : null,
provideDocumentSemanticTokens: (document, token, next) => externalSources.canQuery(document) ? next(document, token) : null
},
errorHandler: {
error: (error) => {
output.appendLine(`Prometeu LSP transport error: ${error.message}`);
@@ -138,11 +156,18 @@ async function startClient(context) {
};
const nextClient = new node_1.LanguageClient("prometeuStudioLsp", "Prometeu Studio LSP", serverOptions, clientOptions);
client = nextClient;
nextClient.registerFeature({
fillClientCapabilities(capabilities) {
capabilities.experimental = { ...capabilities.experimental, prometeuExternalSources: true };
},
initialize() { }, clear() { }, getState() { return { kind: "static" }; }
});
nextClient.onDidChangeState((event) => {
output.appendLine(`Prometeu LSP state changed: ${node_1.State[event.oldState]} -> ${node_1.State[event.newState]}`);
if (event.newState !== node_1.State.Stopped) {
return;
}
externalSources.disconnected();
if (client === nextClient) {
client = undefined;
}
@@ -152,11 +177,13 @@ async function startClient(context) {
});
try {
await nextClient.start();
await externalSources.connected(nextClient);
await applyPrometeuVisualTheme(nextClient.initializeResult);
output.appendLine("Prometeu Studio LSP client started.");
vscode.window.showInformationMessage("Connected to Prometeu Studio LSP.");
}
catch (error) {
externalSources.resetSession();
if (client === nextClient) {
client = undefined;
}
@@ -0,0 +1,146 @@
"use strict";
var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
if (k2 === undefined) k2 = k;
var desc = Object.getOwnPropertyDescriptor(m, k);
if (!desc || ("get" in desc ? !m.__esModule : desc.writable || desc.configurable)) {
desc = { enumerable: true, get: function() { return m[k]; } };
}
Object.defineProperty(o, k2, desc);
}) : (function(o, m, k, k2) {
if (k2 === undefined) k2 = k;
o[k2] = m[k];
}));
var __setModuleDefault = (this && this.__setModuleDefault) || (Object.create ? (function(o, v) {
Object.defineProperty(o, "default", { enumerable: true, value: v });
}) : function(o, v) {
o["default"] = v;
});
var __importStar = (this && this.__importStar) || (function () {
var ownKeys = function(o) {
ownKeys = Object.getOwnPropertyNames || function (o) {
var ar = [];
for (var k in o) if (Object.prototype.hasOwnProperty.call(o, k)) ar[ar.length] = k;
return ar;
};
return ownKeys(o);
};
return function (mod) {
if (mod && mod.__esModule) return mod;
var result = {};
if (mod != null) for (var k = ownKeys(mod), i = 0; i < k.length; i++) if (k[i] !== "default") __createBinding(result, mod, k[i]);
__setModuleDefault(result, mod);
return result;
};
})();
Object.defineProperty(exports, "__esModule", { value: true });
exports.ExternalSources = exports.externalScheme = void 0;
const vscode = __importStar(require("vscode"));
exports.externalScheme = "prometeu-source";
/** Immutable virtual documents. No filesystem provider and no content-change events. */
class ExternalSources {
client;
consumerId;
documents = new Map();
available = new Set();
closed = new Set();
status = vscode.window.createStatusBarItem(vscode.StatusBarAlignment.Right, 10);
refreshStatus() {
const document = vscode.window.activeTextEditor?.document;
if (document?.uri.scheme === exports.externalScheme && !this.canQuery(document)) {
this.status.text = "Prometeu: external source unavailable";
this.status.tooltip = "Reconnect to Studio or navigate again to obtain the current source revision.";
this.status.show();
}
else
this.status.hide();
}
get initializationOptions() {
return this.consumerId ? { prometeuExternalConsumerId: this.consumerId } : {};
}
canQuery(document) {
const uri = document.uri;
if (uri.scheme !== exports.externalScheme)
return true;
const content = this.documents.get(uri.toString());
return !!this.client && this.available.has(uri.toString()) && !!content
&& document.getText().replace(/\r\n?/g, "\n") === content.text.replace(/\r\n?/g, "\n");
}
async connected(client) {
const experimental = client.initializeResult?.capabilities.experimental;
if (!experimental?.prometeuExternalSources)
return;
this.client = client;
this.consumerId = experimental.prometeuExternalConsumerId;
for (const uri of this.closed)
await this.release(uri);
this.closed.clear();
for (const [uri, old] of this.documents) {
try {
const current = await this.request("revalidateExternalSource", uri);
if (current.descriptor.revision === old.descriptor.revision && current.text === old.text)
this.available.add(uri);
}
catch { /* Keep displayed text; fresh navigation is required. */ }
}
this.refreshStatus();
}
disconnected() { this.client = undefined; this.available.clear(); this.refreshStatus(); }
resetSession() { this.disconnected(); this.consumerId = undefined; }
request(operation, uri) {
if (!this.client || !this.consumerId)
return Promise.reject(new Error("External source unavailable: reconnect to Studio."));
return this.client.sendRequest(`prometeu/${operation}`, { consumerId: this.consumerId, uri });
}
async provideTextDocumentContent(uri, token) {
const key = uri.toString();
const old = this.documents.get(key);
if (old)
return old.text;
const content = await this.request("readExternalSource", key);
if (token.isCancellationRequested) {
await this.release(key);
throw new Error("External source read cancelled.");
}
if (!content.descriptor.readOnly || content.descriptor.uri !== key)
throw new Error("Invalid external source descriptor.");
this.documents.set(key, content);
this.available.add(key);
this.refreshStatus();
return content.text;
}
async opened(document) {
if (document.uri.scheme !== exports.externalScheme)
return;
const content = this.documents.get(document.uri.toString());
if (content && content.descriptor.languageId !== document.languageId)
await vscode.languages.setTextDocumentLanguage(document, content.descriptor.languageId);
}
async closeDocument(document) {
const uri = document.uri.toString();
if (!this.documents.has(uri))
return;
// Changing language also fires close/open; retain while VS Code still owns the document.
await new Promise(resolve => setTimeout(resolve, 0));
if (vscode.workspace.textDocuments.some(open => open.uri.toString() === uri && !open.isClosed))
return;
this.documents.delete(uri);
this.available.delete(uri);
if (this.client)
await this.release(uri);
else
this.closed.add(uri);
}
async release(uri) {
if (this.client && this.consumerId)
await this.client.sendRequest("prometeu/releaseExternalSource", { consumerId: this.consumerId, uri });
}
async shutdown() {
if (this.client && this.consumerId)
await this.client.sendRequest("prometeu/releaseExternalSource", { consumerId: this.consumerId, uri: null });
this.resetSession();
this.documents.clear();
}
dispose() { this.disconnected(); this.documents.clear(); this.status.dispose(); }
}
exports.ExternalSources = ExternalSources;
//# sourceMappingURL=externalSources.js.map
+29 -2
View File
@@ -1,5 +1,6 @@
import * as net from "node:net";
import * as vscode from "vscode";
import { ExternalSources, externalScheme } from "./externalSources";
import {
CloseAction,
@@ -17,6 +18,7 @@ let reconnectTimer: NodeJS.Timeout | undefined;
let reconnectContext: vscode.ExtensionContext | undefined;
let stoppingClient = false;
let lastAppliedThemeSignature: string | undefined;
const externalSources = new ExternalSources();
const RECONNECT_DELAY_MS = 2000;
@@ -76,6 +78,11 @@ export function activate(context: vscode.ExtensionContext): void {
reconnectContext = context;
output = vscode.window.createOutputChannel("Prometeu LSP");
output.appendLine("Prometeu VS Code extension activated.");
context.subscriptions.push(externalSources,
vscode.workspace.registerTextDocumentContentProvider(externalScheme, externalSources),
vscode.window.onDidChangeActiveTextEditor(() => externalSources.refreshStatus()),
vscode.workspace.onDidOpenTextDocument(document => void externalSources.opened(document)),
vscode.workspace.onDidCloseTextDocument(document => void externalSources.closeDocument(document)));
const connectCommand = vscode.commands.registerCommand(
"prometeu.connectStudioLsp",
@@ -98,7 +105,8 @@ export function activate(context: vscode.ExtensionContext): void {
}
export async function deactivate(): Promise<void> {
await stopClient();
try { await externalSources.shutdown(); }
finally { await stopClient(); }
}
async function startClient(context: vscode.ExtensionContext): Promise<void> {
@@ -167,7 +175,7 @@ async function startClient(context: vscode.ExtensionContext): Promise<void> {
{
scheme: "file",
language: "pbs"
}
}, { scheme: externalScheme, language: "pbs" }
],
synchronize: {
fileEvents: [
@@ -175,10 +183,20 @@ async function startClient(context: vscode.ExtensionContext): Promise<void> {
]
},
initializationOptions: {
...externalSources.initializationOptions,
clientKind: "vscode",
prometeuClientVersion: context.extension.packageJSON.version
},
outputChannel: output,
middleware: {
prepareRename: (document, position, token, next) => document.uri.scheme === externalScheme ? null : next(document, position, token),
provideRenameEdits: (document, position, name, token, next) => document.uri.scheme === externalScheme ? null : next(document, position, name, token),
provideDocumentFormattingEdits: (document, options, token, next) => document.uri.scheme === externalScheme ? [] : next(document, options, token),
provideCodeActions: (document, range, context, token, next) => document.uri.scheme === externalScheme ? [] : next(document, range, context, token),
provideHover: (document, position, token, next) => externalSources.canQuery(document) ? next(document, position, token) : null,
provideDefinition: (document, position, token, next) => externalSources.canQuery(document) ? next(document, position, token) : null,
provideDocumentSemanticTokens: (document, token, next) => externalSources.canQuery(document) ? next(document, token) : null
},
errorHandler: {
error: (error) => {
output.appendLine(`Prometeu LSP transport error: ${error.message}`);
@@ -198,11 +216,18 @@ async function startClient(context: vscode.ExtensionContext): Promise<void> {
clientOptions
);
client = nextClient;
nextClient.registerFeature({
fillClientCapabilities(capabilities) {
capabilities.experimental = { ...capabilities.experimental, prometeuExternalSources: true };
},
initialize() {}, clear() {}, getState() { return { kind: "static" }; }
});
nextClient.onDidChangeState((event) => {
output.appendLine(`Prometeu LSP state changed: ${State[event.oldState]} -> ${State[event.newState]}`);
if (event.newState !== State.Stopped) {
return;
}
externalSources.disconnected();
if (client === nextClient) {
client = undefined;
}
@@ -213,11 +238,13 @@ async function startClient(context: vscode.ExtensionContext): Promise<void> {
try {
await nextClient.start();
await externalSources.connected(nextClient);
await applyPrometeuVisualTheme(nextClient.initializeResult);
output.appendLine("Prometeu Studio LSP client started.");
vscode.window.showInformationMessage("Connected to Prometeu Studio LSP.");
} catch (error) {
externalSources.resetSession();
if (client === nextClient) {
client = undefined;
}
@@ -0,0 +1,106 @@
import * as vscode from "vscode";
import { LanguageClient } from "vscode-languageclient/node";
export const externalScheme = "prometeu-source";
export type ExternalContent = {
descriptor: { uri: string; revision: string; origin: string; name: string; languageId: string; readOnly: true };
text: string;
};
/** Immutable virtual documents. No filesystem provider and no content-change events. */
export class ExternalSources implements vscode.TextDocumentContentProvider, vscode.Disposable {
private client?: LanguageClient;
private consumerId?: string;
private readonly documents = new Map<string, ExternalContent>();
private readonly available = new Set<string>();
private readonly closed = new Set<string>();
private readonly status = vscode.window.createStatusBarItem(vscode.StatusBarAlignment.Right, 10);
refreshStatus(): void {
const document = vscode.window.activeTextEditor?.document;
if (document?.uri.scheme === externalScheme && !this.canQuery(document)) {
this.status.text = "Prometeu: external source unavailable";
this.status.tooltip = "Reconnect to Studio or navigate again to obtain the current source revision.";
this.status.show();
} else this.status.hide();
}
get initializationOptions(): object {
return this.consumerId ? { prometeuExternalConsumerId: this.consumerId } : {};
}
canQuery(document: vscode.TextDocument): boolean {
const uri = document.uri;
if (uri.scheme !== externalScheme) return true;
const content = this.documents.get(uri.toString());
return !!this.client && this.available.has(uri.toString()) && !!content
&& document.getText().replace(/\r\n?/g, "\n") === content.text.replace(/\r\n?/g, "\n");
}
async connected(client: LanguageClient): Promise<void> {
const experimental = client.initializeResult?.capabilities.experimental;
if (!experimental?.prometeuExternalSources) return;
this.client = client;
this.consumerId = experimental.prometeuExternalConsumerId;
for (const uri of this.closed) await this.release(uri);
this.closed.clear();
for (const [uri, old] of this.documents) {
try {
const current = await this.request("revalidateExternalSource", uri);
if (current.descriptor.revision === old.descriptor.revision && current.text === old.text) this.available.add(uri);
} catch { /* Keep displayed text; fresh navigation is required. */ }
}
this.refreshStatus();
}
disconnected(): void { this.client = undefined; this.available.clear(); this.refreshStatus(); }
resetSession(): void { this.disconnected(); this.consumerId = undefined; }
private request(operation: string, uri: string): Promise<ExternalContent> {
if (!this.client || !this.consumerId) return Promise.reject(new Error("External source unavailable: reconnect to Studio."));
return this.client.sendRequest<ExternalContent>(`prometeu/${operation}`, { consumerId: this.consumerId, uri });
}
async provideTextDocumentContent(uri: vscode.Uri, token: vscode.CancellationToken): Promise<string> {
const key = uri.toString();
const old = this.documents.get(key);
if (old) return old.text;
const content = await this.request("readExternalSource", key);
if (token.isCancellationRequested) { await this.release(key); throw new Error("External source read cancelled."); }
if (!content.descriptor.readOnly || content.descriptor.uri !== key) throw new Error("Invalid external source descriptor.");
this.documents.set(key, content);
this.available.add(key);
this.refreshStatus();
return content.text;
}
async opened(document: vscode.TextDocument): Promise<void> {
if (document.uri.scheme !== externalScheme) return;
const content = this.documents.get(document.uri.toString());
if (content && content.descriptor.languageId !== document.languageId)
await vscode.languages.setTextDocumentLanguage(document, content.descriptor.languageId);
}
async closeDocument(document: vscode.TextDocument): Promise<void> {
const uri = document.uri.toString();
if (!this.documents.has(uri)) return;
// Changing language also fires close/open; retain while VS Code still owns the document.
await new Promise(resolve => setTimeout(resolve, 0));
if (vscode.workspace.textDocuments.some(open => open.uri.toString() === uri && !open.isClosed)) return;
this.documents.delete(uri);
this.available.delete(uri);
if (this.client) await this.release(uri); else this.closed.add(uri);
}
private async release(uri: string): Promise<void> {
if (this.client && this.consumerId)
await this.client.sendRequest("prometeu/releaseExternalSource", { consumerId: this.consumerId, uri });
}
async shutdown(): Promise<void> {
if (this.client && this.consumerId)
await this.client.sendRequest("prometeu/releaseExternalSource", { consumerId: this.consumerId, uri: null });
this.resetSession();
this.documents.clear();
}
dispose(): void { this.disconnected(); this.documents.clear(); this.status.dispose(); }
}
@@ -0,0 +1,105 @@
import * as assert from "node:assert/strict";
import * as vscode from "vscode";
import { ExternalSources, ExternalContent } from "../externalSources";
import type { LanguageClient } from "vscode-languageclient/node";
async function providerLifecycle(): Promise<void> {
const provider = new ExternalSources();
const uri = vscode.Uri.parse("prometeu-source://test/revision/main.pbs");
let content: ExternalContent = { descriptor: { uri: uri.toString(), revision: "one", origin: "dependency", name: "main.pbs", languageId: "pbs", readOnly: true }, text: "// á 😀\r\nfn f() {}\r\n" };
const requests: string[] = [];
const fake = { initializeResult: { capabilities: { experimental: { prometeuExternalSources: true, prometeuExternalConsumerId: "consumer" } } },
sendRequest: async (method: string) => { requests.push(method); return content; } } as unknown as LanguageClient;
await provider.connected(fake);
const cancellation = new vscode.CancellationTokenSource();
const original = await provider.provideTextDocumentContent(uri, cancellation.token);
const document = { uri, getText: () => original.replace(/\r\n/g, "\n") } as vscode.TextDocument;
assert.equal(provider.canQuery(document), true);
provider.disconnected();
assert.equal(provider.canQuery(document), false);
assert.equal(await provider.provideTextDocumentContent(uri, cancellation.token), original);
await provider.connected(fake);
assert.equal(provider.canQuery(document), true);
provider.disconnected();
content = { descriptor: { ...content.descriptor, revision: "two" }, text: "different" };
await provider.connected(fake);
assert.equal(provider.canQuery(document), false);
assert.equal(await provider.provideTextDocumentContent(uri, cancellation.token), original);
await provider.closeDocument(document);
assert.ok(requests.includes("prometeu/releaseExternalSource"));
cancellation.cancel();
await assert.rejects(provider.provideTextDocumentContent(uri, cancellation.token), /cancelled/);
await provider.shutdown();
provider.dispose();
cancellation.dispose();
}
async function eventually<T>(query: () => PromiseLike<T>, accept: (value: T) => boolean): Promise<T> {
const deadline = Date.now() + 20000;
while (true) {
const value = await query();
if (accept(value)) return value;
if (Date.now() >= deadline) throw new Error("Timed out waiting for development LSP");
await new Promise(resolve => setTimeout(resolve, 100));
}
}
/** Executed by the real Extension Development Host; no packaged Studio involved. */
export async function run(): Promise<void> {
await providerLifecycle();
const root = vscode.workspace.workspaceFolders![0].uri;
const project = await vscode.workspace.openTextDocument(vscode.Uri.joinPath(root, "src/main.pbs"));
await vscode.window.showTextDocument(project);
const use = project.positionAt(project.getText().indexOf("Gfx.clear") + 4);
const definitions = await eventually(
() => vscode.commands.executeCommand<vscode.Location[]>("vscode.executeDefinitionProvider", project.uri, use),
result => !!result?.length);
assert.equal(definitions[0].uri.scheme, "prometeu-source");
const external = await vscode.workspace.openTextDocument(definitions[0].uri);
const editor = await vscode.window.showTextDocument(external, { selection: definitions[0].range });
await eventually(async () => external.languageId, language => language === "pbs");
assert.equal(external.isUntitled, false);
assert.equal(external.isDirty, false);
const original = external.getText();
assert.ok(external.getText(definitions[0].range).includes("clear"));
await vscode.commands.executeCommand("type", { text: "changed" });
assert.equal(external.getText(), original);
assert.equal(external.isDirty, false);
const call = external.positionAt(original.indexOf("LowGfx.clear") + 7);
const hover = await eventually(
() => vscode.commands.executeCommand<vscode.Hover[]>("vscode.executeHoverProvider", external.uri, call),
result => !!result?.length);
assert.ok(hover.flatMap(item => item.contents).some(content => (typeof content === "string" ? content : content.value).includes("clear")));
const host = await vscode.commands.executeCommand<vscode.Location[]>("vscode.executeDefinitionProvider", external.uri, call);
assert.ok(host.length);
const primitive = await vscode.workspace.openTextDocument(host[0].uri);
assert.ok(primitive.getText(host[0].range).includes("clear"));
const tokens = await eventually(
() => vscode.commands.executeCommand<vscode.SemanticTokens>("vscode.provideDocumentSemanticTokens", external.uri),
result => !!result?.data.length);
assert.ok(tokens.data.length > 0);
const libraryUse = project.positionAt(project.getText().lastIndexOf("answer()"));
const libraryDefinitions = await vscode.commands.executeCommand<vscode.Location[]>("vscode.executeDefinitionProvider", project.uri, libraryUse);
assert.equal(libraryDefinitions[0].uri.scheme, "prometeu-source");
const library = await vscode.workspace.openTextDocument(libraryDefinitions[0].uri);
await vscode.window.showTextDocument(library, { selection: libraryDefinitions[0].range });
assert.ok(library.getText(libraryDefinitions[0].range).includes("answer"));
assert.equal(libraryDefinitions[0].range.start.line, 1);
const nestedUse = library.positionAt(library.getText().indexOf("helper();"));
const nested = await eventually(
() => vscode.commands.executeCommand<vscode.Location[]>("vscode.executeDefinitionProvider", library.uri, nestedUse),
result => !!result?.length);
assert.equal(nested[0].range.start.line, 2);
await vscode.commands.executeCommand("prometeu.restartStudioLsp");
assert.equal(external.getText(), original);
const afterReconnect = await eventually(
() => vscode.commands.executeCommand<vscode.Hover[]>("vscode.executeHoverProvider", external.uri, call),
result => !!result?.length);
assert.ok(afterReconnect.flatMap(item => item.contents).some(content => (typeof content === "string" ? content : content.value).includes("clear")));
// Project editing remains functional in the same editor session.
const projectEditor = await vscode.window.showTextDocument(project);
assert.equal(await projectEditor.edit(edit => edit.insert(new vscode.Position(0, 0), "// editable\n")), true);
await vscode.commands.executeCommand("workbench.action.revertAndCloseActiveEditor");
await vscode.commands.executeCommand("workbench.action.closeAllEditors");
console.log("PLN-0141 VS Code development acceptance passed: readonly, syntax, positions, hover, continued definition, project editing.");
}