package PBS references and expose paginated MCP reads
This commit is contained in:
+33
@@ -323,3 +323,36 @@ The current Studio direction is:
|
|||||||
4. the compiler should use explicit abstractions such as `StdlibEnvironment` and `StdlibModuleResolver`,
|
4. the compiler should use explicit abstractions such as `StdlibEnvironment` and `StdlibModuleResolver`,
|
||||||
5. stdlib interface modules are parsed with the normal PBS parser and validated in interface-module mode,
|
5. stdlib interface modules are parsed with the normal PBS parser and validated in interface-module mode,
|
||||||
6. reserved attributes such as `[Host(...)]` are extracted as compile-time metadata for later lowering.
|
6. reserved attributes such as `[Host(...)]` are extracted as compile-time metadata for later lowering.
|
||||||
|
|
||||||
|
## 13. Installed Language and SDK Reference Bundle
|
||||||
|
|
||||||
|
The PBS frontend distribution MUST provide local, indexed documentation through
|
||||||
|
the frontend reference-provider contract. This packaging does not create a new
|
||||||
|
normative specification: the numbered PBS documents in their owner directory
|
||||||
|
remain authoritative, and the quickstart/example remain informative.
|
||||||
|
|
||||||
|
The build MUST generate the reference index from owner-maintained sources:
|
||||||
|
|
||||||
|
- this directory's numbered specifications and README quickstart;
|
||||||
|
- the frontend's example manifest, PBS source and barrel;
|
||||||
|
- the actual packaged SDK `.pbs` and `.barrel` files, including their documentation.
|
||||||
|
|
||||||
|
Generated copies MUST NOT be edited independently. SDK declarations MUST NOT be
|
||||||
|
rewritten to manufacture a documentation-only API. The index MUST contain stable
|
||||||
|
reference identities, titles, kinds, packaged-resource identities, content hashes
|
||||||
|
and applicable SDK major/target. The loader MUST reject missing/corrupted indexed
|
||||||
|
content instead of advertising unreadable references or consulting a checkout.
|
||||||
|
|
||||||
|
For the current distribution, SDK/example selection is Game / major 1. The general
|
||||||
|
PBS specifications remain available for other selections, but Game SDK 1 references
|
||||||
|
MUST NOT be presented as another SDK version or target. An absent SDK reference
|
||||||
|
set MUST be explicit to consumers.
|
||||||
|
|
||||||
|
Language, compiler, SDK and index-schema versions remain independent. General
|
||||||
|
specification references use an unknown release version when owner release
|
||||||
|
metadata is absent; SDK-associated references identify the installed SDK major.
|
||||||
|
Content hashes identify the packaged texts, not a semantic language release.
|
||||||
|
|
||||||
|
Conformance MUST include reading all indexed references from installed JARs outside
|
||||||
|
the repository and compiling the shipped example against the installed SDK. No
|
||||||
|
network access or source checkout may be required by that reference-read path.
|
||||||
|
|||||||
@@ -2,6 +2,73 @@
|
|||||||
|
|
||||||
This directory contains the normative PBS specification set.
|
This directory contains the normative PBS specification set.
|
||||||
|
|
||||||
|
## Installed Quickstart (Informative)
|
||||||
|
|
||||||
|
This guide and the numbered specifications are packaged with the PBS frontend.
|
||||||
|
They describe the installed distribution, not a promise of a numbered language
|
||||||
|
release. A spec's `Draft v1` status is not the compiler version or SDK version.
|
||||||
|
The project manifest selects the SDK major with `stdlib`; current packaged
|
||||||
|
Game SDK resources use major `1`. Unknown language/compiler versions remain
|
||||||
|
unknown in Studio's context.
|
||||||
|
|
||||||
|
For an external coding agent:
|
||||||
|
|
||||||
|
1. Request `studio_project_context` for the absolute root of the open project.
|
||||||
|
2. Read `pbs:quickstart`, `pbs:spec:3` (syntax), `pbs:spec:4` (static semantics)
|
||||||
|
and the relevant SDK reference IDs returned in that context. Continue pages
|
||||||
|
until `nextOffset` is null. References are IDs, not filesystem paths.
|
||||||
|
3. Inspect the project and its dependencies before changing source. Only edit
|
||||||
|
the project's editable scope, excluding read-only SDK/external libraries.
|
||||||
|
4. Use the installed LSP and compiler to check changes. When the corresponding
|
||||||
|
MCP tools are available, acquire project control after the developer's DEV
|
||||||
|
grant, then use the advertised language/build tools. Documentation reads do
|
||||||
|
not require that grant. Missing tools/capabilities must not be assumed.
|
||||||
|
5. Build and test small changes. Runtime execution also requires a compatible
|
||||||
|
configured runtime; reading references does not install one.
|
||||||
|
|
||||||
|
### Small Game Project
|
||||||
|
|
||||||
|
The installed example has three separately indexed files:
|
||||||
|
|
||||||
|
- `pbs:example:game:1:prometeu.json`: a Game manifest with `language: pbs`,
|
||||||
|
`stdlib: 1` (serialized as a string), and no external dependencies;
|
||||||
|
- `pbs:example:game:1:src/main.pbs`: `[Init]` and `[Frame]` functions drawing a
|
||||||
|
moving rectangle through `Gfx` and constructing colors through `Color`;
|
||||||
|
- `pbs:example:game:1:src/mod.barrel`: the module's public declarations.
|
||||||
|
|
||||||
|
Copy these reference texts to the corresponding paths only when creating that
|
||||||
|
example in your editable project. The example is compiled in the test suite;
|
||||||
|
it is not a complete game or a claim that every requested gameplay abstraction
|
||||||
|
already exists. If a project has no external dependencies, no registry is needed.
|
||||||
|
For dependencies, follow the installed compiler's manifest/registry requirements.
|
||||||
|
|
||||||
|
PBS source lives under `src` for the installed provider. Functions use `fn`,
|
||||||
|
typed parameters and `->` return types. `let` declares a local binding;
|
||||||
|
`declare global` declares persistent global state. Imports name exports and a
|
||||||
|
module address, for example `import { Gfx } from @sdk:gfx;`. A `mod.barrel`
|
||||||
|
describes a module's public surface. Read the syntax/static-semantics chapters
|
||||||
|
for exact rules rather than inferring compatibility with another language.
|
||||||
|
|
||||||
|
### SDK Navigation and Limits
|
||||||
|
|
||||||
|
`pbs:sdk:game:1:sdk/gfx/main.pbs` exposes the actual installed Gfx source,
|
||||||
|
including documentation comments and signatures. Its `mod.barrel` reference
|
||||||
|
defines exports. Equivalent references exist for `sdk/asset`, `sdk/composer`,
|
||||||
|
`sdk/input`, `sdk/log`, and `core/color`. Read those interfaces before calling
|
||||||
|
them. Game SDK 1 does not currently ship `audio` or `core/math` modules.
|
||||||
|
|
||||||
|
Ordinary game source uses public services such as `Gfx`; it must not manufacture
|
||||||
|
host bindings or mutate SDK declarations. Asset/composer workflows require their
|
||||||
|
real asset/scene data and runtime capabilities. Reference availability does not
|
||||||
|
guarantee those project resources exist. SDK or tooling limitations should be
|
||||||
|
reported for separate work, not patched in read-only dependencies.
|
||||||
|
|
||||||
|
The reference index is generated from these owner-maintained documents and the
|
||||||
|
frontend's SDK/example resources. The index schema revision is not a language
|
||||||
|
version. Source hashes bind texts to the installed bundle; no repository checkout
|
||||||
|
or network is needed to read them. The numbered specifications remain normative;
|
||||||
|
this onboarding section and example are explanatory, not a second specification.
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
|
|
||||||
Specs define the official PBS language/frontend contract.
|
Specs define the official PBS language/frontend contract.
|
||||||
|
|||||||
@@ -184,7 +184,8 @@ Reference reads take an opaque ID from that index, not a path or arbitrary URI.
|
|||||||
Offset defaults to 0; limit defaults to 4000 and accepts 1–16000 Unicode code
|
Offset defaults to 0; limit defaults to 4000 and accepts 1–16000 Unicode code
|
||||||
points. A page returns referenceId, offset, total, text and nextOffset (null at
|
points. A page returns referenceId, offset, total, text and nextOffset (null at
|
||||||
end). Invalid pagination fails; unknown/traversal-like IDs cannot escape the
|
end). Invalid pagination fails; unknown/traversal-like IDs cannot escape the
|
||||||
index. Installed packaging and real provider resolution are steps 17–18.
|
index. Installed reference content is resolved through the frontend's owner-maintained
|
||||||
|
reference provider, not a Studio-maintained language specification.
|
||||||
|
|
||||||
### 5.1 Manifest and Provider Resolution
|
### 5.1 Manifest and Provider Resolution
|
||||||
|
|
||||||
@@ -230,9 +231,44 @@ symlink targets. SDK entries are always read-only; packaged SDK resources need n
|
|||||||
have filesystem roots. Scope metadata does not sandbox an external agent's own
|
have filesystem roots. Scope metadata does not sandbox an external agent's own
|
||||||
filesystem tools; the developer remains responsible for those tools and Git.
|
filesystem tools; the developer remains responsible for those tools and Git.
|
||||||
|
|
||||||
Paths in MCP results are absolute filesystem strings, not `file:` URIs. Until
|
Paths in MCP results are absolute filesystem strings, not `file:` URIs.
|
||||||
step 18 packages the owner-maintained index, references/reference IDs are empty
|
|
||||||
with `DOCUMENTATION_UNAVAILABLE`; no guessed path or reference is advertised.
|
### 5.2 Installed References and Pagination
|
||||||
|
|
||||||
|
An installed frontend MAY implement `FrontendReferenceProvider` through
|
||||||
|
`FrontendProvider.documentation()`. That compiler-side contract supplies indexed,
|
||||||
|
owner-maintained texts for an SDK major and project target. It does not depend on
|
||||||
|
Studio API or MCP. Studio projects these into neutral reference descriptors;
|
||||||
|
full texts MUST NOT be included in project context. Reference IDs are namespaced
|
||||||
|
by language (`<languageId>:`), unique within the selected project index and opaque
|
||||||
|
to callers. Studio MUST NOT interpret an ID as a filename, URI or URL.
|
||||||
|
|
||||||
|
PBS packages its numbered specifications and informative quickstart from their
|
||||||
|
owner directory, plus a compilable Game example and actual SDK sources/barrels.
|
||||||
|
A build-generated index binds these texts to the installed frontend using content
|
||||||
|
hashes. Distribution-only reads MUST work without a checkout or network. SDK and
|
||||||
|
example references identify their SDK major and target; unsupported selections
|
||||||
|
MUST NOT silently reuse another SDK/target. Numbered spec/draft revisions and the
|
||||||
|
reference-index schema revision MUST NOT be reported as language/compiler versions.
|
||||||
|
|
||||||
|
`studio_reference_read` is authenticated but does not require a control grant or
|
||||||
|
task. It resolves the explicit open root and its current saved manifest selection
|
||||||
|
on every read. Changing SDK/target can invalidate previously advertised IDs.
|
||||||
|
Only IDs in that project's installed index can be read, including when other
|
||||||
|
languages/providers are installed globally. SDK and external-library texts remain
|
||||||
|
consultable, not editable through this surface.
|
||||||
|
|
||||||
|
Pagination counts Unicode code points: an astral character occupies one offset,
|
||||||
|
not two UTF-16 units. Offset 0/limit 4000 are defaults; limit MUST be 1–16000.
|
||||||
|
An offset equal to total returns empty text and null nextOffset. Offsets beyond
|
||||||
|
total, negative offsets, nonintegral values and integers beyond the implementation's
|
||||||
|
signed 32-bit range are invalid parameters, never wrapped/truncated to another page.
|
||||||
|
|
||||||
|
Missing or unreadable provider documentation yields `DOCUMENTATION_UNAVAILABLE`
|
||||||
|
limitations, not promised references. An uninstalled primary language yields
|
||||||
|
`UNSUPPORTED_LANGUAGE` for reference reads; an available index without the requested
|
||||||
|
ID yields `REFERENCE_NOT_FOUND`. Closed projects yield `PROJECT_NOT_OPEN` before
|
||||||
|
reference lookup. These failures do not grant/revoke task authority.
|
||||||
|
|
||||||
## 6. Image Contract
|
## 6. Image Contract
|
||||||
|
|
||||||
@@ -258,7 +294,8 @@ authentication, Origin, endpoint/method/version handling, tool dispatch, malform
|
|||||||
or oversized input, closure, restart and old-token rejection. Provider doubles
|
or oversized input, closure, restart and old-token rejection. Provider doubles
|
||||||
exist only in tests. Composition starts/closes transport with Studio.
|
exist only in tests. Composition starts/closes transport with Studio.
|
||||||
|
|
||||||
The production registry composes `studio_project_context` as of step 17; other
|
The production registry composes `studio_project_context` and `studio_reference_read`
|
||||||
|
as of step 18; other
|
||||||
tools are advertised only when their functional handlers are connected.
|
tools are advertised only when their functional handlers are connected.
|
||||||
DEV configuration-copy stays unavailable until step 21. Passing this transport
|
DEV configuration-copy stays unavailable until step 21. Passing this transport
|
||||||
gate is not the phase 3 or final external-agent acceptance gate. API/Core and
|
gate is not the phase 3 or final external-agent acceptance gate. API/Core and
|
||||||
|
|||||||
@@ -31,7 +31,7 @@ javafx {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// Opt-in desktop gate: real JavaFX toolkit and windows, no display required by normal unit tests.
|
// Opt-in desktop gate: real JavaFX toolkit and windows, no display required by normal unit tests.
|
||||||
tasks.named<Test>("test") { exclude("**/DevDesktopTest.class") }
|
tasks.named<Test>("test") { exclude("**/DevDesktopTest.class", "**/InstalledKnowledgeTest.class") }
|
||||||
tasks.register<Test>("devDesktopTest") {
|
tasks.register<Test>("devDesktopTest") {
|
||||||
group = "verification"
|
group = "verification"
|
||||||
description = "Runs the DEV/Debug desktop lifecycle and handoff smoke tests (requires a display)."
|
description = "Runs the DEV/Debug desktop lifecycle and handoff smoke tests (requires a display)."
|
||||||
@@ -41,3 +41,16 @@ tasks.register<Test>("devDesktopTest") {
|
|||||||
include("**/DevDesktopTest.class")
|
include("**/DevDesktopTest.class")
|
||||||
systemProperty("java.awt.headless", "false")
|
systemProperty("java.awt.headless", "false")
|
||||||
}
|
}
|
||||||
|
|
||||||
|
tasks.register<Test>("installedKnowledgeTest") {
|
||||||
|
group = "verification"
|
||||||
|
description = "Reads installed references and compiles their example in an isolated JVM outside the checkout."
|
||||||
|
dependsOn(tasks.named("installDist"))
|
||||||
|
testClassesDirs = sourceSets["test"].output.classesDirs
|
||||||
|
classpath = sourceSets["test"].runtimeClasspath
|
||||||
|
useJUnitPlatform()
|
||||||
|
include("**/InstalledKnowledgeTest.class")
|
||||||
|
val distribution = layout.buildDirectory.dir("install/prometeu-app")
|
||||||
|
inputs.dir(distribution)
|
||||||
|
systemProperty("studio.test.distribution", distribution.get().asFile.absolutePath)
|
||||||
|
}
|
||||||
|
|||||||
@@ -304,7 +304,7 @@ public class StudioApplicationConfiguration {
|
|||||||
@Bean
|
@Bean
|
||||||
p.studio.app.mcp.McpToolRegistry mcpToolRegistry(p.studio.app.mcp.McpToolCatalog catalog, ObjectMapper mapper,
|
p.studio.app.mcp.McpToolRegistry mcpToolRegistry(p.studio.app.mcp.McpToolCatalog catalog, ObjectMapper mapper,
|
||||||
p.studio.api.automation.ProjectKnowledge knowledge) {
|
p.studio.api.automation.ProjectKnowledge knowledge) {
|
||||||
// Further providers are connected in PLN-0139 steps 18–20. Never advertise placeholders.
|
// Further providers are connected in PLN-0139 steps 19–20. Never advertise placeholders.
|
||||||
return new p.studio.app.mcp.McpToolRegistry(catalog, mapper,
|
return new p.studio.app.mcp.McpToolRegistry(catalog, mapper,
|
||||||
p.studio.app.mcp.McpProjectKnowledgeTools.handlers(knowledge, mapper));
|
p.studio.app.mcp.McpProjectKnowledgeTools.handlers(knowledge, mapper));
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -20,6 +20,8 @@ import java.util.*;
|
|||||||
|
|
||||||
/** Read-only manifest/provider discovery. Never selects a default frontend or starts a build. */
|
/** Read-only manifest/provider discovery. Never selects a default frontend or starts a build. */
|
||||||
public final class CompilerProjectKnowledge implements ProjectKnowledge {
|
public final class CompilerProjectKnowledge implements ProjectKnowledge {
|
||||||
|
private record OwnedReference(String owner, p.studio.compiler.services.FrontendReferenceProvider.Reference reference) {}
|
||||||
|
private record Discovery(Context context, Map<String, OwnedReference> references) {}
|
||||||
private static final int MAX_PROJECTS = 256;
|
private static final int MAX_PROJECTS = 256;
|
||||||
private final ProjectAccessRegistry openProjects;
|
private final ProjectAccessRegistry openProjects;
|
||||||
private final ProjectCatalog projects;
|
private final ProjectCatalog projects;
|
||||||
@@ -33,6 +35,10 @@ public final class CompilerProjectKnowledge implements ProjectKnowledge {
|
|||||||
}
|
}
|
||||||
|
|
||||||
@Override public Context context(Path openProjectRoot) {
|
@Override public Context context(Path openProjectRoot) {
|
||||||
|
return discover(openProjectRoot).context();
|
||||||
|
}
|
||||||
|
|
||||||
|
private Discovery discover(Path openProjectRoot) {
|
||||||
Path root = openProjects.requireOpenRoot(openProjectRoot);
|
Path root = openProjects.requireOpenRoot(openProjectRoot);
|
||||||
var limitations = new ArrayList<String>();
|
var limitations = new ArrayList<String>();
|
||||||
var discovered = new LinkedHashMap<Path, ProjectReference>();
|
var discovered = new LinkedHashMap<Path, ProjectReference>();
|
||||||
@@ -128,12 +134,15 @@ public final class CompilerProjectKnowledge implements ProjectKnowledge {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
libraries.addAll(unresolved);
|
libraries.addAll(unresolved);
|
||||||
|
var references = new LinkedHashMap<String, OwnedReference>();
|
||||||
|
discovered.values().forEach(project -> collectReferences(project, references, limitations));
|
||||||
var languages = new ArrayList<Language>();
|
var languages = new ArrayList<Language>();
|
||||||
String primary = discovered.get(root).languageId();
|
String primary = discovered.get(root).languageId();
|
||||||
if (byLanguage.size() > 1) limitations.add("The installed build pipeline requires the same language across the dependency graph");
|
if (byLanguage.size() > 1) limitations.add("The installed build pipeline requires the same language across the dependency graph");
|
||||||
for (var entry : byLanguage.entrySet()) {
|
for (var entry : byLanguage.entrySet()) {
|
||||||
var language = language(entry.getKey(), entry.getValue(), primary.equals(entry.getKey()),
|
var language = language(entry.getKey(), entry.getValue(), primary.equals(entry.getKey()),
|
||||||
dependenciesComplete && byLanguage.size() == 1, limitations);
|
dependenciesComplete && byLanguage.size() == 1, limitations,
|
||||||
|
references.entrySet().stream().filter(ref -> ref.getValue().owner().equals(entry.getKey())).map(Map.Entry::getKey).toList());
|
||||||
languages.add(language);
|
languages.add(language);
|
||||||
for (Path source : language.sourceRoots()) if (!source.startsWith(root)) readOnly.add(source);
|
for (Path source : language.sourceRoots()) if (!source.startsWith(root)) readOnly.add(source);
|
||||||
for (var project : entry.getValue()) {
|
for (var project : entry.getValue()) {
|
||||||
@@ -144,15 +153,19 @@ public final class CompilerProjectKnowledge implements ProjectKnowledge {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
limitations.add("SDK resources are provider-owned, read-only resources, not editable project files");
|
limitations.add("SDK resources are provider-owned, read-only resources, not editable project files");
|
||||||
limitations.add("DOCUMENTATION_UNAVAILABLE: installed language/SDK reference index is not yet available");
|
|
||||||
// Closing a project during discovery must not publish a successful context for that closed project.
|
// Closing a project during discovery must not publish a successful context for that closed project.
|
||||||
openProjects.requireOpenRoot(root);
|
openProjects.requireOpenRoot(root);
|
||||||
return new Context(root, discovered.get(root).name(), languages, libraries,
|
var descriptors = references.values().stream().map(owned -> {
|
||||||
List.of(root), List.copyOf(readOnly), List.of(), limitations);
|
var ref = owned.reference();
|
||||||
|
return new Reference(ref.id(), ref.title(), owned.owner(), ReferenceKind.valueOf(ref.kind().name()),
|
||||||
|
new Version(ref.version(), ref.versionSource()));
|
||||||
|
}).toList();
|
||||||
|
return new Discovery(new Context(root, discovered.get(root).name(), languages, libraries,
|
||||||
|
List.of(root), List.copyOf(readOnly), descriptors, limitations.stream().distinct().toList()), Map.copyOf(references));
|
||||||
}
|
}
|
||||||
|
|
||||||
private Language language(String id, List<ProjectReference> projects, boolean primary,
|
private Language language(String id, List<ProjectReference> projects, boolean primary,
|
||||||
boolean graphBuildable, List<String> limitations) {
|
boolean graphBuildable, List<String> limitations, List<String> referenceIds) {
|
||||||
var sdkVersions = projects.stream().map(ProjectReference::stdlibVersion).distinct().toList();
|
var sdkVersions = projects.stream().map(ProjectReference::stdlibVersion).distinct().toList();
|
||||||
Version sdk = sdkVersions.size() == 1
|
Version sdk = sdkVersions.size() == 1
|
||||||
? new Version(sdkVersions.getFirst().toString(), "prometeu.json#stdlib (requested)") : Version.unknown();
|
? new Version(sdkVersions.getFirst().toString(), "prometeu.json#stdlib (requested)") : Version.unknown();
|
||||||
@@ -161,7 +174,7 @@ public final class CompilerProjectKnowledge implements ProjectKnowledge {
|
|||||||
if (provider.isEmpty()) {
|
if (provider.isEmpty()) {
|
||||||
limitations.add("UNSUPPORTED_LANGUAGE: no installed frontend for " + id);
|
limitations.add("UNSUPPORTED_LANGUAGE: no installed frontend for " + id);
|
||||||
return new Language(id, primary, Version.unknown(), Version.unknown(), sdk, List.of(), List.of(),
|
return new Language(id, primary, Version.unknown(), Version.unknown(), sdk, List.of(), List.of(),
|
||||||
List.of(capability("build", false), capability("lsp", false)), List.of());
|
List.of(capability("build", false), capability("lsp", false)), referenceIds);
|
||||||
}
|
}
|
||||||
var spec = provider.get().specification();
|
var spec = provider.get().specification();
|
||||||
var roots = new LinkedHashSet<Path>();
|
var roots = new LinkedHashSet<Path>();
|
||||||
@@ -176,7 +189,35 @@ public final class CompilerProjectKnowledge implements ProjectKnowledge {
|
|||||||
String compilerVersion = provider.get().getClass().getPackage().getImplementationVersion();
|
String compilerVersion = provider.get().getClass().getPackage().getImplementationVersion();
|
||||||
return new Language(id, primary, Version.unknown(), compilerVersion == null || compilerVersion.isBlank()
|
return new Language(id, primary, Version.unknown(), compilerVersion == null || compilerVersion.isBlank()
|
||||||
? Version.unknown() : new Version(compilerVersion, "installed frontend package Implementation-Version"), sdk,
|
? Version.unknown() : new Version(compilerVersion, "installed frontend package Implementation-Version"), sdk,
|
||||||
spec.getAllowedExtensions().stream().sorted().toList(), List.copyOf(roots), capabilities, List.of());
|
spec.getAllowedExtensions().stream().sorted().toList(), List.copyOf(roots), capabilities, referenceIds);
|
||||||
|
}
|
||||||
|
|
||||||
|
private void collectReferences(ProjectReference project, Map<String, OwnedReference> references, List<String> limitations) {
|
||||||
|
String owner = project.languageId();
|
||||||
|
String selection = owner + " / " + project.target() + " / SDK " + project.stdlibVersion();
|
||||||
|
try {
|
||||||
|
var documentation = FrontendRegistryService.getProvider(owner).flatMap(p.studio.compiler.services.FrontendProvider::documentation);
|
||||||
|
if (documentation.isEmpty()) {
|
||||||
|
limitations.add("DOCUMENTATION_UNAVAILABLE: no installed reference provider for " + owner);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
var selected = documentation.get().references(project.stdlibVersion(), p.studio.AppMode.valueOf(project.target().name()));
|
||||||
|
var additions = new LinkedHashMap<String, OwnedReference>();
|
||||||
|
for (var reference : selected) {
|
||||||
|
var owned = new OwnedReference(owner, reference);
|
||||||
|
if (!reference.id().startsWith(owner + ":")
|
||||||
|
|| references.containsKey(reference.id()) && !references.get(reference.id()).equals(owned)
|
||||||
|
|| additions.putIfAbsent(reference.id(), owned) != null)
|
||||||
|
throw new IllegalStateException("Ambiguous or invalid reference ID");
|
||||||
|
}
|
||||||
|
references.putAll(additions);
|
||||||
|
if (selected.isEmpty()) limitations.add("DOCUMENTATION_UNAVAILABLE: no installed references for " + selection);
|
||||||
|
if (selected.stream().noneMatch(ref -> ref.kind() == p.studio.compiler.services.FrontendReferenceProvider.Kind.SDK
|
||||||
|
&& Integer.toString(project.stdlibVersion()).equals(ref.version())))
|
||||||
|
limitations.add("DOCUMENTATION_UNAVAILABLE: no installed SDK reference for " + selection);
|
||||||
|
} catch (RuntimeException unavailable) {
|
||||||
|
limitations.add("DOCUMENTATION_UNAVAILABLE: cannot load installed references for " + selection);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
private void addLspCapabilities(String id, ProjectReference project, List<Capability> capabilities, List<String> limitations) {
|
private void addLspCapabilities(String id, ProjectReference project, List<Capability> capabilities, List<String> limitations) {
|
||||||
@@ -224,6 +265,21 @@ public final class CompilerProjectKnowledge implements ProjectKnowledge {
|
|||||||
|
|
||||||
@Override public ReferencePage readReference(Path openProjectRoot, String referenceId, int offset, int limit) {
|
@Override public ReferencePage readReference(Path openProjectRoot, String referenceId, int offset, int limit) {
|
||||||
openProjects.requireOpenRoot(openProjectRoot);
|
openProjects.requireOpenRoot(openProjectRoot);
|
||||||
throw new AutomationException(AutomationFailure.Code.DOCUMENTATION_UNAVAILABLE, "No installed reference index is available");
|
if (referenceId == null || referenceId.isBlank() || offset < 0 || limit < 1 || limit > 16000)
|
||||||
|
throw new IllegalArgumentException("Invalid reference ID or pagination");
|
||||||
|
var discovery = discover(openProjectRoot);
|
||||||
|
var reference = discovery.references().get(referenceId);
|
||||||
|
if (reference == null) {
|
||||||
|
var primary = discovery.context().languages().stream().filter(Language::primary).findFirst().orElseThrow();
|
||||||
|
var code = FrontendRegistryService.getProvider(primary.id()).isEmpty() ? AutomationFailure.Code.UNSUPPORTED_LANGUAGE
|
||||||
|
: discovery.references().isEmpty() ? AutomationFailure.Code.DOCUMENTATION_UNAVAILABLE : AutomationFailure.Code.REFERENCE_NOT_FOUND;
|
||||||
|
throw new AutomationException(code, "Reference is not available in this project's installed index");
|
||||||
|
}
|
||||||
|
String text = reference.reference().text();
|
||||||
|
int total = text.codePointCount(0, text.length());
|
||||||
|
if (offset > total) throw new IllegalArgumentException("Reference offset exceeds its length");
|
||||||
|
int next = offset + Math.min(limit, total - offset);
|
||||||
|
String page = text.substring(text.offsetByCodePoints(0, offset), text.offsetByCodePoints(0, next));
|
||||||
|
return new ReferencePage(referenceId, offset, total, next == total ? null : next, page);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -19,6 +19,16 @@ public final class McpProjectKnowledgeTools {
|
|||||||
var paths = new SimpleModule().addSerializer(Path.class, new ToStringSerializer());
|
var paths = new SimpleModule().addSerializer(Path.class, new ToStringSerializer());
|
||||||
var wireMapper = mapper.copy().registerModule(paths);
|
var wireMapper = mapper.copy().registerModule(paths);
|
||||||
return Map.of("studio_project_context", arguments -> McpToolResult.success(mapper,
|
return Map.of("studio_project_context", arguments -> McpToolResult.success(mapper,
|
||||||
wireMapper.valueToTree(knowledge.context(Path.of(arguments.path("projectRoot").asText())))));
|
wireMapper.valueToTree(knowledge.context(Path.of(arguments.path("projectRoot").asText())))),
|
||||||
|
"studio_reference_read", arguments -> McpToolResult.success(mapper, wireMapper.valueToTree(
|
||||||
|
knowledge.readReference(Path.of(arguments.path("projectRoot").asText()), arguments.path("referenceId").asText(),
|
||||||
|
integer(arguments, "offset", 0), integer(arguments, "limit", 4000)))));
|
||||||
|
}
|
||||||
|
|
||||||
|
private static int integer(JsonNode arguments, String name, int fallback) {
|
||||||
|
var value = arguments.get(name);
|
||||||
|
if (value == null) return fallback;
|
||||||
|
if (!value.isIntegralNumber() || !value.canConvertToInt()) throw new IllegalArgumentException("Invalid " + name);
|
||||||
|
return value.intValue();
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -111,6 +111,7 @@
|
|||||||
"offset": {
|
"offset": {
|
||||||
"type": "integer",
|
"type": "integer",
|
||||||
"minimum": 0,
|
"minimum": 0,
|
||||||
|
"maximum": 2147483647,
|
||||||
"default": 0
|
"default": 0
|
||||||
},
|
},
|
||||||
"limit": {
|
"limit": {
|
||||||
@@ -183,12 +184,12 @@
|
|||||||
},
|
},
|
||||||
"exampleArguments": {
|
"exampleArguments": {
|
||||||
"projectRoot": "/projects/game",
|
"projectRoot": "/projects/game",
|
||||||
"referenceId": "pbs.quickstart",
|
"referenceId": "pbs:quickstart",
|
||||||
"offset": 0,
|
"offset": 0,
|
||||||
"limit": 4000
|
"limit": 4000
|
||||||
},
|
},
|
||||||
"exampleValue": {
|
"exampleValue": {
|
||||||
"referenceId": "pbs.quickstart",
|
"referenceId": "pbs:quickstart",
|
||||||
"offset": 0,
|
"offset": 0,
|
||||||
"total": 7,
|
"total": 7,
|
||||||
"nextOffset": null,
|
"nextOffset": null,
|
||||||
|
|||||||
@@ -30,8 +30,9 @@ class StudioApplicationConfigurationTest {
|
|||||||
assertFalse(executor.isShutdown());
|
assertFalse(executor.isShutdown());
|
||||||
assertTrue(mcp.isRunning());
|
assertTrue(mcp.isRunning());
|
||||||
var tools = context.getBean(p.studio.app.mcp.McpToolRegistry.class).descriptors();
|
var tools = context.getBean(p.studio.app.mcp.McpToolRegistry.class).descriptors();
|
||||||
org.junit.jupiter.api.Assertions.assertEquals(1, tools.size());
|
org.junit.jupiter.api.Assertions.assertEquals(2, tools.size());
|
||||||
org.junit.jupiter.api.Assertions.assertEquals("studio_project_context", tools.get(0).path("name").asText());
|
org.junit.jupiter.api.Assertions.assertEquals("studio_project_context", tools.get(0).path("name").asText());
|
||||||
|
org.junit.jupiter.api.Assertions.assertEquals("studio_reference_read", tools.get(1).path("name").asText());
|
||||||
}
|
}
|
||||||
assertTrue(executor.isShutdown());
|
assertTrue(executor.isShutdown());
|
||||||
assertFalse(mcp.isRunning());
|
assertFalse(mcp.isRunning());
|
||||||
|
|||||||
@@ -45,8 +45,10 @@ class CompilerProjectKnowledgeTest {
|
|||||||
assertEquals(List.of(root), result.editableRoots());
|
assertEquals(List.of(root), result.editableRoots());
|
||||||
assertFalse(result.libraries().getFirst().editable());
|
assertFalse(result.libraries().getFirst().editable());
|
||||||
assertTrue(result.libraries().getFirst().id().startsWith("sdk:pbs:"));
|
assertTrue(result.libraries().getFirst().id().startsWith("sdk:pbs:"));
|
||||||
assertTrue(result.references().isEmpty());
|
assertFalse(result.references().isEmpty());
|
||||||
assertTrue(result.limitations().stream().anyMatch(value -> value.contains("DOCUMENTATION_UNAVAILABLE")));
|
assertTrue(result.references().stream().anyMatch(value -> value.id().equals("pbs:quickstart")));
|
||||||
|
assertTrue(language.referenceIds().contains("pbs:quickstart"));
|
||||||
|
assertFalse(result.limitations().stream().anyMatch(value -> value.contains("DOCUMENTATION_UNAVAILABLE")));
|
||||||
assertFalse(Files.exists(root.resolve(".workspace")));
|
assertFalse(Files.exists(root.resolve(".workspace")));
|
||||||
assertFalse(Files.exists(root.resolve(".studio")));
|
assertFalse(Files.exists(root.resolve(".studio")));
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,62 @@
|
|||||||
|
package p.studio.app.compiler;
|
||||||
|
|
||||||
|
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||||
|
import p.studio.api.automation.ProjectKnowledge;
|
||||||
|
import p.studio.compiler.FrontendRegistryService;
|
||||||
|
import p.studio.compiler.messages.BuilderPipelineConfig;
|
||||||
|
import p.studio.compiler.models.BuilderPipelineContext;
|
||||||
|
import p.studio.compiler.workspaces.BuilderPipelineService;
|
||||||
|
import p.studio.core.project.*;
|
||||||
|
import p.studio.utilities.logs.LogAggregator;
|
||||||
|
|
||||||
|
import java.nio.file.Files;
|
||||||
|
import java.nio.file.Path;
|
||||||
|
|
||||||
|
/** Separate JVM gate: product classes/resources must resolve exclusively from installDist JARs. */
|
||||||
|
public final class InstalledKnowledgeProbe {
|
||||||
|
public static void main(String[] arguments) throws Exception {
|
||||||
|
var provider = FrontendRegistryService.require("pbs");
|
||||||
|
check(provider.getClass().getProtectionDomain().getCodeSource().getLocation().getPath().endsWith(".jar"), "Frontend must come from installed JAR");
|
||||||
|
check(CompilerProjectKnowledge.class.getProtectionDomain().getCodeSource().getLocation().getPath().endsWith(".jar"), "App must come from installed JAR");
|
||||||
|
check(!Files.exists(Path.of("settings.gradle.kts")) && !Files.exists(Path.of("docs")), "No repository in working directory");
|
||||||
|
Path root = Files.createDirectory(Path.of("example")).toRealPath();
|
||||||
|
var docs = provider.documentation().orElseThrow().references(1, p.studio.AppMode.Game);
|
||||||
|
for (var reference : docs) {
|
||||||
|
if (reference.id().startsWith("pbs:example:game:1:")) {
|
||||||
|
Path file = root.resolve(reference.id().substring("pbs:example:game:1:".length()));
|
||||||
|
Files.createDirectories(file.getParent());
|
||||||
|
Files.writeString(file, reference.text());
|
||||||
|
}
|
||||||
|
}
|
||||||
|
var mapper = new ObjectMapper();
|
||||||
|
var access = new ProjectAccessRegistry();
|
||||||
|
var registration = access.register(root, () -> { throw new AssertionError("References require no grant"); }, () -> true);
|
||||||
|
var catalog = new ProjectCatalogService(root, mapper, new ProjectLocalStudioSetupStore(mapper));
|
||||||
|
var knowledge = new CompilerProjectKnowledge(access, catalog, mapper);
|
||||||
|
var context = knowledge.context(root);
|
||||||
|
for (var kind : java.util.List.of(ProjectKnowledge.ReferenceKind.QUICKSTART, ProjectKnowledge.ReferenceKind.SYNTAX,
|
||||||
|
ProjectKnowledge.ReferenceKind.SEMANTICS, ProjectKnowledge.ReferenceKind.EXAMPLE, ProjectKnowledge.ReferenceKind.SDK))
|
||||||
|
check(context.references().stream().anyMatch(ref -> ref.kind() == kind), "Missing " + kind);
|
||||||
|
for (var reference : docs) {
|
||||||
|
var actual = new StringBuilder();
|
||||||
|
int offset = 0;
|
||||||
|
while (true) {
|
||||||
|
var page = knowledge.readReference(root, reference.id(), offset, 997);
|
||||||
|
actual.append(page.text());
|
||||||
|
if (page.nextOffset() == null) break;
|
||||||
|
offset = page.nextOffset();
|
||||||
|
}
|
||||||
|
check(actual.toString().equals(reference.text()), "Incorrect installed reference: " + reference.id());
|
||||||
|
}
|
||||||
|
var result = BuilderPipelineService.INSTANCE.compile(
|
||||||
|
BuilderPipelineContext.fromConfig(new BuilderPipelineConfig(false, root.toString())), LogAggregator.empty());
|
||||||
|
check(result.diagnostics().stream().noneMatch(issue -> issue.isError()), "Example must compile without errors");
|
||||||
|
check(result.bytecodeBytes().length > 0, "Example must produce bytecode");
|
||||||
|
registration.unsubscribe();
|
||||||
|
System.out.println("Installed knowledge verified: " + docs.size() + " references; example compiled");
|
||||||
|
}
|
||||||
|
|
||||||
|
private static void check(boolean condition, String message) {
|
||||||
|
if (!condition) throw new AssertionError(message);
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,33 @@
|
|||||||
|
package p.studio.app.compiler;
|
||||||
|
|
||||||
|
import org.junit.jupiter.api.Test;
|
||||||
|
import org.junit.jupiter.api.io.TempDir;
|
||||||
|
import java.nio.file.Files;
|
||||||
|
import java.nio.file.Path;
|
||||||
|
import java.util.concurrent.TimeUnit;
|
||||||
|
import java.util.stream.Collectors;
|
||||||
|
import static org.junit.jupiter.api.Assertions.*;
|
||||||
|
|
||||||
|
class InstalledKnowledgeTest {
|
||||||
|
@TempDir Path outsideRepository;
|
||||||
|
|
||||||
|
@Test void installedJarsProvideAllReferencesAndCompileTheExampleOutsideTheCheckout() throws Exception {
|
||||||
|
Path distribution = Path.of(System.getProperty("studio.test.distribution"));
|
||||||
|
String jars;
|
||||||
|
try (var files = Files.list(distribution.resolve("lib"))) {
|
||||||
|
jars = files.filter(file -> file.toString().endsWith(".jar")).sorted()
|
||||||
|
.map(Path::toString).collect(Collectors.joining(java.io.File.pathSeparator));
|
||||||
|
}
|
||||||
|
String probeClasses = Path.of(InstalledKnowledgeProbe.class.getProtectionDomain().getCodeSource().getLocation().toURI()).toString();
|
||||||
|
Path output = outsideRepository.resolve("probe.log");
|
||||||
|
var process = new ProcessBuilder(Path.of(System.getProperty("java.home"), "bin", "java").toString(),
|
||||||
|
"-cp", jars + java.io.File.pathSeparator + probeClasses, InstalledKnowledgeProbe.class.getName())
|
||||||
|
.directory(outsideRepository.toFile()).redirectErrorStream(true).redirectOutput(output.toFile()).start();
|
||||||
|
try {
|
||||||
|
assertTrue(process.waitFor(45, TimeUnit.SECONDS), "Installed knowledge probe timed out");
|
||||||
|
String log = Files.readString(output);
|
||||||
|
assertEquals(0, process.exitValue(), log);
|
||||||
|
assertTrue(log.contains("references; example compiled"), log);
|
||||||
|
} finally { if (process.isAlive()) process.destroyForcibly().waitFor(); }
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,143 @@
|
|||||||
|
package p.studio.app.compiler;
|
||||||
|
|
||||||
|
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||||
|
import org.junit.jupiter.api.Test;
|
||||||
|
import org.junit.jupiter.api.io.TempDir;
|
||||||
|
import p.studio.api.automation.*;
|
||||||
|
import p.studio.compiler.FrontendRegistryService;
|
||||||
|
import p.studio.compiler.models.FrontendSpec;
|
||||||
|
import p.studio.compiler.services.*;
|
||||||
|
import p.studio.core.project.*;
|
||||||
|
import p.studio.utilities.structures.ReadOnlySet;
|
||||||
|
|
||||||
|
import java.nio.file.Files;
|
||||||
|
import java.nio.file.Path;
|
||||||
|
import java.util.*;
|
||||||
|
import static org.junit.jupiter.api.Assertions.*;
|
||||||
|
|
||||||
|
class ProjectReferencesTest {
|
||||||
|
@TempDir Path root;
|
||||||
|
final ObjectMapper mapper = new ObjectMapper();
|
||||||
|
final ProjectAccessRegistry access = new ProjectAccessRegistry();
|
||||||
|
|
||||||
|
CompilerProjectKnowledge knowledge(String language, String sdk) throws Exception {
|
||||||
|
writeManifest(language, sdk);
|
||||||
|
access.register(root, () -> { throw new AssertionError("Read needs no grant"); }, () -> true);
|
||||||
|
return new CompilerProjectKnowledge(access, new ProjectCatalogService(root, mapper, new ProjectLocalStudioSetupStore(mapper)), mapper);
|
||||||
|
}
|
||||||
|
|
||||||
|
void writeManifest(String language, String sdk) throws Exception {
|
||||||
|
mapper.writeValue(root.resolve("prometeu.json").toFile(), Map.of("name", "References", "version", "1.0.0",
|
||||||
|
"language", language, "stdlib", sdk, "target", "Game", "dependencies", List.of()));
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test void pbsReferenceKindsAndVersionsMatchTheInstalledProviderAndPaginateExactly() throws Exception {
|
||||||
|
var knowledge = knowledge("pbs", "1");
|
||||||
|
var context = knowledge.context(root);
|
||||||
|
var kinds = context.references().stream().map(ProjectKnowledge.Reference::kind).collect(java.util.stream.Collectors.toSet());
|
||||||
|
assertTrue(kinds.containsAll(List.of(ProjectKnowledge.ReferenceKind.QUICKSTART, ProjectKnowledge.ReferenceKind.SYNTAX,
|
||||||
|
ProjectKnowledge.ReferenceKind.SEMANTICS, ProjectKnowledge.ReferenceKind.EXAMPLE, ProjectKnowledge.ReferenceKind.SDK)));
|
||||||
|
assertEquals(context.references().size(), context.languages().getFirst().referenceIds().size());
|
||||||
|
for (var reference : context.references()) {
|
||||||
|
assertEquals("pbs", reference.owner());
|
||||||
|
if (reference.kind() == ProjectKnowledge.ReferenceKind.SDK || reference.kind() == ProjectKnowledge.ReferenceKind.EXAMPLE) {
|
||||||
|
assertEquals("1", reference.version().value());
|
||||||
|
assertTrue(reference.version().source().contains("SDK"));
|
||||||
|
} else assertFalse(reference.version().known());
|
||||||
|
}
|
||||||
|
var sdk = "pbs:sdk:game:1:sdk/gfx/main.pbs";
|
||||||
|
var expected = FrontendRegistryService.require("pbs").documentation().orElseThrow().references(1, p.studio.AppMode.Game)
|
||||||
|
.stream().filter(ref -> ref.id().equals(sdk)).findFirst().orElseThrow().text();
|
||||||
|
var actual = new StringBuilder();
|
||||||
|
int offset = 0;
|
||||||
|
do {
|
||||||
|
var page = knowledge.readReference(root, sdk, offset, 97);
|
||||||
|
assertEquals(expected.codePointCount(0, expected.length()), page.total());
|
||||||
|
actual.append(page.text());
|
||||||
|
if (page.nextOffset() == null) break;
|
||||||
|
offset = page.nextOffset();
|
||||||
|
} while (true);
|
||||||
|
assertEquals(expected, actual.toString());
|
||||||
|
assertTrue(actual.toString().contains("fn fill_rect"));
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test void unknownAndTraversalLikeIdsNeverReadFilesOrUrls() throws Exception {
|
||||||
|
var knowledge = knowledge("pbs", "1");
|
||||||
|
Path secret = root.resolve("secret.txt");
|
||||||
|
Files.writeString(secret, "not a reference");
|
||||||
|
for (String id : List.of(secret.toString(), secret.toUri().toString(), "../secret.txt", "pbs:../secret.txt",
|
||||||
|
"pbs-knowledge/index.json", "pbs:%2e%2e/secret.txt", "https://example.invalid", "pbs:unknown")) {
|
||||||
|
assertEquals(AutomationFailure.Code.REFERENCE_NOT_FOUND,
|
||||||
|
assertThrows(AutomationException.class, () -> knowledge.readReference(root, id, 0, 100)).failure().code());
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test void changedSdkCannotReadPreviouslyAdvertisedSdkOrExampleIds() throws Exception {
|
||||||
|
var knowledge = knowledge("pbs", "1");
|
||||||
|
assertTrue(knowledge.context(root).languages().getFirst().referenceIds().contains("pbs:sdk:game:1:sdk/gfx/main.pbs"));
|
||||||
|
writeManifest("pbs", "999");
|
||||||
|
var context = knowledge.context(root);
|
||||||
|
assertTrue(context.references().stream().noneMatch(ref -> ref.kind() == ProjectKnowledge.ReferenceKind.SDK || ref.kind() == ProjectKnowledge.ReferenceKind.EXAMPLE));
|
||||||
|
assertTrue(context.limitations().stream().anyMatch(value -> value.contains("no installed SDK reference")));
|
||||||
|
assertEquals(AutomationFailure.Code.REFERENCE_NOT_FOUND, assertThrows(AutomationException.class,
|
||||||
|
() -> knowledge.readReference(root, "pbs:sdk:game:1:sdk/gfx/main.pbs", 0, 100)).failure().code());
|
||||||
|
assertFalse(knowledge.readReference(root, "pbs:quickstart", 0, 100).text().isEmpty());
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test void paginationCountsUnicodeCodePointsAndAcceptsTheExactEnd() throws Exception {
|
||||||
|
String id = "reference-test";
|
||||||
|
register(id, Optional.of((sdk, target) -> List.of(new FrontendReferenceProvider.Reference(
|
||||||
|
id + ":text", "Unicode text", FrontendReferenceProvider.Kind.QUICKSTART, null, "unknown", "A😀é\nB"))));
|
||||||
|
try {
|
||||||
|
var knowledge = knowledge(id, "1");
|
||||||
|
var emoji = knowledge.readReference(root, id + ":text", 1, 1);
|
||||||
|
assertEquals("😀", emoji.text());
|
||||||
|
assertEquals(5, emoji.total());
|
||||||
|
assertEquals(2, emoji.nextOffset());
|
||||||
|
var end = knowledge.readReference(root, id + ":text", 5, 100);
|
||||||
|
assertEquals("", end.text()); assertNull(end.nextOffset());
|
||||||
|
for (int offset : List.of(-1, 6, Integer.MAX_VALUE))
|
||||||
|
assertThrows(IllegalArgumentException.class, () -> knowledge.readReference(root, id + ":text", offset, 1));
|
||||||
|
for (int limit : List.of(0, -1, 16001))
|
||||||
|
assertThrows(IllegalArgumentException.class, () -> knowledge.readReference(root, id + ":text", 0, limit));
|
||||||
|
} finally { FrontendRegistryService.unregister(id); }
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test void missingDocumentationAndUnsupportedLanguageHaveExplicitFailures() throws Exception {
|
||||||
|
String id = "reference-test";
|
||||||
|
register(id, Optional.empty());
|
||||||
|
try {
|
||||||
|
var knowledge = knowledge(id, "1");
|
||||||
|
assertTrue(knowledge.context(root).references().isEmpty());
|
||||||
|
assertEquals(AutomationFailure.Code.DOCUMENTATION_UNAVAILABLE, assertThrows(AutomationException.class,
|
||||||
|
() -> knowledge.readReference(root, id + ":text", 0, 100)).failure().code());
|
||||||
|
writeManifest("uninstalled", "1");
|
||||||
|
assertEquals(AutomationFailure.Code.UNSUPPORTED_LANGUAGE, assertThrows(AutomationException.class,
|
||||||
|
() -> knowledge.readReference(root, id + ":text", 0, 100)).failure().code());
|
||||||
|
} finally { FrontendRegistryService.unregister(id); }
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test void failingProviderDoesNotAdvertiseUnreadableReferences() throws Exception {
|
||||||
|
String id = "reference-test";
|
||||||
|
register(id, Optional.of((sdk, target) -> { throw new IllegalStateException("Missing installed index"); }));
|
||||||
|
try {
|
||||||
|
var knowledge = knowledge(id, "1");
|
||||||
|
var context = knowledge.context(root);
|
||||||
|
assertTrue(context.references().isEmpty());
|
||||||
|
assertTrue(context.limitations().stream().anyMatch(value -> value.contains("cannot load installed references")));
|
||||||
|
assertEquals(AutomationFailure.Code.DOCUMENTATION_UNAVAILABLE, assertThrows(AutomationException.class,
|
||||||
|
() -> knowledge.readReference(root, id + ":text", 0, 100)).failure().code());
|
||||||
|
} finally { FrontendRegistryService.unregister(id); }
|
||||||
|
}
|
||||||
|
|
||||||
|
private void register(String id, Optional<FrontendReferenceProvider> documentation) {
|
||||||
|
FrontendRegistryService.register(new FrontendProvider() {
|
||||||
|
@Override public FrontendSpec specification() {
|
||||||
|
return FrontendSpec.builder().languageId(id).allowedExtensions(ReadOnlySet.from("alt"))
|
||||||
|
.sourceRoots(ReadOnlySet.from("src")).stdlibVersions(List.of(FrontendSpec.Stdlib.of(1))).build();
|
||||||
|
}
|
||||||
|
@Override public FrontendPhaseService compiler() { throw new AssertionError("No compilation for reference reads"); }
|
||||||
|
@Override public Optional<FrontendReferenceProvider> documentation() { return documentation; }
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -129,6 +129,20 @@ class LocalMcpServerTest {
|
|||||||
assertEquals("pbs", value.path("languages").get(0).path("id").asText());
|
assertEquals("pbs", value.path("languages").get(0).path("id").asText());
|
||||||
assertTrue(value.path("languages").get(0).path("languageVersion").path("value").isNull());
|
assertTrue(value.path("languages").get(0).path("languageVersion").path("value").isNull());
|
||||||
assertFalse(java.nio.file.Files.exists(root.resolve(".studio")));
|
assertFalse(java.nio.file.Files.exists(root.resolve(".studio")));
|
||||||
|
var params = (com.fasterxml.jackson.databind.node.ObjectNode) call.path("params");
|
||||||
|
params.put("name", "studio_reference_read");
|
||||||
|
var arguments = (com.fasterxml.jackson.databind.node.ObjectNode) params.path("arguments");
|
||||||
|
arguments.put("referenceId", "pbs:quickstart");
|
||||||
|
var page = mapper.readTree(post(call.toString()).body()).path("result").path("structuredContent").path("value");
|
||||||
|
assertEquals(0, page.path("offset").asInt());
|
||||||
|
assertEquals(4000, page.path("text").asText().codePointCount(0, page.path("text").asText().length()));
|
||||||
|
assertEquals(4000, page.path("nextOffset").asInt());
|
||||||
|
arguments.put("offset", 4294967296L);
|
||||||
|
assertEquals(-32602, mapper.readTree(post(call.toString()).body()).path("error").path("code").asInt());
|
||||||
|
arguments.remove("offset");
|
||||||
|
arguments.put("referenceId", "../../prometeu.json");
|
||||||
|
var missing = mapper.readTree(post(call.toString()).body()).path("result");
|
||||||
|
assertEquals("REFERENCE_NOT_FOUND", missing.path("structuredContent").path("error").path("code").asText());
|
||||||
registration.unsubscribe();
|
registration.unsubscribe();
|
||||||
var denied = mapper.readTree(post(call.toString()).body()).path("result");
|
var denied = mapper.readTree(post(call.toString()).body()).path("result");
|
||||||
assertTrue(denied.path("isError").asBoolean());
|
assertTrue(denied.path("isError").asBoolean());
|
||||||
|
|||||||
@@ -1,3 +1,5 @@
|
|||||||
|
import java.security.MessageDigest
|
||||||
|
|
||||||
plugins {
|
plugins {
|
||||||
id("gradle.java-library-conventions")
|
id("gradle.java-library-conventions")
|
||||||
}
|
}
|
||||||
@@ -8,3 +10,47 @@ dependencies {
|
|||||||
implementation(project(":prometeu-compiler:prometeu-frontend-api"))
|
implementation(project(":prometeu-compiler:prometeu-frontend-api"))
|
||||||
implementation(project(":prometeu-compiler:prometeu-compiler-core"))
|
implementation(project(":prometeu-compiler:prometeu-compiler-core"))
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Build-time projection of owner-maintained sources; no second normative copy is edited by Studio.
|
||||||
|
val pbsSpecs = rootProject.layout.projectDirectory.dir("docs/specs/compiler-languages/pbs")
|
||||||
|
val pbsResources = layout.projectDirectory.dir("src/main/resources")
|
||||||
|
val generatePbsKnowledge by tasks.registering(Sync::class) {
|
||||||
|
from(pbsSpecs) { include("*.md"); into("pbs-knowledge/specs") }
|
||||||
|
into(layout.buildDirectory.dir("generated/pbsKnowledge"))
|
||||||
|
inputs.files(fileTree(pbsResources) { include("game/stdlib/**/*.pbs", "game/stdlib/**/*.barrel", "pbs-knowledge/examples/**") })
|
||||||
|
doLast {
|
||||||
|
val entries = mutableListOf<Map<String, Any?>>()
|
||||||
|
fun entry(id: String, title: String, kind: String, resource: String, source: File, sdk: Int? = null, target: String? = null) {
|
||||||
|
check(source.isFile) { "Missing PBS reference: $source" }
|
||||||
|
val digest = MessageDigest.getInstance("SHA-256").digest(source.readBytes())
|
||||||
|
.joinToString("") { "%02x".format(it) }
|
||||||
|
entries.add(linkedMapOf("id" to id, "title" to title, "kind" to kind,
|
||||||
|
"resource" to resource, "sha256" to digest, "sdkVersion" to sdk, "target" to target))
|
||||||
|
}
|
||||||
|
pbsSpecs.asFile.listFiles()!!.filter { it.extension == "md" }.sortedBy { it.name }.forEach { source ->
|
||||||
|
val chapter = source.name.substringBefore(". ")
|
||||||
|
val quickstart = source.name == "README.md"
|
||||||
|
entry(if (quickstart) "pbs:quickstart" else "pbs:spec:$chapter",
|
||||||
|
source.readLines().first { it.startsWith("# ") }.removePrefix("# "),
|
||||||
|
if (quickstart) "QUICKSTART" else if (chapter == "3") "SYNTAX" else "SEMANTICS",
|
||||||
|
"pbs-knowledge/specs/${source.name}", source)
|
||||||
|
}
|
||||||
|
fileTree(pbsResources) { include("game/stdlib/**/*.pbs", "game/stdlib/**/*.barrel") }.files.sorted().forEach { source ->
|
||||||
|
val path = source.relativeTo(pbsResources.asFile).invariantSeparatorsPath
|
||||||
|
val parts = path.split('/')
|
||||||
|
val sdk = parts[2].toInt()
|
||||||
|
val module = parts.drop(3).dropLast(1).joinToString(":")
|
||||||
|
entry("pbs:sdk:game:$sdk:${parts.drop(3).joinToString("/")}", "PBS SDK $sdk @$module / ${source.name}",
|
||||||
|
"SDK", path, source, sdk, "Game")
|
||||||
|
}
|
||||||
|
fileTree(pbsResources.dir("pbs-knowledge/examples/game/1")) { include("**/*") }.files.sorted().forEach { source ->
|
||||||
|
val relative = source.relativeTo(pbsResources.dir("pbs-knowledge/examples/game/1").asFile).invariantSeparatorsPath
|
||||||
|
entry("pbs:example:game:1:$relative", "PBS Game / SDK 1 example: $relative", "EXAMPLE",
|
||||||
|
"pbs-knowledge/examples/game/1/$relative", source, 1, "Game")
|
||||||
|
}
|
||||||
|
check(entries.map { it["id"] }.distinct().size == entries.size) { "Duplicate PBS reference IDs" }
|
||||||
|
destinationDir.resolve("pbs-knowledge/index.json").writeText(groovy.json.JsonOutput.prettyPrint(
|
||||||
|
groovy.json.JsonOutput.toJson(mapOf("schemaVersion" to 1, "owner" to "pbs", "references" to entries))))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
sourceSets.main { resources.srcDir(generatePbsKnowledge) }
|
||||||
|
|||||||
+7
@@ -11,6 +11,8 @@ import java.util.Optional;
|
|||||||
public final class PBSFrontendProvider implements FrontendProvider {
|
public final class PBSFrontendProvider implements FrontendProvider {
|
||||||
private final FrontendPhaseService compiler;
|
private final FrontendPhaseService compiler;
|
||||||
private final PBSFrontendLanguageService languageService;
|
private final PBSFrontendLanguageService languageService;
|
||||||
|
private final p.studio.compiler.services.FrontendReferenceProvider documentation =
|
||||||
|
new p.studio.compiler.pbs.documentation.PbsDocumentation();
|
||||||
|
|
||||||
public PBSFrontendProvider() {
|
public PBSFrontendProvider() {
|
||||||
this(new PBSFrontendPhaseService(), new PBSFrontendLanguageService());
|
this(new PBSFrontendPhaseService(), new PBSFrontendLanguageService());
|
||||||
@@ -41,4 +43,9 @@ public final class PBSFrontendProvider implements FrontendProvider {
|
|||||||
public Optional<FrontendLanguageService> languageService() {
|
public Optional<FrontendLanguageService> languageService() {
|
||||||
return Optional.of(languageService);
|
return Optional.of(languageService);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public Optional<p.studio.compiler.services.FrontendReferenceProvider> documentation() {
|
||||||
|
return Optional.of(documentation);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
+78
@@ -0,0 +1,78 @@
|
|||||||
|
package p.studio.compiler.pbs.documentation;
|
||||||
|
|
||||||
|
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||||
|
import p.studio.AppMode;
|
||||||
|
import p.studio.compiler.services.FrontendReferenceProvider;
|
||||||
|
|
||||||
|
import java.io.IOException;
|
||||||
|
import java.nio.ByteBuffer;
|
||||||
|
import java.nio.charset.CodingErrorAction;
|
||||||
|
import java.nio.charset.StandardCharsets;
|
||||||
|
import java.security.MessageDigest;
|
||||||
|
import java.util.*;
|
||||||
|
import java.io.InputStream;
|
||||||
|
import java.util.function.Function;
|
||||||
|
|
||||||
|
/** Loads only the build-generated PBS index and its packaged resources, never user-supplied paths. */
|
||||||
|
public final class PbsDocumentation implements FrontendReferenceProvider {
|
||||||
|
private record Index(int schemaVersion, String owner, List<Entry> references) {}
|
||||||
|
private record Entry(String id, String title, Kind kind, String resource, String sha256,
|
||||||
|
Integer sdkVersion, String target) {}
|
||||||
|
private record Installed(Entry entry, Reference reference) {}
|
||||||
|
private final Function<String, InputStream> resources;
|
||||||
|
private volatile List<Installed> installed;
|
||||||
|
|
||||||
|
public PbsDocumentation() { this(PbsDocumentation.class.getClassLoader()::getResourceAsStream); }
|
||||||
|
PbsDocumentation(Function<String, InputStream> resources) { this.resources = Objects.requireNonNull(resources); }
|
||||||
|
|
||||||
|
@Override public List<Reference> references(int stdlibVersion, AppMode target) {
|
||||||
|
Objects.requireNonNull(target);
|
||||||
|
return installed().stream()
|
||||||
|
.filter(item -> item.entry().sdkVersion() == null || item.entry().sdkVersion() == stdlibVersion)
|
||||||
|
.filter(item -> item.entry().target() == null || item.entry().target().equals(target.name()))
|
||||||
|
.map(Installed::reference).toList();
|
||||||
|
}
|
||||||
|
|
||||||
|
private List<Installed> installed() {
|
||||||
|
if (installed != null) return installed;
|
||||||
|
synchronized (this) {
|
||||||
|
if (installed != null) return installed;
|
||||||
|
try {
|
||||||
|
var index = new ObjectMapper().readValue(resource("pbs-knowledge/index.json"), Index.class);
|
||||||
|
if (index.schemaVersion() != 1 || !"pbs".equals(index.owner()) || index.references() == null)
|
||||||
|
throw new IOException("Invalid PBS reference index");
|
||||||
|
var entries = new ArrayList<Installed>();
|
||||||
|
var ids = new HashSet<String>();
|
||||||
|
for (var entry : index.references()) {
|
||||||
|
if (entry.id() == null || !entry.id().startsWith("pbs:") || !ids.add(entry.id())
|
||||||
|
|| entry.sdkVersion() != null && entry.sdkVersion() < 1)
|
||||||
|
throw new IOException("Invalid or duplicate PBS reference identity");
|
||||||
|
byte[] bytes = resource(entry.resource());
|
||||||
|
if (!HexFormat.of().formatHex(MessageDigest.getInstance("SHA-256").digest(bytes)).equals(entry.sha256()))
|
||||||
|
throw new IOException("PBS reference does not match its installed index");
|
||||||
|
String content = StandardCharsets.UTF_8.newDecoder().onMalformedInput(CodingErrorAction.REPORT)
|
||||||
|
.onUnmappableCharacter(CodingErrorAction.REPORT).decode(ByteBuffer.wrap(bytes)).toString();
|
||||||
|
entries.add(new Installed(entry, new Reference(entry.id(), entry.title(), entry.kind(),
|
||||||
|
entry.sdkVersion() == null ? null : entry.sdkVersion().toString(),
|
||||||
|
entry.sdkVersion() == null ? "unknown" : "installed PBS SDK major; target=" + entry.target(), content)));
|
||||||
|
}
|
||||||
|
installed = List.copyOf(entries);
|
||||||
|
return installed;
|
||||||
|
} catch (Exception failure) {
|
||||||
|
throw new IllegalStateException("Installed PBS documentation is unavailable", failure);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private byte[] resource(String path) throws IOException {
|
||||||
|
if (path == null || path.contains("..") || path.contains(":") || path.contains("\\")
|
||||||
|
|| !(path.startsWith("pbs-knowledge/") || path.startsWith("game/stdlib/")))
|
||||||
|
throw new IOException("Invalid PBS reference resource");
|
||||||
|
try (var stream = resources.apply(path)) {
|
||||||
|
if (stream == null) throw new IOException("Missing PBS reference resource");
|
||||||
|
byte[] bytes = stream.readNBytes(4 * 1024 * 1024 + 1);
|
||||||
|
if (bytes.length > 4 * 1024 * 1024) throw new IOException("PBS reference resource is too large");
|
||||||
|
return bytes;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
+8
@@ -0,0 +1,8 @@
|
|||||||
|
{
|
||||||
|
"name": "pbs-reference-example",
|
||||||
|
"version": "1.0.0",
|
||||||
|
"language": "pbs",
|
||||||
|
"stdlib": "1",
|
||||||
|
"target": "Game",
|
||||||
|
"dependencies": []
|
||||||
|
}
|
||||||
+19
@@ -0,0 +1,19 @@
|
|||||||
|
import { Color } from @core:color;
|
||||||
|
import { Gfx } from @sdk:gfx;
|
||||||
|
|
||||||
|
declare global x: int = 0;
|
||||||
|
|
||||||
|
[Init]
|
||||||
|
fn init() -> void {
|
||||||
|
x = 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
[Frame]
|
||||||
|
fn frame() -> void {
|
||||||
|
Gfx.clear(Color.rgb(16, 20, 28));
|
||||||
|
Gfx.fill_rect(x, 32, 16, 16, Color.rgb(240, 160, 48));
|
||||||
|
x = x + 1;
|
||||||
|
if (x > 120) {
|
||||||
|
x = 0;
|
||||||
|
}
|
||||||
|
}
|
||||||
+2
@@ -0,0 +1,2 @@
|
|||||||
|
pub fn init() -> void;
|
||||||
|
pub fn frame() -> void;
|
||||||
+60
@@ -0,0 +1,60 @@
|
|||||||
|
package p.studio.compiler.pbs.documentation;
|
||||||
|
|
||||||
|
import org.junit.jupiter.api.Test;
|
||||||
|
import p.studio.AppMode;
|
||||||
|
import p.studio.compiler.services.FrontendReferenceProvider;
|
||||||
|
import java.io.ByteArrayInputStream;
|
||||||
|
import java.nio.charset.StandardCharsets;
|
||||||
|
import java.util.*;
|
||||||
|
import static org.junit.jupiter.api.Assertions.*;
|
||||||
|
|
||||||
|
class PbsDocumentationTest {
|
||||||
|
@Test void installedIndexHasAllKindsUniqueIdsAndExactSdkSources() throws Exception {
|
||||||
|
var references = new PbsDocumentation().references(1, AppMode.Game);
|
||||||
|
assertEquals(references.size(), references.stream().map(FrontendReferenceProvider.Reference::id).distinct().count());
|
||||||
|
for (var kind : List.of(FrontendReferenceProvider.Kind.QUICKSTART, FrontendReferenceProvider.Kind.SYNTAX,
|
||||||
|
FrontendReferenceProvider.Kind.SEMANTICS, FrontendReferenceProvider.Kind.EXAMPLE, FrontendReferenceProvider.Kind.SDK))
|
||||||
|
assertTrue(references.stream().anyMatch(ref -> ref.kind() == kind));
|
||||||
|
for (var reference : references) {
|
||||||
|
assertFalse(reference.text().isBlank());
|
||||||
|
if (reference.kind() != FrontendReferenceProvider.Kind.SDK) continue;
|
||||||
|
try (var stream = getClass().getClassLoader().getResourceAsStream("game/stdlib/1/" + reference.id().substring("pbs:sdk:game:1:".length()))) {
|
||||||
|
assertNotNull(stream);
|
||||||
|
assertEquals(new String(stream.readAllBytes(), StandardCharsets.UTF_8), reference.text());
|
||||||
|
assertEquals("1", reference.version());
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test void unsupportedSdkOrTargetDoesNotBorrowAnotherSdkOrExample() {
|
||||||
|
var docs = new PbsDocumentation();
|
||||||
|
for (var selection : List.of(docs.references(999, AppMode.Game), docs.references(1, AppMode.Shell))) {
|
||||||
|
assertFalse(selection.isEmpty());
|
||||||
|
assertTrue(selection.stream().noneMatch(ref -> ref.kind() == FrontendReferenceProvider.Kind.SDK || ref.kind() == FrontendReferenceProvider.Kind.EXAMPLE));
|
||||||
|
assertTrue(selection.stream().allMatch(ref -> ref.version() == null && ref.versionSource().equals("unknown")));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test void missingIndexAndCorruptedContentFailExplicitly() {
|
||||||
|
assertThrows(IllegalStateException.class, () -> new PbsDocumentation(path -> null).references(1, AppMode.Game));
|
||||||
|
var corrupt = new PbsDocumentation(path -> path.equals("pbs-knowledge/index.json")
|
||||||
|
? getClass().getClassLoader().getResourceAsStream(path)
|
||||||
|
: new ByteArrayInputStream("corrupted resource".getBytes(StandardCharsets.UTF_8)));
|
||||||
|
assertThrows(IllegalStateException.class, () -> corrupt.references(1, AppMode.Game));
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test void invalidIndexResourceCannotReachTheResourceLoader() {
|
||||||
|
var opened = new ArrayList<String>();
|
||||||
|
String invalidIndex = """
|
||||||
|
{"schemaVersion":1,"owner":"pbs","references":[
|
||||||
|
{"id":"pbs:bad","title":"Bad","kind":"SYNTAX","resource":"../private.txt","sha256":"invalid"}
|
||||||
|
]}
|
||||||
|
""";
|
||||||
|
var docs = new PbsDocumentation(path -> {
|
||||||
|
opened.add(path);
|
||||||
|
return new ByteArrayInputStream(invalidIndex.getBytes(StandardCharsets.UTF_8));
|
||||||
|
});
|
||||||
|
assertThrows(IllegalStateException.class, () -> docs.references(1, AppMode.Game));
|
||||||
|
assertEquals(List.of("pbs-knowledge/index.json"), opened);
|
||||||
|
}
|
||||||
|
}
|
||||||
+4
@@ -12,4 +12,8 @@ public interface FrontendProvider {
|
|||||||
default Optional<FrontendLanguageService> languageService() {
|
default Optional<FrontendLanguageService> languageService() {
|
||||||
return Optional.empty();
|
return Optional.empty();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
default Optional<FrontendReferenceProvider> documentation() {
|
||||||
|
return Optional.empty();
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
+21
@@ -0,0 +1,21 @@
|
|||||||
|
package p.studio.compiler.services;
|
||||||
|
|
||||||
|
import p.studio.AppMode;
|
||||||
|
import java.util.List;
|
||||||
|
import java.util.Objects;
|
||||||
|
|
||||||
|
/** Owner-maintained, installed reference texts. This is not a filesystem or network reader. */
|
||||||
|
public interface FrontendReferenceProvider {
|
||||||
|
List<Reference> references(int stdlibVersion, AppMode target);
|
||||||
|
|
||||||
|
enum Kind { QUICKSTART, SYNTAX, SEMANTICS, EXAMPLE, SDK, LIBRARY }
|
||||||
|
|
||||||
|
record Reference(String id, String title, Kind kind, String version, String versionSource, String text) {
|
||||||
|
public Reference {
|
||||||
|
Objects.requireNonNull(id); Objects.requireNonNull(title); Objects.requireNonNull(kind);
|
||||||
|
Objects.requireNonNull(versionSource); Objects.requireNonNull(text);
|
||||||
|
if (id.isBlank() || title.isBlank() || versionSource.isBlank() || version != null && version.isBlank())
|
||||||
|
throw new IllegalArgumentException("Blank reference metadata");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
Reference in New Issue
Block a user