Skip to content

Commit 0d242c9

Browse files
committed
fix(world): preserve structured spawn hierarchy
1 parent f8ee994 commit 0d242c9

15 files changed

Lines changed: 423 additions & 71 deletions

Assets/CoreAI/CHANGELOG.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,12 @@
22

33
## [Unreleased]
44

5+
- **Structured world spawning.** `world_command` `spawn`, `spawn_batch`, and `change` expose
6+
`worldPositionStays` (default `false`). Parented transforms are local by default; callers can pass `true`
7+
to preserve world space. The tool contract now recommends named `empty` roots and meaningful child
8+
hierarchies for compound objects. The visual benchmark executor now preserves those parent relationships
9+
in generated scene prefabs instead of flattening every spawned object under the benchmark root.
10+
511
## 5.5.0 - R6 resilience, benchmark v2 tooling, CI/package gates (2026-07-11)
612

713
- **Benchmark: model-authored castles export as prefabs.** Every G6 free-build run now saves the built scene

Assets/CoreAI/Docs/LLM_TOOLS.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -30,14 +30,14 @@ agent.WithTool(new ComponentLlmTool(componentExecutor, settings, logger));
3030
| `inventory` | `InventoryLlmTool` | `IInventoryProvider` | Game inventory read/grant. |
3131
| compatibility | `CompatibilityLlmTool` | `CompatibilityChecker` | Crafting compatibility. |
3232
| game config | `GameConfigLlmTool` | `GameConfigPolicy.CreateLlmTool(store, roleId)` | Per-role config slots. |
33-
| `world_command` | `WorldLlmTool` | `ICoreAiWorldCommandExecutor` | Native world edits. `spawn` accepts registered prefab keys or built-in primitives (`cube`, `sphere`, `cylinder`, `capsule`, `plane`, `empty`) when `AllowWorldPrimitives` is on; spawn can ALSO set rotation (`fx/fy/fz` degrees) and uniform `scale` in one call. Models can also move, rotate, scale, and parent objects. Models usually edit the world via Lua `coreai_world_*` instead, so this is optional. |
33+
| `world_command` | `WorldLlmTool` | `ICoreAiWorldCommandExecutor` | Native world edits. `spawn` accepts registered prefab keys or built-in primitives (`cube`, `sphere`, `cylinder`, `capsule`, `plane`, `empty`) when `AllowWorldPrimitives` is on; spawn can set transform and parent in one call. Parented coordinates are local by default; `worldPositionStays=true` preserves world space. Models usually edit the world via Lua `coreai_world_*` instead, so this is optional. |
3434
| `component_command` | `ComponentLlmTool` | `ICoreAiComponentCommandExecutor` | Add/remove/configure supported Unity components through a curated catalog, no reflection. |
3535
| scene query | `SceneLlmTool` | scene | `unity_*`-style scene inspection as a native tool (alternative to Full-tier Lua). |
3636
| `capture_camera` | `CameraLlmTool` | a `Camera` + a **vision-capable model** | Renders a camera to JPEG. Attach the captured `DataContent` (`CameraLlmTool.CaptureCameraImageContent`) to a user message; `MeaiOpenAiChatClient` serializes it to OpenAI `image_url`. See **Vision / multimodal** below for the host send path and autonomous-tool wiring. |
3737

3838
### World and component commands
3939

40-
`world_command` now works without a prefab registry for simple spawn requests when `ICoreAISettings.AllowWorldPrimitives` is enabled (default). `prefabKey` may be a registered prefab key or one of the built-in primitive keys: `cube`, `sphere`, `cylinder`, `capsule`, `plane`, `empty`. During spawn you can ALSO set rotation (fx/fy/fz degrees) and uniform scale in one call, or use separate rotate/set_scale actions later. The action list includes moving, rotating, scaling, and parenting objects.
40+
`world_command` works without a prefab registry for simple spawn requests when `ICoreAISettings.AllowWorldPrimitives` is enabled (default). `prefabKey` may be a registered prefab key or one of the built-in primitive keys: `cube`, `sphere`, `cylinder`, `capsule`, `plane`, `empty`. During spawn, position, rotation, scale and parent can be supplied in one call. With a parent, coordinates are local by default (`worldPositionStays=false`); set it to `true` to preserve a world transform. For compound objects, create an `empty` root first and parent the named parts beneath it instead of leaving a flat scene hierarchy.
4141

4242
`component_command` is separate from `world_command` and uses a curated catalog instead of reflection. Actions are `add`, `remove`, `set`, and `list_components`. Supported `componentType` values are `rigidbody`, `rigidbody2d`, `boxcollider`, `spherecollider`, `capsulecollider`, `meshcollider`, `light`, `audiosource`, `camera`, `linerenderer`, `trailrenderer`, `textmesh`, `meshrenderer`, and `particlesystem`. For `set`, pass `propertyName` plus the matching value field: `floatValue`, `boolValue` (`0` / `1`), `stringValue`, or `x` / `y` / `z`.
4343

Assets/CoreAI/Docs/TOOL_CALLING_BEST_PRACTICES.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -241,6 +241,10 @@ Guidelines that follow from this:
241241
`list_prefabs` lets the model learn valid `prefabKey` values before spawning,
242242
the same way `list_animations` and `list_objects` let it learn valid
243243
`animationName`/`targetName` values.
244+
- **Spawn meaningful hierarchies for compound objects.** Create a named `empty` root before its parts,
245+
then set the parent on posts, roofs, props and decorations. Parented coordinates are local by default;
246+
use `worldPositionStays=true` only when an existing world transform must be preserved. This costs no
247+
extra parent call when the parent is supplied during `spawn` or per `spawn_batch` item.
244248

245249
## Verification Checklist
246250

Assets/CoreAIBenchmark/Tests/PlayMode/Benchmarks/GameCreationBenchmarkHarness.cs

Lines changed: 73 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -399,6 +399,7 @@ public sealed class RecordedWorldCommand
399399
public float FloatValue;
400400
public float Fx, Fy, Fz;
401401
public float ScaleX, ScaleY, ScaleZ;
402+
public bool WorldPositionStays;
402403
public bool HasPosition, HasRotation, HasScale;
403404
public bool HasX, HasY, HasZ;
404405
public bool HasFx, HasFy, HasFz;
@@ -446,6 +447,7 @@ public bool TryExecute(ApplyAiGameCommand cmd)
446447
ScaleX = env.scaleX,
447448
ScaleY = env.scaleY,
448449
ScaleZ = env.scaleZ,
450+
WorldPositionStays = env.worldPositionStays,
449451
HasPosition = env.hasPosition,
450452
HasRotation = env.hasRotation,
451453
HasScale = env.hasScale,
@@ -627,8 +629,9 @@ protected override void OnCommand(RecordedWorldCommand cmd)
627629
bool expected = ExpectedNames.Count == 0 || ExpectedNames.Contains(key);
628630
UnityEngine.GameObject go = BuildVisual(
629631
key, cmd.PrefabKeyOrName, new UnityEngine.Vector3(cmd.X, cmd.Y, cmd.Z),
630-
expected, false);
631-
ApplyInlineTransform(go, cmd);
632+
expected, false, cmd.StringValue, cmd.WorldPositionStays);
633+
ApplyInlineTransform(go, cmd, !string.IsNullOrWhiteSpace(cmd.StringValue)
634+
&& !cmd.WorldPositionStays);
632635
_objects[key] = go;
633636
}
634637
}
@@ -662,15 +665,18 @@ protected override void OnCommand(RecordedWorldCommand cmd)
662665
else if (action == "change"
663666
&& _objects.TryGetValue(name, out UnityEngine.GameObject ch) && ch != null)
664667
{
665-
// 'change' is the current unified update action (position/rotation/scale/parent);
666-
// without this the scene screenshot silently showed the object's original spawn
667-
// transform even when the model correctly repositioned/rescaled/rotated it afterward.
668-
if (cmd.X != 0f || cmd.Y != 0f || cmd.Z != 0f)
668+
bool hasParent = TryResolveParent(cmd.StringValue, out UnityEngine.Transform parent);
669+
bool localTransform = hasParent && !cmd.WorldPositionStays;
670+
if (localTransform)
669671
{
670-
ch.transform.position = new UnityEngine.Vector3(cmd.X, cmd.Y, cmd.Z);
672+
ch.transform.SetParent(parent, false);
671673
}
672674

673-
ApplyInlineTransform(ch, cmd);
675+
ApplyInlineTransform(ch, cmd, localTransform);
676+
if (hasParent && cmd.WorldPositionStays)
677+
{
678+
ch.transform.SetParent(parent, true);
679+
}
674680
}
675681
else if (action == "set_color"
676682
&& _objects.TryGetValue(name, out UnityEngine.GameObject col) && col != null
@@ -697,16 +703,38 @@ protected override void OnCommand(RecordedWorldCommand cmd)
697703
}
698704
}
699705

700-
private static void ApplyInlineTransform(UnityEngine.GameObject go, RecordedWorldCommand cmd)
706+
private static void ApplyInlineTransform(
707+
UnityEngine.GameObject go, RecordedWorldCommand cmd, bool localTransform)
701708
{
702709
if (go == null || cmd == null)
703710
{
704711
return;
705712
}
706713

707-
if (cmd.Fx != 0f || cmd.Fy != 0f || cmd.Fz != 0f)
714+
if (cmd.HasPosition || cmd.X != 0f || cmd.Y != 0f || cmd.Z != 0f)
708715
{
709-
go.transform.rotation = UnityEngine.Quaternion.Euler(cmd.Fx, cmd.Fy, cmd.Fz);
716+
UnityEngine.Vector3 position = new(cmd.X, cmd.Y, cmd.Z);
717+
if (localTransform)
718+
{
719+
go.transform.localPosition = position;
720+
}
721+
else
722+
{
723+
go.transform.position = position;
724+
}
725+
}
726+
727+
if (cmd.HasRotation || cmd.Fx != 0f || cmd.Fy != 0f || cmd.Fz != 0f)
728+
{
729+
UnityEngine.Quaternion rotation = UnityEngine.Quaternion.Euler(cmd.Fx, cmd.Fy, cmd.Fz);
730+
if (localTransform)
731+
{
732+
go.transform.localRotation = rotation;
733+
}
734+
else
735+
{
736+
go.transform.rotation = rotation;
737+
}
710738
}
711739

712740
if (cmd.FloatValue > 0f || cmd.ScaleX > 0f || cmd.ScaleY > 0f || cmd.ScaleZ > 0f)
@@ -781,19 +809,35 @@ private static float Safe(float v)
781809
}
782810

783811
private UnityEngine.GameObject BuildVisual(
784-
string key, string prefabKey, UnityEngine.Vector3 pos, bool expected, bool ghost)
812+
string key, string prefabKey, UnityEngine.Vector3 pos, bool expected, bool ghost,
813+
string parentName = "", bool worldPositionStays = false)
785814
{
786815
(UnityEngine.PrimitiveType prim, UnityEngine.Vector3 scale) = ShapeFor(prefabKey);
787-
UnityEngine.GameObject go = UnityEngine.GameObject.CreatePrimitive(prim);
816+
bool isEmpty = string.Equals(prefabKey?.Trim(), "empty", StringComparison.OrdinalIgnoreCase);
817+
if (isEmpty)
818+
{
819+
scale = UnityEngine.Vector3.one;
820+
}
821+
UnityEngine.GameObject go = isEmpty
822+
? new UnityEngine.GameObject()
823+
: UnityEngine.GameObject.CreatePrimitive(prim);
788824
go.name = ghost ? $"ghost:{key}" : key;
789825
UnityEngine.Collider col = go.GetComponent<UnityEngine.Collider>();
790826
if (col != null)
791827
{
792828
UnityEngine.Object.DestroyImmediate(col);
793829
}
794830

795-
go.transform.SetParent(Root, false);
796-
go.transform.position = pos;
831+
bool hasParent = TryResolveParent(parentName, out UnityEngine.Transform parent);
832+
go.transform.SetParent(hasParent ? parent : Root, hasParent && worldPositionStays);
833+
if (hasParent && !worldPositionStays)
834+
{
835+
go.transform.localPosition = pos;
836+
}
837+
else
838+
{
839+
go.transform.position = pos;
840+
}
797841
go.transform.localScale = scale;
798842

799843
UnityEngine.Color objColor =
@@ -837,6 +881,20 @@ private UnityEngine.GameObject BuildVisual(
837881
return go;
838882
}
839883

884+
private bool TryResolveParent(string parentName, out UnityEngine.Transform parent)
885+
{
886+
parent = null;
887+
if (string.IsNullOrWhiteSpace(parentName)
888+
|| !_objects.TryGetValue(parentName.Trim(), out UnityEngine.GameObject parentObject)
889+
|| parentObject == null)
890+
{
891+
return false;
892+
}
893+
894+
parent = parentObject.transform;
895+
return true;
896+
}
897+
840898
/// <summary>Adds a faint placeholder for every expected object the model never spawned, so the
841899
/// picture shows what is MISSING. Cosmetic, runs after grading.</summary>
842900
public void AddMissingGhosts()
Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
#if !COREAI_NO_LUA
2+
#if !COREAI_NO_LLM && !UNITY_WEBGL
3+
using CoreAI.Ai;
4+
using CoreAI.Infrastructure.World;
5+
using CoreAI.Messaging;
6+
using Newtonsoft.Json;
7+
using NUnit.Framework;
8+
using UnityEngine;
9+
using static CoreAI.Tests.PlayMode.Benchmarks.GameCreationBenchmarkHarness;
10+
11+
namespace CoreAI.Tests.PlayMode.Benchmarks
12+
{
13+
public sealed class VisualBenchmarkWorldExecutorPlayModeTests
14+
{
15+
[Test]
16+
public void SpawnAndChange_HonorParentCoordinateSpace()
17+
{
18+
VisualBenchmarkWorldExecutor executor = new() { HideLabels = true };
19+
try
20+
{
21+
Execute(executor, Spawn("group", "empty", 10f, 0f, 0f));
22+
Execute(executor, Spawn("local", "cube", 1f, 2f, 3f, "group"));
23+
Execute(executor, Spawn("world", "cube", 2f, 3f, 4f, "group", true));
24+
25+
Transform group = executor.Root.Find("group");
26+
Transform local = group.Find("local");
27+
Transform world = group.Find("world");
28+
Assert.That(local.localPosition, Is.EqualTo(new Vector3(1f, 2f, 3f)));
29+
Assert.That(world.position, Is.EqualTo(new Vector3(2f, 3f, 4f)));
30+
31+
CoreAiWorldCommandEnvelope change = new()
32+
{
33+
action = "change",
34+
targetName = "world",
35+
stringValue = "group",
36+
x = 4f,
37+
y = 5f,
38+
z = 6f,
39+
hasPosition = true
40+
};
41+
Execute(executor, change);
42+
43+
Assert.That(world.localPosition, Is.EqualTo(new Vector3(4f, 5f, 6f)));
44+
}
45+
finally
46+
{
47+
executor.Cleanup();
48+
}
49+
}
50+
51+
private static CoreAiWorldCommandEnvelope Spawn(
52+
string name, string prefab, float x, float y, float z,
53+
string parent = "", bool worldPositionStays = false)
54+
{
55+
return new CoreAiWorldCommandEnvelope
56+
{
57+
action = "spawn",
58+
targetName = name,
59+
prefabKeyOrName = prefab,
60+
stringValue = parent,
61+
worldPositionStays = worldPositionStays,
62+
x = x,
63+
y = y,
64+
z = z,
65+
hasPosition = true
66+
};
67+
}
68+
69+
private static void Execute(
70+
VisualBenchmarkWorldExecutor executor, CoreAiWorldCommandEnvelope envelope)
71+
{
72+
Assert.IsTrue(executor.TryExecute(new ApplyAiGameCommand
73+
{
74+
CommandTypeId = AiGameCommandTypeIds.WorldCommand,
75+
JsonPayload = JsonConvert.SerializeObject(envelope)
76+
}));
77+
}
78+
}
79+
}
80+
#endif
81+
#endif

Assets/CoreAIBenchmark/Tests/PlayMode/Benchmarks/VisualBenchmarkWorldExecutorPlayModeTests.cs.meta

Lines changed: 2 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

Assets/CoreAiUnity/CHANGELOG.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,11 @@ Unity host: **CoreAI.Source** build, EditMode / PlayMode tests, Editor menus, do
44

55
## [Unreleased]
66

7+
- **Structured world spawning.** Added `worldPositionStays` to `world_command` `spawn`, `spawn_batch`, and
8+
`change`. The default is `false`, so parented position/rotation values are local; `true` preserves the
9+
world transform. Batch items can override the call-level default. Tool guidance now recommends named
10+
`empty` roots for wells, stalls, buildings, and other compound objects.
11+
712
## 5.5.0 - R6 resilience, benchmark v2 tooling, CI/package gates (2026-07-11)
813

914
- **R6 resilience wiring.** `LlmPipelineInstaller` now composes `Timeout( Logging( RetryingStreaming( routed )))`,

Assets/CoreAiUnity/Docs/JSON_COMMAND_FORMAT.md

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -165,7 +165,14 @@ unsupported failure when `SecureLuaEnvironment.IsSupported` is `false`.
165165
}
166166
```
167167

168-
`prefabKey` and `targetName` are required. Position, rotation (`fx/fy/fz`), uniform `scale`, per-axis `scaleX/scaleY/scaleZ`, and parent (`stringValue`) are optional. `scale` is a uniform fallback; use per-axis scale for meter-accurate dimensions.
168+
`prefabKey` and `targetName` are required. Position, rotation (`fx/fy/fz`), uniform `scale`, per-axis
169+
`scaleX/scaleY/scaleZ`, parent (`stringValue`), and `worldPositionStays` are optional. When a parent is set,
170+
coordinates are local by default (`worldPositionStays: false`); `true` preserves world space. Without a
171+
parent, local and world coordinates are identical. `scale` is a uniform fallback; use per-axis scale for
172+
meter-accurate dimensions.
173+
174+
For compound objects, create a named `empty` root first and parent related pieces beneath it instead of
175+
leaving every piece at scene root.
169176

170177
`prefabKey` may be a registered prefab key or a built-in primitive key (`cube`, `sphere`, `cylinder`, `capsule`, `plane`, `empty`) when `AllowWorldPrimitives` is enabled. Registered prefabs take precedence.
171178

Assets/CoreAiUnity/Docs/TOOL_CALL_SPEC.md

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -229,15 +229,16 @@ Since **v1.5.0**, `ToolExecutionPolicy`, `SmartToolCallingChatClient`, `LoggingL
229229
| `scale` | float | Optional uniform scale fallback. |
230230
| `scaleX`, `scaleY`, `scaleZ` | float | Optional per-axis scale for meter-accurate dimensions. |
231231
| `stringValue` | string | Scene name, search query, parent name, HTML colour, or clip key depending on action. |
232+
| `worldPositionStays` | bool | Parenting policy for `spawn`/`change`. Default `false`: coordinates are local to the named parent. `true`: preserve world transform. |
232233
| `animationName` | string | Animation name. |
233234
| `textToDisplay` | string | Text to display. |
234235
| `volume` | float | Audio volume. |
235236

236237
**Examples:**
237238

238239
```json
239-
// Spawn with optional position, rotation, non-uniform scale, and parent
240-
{"name": "world_command", "arguments": {"action": "spawn", "prefabKey": "cube", "targetName": "wall_1", "x": 0, "y": 1, "z": 0, "fy": 90, "scaleX": 8, "scaleY": 2, "scaleZ": 0.5, "stringValue": "CastleRoot"}}
240+
// Spawn with optional local position, rotation, non-uniform scale, and parent
241+
{"name": "world_command", "arguments": {"action": "spawn", "prefabKey": "cube", "targetName": "wall_1", "x": 0, "y": 1, "z": 0, "fy": 90, "scaleX": 8, "scaleY": 2, "scaleZ": 0.5, "stringValue": "CastleRoot", "worldPositionStays": false}}
241242

242243
// Change only supplied fields
243244
{"name": "world_command", "arguments": {"action": "change", "targetName": "wall_1", "x": 2, "fy": 180, "scaleY": 3, "stringValue": "none"}}
@@ -256,6 +257,11 @@ Since **v1.5.0**, `ToolExecutionPolicy`, `SmartToolCallingChatClient`, `LoggingL
256257
```
257258

258259
**When to use:** Creator / Designer AI that dynamically drives the world. For scene inspection and instance-level scene operations, use `scene_tool`; it is separate from `world_command`.
260+
261+
For compound spawned objects, create a named `empty` root first and parent related pieces beneath it. Use
262+
local coordinates (the default) for parts such as posts, roofs and props; use `worldPositionStays: true`
263+
only when attaching an already world-positioned object. This keeps the hierarchy meaningful and lets the
264+
whole object move as one unit.
259265
### 5. Component command tool
260266

261267
**Purpose:** Add, remove, configure, and list Unity components on existing GameObjects through a curated catalog. This tool does not use reflection.

0 commit comments

Comments
 (0)