Skip to content

fix(build-agent): stop hardcoding cloud host in runtime remediation#13

Open
bekossy wants to merge 5 commits into
mainfrom
fix/remediation-host
Open

fix(build-agent): stop hardcoding cloud host in runtime remediation#13
bekossy wants to merge 5 commits into
mainfrom
fix/remediation-host

Conversation

@bekossy

@bekossy bekossy commented Jul 6, 2026

Copy link
Copy Markdown
Member

Context

SKILL.md's "First run: credentials" section uses https://cloud.agenta.ai as the example for initial API-key setup. When a later runtime failure calls for telling the user to go add something to their Agenta project (for example, a missing provider key in the vault), that same cloud URL was getting reused out of context instead of the user's actual configured $AGENTA_HOST, sending a self-hosted user to look in the wrong place entirely. Reproduced 3 out of 3 times against a self-hosted instance (bighetzner) during QA testing: every time a runtime remediation message came up, it referenced cloud.agenta.ai instead of the project's real host.

To be precise about what is proven here: the symptom (guidance pointing at the wrong host) is confirmed 3/3 times. The root cause described below (the model reusing the credentials section's initial-setup wording out of context for later remediation) is the most likely explanation given how SKILL.md was written, not something proven mechanistically, since an LLM's output isn't fully traceable to one line of prompt text. The fix addresses the wording regardless, since it removes the only plausible cause we could find, and costs nothing if some other cause also contributed.

Changes

Before, SKILL.md's credentials section introduced the cloud URL with no scope note:

The scripts need AGENTA_API_KEY, and AGENTA_HOST only if the user self-hosts (it defaults to Agenta cloud, https://cloud.agenta.ai).

Nothing elsewhere in the file said that URL should not be reused later.

After, the same section explicitly scopes that URL to initial setup only, and a new hard rule states that any later remediation must reference the user's actual $AGENTA_HOST:

https://cloud.agenta.ai above is a one-time example for INITIAL setup only. If a later runtime error tells you to add a credential to the project vault (or anything else that means "go to your Agenta project"), point the user at their actual configured $AGENTA_HOST...

Tests

This is a documentation-only change (no script logic to unit test). Verified by reading the updated SKILL.md end to end to confirm the new wording is unambiguous, and that it appears both where the URL is first introduced and in the hard rules section, where a caller under time pressure is more likely to be scanning rather than reading linearly.

What to QA

  • Point a coding agent at a self-hosted Agenta instance and get it to build an agent that needs a vault credential the project doesn't have yet (e.g. a provider key). When it reports the failure, it should tell you to go to your own $AGENTA_HOST, not cloud.agenta.ai.
  • Regression: confirm initial credential setup still mentions cloud.agenta.ai as the example for a cloud user's first-time setup.

Closes #5.

Traced to SKILL.md's "First run: credentials" section, which uses
https://cloud.agenta.ai as the example for initial API-key setup. When a
later runtime failure calls for telling the user to go add something to
their Agenta project (e.g. a vault key), that same cloud URL was getting
reused out of context instead of the user's actual configured $AGENTA_HOST -
sending a self-hosted user to look in the wrong place entirely. Reproduced
3/3 times against a self-hosted instance during QA testing.

SKILL.md now explicitly marks that URL as initial-setup-only, right where
it's introduced, and adds a hard rule: any later remediation must reference
the user's actual $AGENTA_HOST, not a hardcoded cloud URL.

@mmabrouk mmabrouk left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @bekossy please check https://agentagroup.slack.com/archives/C0BBU3T45S5/p1783330940127549 and sync with jp on how to fix this

bekossy added 2 commits July 7, 2026 10:46
Per review feedback (jp asked why the skill uses AGENTA_HOST instead of
AGENTA_API_URL): these scripts curl the public REST API directly from
outside Agenta's own infrastructure, the same way the SDK derives its own
api_url from host + "/api" (sdks/python/agenta/sdk/utils/init.py).
AGENTA_API_URL is a different, internal override used when a workflow
running INSIDE Agenta's infra needs to reach the API at a different address
than the public host (e.g. docker-internal networking on a self-hosted
install) - it doesn't apply to an external caller hitting the public REST
API, which is all this skill ever does. Added a clarifying note in both
SKILL.md and lib.sh's header comment so this doesn't get relitigated.

Also merges main to pick up #12/#14, already merged.
@bekossy

bekossy commented Jul 7, 2026

Copy link
Copy Markdown
Member Author

Dug into the AGENTA_HOST vs AGENTA_API_URL question directly in the agenta codebase before syncing, so I could bring a concrete answer rather than just a question:

AGENTA_HOST is correct for this skill. These scripts curl the public REST API directly from outside Agenta's own infra, the same way the SDK derives its own api_url from host + "/api" (sdks/python/agenta/sdk/utils/init.py:87-102). AGENTA_API_URL is a different, internal override the SDK uses when a workflow running INSIDE Agenta's infra needs to reach the API at a different address than the public host (e.g. docker-internal networking on a self-hosted install) - it doesn't apply to an external caller hitting the public REST API, which is all this skill ever does.

Added a clarifying note in both SKILL.md and lib.sh's header comment explaining this, so it doesn't get relitigated. Happy to loop in @jp-agenta directly if this doesn't match his read, flagging here first since I found a concrete technical answer rather than just relaying the question.

…y fallback

My previous commit claimed AGENTA_HOST was correct and AGENTA_API_URL was an
internal-only override. That was wrong, confirmed by direct correction from
jp plus a closer re-read of sdks/python/agenta/sdk/utils/init.py:87-102: the
SDK actually checks AGENTA_API_INTERNAL_URL then AGENTA_API_URL FIRST, and
only derives from AGENTA_HOST as a fallback when neither is set. I had
conflated AGENTA_API_INTERNAL_URL (genuinely internal-only) with
AGENTA_API_URL (the public API URL, confirmed by jp's own examples:
https://eu.cloud.agenta.ai/api, http://localhost/api).

lib.sh now accepts AGENTA_API_URL (already includes /api) as the primary
variable, matching the SDK's own precedence, and falls back to deriving it
from AGENTA_HOST (a bare host, legacy) only when AGENTA_API_URL is not set.
Normalizes either input down to a bare-host internal AGENTA_HOST so every
other script's existing /api/... and /services/... path literals stay
unchanged - this is a config-parsing change in lib.sh only, not a rewrite of
every script's call sites.

Verified the derivation logic locally across all four cases (AGENTA_API_URL
set, AGENTA_HOST-only legacy fallback, neither set defaulting to cloud, both
set with AGENTA_API_URL correctly winning). Could not verify live against
bighetzner this round: the instance is unreachable from this environment
right now (DNS resolves, but the connection times out completely, even for
a bare GET to /) while general internet access is otherwise fine (confirmed
against cloud.agenta.ai) - flagged for a live check once reachable.
@bekossy

bekossy commented Jul 7, 2026

Copy link
Copy Markdown
Member Author

@jp you're right, I had it backwards, thanks for catching it directly. Re-read sdks/python/agenta/sdk/utils/init.py:87-102 more carefully: AGENTA_API_INTERNAL_URL and AGENTA_API_URL are checked FIRST, AGENTA_HOST is only the fallback derivation when neither is set. I'd conflated AGENTA_API_INTERNAL_URL (genuinely internal-only) with AGENTA_API_URL (the public API URL, your own examples confirm it: https://eu.cloud.agenta.ai/api, http://localhost/api).

Pushed a fix: lib.sh now takes AGENTA_API_URL as the primary variable (matching the SDK's own precedence), with AGENTA_HOST kept only as a legacy fallback for anyone with an existing .env, so nothing already working breaks. Updated SKILL.md accordingly. Verified the fallback/precedence logic locally across all four cases; couldn't verify live against bighetzner this round since it's unreachable from my environment right now, will recheck once it's back.

@mmabrouk this is now sync'd with jp, ready for another look.

@bekossy
bekossy requested review from jp-agenta and mmabrouk July 7, 2026 10:43
Per jp: this skill hasn't shipped to any real users yet, so there is no
back-compat burden to preserve. Removed AGENTA_HOST entirely instead of
keeping it as a legacy fallback.

lib.sh now only reads AGENTA_API_URL (defaults to
https://cloud.agenta.ai/api), derives an internal bare-root variable for the
one call site that needs a non-/api path (test-agent.sh's
/services/agent/v0/invoke), and every other script's /api/... path literals
are untouched. No AGENTA_HOST references remain anywhere in the skill.

Verified end to end against bighetzner: build.sh (create + invoke) succeeds
with only AGENTA_API_URL set, RESOLVED prints correctly, throwaway agent
archived afterward. Also updated the two local .env files in
~/Desktop/agenta-skill-test that still had AGENTA_HOST.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

build-agent: post-failure remediation guidance links to cloud.agenta.ai instead of configured AGENTA_HOST

2 participants