Add local inference support and improve Neo4j troubleshooting by toughcoding · Pull Request #5 · neo4j-labs/create-context-graph

toughcoding · 2026-03-29T00:45:52Z

Add support for localhost model inference (local LLM servers)
Extend CLI and wizard to accept local model endpoints
Add Google/Gemini API key support for google-adk framework
Add --demo shortcut flag for quick demo setup
Improve README with comprehensive Neo4j authentication troubleshooting

- Add support for localhost model inference (local LLM servers) - Extend CLI and wizard to accept local model endpoints - Add Google/Gemini API key support for google-adk framework - Add --demo shortcut flag for quick demo setup - Improve README with comprehensive Neo4j authentication troubleshooting

Copilot

Pull request overview

Adds support for configuring Anthropic-compatible local endpoints across scaffolding (CLI/wizard/templates) and improves troubleshooting/docs around Neo4j, while also extending ontology loading to search user-local custom domains.

Changes:

Extend CLI + wizard to accept optional inputs (including --anthropic-base-url) and include the value in generated project config/env.
Update backend templates to expose ANTHROPIC_BASE_URL and wire it into multiple agent frameworks.
Improve domain loading (fallback to custom domains dir) and expand README with Neo4j auth troubleshooting.

Reviewed changes

Copilot reviewed 17 out of 17 changed files in this pull request and generated 11 comments.

Show a summary per file

File	Description
src/create_context_graph/wizard.py	Adds non-interactive parameters and collects/stores `anthropic_base_url` + Neo4j options.
src/create_context_graph/templates/base/dot_env_example.j2	Documents `ANTHROPIC_BASE_URL` in `.env.example`.
src/create_context_graph/templates/base/dot_env.j2	Emits `ANTHROPIC_BASE_URL` into generated `.env` when configured.
src/create_context_graph/templates/base/docker-compose.prod.yml.j2	Passes `ANTHROPIC_BASE_URL` into backend container env.
src/create_context_graph/templates/backend/shared/config.py.j2	Adds `anthropic_base_url` to backend Settings.
src/create_context_graph/templates/backend/agents/strands/agent.py.j2	Attempts to set `ANTHROPIC_BASE_URL` for Strands agent.
src/create_context_graph/templates/backend/agents/pydanticai/agent.py.j2	Attempts to set `ANTHROPIC_BASE_URL` for PydanticAI agent.
src/create_context_graph/templates/backend/agents/langgraph/agent.py.j2	Adds optional base URL wiring for LangChain Anthropic client.
src/create_context_graph/templates/backend/agents/claude_agent_sdk/agent.py.j2	Adds optional base URL wiring for Anthropic SDK client.
src/create_context_graph/templates/backend/agents/anthropic_tools/agent.py.j2	Attempts to set `ANTHROPIC_BASE_URL` for Anthropic Tools agent.
src/create_context_graph/renderer.py	Adds `anthropic_base_url` to template context and tweaks exception handling.
src/create_context_graph/ontology.py	Falls back to user-local custom domains directory when loading domains.
src/create_context_graph/generator.py	Adds base URL support to Anthropic client init and adjusts response parsing.
src/create_context_graph/custom_domain.py	Threads `base_url` through custom domain generation.
src/create_context_graph/config.py	Adds `anthropic_base_url` to `ProjectConfig`.
src/create_context_graph/cli.py	Adds `--anthropic-base-url` flag and passes it into wizard/custom domain flow.
README.md	Adds Neo4j authentication troubleshooting section.

Comments suppressed due to low confidence (1)

src/create_context_graph/renderer.py:213

except Exception as e introduces an unused variable (e) and will be flagged by Ruff (F841) in this repo’s make lint. Either drop as e or log/use the exception before falling back to the stub so failures are diagnosable.

        try:
            self._render_template(agent_template, backend_dir / "app" / "agent.py", ctx)
        except Exception as e:
            # Fallback: render a minimal agent stub
            self._render_template(
                "backend/shared/agent_stub.py.j2",
                backend_dir / "app" / "agent.py",
                ctx,
            )

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

- renderer.py: log exception in agent template fallback (fixes Ruff F841) - langgraph/agent.py.j2: remove unused `import langgraph`, restore missing {% raw %} blocks around run_cypher/get_graph_schema - docker-compose.prod.yml.j2: make ANTHROPIC_BASE_URL conditional - dot_env.j2: fix trailing blank line when anthropic_base_url is set - strands/agent.py.j2: clean up empty raw/endraw block - pydanticai/agent.py.j2: clean up empty raw/endraw block - anthropic_tools/agent.py.j2: clean up empty raw/endraw block

When a user provides an Anthropic base URL during custom domain generation, that value was not carried forward. The wizard would prompt again and the generated project could end up with a different (or empty) endpoint. Assign anthropic_base_url = custom_base_url after the custom domain flow so the later prompt is skipped and the project config stays consistent.

If the Anthropic SDK returns an empty content list, indexing response.content[0] would raise IndexError. Add an early return for this edge case.

The base URL was baked as a literal string at scaffold time, preventing users from changing it via .env after generation. Read from settings.anthropic_base_url at runtime instead, matching how api_key is already handled.

…template Same as the langgraph fix: read base_url from settings at runtime instead of baking a literal at scaffold time.

The README said bolt:// while .env.example uses neo4j://. Note both forms are accepted to avoid confusion.

toughcoding · 2026-04-03T11:22:33Z

Screenshot 2026-04-03 at 13-22-13 Localmodel · toughcoding_create-context-graph-local@2b63b0f

fixed

Copilot

Pull request overview

Copilot reviewed 17 out of 17 changed files in this pull request and generated 7 comments.

Comments suppressed due to low confidence (1)

src/create_context_graph/wizard.py:414

The wizard prompts for openai_api_key twice: once unconditionally at Step 6 (lines 385-392) and then again in the framework-specific block (lines 403-414), overwriting any provided openai_api_key (including values passed from the CLI). This also breaks non-interactive usage of run_wizard with a provided OpenAI key. Consolidate to a single prompt that only runs when openai_api_key is not already set, and only require it when framework == "openai-agents".

    if openai_api_key:
        # Use provided OpenAI API key
        pass
    else:
        openai_api_key = questionary.password(
            "OpenAI API key (for embeddings, or Enter to skip):",
            default="",
        ).ask()

    if anthropic_base_url:
        # Use provided base URL
        pass
    else:
        anthropic_base_url = questionary.text(
            "Anthropic-compatible API base URL (Enter to use default):",
            default="",
        ).ask()

    if framework == "openai-agents":
        openai_api_key = questionary.password(
            "OpenAI API key (required for OpenAI Agents SDK):",
            default="",
        ).ask()
        if not openai_api_key:
            console.print("[yellow]Warning:[/yellow] OpenAI Agents SDK requires OPENAI_API_KEY. Set it in your .env file.")
    else:
        openai_api_key = questionary.password(
            "OpenAI API key (optional — for OpenAI embeddings, or Enter to skip):",
            default="",
        ).ask()

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-05T17:46:53Z

    # Step 2: Data source
-    data_source = questionary.select(
-        "How would you like to populate your context graph?",
-        choices=[
-            questionary.Choice("Generate demo data (synthetic documents & entities)", value="demo"),
-            questionary.Choice("Connect to SaaS services (Gmail, Slack, Jira, etc.)", value="saas"),
-        ],
-    ).ask()
-    if not data_source:
-        raise SystemExit("Aborted.")
+    if demo_data or connector:
+        # Use provided data source
+        data_source = "saas" if connector else ("demo" if demo_data else "none")
+    else:


generate_data is derived from data_source == "demo", but data_source is forced to "saas" whenever any connector is provided (even if demo_data is also true). This makes --demo-data ineffective when combined with --connector in the wizard path. Consider treating demo generation as independent of the SaaS selection (e.g., set generate_data from the demo_data flag when provided, or add a separate wizard prompt to enable demo data alongside connectors).

@copilot generate a new github issue based on this feedback

Copilot · 2026-04-05T17:46:54Z

+    if neo4j_aura_env:
+        neo4j_type = "aura"
+    elif neo4j_local:
+        neo4j_type = "local"
+    elif neo4j_uri and "aura" in (neo4j_uri or ""):
+        neo4j_type = "aura"
+    elif neo4j_uri:
+        neo4j_type = "existing"


neo4j_type auto-detection uses "aura" in neo4j_uri, but Aura URIs are typically like neo4j+s://<id>.databases.neo4j.io and won’t contain the substring "aura". This can misclassify an Aura URI as existing and change the subsequent prompts/defaults. Consider detecting Aura by scheme (neo4j+s://) or hostname (.databases.neo4j.io) instead, or only setting neo4j_type="aura" when --neo4j-aura-env is provided.

@copilot apply changes based on this feedback

Copilot · 2026-04-05T17:46:54Z

+        neo4j_password = neo4j_password or "password"
        console.print(
            "[dim]Will use [bold]@johnymontana/neo4j-local[/bold] — "
            "run [bold]make neo4j-start[/bold] to launch Neo4j (requires Node.js)[/dim]"


For neo4j_type == "local", the wizard defaults neo4j_password to "password", but the repository’s Neo4j-local docs state the default credentials are neo4j/neo4j (with a first-login password change prompt). This default is likely to cause immediate auth failures for users following the --neo4j-local path. Consider defaulting to "neo4j" (or prompting the user), and ensure the README/docs/wizard agree on the expected credential flow.

Suggested change

neo4j_password = neo4j_password or "password"

console.print(

"[dim]Will use [bold]@johnymontana/neo4j-local[/bold] — "

"run [bold]make neo4j-start[/bold] to launch Neo4j (requires Node.js)[/dim]"

neo4j_password = neo4j_password or "neo4j"

console.print(

"[dim]Will use [bold]@johnymontana/neo4j-local[/bold] — "

"run [bold]make neo4j-start[/bold] to launch Neo4j (requires Node.js). "

"Default credentials are [bold]neo4j/neo4j[/bold]; if prompted to change the "

"password on first login, use your updated password afterward.[/dim]"

Copilot · 2026-04-05T17:46:54Z

+If Neo4j connection fails or the password is lost, reset the database:
+
+```bash
+make neo4j-stop && rm -rf ~/.local/share/neo4j-local/default && make neo4j-start
+cat /tmp/neo4j-local.log  # View generated password
+```
+
+**Connection Details:**
+- URI: `neo4j://localhost:7687` (the `bolt://` form is also supported)
+- Username: `neo4j`
+- Database: `neo4j`
+- Password: Generated automatically (check logs)
+


The neo4j-local troubleshooting guidance says the password is “Generated automatically (check logs)” and references ~/.local/share/neo4j-local/default, but the existing docs/docs/how-to/use-neo4j-local.md states default credentials are neo4j/neo4j and points to a different data directory (usually ~/.neo4j-local/data/). Please align these instructions with the documented behavior used elsewhere in the repo to avoid sending users to the wrong reset location / credentials.

@copilot apply changes based on this feedback

Copilot · 2026-04-05T17:46:54Z

+{% if anthropic_base_url %}
+      ANTHROPIC_BASE_URL: ${ANTHROPIC_BASE_URL}
+{% endif %}
 {% endif %}


docker-compose.prod.yml only includes ANTHROPIC_BASE_URL when anthropic_base_url was set at scaffold time. This prevents enabling a local endpoint later by setting ANTHROPIC_BASE_URL in the environment, because Compose won’t pass it through at all. Since other keys (e.g., OPENAI_API_KEY) are always wired through regardless of whether they were set during generation, consider always including ANTHROPIC_BASE_URL: ${ANTHROPIC_BASE_URL:-} (or similar) when an Anthropic-based framework is selected.

Suggested change

{% if anthropic_base_url %}

ANTHROPIC_BASE_URL: ${ANTHROPIC_BASE_URL}

{% endif %}

{% endif %}

ANTHROPIC_BASE_URL: ${ANTHROPIC_BASE_URL:-}

{% endif %}

Copilot · 2026-04-05T17:46:55Z

 @click.option("--neo4j-aura-env", type=click.Path(exists=True), help="Path to Neo4j Aura .env file with credentials")
 @click.option("--neo4j-local", is_flag=True, help="Use @johnymontana/neo4j-local for local Neo4j (no Docker)")
 @click.option("--anthropic-api-key", envvar="ANTHROPIC_API_KEY", help="Anthropic API key for LLM generation")
+@click.option("--anthropic-base-url", envvar="ANTHROPIC_BASE_URL", help="Anthropic-compatible API base URL (e.g., http://127.0.0.1:8082)")
 @click.option("--openai-api-key", envvar="OPENAI_API_KEY", help="OpenAI API key for LLM generation")
 @click.option("--google-api-key", envvar="GOOGLE_API_KEY", help="Google/Gemini API key (required for google-adk framework)")


A new CLI option --anthropic-base-url is introduced here, but the CLI test suite currently has coverage for --google-api-key and --openai-api-key flowing into the rendered .env (see tests/test_cli.py). Consider adding a similar test to assert ANTHROPIC_BASE_URL=... is present in the generated .env when this flag is provided, to prevent regressions in local inference support.

@copilot apply changes based on this feedback

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 17 out of 17 changed files in this pull request and generated 10 comments.

 {% endraw %}
+{% if anthropic_base_url %}
+os.environ["ANTHROPIC_BASE_URL"] = "{{ anthropic_base_url }}"


+# Set ANTHROPIC_BASE_URL if configured
 {% endraw %}
+{% if anthropic_base_url %}
+os.environ["ANTHROPIC_BASE_URL"] = "{{ anthropic_base_url }}"
+{% endif %}


 @click.option("--neo4j-aura-env", type=click.Path(exists=True), help="Path to Neo4j Aura .env file with credentials")
 @click.option("--neo4j-local", is_flag=True, help="Use @johnymontana/neo4j-local for local Neo4j (no Docker)")
 @click.option("--anthropic-api-key", envvar="ANTHROPIC_API_KEY", help="Anthropic API key for LLM generation")
+@click.option("--anthropic-base-url", envvar="ANTHROPIC_BASE_URL", help="Anthropic-compatible API base URL (e.g., http://127.0.0.1:8082)")



    def _render_data(self, data_dir: Path, ctx: dict) -> None:
        """Copy ontology and create data directory structure."""
+        from create_context_graph.ontology import _get_domains_path


+        if connector:
+            # Use provided connectors
+            selected_connectors = list(connector)
+        else:
+            from create_context_graph.connectors import list_connectors, get_connector
+            from create_context_graph.connectors.oauth import check_gws_cli, install_gws_cli
+


+        config = run_wizard(
+            project_name=project_name,
+            domain=domain,
+            framework=framework,
+            anthropic_api_key=anthropic_api_key,


+    if neo4j_aura_env:
+        neo4j_type = "aura"
+    elif neo4j_local:
+        neo4j_type = "local"
+    elif neo4j_uri and "aura" in (neo4j_uri or ""):


+If Neo4j connection fails or the password is lost, reset the database:
+
+```bash
+make neo4j-stop && rm -rf ~/.local/share/neo4j-local/default && make neo4j-start
+cat /tmp/neo4j-local.log  # View generated password


+
+    if not domain_path.exists():
+        # Try custom domains directory
+        custom_domains_dir = _get_custom_domains_path()
+        domain_path = custom_domains_dir / f"{domain_id}.yaml"
+


+    if openai_api_key:
+        # Use provided OpenAI API key
+        pass
+    else:
+        openai_api_key = questionary.password(


toughcoding force-pushed the localmodel branch from c7eee7d to 29fba0c Compare March 29, 2026 00:50

toughcoding force-pushed the localmodel branch from 29fba0c to eb5111f Compare March 29, 2026 00:55

johnymontana requested a review from Copilot March 29, 2026 03:56

Copilot started reviewing on behalf of johnymontana March 29, 2026 03:56 View session

Copilot AI reviewed Mar 29, 2026

View reviewed changes

toughcoding and others added 8 commits March 29, 2026 23:49

Merge branch 'main' into localmodel

5bcde57

fix: guard against empty response.content in _llm_generate

b376160

If the Anthropic SDK returns an empty content list, indexing response.content[0] would raise IndexError. Add an early return for this edge case.

fix: use runtime settings for anthropic_base_url in claude_agent_sdk …

26ed918

…template Same as the langgraph fix: read base_url from settings at runtime instead of baking a literal at scaffold time.

docs: align Neo4j URI in README with generated .env.example

c955fe5

The README said bolt:// while .env.example uses neo4j://. Note both forms are accepted to avoid confusion.

removing bug

2b63b0f

johnymontana requested a review from Copilot April 5, 2026 17:40

Copilot started reviewing on behalf of johnymontana April 5, 2026 17:40 View session

Copilot AI reviewed Apr 5, 2026

View reviewed changes

Update src/create_context_graph/renderer.py

7a51cd2

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

johnymontana requested a review from Copilot May 18, 2026 18:27

Copilot started reviewing on behalf of johnymontana May 18, 2026 18:28 View session

Copilot AI reviewed May 18, 2026

View reviewed changes

Uh oh!

Conversation

toughcoding commented Mar 29, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

toughcoding commented Apr 3, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Apr 5, 2026

Choose a reason for hiding this comment

Uh oh!

jexp May 11, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 5, 2026

Choose a reason for hiding this comment

Uh oh!

jexp May 11, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 5, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 5, 2026

Choose a reason for hiding this comment

Uh oh!

jexp May 11, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 5, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI Apr 5, 2026

Choose a reason for hiding this comment

Uh oh!

johnymontana Apr 28, 2026

Choose a reason for hiding this comment

Uh oh!

jexp May 11, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants