Widen ndarray linear index to int64 by paveltc · Pull Request #776 · Genesis-Embodied-AI/quadrants

paveltc · 2026-07-11T05:57:14Z

Summary

Backport of ROCm/quadrants@e0ec4416a.
Widens index values to int64 in the LLVM codegen path (ndarray ExternalPtrStmt
linear indexing), makes Ndarray::nelement_ a size_t accumulation, and removes
the obsolete int32-boundary warning logic. This does not make Quadrants globally
int64-indexed; it only prevents int32 overflow in the generated linear-index
arithmetic for large ndarrays.
Cross-backend change (CPU + AMDGPU); conflicts on cherry-pick were purely
clang-format cosmetic and resolved to match the original commit.

Tests

Result: 5098 passed, 142 skipped, 28 xfailed. The 21 failures are pre-existing
known-flaky amdgpu block-scan tests (test_block_inclusive,
test_exclusive_scan_composition) also present in the base branch — no new regressions.
New CPU test test_ndarray_external_ptr_uses_i64_linear_index passes, verifying
i64 linear indexing in the generated LLVM IR.

To make the motivation concrete, I added two tests to tests/python/test_ndarray_indexing_i64.py that show why the linear index must be accumulated in i64.

visit(ExternalPtrStmt) flattens an N-D ndarray access into a single linear element index — for a 2-D array of shape (D0, D1) it emits linear = i * D1 + j. Before this PR that accumulation was done in i32 and only sign-extended to i64 for the final GEP, so any array with more than 2**31 elements overflowed before the extend and produced a wrong (often negative) address. The smallest 2-D shape that triggers it is (2, 2**30 + 1), indexed at [1, 2**30], where the true linear index is 2**31 + 1 — just past INT32_MAX.

test_i32_linear_index_overflows_but_i64_is_correct (always runs, CPU-only) — reproduces the exact i * D1 + j arithmetic with two kernels, one accumulating in i32 (pre-fix behavior) and one in i64 (the fix), and asserts the i32 version wraps to a negative value while the i64 version stays correct:
```
TRUE      = 2147483649
i32 accum = -2147483647   # wraps negative -> wrong address
i64 accum = 2147483649    # correct
```
test_ndarray_read_past_int32_index_boundary (end-to-end regression guard) — allocates a real >2**31-element int8 ndarray, writes a sentinel at [1, 2**30], and reads it back through the actual ExternalPtr codegen path. On the pre-fix code the wrapped negative offset returns garbage; with this PR it correctly returns the sentinel. It's gated with @pytest.mark.skipif on available memory (needs ~2 GB backing + headroom) so it self-skips on small runners. This test adds a psutil import, consistent with the existing tests/python/test_memory.py.

How to run

# both tests
python3 tests/run_tests.py tests/python/test_ndarray_indexing_i64.py -v

# just the always-on simulation test (no large allocation)
python3 tests/run_tests.py -k test_i32_linear_index_overflows_but_i64_is_correct -v

# force the memory-gated end-to-end test to run (needs ~2GB+ free RAM)
python3 tests/run_tests.py -k test_ndarray_read_past_int32_index_boundary -v

Or directly with pytest:

python3 -m pytest tests/python/test_ndarray_indexing_i64.py -v

Verified against this branch's build: the simulation test passes and the end-to-end read returns the correct sentinel; on the pre-fix codegen the i32 accumulation wraps negative, which is exactly the failure this PR resolves.

…d removes some warning logic around int32. It does not make quadrants globally indexed by int64. Co-authored-by: Cursor <cursoragent@cursor.com>

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 364b86d25b

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

chatgpt-codex-connector · 2026-07-11T06:02:25Z

            else:
                element_dim = needed_arg_dtype.ndim
                array_shape = v.shape[element_dim:] if is_soa else v.shape[:-element_dim]
+            if any(dim > np.iinfo(np.int32).max for dim in array_shape):


Keep product warning for SPIR-V backends

When this runs on Vulkan/Metal, ndarray addressing is still flattened in SPIR-V as an i32 value (quadrants/codegen/spirv/spirv_codegen.cpp:864-894), so a shape like (65536, 65536) has no single dimension above int32 but still wraps the linear offset and reads/writes the wrong element. This branch handles external NumPy/Torch arrays for every arch, so replacing the old product check with only a per-dimension check removes the only warning for unsupported SPIR-V launches; please keep the product warning for non-LLVM backends or widen the SPIR-V path too.

Useful? React with 👍 / 👎.

hughperkins

Using a different indexing bitwidth on different platforms conflicts with the ideal that Quadrants code should run unchanged across all platforms.

We could consider globally migrate all indices to int64, but:

currently there is no obvious end-user need, and
likely will have a performance impact

Adds a deterministic simulation of the ExternalPtrStmt linear-index arithmetic (i * D1 + j) showing i32 accumulation wraps negative while i64 stays correct, plus a memory-gated end-to-end test that reads an ndarray with more than 2**31 elements at the overflow-triggering index. Co-authored-by: Cursor <cursoragent@cursor.com>

paveltc · 2026-07-13T20:38:01Z

@hughperkins
Thanks — I agree with the principle, but I think this change is narrower than "different indexing bitwidth on different platforms."
It's not a language-level indexing change. Kernel index args keep their declared types. The PR only touches the internal address arithmetic in TaskCodeGenLLVM::visit(ExternalPtrStmt) — how the flat offset into the ndarray buffer is computed.
It's an overflow fix, not a new i64 mode. The final offset was always 64-bit; the old code accumulated in i32 and did a single CreateSExt at the end, so arrays with >2³¹ elements overflowed before the extend. This just moves the widening earlier so the accumulation itself can't overflow.
To make the overflow concrete, I also pushed two tests (tests/python/test_ndarray_indexing_i64.py): a deterministic simulation showing the i32 accumulation wraps negative while i64 stays correct, and a memory-gated end-to-end read of a >2³¹-element ndarray.
Please let me know what kind of fix you would like to see if you disagree with this.

hughperkins · 2026-07-14T12:41:13Z

The criteria is that correctness should be ~identical across all GPU platforms.

Example of what happens if this is not the case:

User writes a script that runs just fine on AMD
- fails to run correctly on other platforms, such as CUDA or Metal
User writes a script that runs just fne on CUDA
- fails to run correctly on other platforms, such as AMDGPU

To justify changing the i64 for AMDGPU, you would need a platform-agnostic test that, on main, without your changes:

passes on CUDA
fails on AMDGPU
... and then provide a fix which narrows the correctness delta between AMDGPU and other GPU platforms.

Restore a product-of-dimensions overflow warning for the SPIR-V backends (Vulkan/Metal/...), which still flatten the ndarray ExternalPtr linear offset in int32. The int64 accumulation fix only applies to the LLVM codegen backends (CPU/CUDA/AMDGPU), so without this those backends would silently read the wrong element for ndarrays whose element count exceeds int32. Addresses the Codex P2 review comment.

paveltc · 2026-07-16T23:10:21Z

Thanks @hughperkins. Digging into where the change actually lives, I think the framing shifts a bit:

The fix is in the shared LLVM codegen — TaskCodeGenLLVM::visit(ExternalPtrStmt) in codegen/llvm/codegen_llvm.cpp — which is used identically by CPU, CUDA, and AMDGPU. On main, that path accumulates the linear index in i32 and only SExts to i64 at the end, so CUDA overflows exactly the same way AMDGPU does. Because of that, the test you asked for (on main: passes on CUDA, fails on AMDGPU) can't really exist — CUDA isn't correct here either. This isn't an AMD-vs-CUDA bitwidth divergence; it's a universal overflow in the LLVM path, and the fix makes CPU/CUDA/AMDGPU consistent. (The [AMDGPU] tag is misleading — I'll retitle.)

The real cross-platform delta your principle is pointing at is LLVM backends vs SPIR-V backends: Vulkan/Metal (spirv_codegen.cpp) still flatten the offset in i32 and would silently overflow. To keep correctness parity, I've restored a product-of-dimensions overflow warning for the SPIR-V backends (also addressing Codex's P2), so no backend silently returns the wrong element. Widening the SPIR-V path to i64 is a larger follow-up (needs the Int64 device capability and has perf implications), which I'm happy to do separately if you'd prefer full parity now.

paveltc · 2026-07-16T23:26:43Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 50e90dccd3

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

chatgpt-codex-connector · 2026-07-16T23:30:36Z

+  ndarray_alloc_ = prog->allocate_memory_on_device(nelement_ * element_size_,
+                                                   prog->result_buffer);


Preserve overflow warning for owned ndarrays

On Vulkan/Metal, TaskCodegen::visit(ExternalPtrStmt) still flattens ndarray offsets in i32, but this constructor no longer emits the old total_shape_ overflow warning. The new Python warning only runs when launching numpy/torch external arrays; qd.ndarray(...) objects take the earlier Ndarray branch in _recursive_set_args and are passed through set_arg_ndarray_impl, so a shape like (2, 2**30 + 1) can now be created and indexed on SPIR-V without any warning even though the generated offset wraps. Please keep a backend-specific warning for Quadrants-owned ndarrays as well.

Useful? React with 👍 / 👎.

… backends The previous push restored the overflow warning only for external numpy/torch arrays (the Python launch path). Quadrants-owned qd.ndarray() objects are constructed via Ndarray(Program*, ...) and bypass that check, so on SPIR-V backends (Vulkan/Metal) an owned ndarray whose element count exceeds int32 could be created and indexed with no warning even though the generated linear offset wraps. Restore the total_shape_ product-of-dimensions overflow warning in that constructor, gated on !arch_uses_llvm() so it only fires on the non-LLVM backends (the LLVM backends now accumulate the offset in int64). Addresses the follow-up Codex P2 review comment.

paveltc · 2026-07-17T04:02:32Z

@codex review

chatgpt-codex-connector · 2026-07-17T04:08:11Z

Codex Review: Didn't find any major issues. Already looking forward to the next diff.

Reviewed commit: 50c6556831

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

hughperkins · 2026-07-17T13:37:32Z

Ok, so it sounds like this PR is not addressing a difference in correctness between back-ends?

Could you give me a little more background/details into the motivations of this PR? Like, sure, it's possible to construct an artificial test that fails without this PR, but what is your actual underlying motivations/intentions?

In genesis-world for example, we are not hitting this limitation, because physics simulations get slow well before we hit such tensor sizes.

Things that I'm wary about:

changes without a clear concrete benefit, since any change involves risk, of adding new bugs, and of performance regressions
performance regressions, since 64-bit types tend to be slower than 32-bit types, e.g. because use more memory bandwidth
correctness regressions, and crash bugs, which changing bit-widths can definitely both create, especially since we'd need to change this across all platforms

paveltc · 2026-07-17T20:20:55Z

You're right that this isn't a cross-backend correctness delta, and you're right that genesis-world doesn't hit it. Physics sims get slow long before a single ndarray exceeds ~2.1B elements, so for the primary workload this limit is effectively unreachable. I don't want to oversell it as something Genesis needs today.

The concrete benefit is a latent memory-safety bug, not "large-tensor support." There are actually two failure modes on main for an ndarray whose element count exceeds INT32_MAX, and both are silent:

Wrong-address reads/writes — the linear index i*D1 + j is accumulated in i32 and only widened to i64 for the final GEP, so it wraps (often negative) before the extend.
Under-allocation → heap corruption — nelement_ is folded with an int accumulator (std::accumulate(..., 1, ...)), so it overflows before it's stored, and nelement_ * element_size_ then allocates a buffer that's too small. Subsequent indexing writes out of bounds.
The second one is the part I'd weight most: it's silent memory unsafety in the shared LLVM path (CPU/CUDA/AMDGPU all affected identically), not just a wrong value. Even if it's rare, "silently corrupts the heap" is the kind of latent bug that's worth closing.

Please let me know if you have any more questions.

hughperkins · 2026-07-17T20:28:25Z

Yes, the fact that it is silent is a great point.

hughperkins · 2026-07-17T20:29:32Z

I'll try running Genesis benchmarks against this branch.

hughperkins · 2026-07-17T20:29:38Z

@codex review

hughperkins · 2026-07-17T20:31:25Z


+# The ndarray ExternalPtrStmt linear index is accumulated in int64 only in the LLVM codegen backends
+# (see TaskCodeGenLLVM::visit(ExternalPtrStmt)). The SPIR-V backends (Vulkan/Metal/...) still flatten the
+# offset in int32, so an ndarray whose total element count exceeds int32 can silently overflow there. We


this states that spirv is still broken.

strong preference that we fix across all backends please.

(or alterntaively, put an overflow warnign for all backends)

Follow-up: widen the SPIR-V linear index to int64 (commit 55ab538)
This extends the fix to the SPIR-V backends (Vulkan/Metal), closing the cross-backend parity gap.

Change. TaskCodegen::visit(ExternalPtrStmt) previously flattened the multi-dimensional ndarray index into a linear offset in int32, so on SPIR-V an ndarray with more than INT32_MAX elements would wrap and silently address the wrong element — the same overflow the earlier commits fixed for the LLVM backends. It now accumulates the offset (and the derived byte offset / element index) in int64 when the device advertises DeviceCapability::spirv_has_int64, feeding the 64-bit offset directly into the physical-storage-buffer address add and OpPtrAccessChain. Devices without shaderInt64 keep the historical int32 arithmetic.

Warnings. The C++ Ndarray(Program*) constructor and the Python external-array launch path now only warn on a SPIR-V device that actually lacks shaderInt64 — every other case (LLVM backends, or SPIR-V with int64) now indexes correctly, so the diagnostic is no longer misleading.

What I validated locally

Compiles in a Vulkan-enabled build (QD_WITH_VULKAN=ON, alongside AMDGPU + CUDA).
Ran on a software Vulkan device (Mesa lavapipe, which reports shaderInt64 + bufferDeviceAddress): multi-dimensional qd.ndarray indexing matches NumPy, and an int64 reduction over the array is correct.
Confirmed via SPIR-V disassembly that the generated offset math is now 64-bit (OpSConvert/OpIMul/OpIAdd/OpShiftLeftLogical on %long, then OpBitcast to %ulong), versus the previous int32 + OpSConvert path.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: af6db1987f

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

chatgpt-codex-connector · 2026-07-17T20:32:00Z

+    auto index = builder->CreateSExtOrBitCast(llvm_val[stmt->indices[i]], i64_ty);
+    linear_index = builder->CreateAdd(linear_index, index);


Document backend-specific large-array indexing

This change expands the user-visible ndarray indexing range beyond INT32_MAX on LLVM backends while leaving Vulkan/Metal limited to int32 linear offsets, but it makes no corresponding change under docs/. The repository's AGENTS.md explicitly requires documentation for public API or usage changes; add end-user documentation describing the newly supported large-array behavior and the backend-specific limitation.

Useful? React with 👍 / 👎.

hughperkins · 2026-07-17T20:33:01Z

            # array shapes.
            is_soa = needed_arg_type.layout == Layout.SOA
            array_shape = v.shape
-            if math.prod(array_shape) > np.iinfo(np.int32).max:


wait, there was already a warning?

Yes — main had an unconditional one here: warnings.warn("...int64 indexing is not supported yet."). But it was only a warning (the offset still overflowed and returned the wrong element), and its message is now false for CPU/CUDA/AMDGPU since those accumulate in int64. Keeping it as-is would warn on exactly the backends this PR fixes. So I kept a warning but scoped it to the backends that still flatten in int32 — and with the new SPIR-V int64 commit, only to devices without shaderInt64.

I would prefer to just keep the warning as-is, without changing anything, without a strong reason.

What is your motivation for creating this PR? I'm sensing the motivation is primarily 'this is broken and it should be fixed'?

(I'm happy to upgrade from warning to throwing an exception, if that would make you more comfortable?)

Here is the motivation from a colleague of mine who worked on this:

Things that I'm wary about:

changes without a clear concrete benefit, since any change involves risk, of adding new bugs, and of performance regressions
They benefit was that they enabled workloads with more than 32k envs (which we explicitly needed)

performance regressions, since 64-bit types tend to be slower than 32-bit types, e.g. because use more memory bandwidth
Sure 64 bit types are slow, but we explicitly don't switch to using int64 for indexes, we narrowed the scope and used it for things like flattened indices which show up a lot and can easily exceed the bounds of int32

correctness regressions, and crash bugs, which changing bit-widths can definitely both create, especially since we'd need to change this across all platforms
The hope was that the limited scope of these changes would also reduce or eliminate these types of problems.

Quick update, and a correction to my earlier comment. When I said genesis-world doesn't hit this — that physics slows down before an ndarray crosses ~2.1B elements — I was reasoning from my own assumptions. Since then a colleague who worked on the original change filled me in: they did hit it in practice, on a real workload running >32k parallel envs, where the aggregate ndarray crosses INT32_MAX even though the per-env tensors are modest. So there is a concrete, non-artificial workload behind this, not just the memory-safety argument — I was wrong to imply nobody needed the capability.

That also speaks to the warning-vs-fix question. For that workload, neither the existing warning nor upgrading it to an exception helps — it needs to run at that size, and that requires the flattened-offset arithmetic not to overflow. I'm fully on board with your instinct that silent is the worst case: making overflow throw instead of warn is strictly better where we can't support the size (e.g. a device without shaderInt64). But where we can support it, the fix lets the workload actually execute rather than failing loudly.

So the two motivations are complementary:

Concrete capability — a real >32k-env workload that overflows INT32_MAX on main and works with this PR.

Latent memory-safety — the silent wrong-address / under-allocation bug in the shared LLVM path (the point you flagged as a good one), which the same change closes for CPU/CUDA/AMDGPU.

And the SPIR-V commit (55ab538) extends the same narrow widening to Vulkan/Metal (capability-gated on shaderInt64, i32 + warning fallback otherwise), which is the cross-backend parity you asked for — so no backend is left silently overflowing.

hmmm, this is ndarray only. what happens if we change between ndarray and field? (we do this in genesis-world for example: ndarray for development mode, compiles quickly; field for performance mode, runs quickly, but compiles slowly). Will this introduce a correctness disparity between ndarray and field?

hughperkins · 2026-07-17T20:33:38Z

Wait, looking a the code, it looks like there is already a warning, in main?

warnings.warn("Ndarray index might be out of int32 boundary but int64 indexing is not supported yet.")

… is available TaskCodegen::visit(ExternalPtrStmt) flattened the multi-dimensional ndarray index into a linear offset in int32, so on the SPIR-V backends (Vulkan/Metal) an ndarray whose element count exceeds int32 would wrap and silently address the wrong element -- the same overflow the earlier commits fixed for the LLVM backends (CPU/CUDA/AMDGPU). Accumulate the linear offset (and the derived byte offset / element index) in int64 when the device advertises DeviceCapability::spirv_has_int64. Sizes and indices are widened with IRBuilder::cast (sign-extend), and the physical-storage -buffer add/OpPtrAccessChain now consume the 64-bit offset directly (cast() bitcasts i64->u64 and sign-extends i32->u64). Devices without shaderInt64 keep the historical int32 arithmetic. Re-scope the product-of-dimensions overflow warnings accordingly: the C++ Ndarray(Program*) constructor and the Python external-array launch path now only warn on a SPIR-V device that actually lacks shaderInt64, since every other case (LLVM backends, or SPIR-V with int64) now indexes correctly. Verified in a Vulkan-enabled build on a software Vulkan device (Mesa lavapipe, shaderInt64+bufferDeviceAddress): multi-dimensional ndarray indexing matches numpy, and the generated SPIR-V computes the linear offset in 64-bit (OpIMul/ OpIAdd/OpShiftLeftLogical on %long, OpBitcast to %ulong) instead of the previous int32 + OpSConvert. Real AMD/NVIDIA-GPU Vulkan perf and Metal are left to CI and hardware owners. Co-authored-by: Cursor <cursoragent@cursor.com>

paveltc · 2026-07-17T22:06:03Z

@codex review

chatgpt-codex-connector · 2026-07-17T22:07:57Z

Codex Review: Didn't find any major issues. Chef's kiss.

Reviewed commit: 55ab538978

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

hughperkins · 2026-07-20T12:46:41Z

Seems Genesis rigid benchmark perf is ok:

hughperkins · 2026-07-20T12:48:20Z

Oh this is for ndarray, and I ran for field. I'll run for ndarray.

hughperkins · 2026-07-20T13:06:13Z

(Note: how to read the benchmark report:

each row is a differnet benchmark
the right hand column shows the percentage FPS (speed) delta between this branch and main
- positive numbers mean this branch is faster
- negative numbers mean this branch is slower
- there is some variance
- the rows are sorted by the percentage delta
- very easy to eyeball if there is a performance delta: look for the zero-crossing row (in this case between go2 and dex_hand)
  - if it's towards the top, the branch increases speed (good)
  - if it's towards the bottom, the branch decreases speed (bad)

hughperkins · 2026-07-20T13:08:45Z

Here are the Genesis benchmarks using ndarray:

In this case, there is a regression of around ~2% FPS.

Widening index values to int64 for llvm codegen, ndarray nelement, an…

364b86d

…d removes some warning logic around int32. It does not make quadrants globally indexed by int64. Co-authored-by: Cursor <cursoragent@cursor.com>

paveltc changed the title ~~codegen: widen ndarray linear index to int64~~ [AMDGPU]: widen ndarray linear index to int64 Jul 11, 2026

paveltc changed the title ~~[AMDGPU]: widen ndarray linear index to int64~~ [AMDGPU] Widen ndarray linear index to int64 Jul 11, 2026

chatgpt-codex-connector Bot reviewed Jul 11, 2026

View reviewed changes

hughperkins requested changes Jul 13, 2026

View reviewed changes

hughperkins added the awaiting-contributor-action awaiting-contributor-action label Jul 13, 2026

paveltc requested a review from hughperkins July 14, 2026 05:51

paveltc changed the title ~~[AMDGPU] Widen ndarray linear index to int64~~ Widen ndarray linear index to int64 Jul 16, 2026

chatgpt-codex-connector Bot reviewed Jul 16, 2026

View reviewed changes

Merge branch 'main' into fix/ndarray-i64-linear-index

af6db19

hughperkins reviewed Jul 17, 2026

View reviewed changes

chatgpt-codex-connector Bot reviewed Jul 17, 2026

View reviewed changes

hughperkins reviewed Jul 17, 2026

View reviewed changes

paveltc requested a review from hughperkins July 17, 2026 21:38

		ndarray_alloc_ = prog->allocate_memory_on_device(nelement_ * element_size_,
		prog->result_buffer);

		auto index = builder->CreateSExtOrBitCast(llvm_val[stmt->indices[i]], i64_ty);
		linear_index = builder->CreateAdd(linear_index, index);

Uh oh!

Conversation

paveltc commented Jul 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Tests

How to run

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Jul 11, 2026

Choose a reason for hiding this comment

Uh oh!

hughperkins left a comment

Choose a reason for hiding this comment

Uh oh!

paveltc commented Jul 13, 2026

Uh oh!

hughperkins commented Jul 14, 2026

Uh oh!

paveltc commented Jul 16, 2026

Uh oh!

paveltc commented Jul 16, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Jul 16, 2026

Choose a reason for hiding this comment

Uh oh!

paveltc commented Jul 17, 2026

Uh oh!

chatgpt-codex-connector Bot commented Jul 17, 2026

Uh oh!

hughperkins commented Jul 17, 2026

Uh oh!

paveltc commented Jul 17, 2026

Uh oh!

hughperkins commented Jul 17, 2026

Uh oh!

hughperkins commented Jul 17, 2026

Uh oh!

hughperkins commented Jul 17, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Jul 17, 2026

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

paveltc Jul 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

paveltc Jul 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hughperkins commented Jul 17, 2026

Uh oh!

paveltc commented Jul 11, 2026 •

edited

Loading

paveltc Jul 18, 2026 •

edited

Loading

paveltc Jul 18, 2026 •

edited

Loading