Drop AccessStruct#242
Open
reinauer wants to merge 41 commits into
Open
Conversation
AmiFuse starts handlers before a scheduler-backed task exists, so signal allocation and message ports still need to work when ctx.task is unset. Teach the lexec helpers to keep fallback signal state, preserve the historical signal 0 CreateMsgPort() behavior, auto-track port-manager queues, and make empty GetMsg() return None. Add a pytask that exercises this no-task path directly through vamos so the handler startup semantics stay covered.
Run the fallback exec pytask with a temporary HOME so vamos creates its ram: volume tree in a writable location. This keeps the new coverage working under normal pytest runs instead of depending on ~/.vamos being writable.
AmigaOS does not fail an otherwise valid ClockData just because the weekday field is invalid or inconsistent. Write Sunday-based weekdays when filling ClockData so the stored value matches Amiga numbering, and stop rejecting valid dates when the stored weekday does not match Python's weekday numbering. Update the util_date coverage and add unit tests for the weekday mapping and the relaxed read path.
Handler startup reaches exec.library before the scheduler-backed wait and message paths are fully available. Initialize the remaining exec global lists, preserve the fallback Wait() bookkeeping, zero new IORequests, record their reply port metadata, and add the generic device I/O helpers for DoIO/SendIO/CheckIO/WaitIO. Add pytask coverage for exec base list setup, IORequest initialization, and the remaining fallback Wait() path.
Serve timer.device requests through BeginIO so exec.library's new DoIO/SendIO path can drive TimeRequest-based callers. Handle TR_GETSYSTIME via the TimeRequest layout and keep the request state consistent for reply-based I/O. Add pytask coverage for DoIO/SendIO/WaitIO against timer.device and keep the existing timer pytask and gcc test_devtimer suite green.
Some handler paths open input.device and expect BeginIO to complete successfully even though they do not rely on richer input semantics. Register input.device, provide the minimal BeginIO/AbortIO implementation, and add pytask coverage that opens the device through exec.library and completes a request.
WaitPkt needs to pull packets from the current process message port, and ReplyPkt must send the reply back to the packet's original reply port even after dp_Port is rewritten for the current process. Restore WaitPkt on top of the exec port manager, preserve the original reply destination in ReplyPkt, normalize signed result values, and keep the message links consistent when packets move through the queue. Add a process-mode pytask that exercises empty WaitPkt, message delivery, and ReplyPkt routing.
Implement the DOS list helpers needed by handler startup and add a minimal CreateNewProc() path for child process setup. Cover FindDosEntry(), AttemptLockDosList(), MakeDosEntry(), AddDosEntry(), RemDosEntry(), FreeDosEntry(), and CreateNewProc() with pytask coverage.
Keep invalid ClockData failures out of normal output by logging Date2Amiga() and CheckDate() invalid-date diagnostics at debug level instead. Add unit coverage for the invalid ClockData path so normal log levels stay quiet.
Use each partition's own DosEnvec geometry when computing partition block counts instead of the RDB header's global cyl_blks value. Also report partition ratios against the physical disk total and add unit coverage for the mixed-geometry case.
SFS starts a helper process named "SFS DosList handler" and then polls FindPort() for that name during startup. The rebased tree registered the child pr_MsgPort for message delivery, but never gave it a name and only searched the public port list in FindPort(). That left SFS spinning forever waiting for a port that was already present in the port manager. Name child message ports from NP_Name and fall back to registered-port lookup by mp_Node.ln_Name when FindPort() misses the public list.
SFS uses intuition.library EasyRequestArgs() during startup when it needs to surface a request. The rebased tree still had the helper with an old single-argument signature, so the stub generator treated it as missing and returned d0=0. Match the fd signature and return success from the compatibility stub so startup can continue.
SFS initializes a large set of MinList structures during startup. The rebased tree left NewMinList() as a missing exec.library stub, so those list heads never became valid and later startup work fell apart. Initialize the MinList header in place and cover it with a pytask regression test.
Wire the private exec vectors at -510 and -516 to their real RawMayGetChar and RawPutChar names, and send RawPutChar output to the host debug stream. This lets handlers that log through RawPutChar show their serial-style debug output under vamos instead of dropping it. RawMayGetChar currently reports no pending input.
Add pytask coverage for a buffered putproc callback that updates a (buffer, remaining) pair through the normal RawDoFmt runner path. While touching RawDoFmt, factor the existing known C-string callback detection into small helpers without changing behavior.
Cache the hot IORequest and MsgPort field offsets during exec setup and use raw memory access in the BeginIO, SendIO, CheckIO, and WaitIO path. This avoids rebuilding AccessStruct wrappers for the same fields on every device I/O call. In the current PFS benchmark it cuts the direct worker profile from about 3.83M calls to 3.02M and reduces the _dispatch_begin_io hotspot noticeably while keeping the matrix and pytest coverage green.
Compile the register-to-argument readers once when building a library stub instead of repeating the issubclass dispatch on every call. This cuts overhead in the hot stub base_func path without changing the wrapped method signatures. In the current PFS benchmark it drops the 3-run average from about 1.265s to 1.137s and reduces the direct worker profile to about 2.97M calls while keeping pytest and the matrix green.
Cache the resolved index path for dotted field lookups so AccessStruct stops splitting names and walking field defs on every access. This keeps the hot struct access path on cached indices while preserving the same external lookup behavior.
Convert FileManager, LockManager and MatchFirstNext to the generated struct wrappers and move Lock onto typed fields. This drops direct AccessStruct use from the DOS-side packet and matcher helpers while keeping Lock compatible with the older FileInfoBlock callers that DosLibrary still drives through AccessStruct. That shrinks the remaining cleanup surface without forcing the wider allocator and DosLibrary work into the same change.
Replace the AccessStruct-based MemHeader, MemChunk and SignalSemaphore accessors with generated struct wrappers. These helpers only need fixed field reads and writes, so the string-based wrapper adds churn without buying anything. Switching now removes more obsolete callers while staying clear of the separate allocator API cleanup.
Convert DosList and MatchFirstNext to generated struct wrappers, and switch the adjacent DosLibrary helpers to the same typed accessors. These paths share the same DOS list, anchor, file info and device proc state, so keeping them together removes another batch of obsolete AccessStruct callers without dragging allocator API work into the change.
Replace the remaining direct AccessStruct callers in DosLibrary with generated struct wrappers for the packet, date, segment, CLI and library state touched by these helpers. This finishes the caller-side cleanup in vamos without changing the allocator API yet. After this, the only remaining non-test AccessStruct users are the wrapper itself and mem.alloc.
alloc_struct(), map_struct(), and alloc_lib() were the last places that still constructed AccessStruct. They also define the Memory object API that the DOS runtime still uses via mem.access, so simply dropping the helper would force a much larger conversion. Replace AccessStruct construction with a small adapter around the generated struct instance and store that typed object on Memory. This keeps mem.access working for old callers while giving new code a direct mem.struct handle and removes allocator-side dependency on AccessStruct.
The remaining AccessStruct cleanup is now concentrated in Process and DosLibrary, but a few smaller DOS helpers still used mem.access for simple field updates. Convert run_command(), CSource, and FileHandle to use the typed struct objects returned by the allocator. Add focused tests for CSource memory round-tripping and FileHandle allocation so these mechanical conversions stay covered while the larger Process cleanup is still pending.
Process was the largest remaining user of mem.access outside DosLibrary. Most of its field updates are direct process, CLI, and packet initialization, so keeping string-path access there only added compatibility overhead. Convert the process runtime helpers to use typed structs directly for CLI setup, shell packet setup, process field initialization, and the input/output/current-dir helpers. This also fixes the stale self.clip reference in the shell startup-file path while preserving the old BPTR-as-address behaviour.
DosLibrary was the last substantial runtime user of mem.access. The remaining call sites were all direct field updates in shell variables, RDArgs, process I/O state, and DOS object/list helpers, so keeping string-path access there only preserved compatibility plumbing. Convert those paths to typed structs and store local shell variables as LocalVarStruct objects instead of access wrappers. Rename the library's typed base field to lib_struct so the remaining access matches are only the allocator compatibility layer.
No in-tree runtime code still depends on Memory.access or the allocator-side access shim. Keeping those wrappers in alloc.py only preserved an extra API layer around typed structs and hid whether callers were still using the old path. Remove Memory.access, MemoryAlloc.access, and the internal struct access compatibility adapter. Keep Memory wrappers for ownership and labels, but make struct allocations expose only the typed struct object. Update the allocator tests to check the direct binding behaviour instead of the removed compatibility surface.
AccessStruct is no longer used by the vamos runtime or allocator. Keeping it in the umbrella astructs import kept the obsolete helper in the default public surface even after the caller cleanup was finished. Stop re-exporting AccessStruct from astructs/__init__.py and update the dedicated unit test to import it from its direct module instead. This keeps the standalone compatibility module available without presenting it as part of the primary typed-struct API.
Teach MemoryAlloc to return bound typed structs directly for callers that want astruct objects instead of Memory wrappers. Keep alloc_struct() for compatibility, but add alloc_astruct() and let TypeBase.alloc() reuse the bound object when no setup kwargs require a second instance. This removes one layer of wrapper churn from hot typed-struct paths without changing raw memory allocation APIs.
Switch the DOS and exec runtime helpers away from wrapper Memory objects where they only allocated a struct and then immediately dereferenced .struct. Use alloc_astruct() for owned allocations and direct typed views for mapped structs so the hot paths stay on typed objects. This keeps the caller code simpler and reduces struct wrapper churn in process, dos list, file handle, packet, and port handling.
Keep the Memory wrapper for compatibility APIs that still expose .struct, but store lightweight ownership records for typed allocations that only need addr, size, and label for free paths. This removes one extra wrapper object from alloc_astruct() and alloc_alib() while preserving allocator bookkeeping, labels, and get_memory() lookup semantics.
Route AmigaStruct.alloc() and Library.alloc() through the typed allocator fast paths instead of allocating a wrapper first and binding a second object on top. This lets typed callers keep a single bound object with the new lightweight ownership record, and the updated unit tests lock down the direct-allocation contract for structs and libraries.
Install descriptors for struct fields so normal attribute access uses precomputed index paths instead of name-based lookup. Also switch internal struct, pointer, and base-type state to object.__setattr__ on hot paths. This cuts Python attribute dispatch and field wrapper churn during load-heavy workloads. This speeds up AmiFUSE load tests by about 32%.
Add a fast bind path for typed struct fields so lazy field creation no longer runs the normal constructor path on every field access. This keeps aliases like List correct via a bind setup hook while reducing typed field dispatch cost. This change speeds up AmiFUSE load tests by about 15%
Bound structs always allocated a free-ref list and ran setup() even when no keyword initialization was requested. That added constructor work to every plain typed view created through the field access path. Use a shared empty tuple until setup kwargs are present. In AmiFuse load tests this improves steady-state medians by about 1.3% on average across PFS3, SFS, and FFS.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Based on top of my other patch... this one cleans up the code base to get rid of AccessStruct dependencies as @cnvogelg mentioned they are obsolete.