[Request] Adding custom TAStudio columns in Lua is very slow; a faster way to do this is needed

[Davidebyzero]

I have made a script to show the current game engine time (for TASing Worms Armageddon in the DOSBox-X core), shown below.

It works, but using tastudio.onqueryitemtext, it redraws the entire visible portion of the input roll after every frame. This makes it quite slow, and the larger the TAStudio window is, the slower it is. Without the custom column, the core (set to 120 fps) runs at about 27 fps, but with it, at the window size I usually want TAStudio to be, it drops to 7 fps. For long fast forwards (if 22% of realtime speed can count as a fast forward) I've been resizing TAStudio down to its minimum height, or minimizing it, as a workaround to avoid the slowness.

I would suggest providing a different way to output to a custom column. Probably the best way would be to provide two built-in tastudio Lua functions, one to write a given string to a given row, and one to read the string from a given row. (I'm guessing you can't just expose the custom column as a Lua array, one element per row, which, when written to by the script, can be trapped by BizHawk to trigger an automatic redraw of that row and column.)

On another note, I'd also like a built-in function to do memory searches, because using a Lua loop to do this is very slow. In the script below it takes about 9 seconds even when the search is successful. (Though it only has to be done once per started game round.) Should I file this as a separate issue?
Edit: Issue now filed.

addr = 0

tastudio.addcolumn("time", "Time", 60)
local cache = {}

local id
id = event.onframeend(function()
    local needle0 = 0x5755D0
    local needle1 = 0x800

    if memory.read_u32_le(addr, "Physical RAM") ~= needle0 then -- or memory.read_u32_le(addr+4, "Physical RAM") ~= needle1 then
        local data = memory.read_bytes_as_array(0, 0x03000000, "Physical RAM")

        addr = 0
        for i = 0, #data - 8, 4 do
            local w0 = data[i+1] + (data[i+2] << 8) + (data[i+3] << 16) + (data[i+4] << 24)
            if w0 == needle0 then
                local w1 = data[i+5] + (data[i+6] << 8) + (data[i+7] << 16) + (data[i+8] << 24)
                if w1 == needle1 then
                    addr = i
                    console.log(string.format("Found at: 0x%08X", addr))
                    break
                end
            end
        end
    end
    if addr == 0 then
        event.unregisterbyid(id)
    else
        local value = memory.read_u32_le(addr + 8, "Physical RAM")
        msg = string.format("%d.%02d", value // 50, (value % 50) * 2)
        cache[emu.framecount()] = msg
        gui.text(1, 27, msg)
    end
end)

tastudio.onqueryitemtext(function(frame, column)
    if column == "time" then
        return cache[frame] or "?"
    end
end)

[YoshiRulz]

Is that Win7? And are you using release 2.11.1?

[Davidebyzero]

Yes, and yes. And I'm quite grateful that 2.11.1 still works in Win7.

[vadosnaprimer]

On another note, I'd also like a built-in function to do memory searches, because using a Lua loop to do this is very slow. In the script below it takes about 9 seconds even when the search is successful. (Those it only has to be done once per started game round.) Should I file this as a separate issue?

Yes.

[SuuperW]

I would suggest providing a different way to output to a custom column. Probably the best way would be to provide two built-in tastudio Lua functions, one to write a given string to a given row, and one to read the string from a given row.

Having Lua write changes using a new API, instead of TAStudio querying the information for each visible cell on each redraw, seems like the best solution. Although as I said on Discord this will require significant code changes in order to ensure efficient redrawing at the correct time.

Why would you want a Lua function to read the string? That seems unnecessary, since you can keep a Lua table with that info in Lua, and accessing that will be faster than calling a BizHawk API method.

We should do something similar with the onqueryitembg and icon methods. I imagine background colors are somewhat more likely to change rapidly or change many cells (since a user may want to color an entire row). So we would probably want to have the color ("bg") one to also have a version that sets the entire row (excluding cursor column and maybe frame column?) and/or a version that accepts a Lua table mapping row+column combinations to new values.

[Davidebyzero]

Why would you want a Lua function to read the string? That seems unnecessary, since you can keep a Lua table with that info in Lua, and accessing that will be faster than calling a BizHawk API method.

I'm glad you asked. I should have explained this in the first place...

If the API is presented in this way, BizHawk will have to store the written elements in its own array. If the Lua program also wants to be able to read elements already written, and there isn't a matching BizHawk function to read them, the Lua program will have to keep its own copy of the array, meaning it'll take at least twice as much RAM. That would be pretty inefficient.

[kalimag]

Another similar approach could be to let the script opt in a column for caching, which would then make BizHawk use greenzone-like logic to maintain the information. I don't know how much code can be reused on the BizHawk side for that, but it would reduce the complexity on the side of the scripts (which would still have to handle invalidation otherwise).

[SuuperW]

Lua can very easily handle invalidations already with tastudio.ongreenzoneinvalidated.