Feature suggestion "Auto Segmented Generation"

[GreatMCGamer]

Yes, I get it you can't force the AI to make a slow song into a 10 star map.
That's not what I'm after.

Add a check box for "Auto Segmented Generation"
This feature will generate a map in 30s segments.

First segment:
Generates 30s of the song

Consecutive segments:
Copy last 8s of previous segment. (counted from last object)
Delete 4 seconds from the end of it.
Generate next 34s.
Repeat.

Finalization:
Merge all segments.

What is the goal of this small script?
Well I've personally grown a liking to generating marathon maps with the AI.
But these maps have a major flaw.
They have slow sections in fast sections just because the "kiai time" already happened.
By the AI not having context of what was already generated other than the last 4 seconds, the AI will always generate "Kiai time" style to the whole fast section, every fast section.

Personally I'll use the feature for generating 1-3 hour long marathon maps that I like to listen to... I mean play. osu! is a game, not a music player with extra steps, surely.

2 hour long marathon map for context:
https://www.youtube.com/live/Ip1m8xZQuGQ?si=VkPEc2YhWACHkB-l

[OliBomby]

What you describe is already how Mapperatorinator generates maps, just with a bigger overlap window. The settings for how much overlap there is is exposed in the inference config files. You could look into these and play with the lookback and lookahead options to see if they have the desired effect.

[GreatMCGamer]

So is the diffusion Settings max_seq_len: int the size of how much to generate at a time?

[OliBomby]

That max_seq_length specific to the diffusion process which isnt used at all for V30

[GreatMCGamer]

After talking with copilot, If I understood correctly, src_seq_len in /osuT5/osuT5/config.py defines the size of the sequence that gets generated...
With the default values, 1 sequence equals around 8 seconds.
If this is true, There has been a major mistake in training of the AI's.

osu roughly defines a maps difficulty based on a minimum of 20s window.
If the AI's were stuck reading examples that are only 8 seconds long, it would read 5* patterns multiple times and be hit on the head for calling it a 5* pattern when it was part of a 20s sequence that creates a 7* pattern.

Even if you had prepared training data as slices that each get individually evaluated for star rating and then fed to the AI.
Those chunks would've been too short to satisfy the full 20s window required for accurate star rating evaluation.
(Minimum ranked map lenght is 30s, add double time mod and it becomes 20s)

[OliBomby]

Yea some models use 8 second windows and other a bit bigger with 32 seconds (V30). However this short sequence window shouldnt significantly affect how the model handels star ratings.

During training a model gets a 8 second sequence of beatmap and the star rating of the full map e.g. 7*. Ofcourse that 8 seconds in isolation is not 7*, but the model understands that its only a small part of the map, and that if you repeat this intensity many times until its 3 minutes long, you will end up at 7*.

During training it is conditioned on the star rating of the full map and the length of the full map.

[GreatMCGamer]

Well, that explains why V30 is "the best model".. And why none of the models have proper understanding of star rating.
They only have loose understanding of Star_rating * music_intensity(0-1 float).
Would need some major restructuring to remove that equation from being embedded to the model.

[OliBomby]

The real issue is lack of context. The current approach works well enough for normal beatmaps, but to truly reach an exact star rating you need to plan the whole song ahead so you know which parts to put the difficulty spikes. It's just very hard to cram that much context into a small model.

[GreatMCGamer]

The model that generates the map should be dumb as a boot.
It should purely generate the desired star rating.
Only context during training the AI needs is the exact star rating of the specific (8-32s) segment it's looking at. Just gotta process the training data to include that context, the AI training handles the rest.

By manual, or future automatic segmentation, the model will appear "intelligent" by having the song segmented and each segment rated.

[OliBomby]

It's not a bad idea, it could make the difficulty a lot more controllable, and training it is definitely possible. The downside is that during inference we need a very detailed intensity graph for the model to follow. This could be a pain for anyone doing it manually, and to automate it we would need a separate model that produces this graph using full context of the song. Also still possible.

What about other controllable parameters though? Should the mapping style also be provided per segment? What about a 2-layer model where the top layer model generates these per-segment ratings and the layer that generates the map is dumb as a boot?

I see a lot of possible directions.