IODA – interexchange format of wavetables between oscilloscope tools

[ffd8]

Hey oscilloscope (+ lasers + vectrex) community!

Next update to XYscope will include an OSC demo for making it possible to export/share the wavetables created via XYscope to other software/tools since maybe it's only one part of a workflow etc etc... But it could/should also have an import!? There have been amazing tools created for a multitude of languages and environments – sooo I thought it would be great to have a small discussion in a space where markdown code-snippets can be included, to pan out what might be a format most (hopefully) can agree upon for shipping these wavetable values (array of numbers for x, y, [z] axis) that could be imported and exported from each toolkit. = IODA (International Oscilloscope Display Association!)

XYscope is made for Processing (Java), and my initial ideas were –

sending a 2D array, ie:
[[x, y], [x, y], [x, y].....]
which can then be looped through once (by default ~ 512 values), then grabbing the x and y out from that inner array of values.
this is a very compact message to send via OSC (avoiding JSON due to extra chars needed).

Or again, having a 2D array that's joining the 2 wavetable arrays in this form:
[[x, x, x, x, x.....], [y, y, y, y, y, y...]
this would allow one to first split the message into the 2 separate arrays, then loop/pass separately to wavetable inputs.

Or would it be better to be sending 2x 1D arrays, one per channel ie:
[x, x, x, x, x, x, x, x...] + [y, y, y, y, y, y, y, y....]
My concern here is if the syncing might be messed up in transmission if not part of the same message...
all depends on if all environments can handle a 2D array?

Perhaps a dash of JSON isn't so bad, ie if going with the 3rd option, something like:

{
    'freq': 50,
    'x' : [x, x, x, x, x, x...],
    'y' : [y, y, y, y, y, y, ...]
}

Rough goals are:

• send as few chars as possible for the OSC message overhead..
• be parsable (import) and packable (export) in all languages being used in the community

I'm sure there's many other techniques, ideas out there – what are they??
@kritzikratzi @macumbista @jameshball @jestern77 + please tag folks that should join this conversation

[kritzikratzi]

generally interleaved is much more practical than planar formats (ie. [x1,y1,...] is preferrable over [x1,x2,...],[y1,y2...]).

another thought is... why not just use the ILDA format

[ffd8]

nice, also a +1 from my side for the [[x1,y1,..], [x2,y2,..], ..] format.
it could vary how many values are included, but use the order: [x, y, z, r, g, b]

as for the ILDA format.. a quick overview suggests its more complex than we need for passing the basic audio-data around?

though interesting the ILDA uses int values (= like floats to fixed prec?):

Extreme left is-32768; extreme right is +32767.

nevertheless, i'd suggest to use the normalized values of a wavetable, ie -1.0, 1.0

[kritzikratzi]

first of all: i want to steer this away from json, as i'm quite certain its too expensive in many ways related to audio processing terms of memory, processing time and also binary representation.

i think it would make sense to first think about the types of data transmitted:

• 1ch => y wavetable
• 2ch => x,y wavetable
• 3ch => x,y,z or x,y,z-modulation wavetable (ambiguity alert!)
• 4ch => x,y,z,z-modulation
• 5ch => x,y,r,g,b wavetable
• 6ch => x,y,z,r,g,b

the only ambiguity are 3ch files, where the last channel could either be either brightness ((r+g+b)/3) or a z-coordinate. maybe this is something that could be standardized, that there is no x,y,z-modulation format. imho it would be the most logical thing to do, otherwise you end up also thinking about "y,z-modulation" which opens up yet another ambiguitiy.

in the oscilloscope simulator i currently resolve this by showing a ◐ icon which lets you use the third channel as either z-modulation or z, ie. shifting the blame to the user :)

ilda, in this regard, is not the most solid choice, because only the formats 4 (x,y,z,r,g,b) and 5 (x,y,r,g,b) are of interest. the formats 0,1,2 have indexed color which i believe no cares about any more at all.

i've made a very quick and rough explanation of how this could look like and attached it as a pdf.

ioda-draft-1.pdf

[kritzikratzi]

what do you think about repurposing wav? i kinda think it makes sense, but then again, i'm not alone in this world :)

[jameshball]

Agree with @kritzikratzi here that it makes sense to repurpose existing stuff especially to prioritise easy implementation. At the very least if we were going to go the custom route, a binary format like bson would be better. My main question though is what benefit do we gain over just using multi-channel .wav directly? Things like 'fps' are inherently baked into a .wav file by just considering the length of the .wav file along with the sample rate.

I also agree it makes sense to keep this pretty generic and don't think we necessarily need to worry about differences between z or z-modulation for example - especially since each software will be different and doesn't need to support 3D coordinates necessarily.

A standardised way of piping outputs between programs would be great, but I wonder whether we are just recreating something that's already done very well with audio plugins in DAWs? I appreciate the VST standard is far from easy to implement, but I think in a perfect world we would all support this instead.

What do you think?

[kritzikratzi]

I also agree it makes sense to keep this pretty generic and don't think we necessarily need to worry about differences between z or z-modulation for example - especially since each software will be different and doesn't need to support 3D coordinates necessarily.

imho the distinction is quite important.

for an application that handles only 2d data you can freely discard all z coordinates and only keep all (x,y) pairs. but if it's a z-modulation value, then you want to discard all samples that have z-modulation=0 and keep the rest.

of course it's not too much of a problem, but i think being clear about what information the data contains is the best thing we can standardize.

@macumbista what do you think about this? i know you sometimes use 3-ch files with the meaning (x,y,z-modulation). would you be willing to move to (x,y,z,z-modulation) and setting z=0? as i said, i think the ambiguity is not too big of a deal, but if we can agree now we will have good times in the future :)

ps. @jameshball transport and vst are a whole other story, also important, but i guess one thing at a time helps us not get completely lost here ^^ especially considering that with a continuous transport you would typically set the parameters only once (sample rate, channel count, bitrate and data format) and then stream the data to keep things minimal. but this is really more the territory of jack audio, interapp audio/audiobus and friends.

[jameshball]

VST is definitely a whole other story - definitely what I think the end goal should be though (and working on it myself) but completely agree that we should take it a step at a time!

I'm not hugely familiar with .wav but I wonder if we can do something sneaky like with laserboys format to make use of unused bits? Could then distinguish z-brightness/coordinate?

[ffd8]

Heyhey - thanks for the insights above @kritzikratzi + @jameshball – though it's probably too early for such a draft PDF until more discussion on here. Very interesting idea about repurposing a .wav, however my initial motivation for this interexchange was for real-time passing of values, ie, sending info via OSC between software tools. While I could pipe audio from Processing into some DAW and take advantage of effects or modulation there– I image there's an advantage to sending the waveform data instead, which could then be parsed and used at any freq or modulated with other shapes/tool-specific-things. Nevertheless, saving this raw-data to a format for async transfer, storage, exchange would also be useful – in which case isn't just a plain .txt file all we need?

As for the order of array values – I think it's important that they always stay the same and work down from there, using 0's where z-modulation is if not being used. I also think we should only consider z to be z-modulation, rather than z-coordinate.. since it's about the waveform/oscillators that would go to an oscilloscope/vectrex/laser. My suggestion would be:

• 2ch [x, y]
• 3ch [x, y, z-mod]
• 6ch [x, y, z-mod, r, g, b]

If one only only cares about 5ch, ie for a laser, then the z-mod should just be filled with 0's and ignored, so one is always using [0] for x, [1] for y, [2] for z, [3-5] for r, g, b.... then there's no confusion about is the 3rd index sometimes z or r. Is there a need for 1ch [y]? What's an example of only using that value?

[kritzikratzi]

only technical remarks:

• z-mod, if not used, should be 1 which would be "everything visible" (opposed to z, where a default 0 would mean 2d)
• i'm not sure the z-mod in (x,y,z-mod,r,g,b) carries information, because z-mod=(r+g+b)/3. like... unless i'm missing something z-mod would only act as a global brightness in this case anyways.

[ffd8]

• Good point – thought '0' would be fine if it's not being used/parsed on the reading end – but 1 makes sense to refer to always on.
• and i'm probably botching terms.. yeah by z-mod i just mean z-axis/blanking. its been a while since i've messed with the z-axis (i know you're quite anti, in a good and impressive way of not needing!). so i'd imagine the main use of that value would be for the 'z-axis' input of a scope, aka global brightness.

[kritzikratzi]

to clarify the second point:

z => z-coordinate/z-axis (ie. 3d)
z-mod => z-modulation (ie. brightness / z-input / blanking)

imho both have their justification.

[ffd8]

sure the difference is clear and my 2cents is to leave the z-axis (coordinate) out of this wavetable transport – the unique aspect of vector synth/oscimusic process. I imagine XYZ coords could/should be transported separately, send the array of drawing shapes, normalized XYZ coords to send between tools, potentially taking advantage of how each technique renders those shapes? thus a 3D array:

[shape][coordsize][x, y, z]
[[[x1, y1, z1], [x2, y2, z2], ...], [x1, y1, z1], [x2, y2, z2], ...]]