Weird results on reading .pixels array

[camelo003]

Most appropriate sub-area of p5.js?

• Accessibility
• Color
• Core/Environment/Rendering
• Data
• DOM
• Events
• Image
• IO
• Math
• Typography
• Utilities
• WebGL
• Build process
• Unit testing
• Internationalization
• Friendly errors
• Other (specify if possible)

p5.js version

v1.11.1

Web browser and version

Mozilla Firefox 136.0 (and others)

Operating system

Linux (and others)

Steps to reproduce this

idw if is actually a bug, but i notice a very different behavior when running a simple sketch on normal and private mode of Firefox. in case im looping trough the .pixel array of a tiny image created with the createImage() function and put them on a both DOM's paragraphs (after converting to binary and hexadecimal values). a mousePressed() function update the image and call the functions that update the paragraphs.

the strange part is that even in the online editor, as in a version hosted locally on my computer, or hosted on a github page i notice the following: in a normal firefox window everything goes as expected. but on a private tab strange results appear in the paraphs.

links to the both online editor and github pages versions:

• https://editor.p5js.org/camelo003/full/BxvoYnL_X
• https://camelo003.github.io/priwiixels/

Steps:

1. open the links above on normal and incognito modes
2. click repeatedly on any of the withe squares
3. check the results on binary and hex paragraphs on each mode

the source on both versions are on the following links, but ill paste the related functions

• https://editor.p5js.org/camelo003/sketches/BxvoYnL_X
• https://github.com/camelo003/priwiixels

Snippet:

function mousePressed() {
  if(mouseX < 25 || mouseX > 375 || mouseY < 25 || mouseY > 375)
    return;
  let cellSize = 350 / 8;
  let x = Math.floor((mouseX - 25) / cellSize);
  let y = Math.floor((mouseY - 25) / cellSize);
  grid.set(x, y, selectedColor);
  grid.updatePixels();
  updateBits();
  updateHexs();
}

function updateBits(){
  bitsStr = "";
  grid.loadPixels();
  for(let i = 0; i < grid.pixels.length; i = i + 1){
    if((i % 4) == 3)
      continue;
    bitsStr += Number(grid.pixels[i]).toString(2).padStart(8, "0") + " ";
  }
  grid.updatePixels();
  if(selector.active){/*weird string manipulation to highlight parts of <p>*/}
  bitsP.html(bitsStr);
}

function updateHexs(){
  hexStr = "";
  grid.loadPixels();
  for(let i = 0; i < grid.pixels.length; i = i + 1){
    if((i % 4) == 3)
      continue;
    hexStr += hex(grid.pixels[i], 2) + " ";
  }
  grid.updatePixels();
  if(selector.active){/*weird string manipulation to highlight parts of <p>*/}
  hexsP.html(hexStr);
}

[welcome]

Welcome! 👋 Thanks for opening your first issue here! And to ensure the community is able to respond to your issue, please make sure to fill out the inputs in the issue forms. Thank you!

[limzykenneth]

I am able to see this bug as well, it's pretty weird that it happens only in private windows though. Will try to find some time to look into it.

[limzykenneth]

Just an initial report, the problem seems to stem from grid.loadPixels() being called more than once in updateBits() and updateHexs() where it should just be called once at the beginning of the sketch only (which is already done). By commenting out those two grid.loadPixels() I no longer see weird pixel values.

As for why that is the case, my current suspicion is that it has to do with image compression artefact and repeatedly modifying the image and then loading its pixels triggers image compression at some point that create subtle compression noise. As for why it happens only in private windows, I don't have an idea yet so still looking.

As a fix, which is good practice anyway, is to only call loadPixels() only once, we can possibly change the internal implementation to not run loadPixels()'s code again when it has been called before though.

[camelo003]

Thanks! That's intressing... I though that i have to call the .loadPixels() method every time i want to read new data from the pixel array. Anyway, im still curious about the behavior seem consistently different on incognito mode.

Also, i had notice that the values are not consistent between the bits and hex, so it maybe something on writing directly on DOM's element? The full implementation of the updateBits() and updateHexs() does some gimmick to span css class to highlights parts os the paragraph (despites the weird results appear even when the highlights are not active).

[ksen0]

@limzykenneth "As a fix, which is good practice anyway, is to only call loadPixels() only once, we can possibly change the internal implementation to not run loadPixels()'s code again when it has been called before though."

Should there be an FES warning (or error) for multiple calls?

[limzykenneth]

I think loadPixels() can just check if pixels is populated and no-op sliently if that is the case. However, I am less sure about that statement myself now that I had some time to think about it, eg. if someone is not exclusively manipulating pixels but also drawing on the canvas with things like ellipse(), loadPixels() possibly update the pixels to include the newly drawn graphics in that case. There is possibly valid use case for calling loadPixels() multiple times.

[limzykenneth]

Here's a minimal reproduction if anyone wants to test:

let img;

function setup() {
  createCanvas(400, 400);
  img = createImage(8, 8);
  img.loadPixels();
  for(let i=0; i<img.pixels.length; i++){
    img.pixels[i] = 255;
  }
  img.updatePixels();
}

function draw() {
  background(220);
  
  image(img, 0, 0, 400, 400);
}

function mouseReleased(){
  img.loadPixels();
  
  img.pixels[0] = 0;
  img.pixels[1] = 0;
  img.pixels[2] = 0;
  console.log(img.pixels);
  
  img.updatePixels();
}

I'm going to try replicating with native Canvas API and if I can also replicate it there (which I suspect likely to be the case), I might just file this against Firefox directly instead.

Edit: Ok, it is the same with just native API so this just a bug on Firefox side and probably not something we can fix directly.

const canvasEl = document.querySelector("#canvas");
const context = canvasEl.getContext("2d");

let imageData = context.getImageData(0, 0, 8, 8);
let data = imageData.data;

for(let i=0; i<data.length; i++){
  data[i] = 255;
}

context.putImageData(imageData, 0, 0);

document.addEventListener("click", () => {
  let imageData = context.getImageData(0, 0, 8, 8);
  let data = imageData.data;

  console.log(data);  
  
  data[0] = 0;
  data[1] = 0;
  data[2] = 0;
  
  context.putImageData(imageData, 0, 0);
});

[camelo003]

Great work! Marking as cause since is a Firefox bug. Thanks you all.

[limzykenneth]

Just an update that the Firefox issue has been logged here: https://bugzilla.mozilla.org/show_bug.cgi?id=1963094

The behavior is somewhat by design as an anti-fingerprinting technique. There may be room to explore the exact implementation but discussion will be in the Firefox issue instead of here since we won't be able to fix this behavior.