Render a video on a Cube #322

[robinroy03]

Render a video on a cube (#322)

This PR adds a new actor - texture_on_cube to fury/actor.py which takes 6 different texture arguments for the different sides of a cube. It also adds the tutorial and tests for the same.

How it is done

I took 6 planes and merged them to form a cube.

How is video rendered

Video is rendered by changing the texture repeatedly.

Simple static texture demo:

from fury import actor, window
import numpy as np

arr_0 = np.full((720, 1280), 0, dtype=np.uint8)
arr_255 = np.full((720, 1280), 255, dtype=np.uint8)

arr_red = np.dstack((arr_255, arr_0, arr_0))
arr_white = np.full((720, 1280, 3), 255, dtype=np.uint8)
arr_aqua = np.dstack((arr_0, arr_255, arr_255))
arr_green = np.dstack((arr_0, arr_255, arr_0))
arr_blue = np.dstack((arr_0, arr_0, arr_255))
arr_yellow = np.dstack((arr_255, arr_255, arr_0))

cube = actor.texture_on_cube(arr_white, arr_red, arr_green, arr_blue, arr_yellow, arr_aqua)
cube2 = actor.texture_on_cube(arr_white, arr_red, arr_green, arr_blue, arr_yellow, arr_aqua, (3, 3, 3))

scene = window.Scene()
scene.add(*cube)
scene.add(*cube2)

window.show(scene, size=(1280, 720))

FURY.0.1.0.dev5378+gea0fb16.2024-03-17.15-17-32.mp4

[pep8speaks]

Hello @robinroy03, Thank you for updating!

Cheers! There are no style issues detected in this Pull Request. 🍻 To test for issues locally, pip install flake8 and then run flake8 fury.

Comment last updated at 2024-05-01 10:27:17 UTC

[skoudoro]

Hi @robinroy03,

Thank you for this, I will try tomorrow and give you an update

[skoudoro]

during this time, can you fix all the pep8 issues?

[robinroy03]

@skoudoro I've fixed all the pep8 issues and have placed some of my thoughts in the code as comments. (and also in the PR description)
I'll add tests and finish this once you're ok with them.

[skoudoro]

Hi @robinroy03,

Thank you for this. This is a good start.

The goal was to show, a different video at each face of the cube. if you want to use the same video, they should start at different time point.

it is ok to use the texture, but you need to control the different part of your texture.

After, I will be able to look deeper at the code.

Thank you in advance for your future update.

[robinroy03]

Hi @skoudoro,

Are there any inbuilt VTK functions that do this? (mapping different textures to different sides of a cube)
Because when I searched I found this (https://discourse.vtk.org/t/how-to-apply-different-textures-to-each-face-of-a-tetrahedron/2333/2), and I'm not sure if I should go this path directly.

Are there any resources for UV-Mapping textures in VTK? (I referred both VTKUsersGuide and VTKTextbook, found no examples)

[robinroy03]

I found this fury/io/load_cubemap_texture function and used it. But I am unable to map the cubemap to the cube. (the below code does not work).

import vtk

from fury import actor, window
from fury.io import load_cubemap_texture


tx = load_cubemap_texture(fnames = [
    "negx.jpg",
    "negy.jpg",
    "negz.jpg",
    "posx.jpg",
    "posy.jpg",
    "posz.jpg"
])

scene = window.Scene()

cubeSource = vtk.vtkCubeSource()

mapper = vtk.vtkPolyDataMapper()
mapper.SetInputConnection(cubeSource.GetOutputPort())

cube_actor = vtk.vtkActor()
cube_actor.SetTexture(tx)
cube_actor.SetMapper(mapper)

scene.add(cube_actor)
window.show(scene)

Can I get some help regarding the general direction?

Or should I take 6 PlaneSources and merge them?

[robinroy03]

@skoudoro,

Please check the new commit, I've used 6 planes to make the cube. I can use vtkAssembly() to make them 1 single actor if needed.

Use a cubemap texture from here to test.

If this is ok, I'll finish this code with some more methods such as texture_update, get_actor etc ... (or should I format the code differently? I believe this class should go inside actor.py)

Demo Video:

FURY.0.9.0.2024-03-03.15-53-38.mp4

This code is not rendering a video, but it'll be done in texture_update. I just wanted to know whether my general direction is correct & feedback about how I'm organizing the code.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 84.46%. Comparing base (14851c6) to head (b26842a).

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #862      +/-   ##
==========================================
+ Coverage   84.43%   84.46%   +0.03%     
==========================================
  Files          44       44              
  Lines       10537    10559      +22     
  Branches     1423     1432       +9     
==========================================
+ Hits         8897     8919      +22     
  Misses       1266     1266              
  Partials      374      374              

Files	Coverage Δ
fury/actor.py	85.85% <100.00%> (+0.25%)	⬆️
fury/lib.py	100.00% <100.00%> (ø)

[robinroy03]

@skoudoro
I've finished everything. Please take a look and let me know if I've to make modifications.

[skoudoro]

Hi @robinroy03,

Thank you for this. See my initial comment. My main concern is about the use of a class. I think everything can be simpler and easier to maintain with function

[robinroy03]

I've made the changes. For test_actors, I only did a test for 1 side colour render. I avoided remaining because it all comes from the same pipeline, so must be similar.

[skoudoro]

Hi @robinroy03,

After skimming the code , it looks much better! Please, check our other tutorials to see how it is formatted. Currently, your tutorial will not render well. You can try to render it locally.

I will try to look deeper on your code later today. Thanks for this

[robinroy03]

@skoudoro, I'm sorry but how do I render it locally? Every time I try I get a segf (it's an open issue #481).

I did create docs (by adding the disable examples flag), but my new code is not shown there. What is the procedure to add my code to the docs?

[skoudoro]

What is the procedure to add my code to the docs?

you need to add it in docs/examples/_valid_examples.toml

Every time I try I get a segf

for now, run it multiple times consecutively

[robinroy03]

All the said modifications are done. I also committed the new file _valid_examples.toml.

I think this is a bug, but viz_play_cube.png will be all black unless I turn interactive = True. It takes a screenshot immediately, even before the actors are rendered. But if we give it some time after turning interactivity on, it works fine (since the actors are rendered).
I also noticed these issues with window.snapshot where they take screenshots very immediately even before some actors are rendered.

[robinroy03]

Hi @skoudoro, PTAL.

Thank you.

[skoudoro]

Hi @robinroy03,

Thank you for this.

6 actors is a lot. I am not sure we would like to have this in actor.py. Maybe the function will be only in the tutorial. I will need the opinion of other members of the team.
See above some comments

[robinroy03]

@skoudoro,

All the requested changes are made. Should I modify the function to be placed inside the tutorial? We'd discussed using 6 different actors to render the cube multiple times.

[skoudoro]

All the requested changes are made.

Thanks!

Should I modify the function to be placed inside the tutorial?

I do not know yet

We'd discussed using 6 different actors to render the cube multiple times.

I know... the fact that we talk about it multiple time show that I am not completely satisfy with this solution because I can see long term issue with it. I have hard time to block time to dig in the texture actor to see if there are alternatives or something that we are missing.

[robinroy03]

ok, I'll also keep researching whether there are better ways to do this.