Feature 1

[UPstartDeveloper]

Build out the first feature (tests then feature)

User story: User is able to move their hand in front of the webcam, to see the controller move on the screen.

[UPstartDeveloper]

This will most likely require using Handpose, and setting up a project similar to the Jenga game on Handsfree.js's website.

[UPstartDeveloper]

There will be 2 main components to this feature, based on the source code of the demo project above:

1. Need to have a JS to utilize Handsfree.js's Handpose model - may not be the hardest thing, except for the gestures I'll need to configure
2. Need to use Three.js to set up the scene, e.g. the ball that the user uses their hand controller to pick up, move around, drop, stack on top of other things, etc.
3. Need to make sure our project also includes HTML/CSS just to make sure the UI looks user-friendly and follows UX conventions.

[UPstartDeveloper]

My personal goal for the next week will be to focus on learning Three.js so we can handle #2,
then come back to read more on Handsfree.js so we can integrate the Handpose model,
and then finally we'll add Bootstrap for the HTML/CSS.

[UPstartDeveloper]

Engineering Standup: January 18, 2021

1. Yesterday: Added the HelloCube project, which now contains the code we can use to add a rotating cube to the scene.

2. Today: will need to go through more of the Three.js fundamentals, and learn more about creating a scene we can use the hand controllers in (from Handsfree.js).

3. Blockers: the milestones left to complete Feature 1 is mainly 1) having interactive Three.js scenes, and 2) being able to move the camera in the scene. Still need to go through more Three.js docs before I understand that though.

[UPstartDeveloper]

Engineering Standup: January 21, 2021

1. Previous: Updated the demo HelloCube project to use CSS style, so the UI is more user-friendly (adding sidebars, and making the site more responsive).

2. For Now: continue through more of the Three.js fundamentals and learn more about creating an interactive scene we can use the hand controllers in (from Handsfree.js).

3. Blockers: Still need to go through more Three.js docs before I understand enough of the details

[UPstartDeveloper]

Engineering Standup: January 31, 2021

Posting a link to the new WebXR Chrome extension for Handsfree.js for better visibility. Hopefully this will enable better gestures to be added later to the project!

No other progress to report currently.

[UPstartDeveloper]

Engineering Standup: February 7, 2021

1. Previous: learning from the sample Jenga game on Handsfree.js.
2. Today: added HelloAnimations, a demo Three.js app that includes interactive controls (press keys 1-8 to make the characters do different movements).
3. Blockers: this demo app helped me learn some new things about the GLTF file format, so I will continue learning how to properly load *.gltf files using Three.js in the near future. Otherwise, the next step will be making a Three.js app which a user can interface with using hand gestures (most likely using Handsfree.js).

[UPstartDeveloper]

Engineering Standup: February 14, 2021

1. Previous: learning about how to do object picking in Three.js
2. Today: adding gestures so we can make hand interactivity a part of Feature 1 for IQ3's product.
3. Blockers: need to determine how to implement our custom hand controllers by extending the Tensorflow Handpose model.

[UPstartDeveloper]

Engineering Standup: February 17, 2021

1. Previous: added a sample app to the repo for picking objects
2. Next Steps: integrating Handsfree.js with the WebXR extension, to enable head gestures for feature 1
3. Blockers: finding a way to run the sample app from the Handsfree.js repo, built to handle this exact task.

[UPstartDeveloper]

Engineering Standup: February 20, 2021

1. Previous: downloaded Handsfree.js Chrome extension( (in dev mode) on my personal laptop
2. Today: added a demo of Handsfree.js face tracking: "Inside the Cube"
3. Next Steps: finding a way to use multiple Handsfree.js models (for tracking the user) at once, and having them manipulate objects.

[UPstartDeveloper]

Engineering Standup: February 20, 2021

1. Previous: studied the Handsfree.js docs on the Weboji model, which was utilized in the "Inside the Cube" demo.
2. Today: Initialized a new demo to do the same thing as the last demo, except it uses Three.js instead of AFrame.
3. Next Steps: improving 1) smoothness of the camera movements, 2) the frame rate, and 3) possibly the responsiveness of Google MediaPipe. Will be keeping track in Fix Latest Demo on Hands and Face Tracking #5

[UPstartDeveloper]

Engineering Standup: March 7, 2021

1. Previous: experimented with different tweeting values through trial and error
2. Today: released "Cube Space", a new sample app for connecting Handsfree face tracking with Three.js.
3. Blockers: not sure how to balance between responsiveness and accuracy - we can make the face tracking more responsive by decreasing the time spent tweening, however, it comes at the price of making the camera tracking more unstable (which is probably the fault of the Weboji model in Handsfree.js). On the other hand, it's also a bad UX to make the face tracking too stable, as it can get so slow the user will feel like it's stopped being responsive.
4. Next Steps: move ahead with object interactions for now, by integrating the Handpose model from Handspose into the app. This way we'll be able to manipulate the coordinates of objects using a finger caster or something like that. If time, we can also add kinematics by using Physijs, a plugin for Three.js.