How to develop a custom Amazon Sumerian Host avatar?

[IvanFarkas]

How to develop a custom Amazon Sumerian Host avatar?

I want to develop a custom Amazon Sumerian Host avatar, like Luke.

• Is there a service that can help me to do that, perhaps ReadyPlayer.me or Wolf3D?
• How do I do that from scratch using Blender, Reallusion Character Creator, Unreal Engine, or any other Creative Tools?
• Are there any documents for the Amazon Sumerian Host avatar 3D model requirements and specifications about bones, rigging, skinning, etc?

[roschler]

@IvanFarkas Bump. This is something that I have wanted to know how to do too for a long time, especially when it comes to connecting the graphics library, in my case ThreeJS, to move the skeleton (including eyes) and Amazon Polly to move the mouth bone(s).

[trusktr]

It isn't very well documented, but the examples show how to load a custom model. Here's the Babylon example:

https://github.com/aws-samples/amazon-sumerian-hosts/blob/mainline2.0/packages/demos-babylon/src/customCharacterDemo.js

To figure it out, we'll need to look at where characterConfig gets passed into createHost, and figure out what things the Sumerian Hosts code expects from the asset files.

From there, we should be able to generate alterantive GLTF (.gltf, .glb) files from a tool like blender that meets the requirements.

We might also open those GLTF files inside a tool like Blender and see if that serves an example. I'm imagining, for example, if we can import those and we can see the animtion in Blender, we can tweak them (f.e. make the character taller, replace the skin with another, etc), then export in the same format.

[trusktr]

Btw, I have a buildless fork (a fork where everything is plain JS modules, requires no build), here:

• [WIP] Convert to vanilla JavaScript modules (ES Modules) #183

[flckv]

hi @trusktr (1) did you choose to use vanilla js to avoid build issues with aws sumerian avatar? Do you think build issues happen with frameworks e.g. react ? #173
(2) I can see you are invested in moving sumerian to vanilla js and thank you for the great work!! I am guessing you tried importing new outfits and new characters with blender( or others) and not worried about issues like #37 ?

[IvanFarkas]

It isn't very well documented, but the examples show how to load a custom model. Here's the Babylon example:

https://github.com/aws-samples/amazon-sumerian-hosts/blob/mainline2.0/packages/demos-babylon/src/customCharacterDemo.js

To figure it out, we'll need to look at where characterConfig gets passed into createHost, and figure out what things the Sumerian Hosts code expects from the asset files.

From there, we should be able to generate alterantive GLTF (.gltf, .glb) files from a tool like blender that meets the requirements.

We might also open those GLTF files inside a tool like Blender and see if that serves an example. I'm imagining, for example, if we can import those and we can see the animtion in Blender, we can tweak them (f.e. make the character taller, replace the skin with another, etc), then export in the same format.

You misunderstood the issue.
Please read the description again.
It's not about code but using graphics tools!

[IvanFarkas]

Btw, I have a buildless fork (a fork where everything is plain JS modules, requires no build), here:

• [WIP] Convert to vanilla JavaScript modules (ES Modules) #183

The real help would be a Typescript version or a wrapper, but the nasty Monkey Patching and blatant overuse of archaic mixins instead of modern and elegant design patterns, it's very hard.
Probably a properly written TS wrapper would be the way to solve it.

[trusktr]

@flckv off topic about the build:

hi @trusktr (1) did you choose to use vanilla js to avoid build issues with aws sumerian avatar?

Yeah, I removed the build (and run as JS modules natively in the browser) to avoid having to spend time solving build issues. It makes maintenance easier, and I'm planning to do some things with this code, so I don't want Webpack to be a blocker.

Once written as plain JS modules, it will work in a browser forever (clone this repo 10 years from now, it will simply still work).

Better if people manage their own build choices in their apps, and they can choose to take on that burden.

Do you think build issues happen with frameworks e.g. react ? #173

Yeah, that issue does not happen with the vanilla approach. If someone adds a build tool to their app, they might run into any number of issues with the build tool, such as in #173. It's more of a webpack issue than a Sumerian.

---

(2) I can see you are invested in moving sumerian to vanilla js and thank you for the great work!! I am guessing you tried importing new outfits and new characters with blender( or others) and not worried about issues like #37 ?

I haven't done this yet, and would like to learn soon! But it is not related to the build, it will be the same process either way.

[trusktr]

You misunderstood the issue.
Please read the description again.
It's not about code but using graphics tools!

I fully understood the issue!

Without documentation, the hard way is

• to look at the assets, and how the code works with them.
• This will inform us what parts are needed from the imported model in the code.
• In turn, this informs us what we need to export into the GLTF.
• This in turn allows us to play with Blender to figure out how to export those things into the GLTF.

It is the reverse engineering approach.

Of course documentation would be better, but so far, I haven't seen any!

[trusktr]

Probably a properly written TS wrapper would be the way to solve it.

What the "it" that that will solve? I wasn't sure what you're referring to there.