WEEK 12 - Faceware LIVE and Looking for Rigs

More Technical Difficulties…

The beginning of this week has unfortunately had a number of issues that have led to some setbacks and the need to reschedule some of my plans for this week.

As expected the lock-down has been extended by another three weeks. Due to this my idea of using the motion capture lab to stream facial data and motion directly into Unreal has fallen short. I will not be able to have access to the lab until after the final deadline of this module. Potentially I will continue these blogs, as I hope to experiment with this idea after the lockdown is over and I have full access to the lab.

At this time my Faceware free-trial with Iclone also ended. I had hoped with the lifting of quarantine I would have access to a new PC (As the free trials work on a per-pc basis) therefore starting the next 30 day trial for use until the hand in in three weeks (As recommended by the faceware Support). However this was no longer an option. After a day of trying to find a solution to the issue, Faceware Live was the next best option.

Faceware Live

The facial capture programme I have been using in this project so far has been ‘Faceware Realtime for Iclone’. Especially designed for use in Iclone it works as a Plug-in only piece of software.

‘Faceware Live’ is the stand alone version of this plugin found on Faceware’s own website and not through Reallusion. It goes into a lot more depth and is far more versatile as a Facial Capture tool.

When Opening ‘Faceware Live Server’ you can see that most of the things that are built into the ‘Motion-Live Plugin’ in Iclone can be found here in a stand alone form. Things such as calibration and facial animation tuning.

What the Live server brings to the table that the plug-in doesn't is its ability to stream directly into Motionbuilder, Iclone, UE4 and Unity. (https://www.facewaretech.com/software/live)

The cross-platform nature of this tool is great and would allow team members to choose to work with the packages they are most familiar with.

Connecting the Server with Iclone

For Iclone the Faceware Live connection is really simple, and works pretty much exactly the same as before. The only difference is the use of an ip when connecting to Faceware Live and using Live as the face dropdown option.

Once all hooked up the quality of the data is just as good as the Iclone plug-in.

A New Female Rig

Now that I’m running into the last few weeks of this project it has become painfully apparent how self-sufficient I need to be to continue work at a good speed.

Therefore I have made the decision to look for a new rig I can use instead of my male one I made using Mixamo. Although good, this rig has too many graphical issues around the mouth when attempting to create phonemes, and as this project is focusing on speech I don't feel it is suitable to continue using. The rig is also a male so, for continuity, would work best with a male actor.

In isolation It would be easier if I only had to rely upon myself to do the facial capture and voice over for the scene. So I have decided to find a new female rig.

Initially I felt that I could create a new one using mixamo again, however the blend shapes around the mouth are just not quite good enough for speech. In a perfect situation I need a rig which has built in phoneme shapes, but a free rig with that kind of attention to detail with a ‘viking’ is hard to find. There was one rig however that I did find which could help with this issue.

I Found this Lithariel rig on RenderHub. (For copyright reasons it’s important to note that I am only using this game rip to better understand how it has been put together and for educational uses only.) She is a professional rig used in the triple A game title ‘Middle Earth: Shadow of Mordor’ so her rigging and texturing not only looks amazing in Unreal but produces excellent animations. She also has a distinct warrior feel, and although I know her hair is designed with elves in mind, the blonde braids work well for the viking in my short scene.

On import into Maya, she had a lot of naming and mesh issues (likely due to the nature of acquisition) but her facial rig was perfect for use in iclone. Above you can see the first problem which was that the mesh smoothing groups did not transfer over to unreal.

I fixed this by taking the rig into maya, using the Mesh > Smooth on all poly surfaces then deleting the history. This of course deleted the skinning on the smoothed mesh so I then had to transfer the skin weights from the blocky mesh back onto the now smooth mesh using the Skin > Copy Skin weights tool. The mouth had a couple of errors on the transfer but nothing a quick skinning touch up couldn't fix.

All In all the rig touch ups took a weekend before I could then bring it into 3D exchange and ready it for use with performance capture in Iclone. This time I have recorded the entire 3D exchange process for use in my breakdown reel, here it is sped up…