iPi Soft Desktop Motion Capture (Interview)

Posted by Michael Muwanguzi on May 23, 2012 | 0 comments

Interview with Michael Nikonov

I was very fortunate to talk with iPi Soft founder and software developer Michael Nikonov before testing out their new motion capture software.

Michael Muwanguzi: Can you tell me a little bit about your background and what led to the trackerless motion capture system?

Michael Nikonov: I don’t actually have a background in animation or movie making. Before designing this system I was a software developer in business. This is my first experience in the animation industry.

This project grew out of my hobby. I was trying to make a video game, and was surprised how difficult it was to create an animation. I wanted to use a motion capture system, but found the available options extremely expensive. So my logic was: I have a video camera, why can’t I do motion capture? Eventually I designed a system that works with off the shelf video cameras.

Now we have a system that supports several popular cameras, including the PlayStation Eye Camera, which is a very inexpensive $30 camera, and you can use up to 6 of them. Or you have a choice of using one or two Kinect cameras.

MM: The current way of capturing motion seems cumbersome and very expensive. What is the difference between these two technologies and what makes markerless motion capture possible?

MN: Motion capture as a concept is not new. A number of technologies existed before markerless motion capture. A very popular technology is optical motion-based motion capture, where you have multiple custom cameras for motion capture. This requires your actor to wear a suit with markers.

There are a number of technologies including motion capture based on inertial sensors where you don’t need cameras. But what they have in common is that you need a suit with markers for sensors and since it requires custom hardware, it is extremely expensive. (A typical cost is about $70,000 for a system.)

With our system, you can use off the shelf hardware and you don’t need a suit for the actor. The actor can wear casual clothing like jeans or shorts and the camera is pretty inexpensive. For a six camera system, for example, the cost would be $30 a piece, which is under $200 total.

So, in terms of comparison to traditional motion capture systems, our markerless solution costs
much lower and the accuracy is actually comparable.

MM: What can users expect from using more than 2 Kinect cameras to capture movement? What’s going on in the technology with the extra data?

MN: As a general rule, more cameras give you better accuracy and better coverage. But with our system, you can still use one or two Kinect cameras, or you have a choice of using Playstation Eye Cameras, which are more traditional 2D cameras. (And you can use up to 6 of them.)

There is a difference due to Kinect hardware limitations: The Kinect gives users a small capture area — around 2×2 meters. We support up to 2 Kinect cameras because it uses much more USB bandwidth. Using one Kinect in the front vector and the other for the back vector will give users 360° of coverage.

The PlayStation Eye Cameras provides a large capture area. We tried 7×7 meters (around 20×20 feet) and find the coverage is quite good.

MM: As far as using 6 PlayStation Eyes or 2 Kinects, would you say the quality of capture is comparable? Is there a setup you prefer?

MN: Yes, the accuracy is very similar. The dual Kinect system is easier to use, but with PlayStation Eyes you can have much bigger capture area.

MM: I have not had a chance to use the dual Kinect motion sensor software yet, but I have gotten a chance to check out some of the videos and final productions. I saw in the FAQ that facial capture isn’t available at the moment. Do you believe that it can be possible through improvements in software, or will we have to wait for the next evolution in the current camera hardware?

MN: Well, with the current level of human technology shooting full body capture and facial capture at the same time, and with the same cameras, is not possible because camera resolution is not good enough.

MM: What if you focused the camera(s) just on the actor’s face for the sole purpose of capturing motion for lip syncing? Would that be feasible?

MN: Yes, technically this is quite possible, we just don’t do that because we are currently concentrating our efforts on the full body motion capture software. Another reason why we don’t do facial motion capture is because there are 2 or 3 other companies that do a very good job of markerless facial motion capture. One of them is Image Metrics, which a lot of video game projects already use. It’s a bit expensive, but it is very good.

I’ve seen a couple new companies that are also doing pretty good markerless facial motion capture software. We don’t want to reinvent the wheel but rather are focusing on what is missing from the market. For markerless full body motion capture, we suggest combining the existing facial capture technology with ours.

MM: Do you think that it is possible to take these cameras to an outdoor location and film in an actual environment? There would be a lot of advantages to being able to capture a practical performance from actors with an HD camera as well as tracking data simultaneously in the future.

MN: Well, this is quite possible and we have some examples of our users doing that. There is video created by one of our users, Nicolas Brunet, and you can actually see it in the video section on our website. It’s a making of video for a CG movie “Danse Kabyle”.

Although this is possible, we do not recommend this for novice users because there can be a number of challenges. For example, it’s very difficult to do markerless motion capture in direct sunlight. You definitely want to do it on a cloudy day.

MM: iPi Soft is a relatively new company focusing on motion capture software technologies. Can you talk about the future of the company and the type of products that people can look forward to from you?

MN: Currently we are concentrating on improving motion capture software and are excited about the release of iPi Soft Motion Capture Version 2 this month. We have found our high-end clients asking for hardware of higher quality. So it is quite possible that we will release our own motion capture cameras in the future and high-end software packages bundled with these cameras.

MM: Do you plan on keeping the camera pricing comparable to the price range of the $150 Kinect camera?

MN: We do not plan to abandon the inexpensive option of our software, but instead introduce a more expensive camera system combined with a professional version of our software. It will cost more money, comparable to existing motion capture systems, but will have larger motion capture volume and more useful features.

Basically, are taking cues to develop products based on the feedback from our current users.


More information about iPi Soft and there products can be found on their site. If you are interested in the results of working with their motion capture software, check out our review here.