2. Frequently Asked Questions (FAQs)¶
2.1. General¶
2.1.1. How does Theia3D work?¶
Theia3D is a markerless motion capture software that estimates 3D human pose from multiple synchronized 2D camera views using deep learning algorithms.
After using a calibration method to determine the position and orientation of every camera in the camera system, deep learning algorithms are applied to the videos on a frame-by-frame basis to identify and track over 100 landmarks on the visible humans in the 2D videos. Using the camera system calibration parameters, those landmark detections from multiple 2D videos are then triangulated into 3D landmark positions, and a skeletal model is scaled and optimized to track the motion of those landmarks. This scaled skeletal model made up of 17 rigid segments represents the 3D pose estimates of the tracked people within the videos, from which numerous kinematic biomechanical signals can be obtained.
2.1.2. Has Theia3D been validated?¶
Theia3D has been used in a growing body of research that includes numerous concurrent validation studies with marker-based motion capture (and other measurement systems) for a variety of tasks.
For an up-to-date research compendium, please contact us at info@theiamarkerless.ca.
2.1.3. How accurate are the results compared to other markerless systems?¶
Because of our dedication to data quality for biomechanics, Theia3D is state of the art in terms of accuracy. To our knowledge, no markerless system has been validated to the degree of Theia3D.
2.2. Hardware¶
2.2.1. What are the camera hardware requirements?¶
Number of cameras: Minimum 6. Recommended 8 or more.
Synchronization: Cameras must be timecode-synchronized.
Resolution: Recommend capturing participants with 500 pixels in height. Resolution selection can vary based on capture volume size and frame rate requirements.
Frame rate: Depends on movement, with faster movements requiring higher frame rates. E.g. 60-120 Hz for walking, 80-180 Hz for running, 180-360 Hz for pitching.
Exposure: Selected based on frame rate. Generally, follow the guideline exposure = 1/(2x frame rate, or higher) seconds. E.g. for 60 Hz frame rate, use 1/125 s = 8000 us or shorter.
2.2.2. What are the computer hardware requirements?¶
Theia3D utilizes local computational power to apply its deep learning algorithms on video data to obtain 3D pose estimates. Therefore, Theia3D has relatively high computational requirements which can be met with high performing consumer-grade computer components. Higher performing components generally reduce processing time and improve overall user experience.
Please see our Documentation (Section 2.1 Installation) for specific computer system requirements.
2.2.3. Can I use my own cameras?¶
Yes you can, however they must collect fully synchronized, high quality video data. We only support data collected using specific camera systems that we have verified the quality and synchronization of the captured videos. So, while you can use your own camera system, if it is not on our list of verified equipment it is your responsibility to ensure the system meets the requirements of Theia3D and we will not be able to provide support for any issues related to data collection or quality.
We support:
Sony RX0 ii cameras
OptiTrack Prime Color cameras
Qualisys Miqus cameras
Vicon Vue cameras
FLIR Blackfly S cameras (operated through Vicon Nexus)
2.2.4. Can I use force plates (or other external devices) with Theia3D?¶
Theia3D is a markerless motion capture solution that outputs kinematic pose data in .c3d or .fbx file formats using synchronized videos as input. If your camera system can record videos that are synchronized with external devices, then the kinematic outputs from Theia3D will be synchronized with the signals from your other external devices. If the signals from those devices can be exported to separate .c3d files, it is possible to merge the kinematic .c3d files from Theia3D with the .c3d files containing other measurements in software such as Visual3D.
The following camera systems can be synchronized with external devices and have been verified to work with Theia3D:
Qualisys Miqus
OptiTrack Prime Color
Vicon Vue
FLIR Blackfly S
2.3. Data Collection¶
2.3.1. How many camera views are required?¶
Theia3D requires six or more synchronized video cameras to calibrate the camera system and produce pose estimates of the people visible in the videos. Video trials with fewer than six cameras cannot be loaded or analyzed using Theia software.
2.3.2. What are the limitations on capture volume? How many cameras do I need?¶
The capture volume size will be limited by the number of cameras. Larger volumes and volumes of more complex shape will require more cameras to provide adequate coverage.
We recommend camera systems have a minimum of 8 cameras, which is generally adequate for capturing a volume approximately 5 m x 5 m. Larger capture volumes will require additional cameras.
2.3.3. What clothing should my participants wear?¶
In general, participants should wear body-fitted clothing that provides rich visual features and allows the different parts of the body to be discerned from each other.
Acceptable clothing can include jeans, joggers, tights, shorts, t-shirts, sweaters, tank tops, etc., and can include casual or athletic attire. Skin-tight clothing is also acceptable, but not required. While these are guidelines only, some factors to consider when selecting or controlling participant attire include: * Color (does the color eliminate visual richness or features?) * Visibility of anatomical features (e.g. can you see the kneecap?) * Visual richness (e.g. is there creasing or folding around the joints?) * Contrast with background (e.g. are they wearing all black against an all black background?) * Participant comfort (e.g. will the movement be inhibited by the attire?)
For more information, you can read our blog post on participant attire.
2.3.4. Can Theia3D track multiple participants at once?¶
Yes, Theia3D can track multiple participants, provided that they are all sufficiently visible in the input video data. The number of people that can be tracked is dependent on their visibility, which is largely determined by the number of cameras in your camera system. Any participant that is sufficiently visible in three or more camera views will be tracked, provided they are not excessively occluded by objects or other people. Under crowded conditions, some people may not be tracked if they are excessively occluded by other people.
2.3.5. Can I collect data in _____ (location/environment)?¶
Theia3D can produce robust pose estimates for people in almost any environment, provided that the data quality and camera system setup meet our recommendations. Some environments where customers have recorded data for analysis in Theia3D include:
laboratories
hospitals
field houses
athletics tracks
courts
rinks
gymnasia
sidewalks
classrooms
clinics
retail spaces
2.3.6. What is the maximum sampling rate and data capture length?¶
The maximum sampling rate and capture length are limitations of the cameras used and thus are different for every camera system. Longer captures will take longer to analyze in Theia3D.
2.3.7. Do I need any software other than Theia3D?¶
Yes, you need software to run the cameras and collect the video data. This software is usually provided as part of a package with your camera system hardware. Theia3D processes the video data saved or exported by this software to produce 3D kinematic data (segments and rotation matrices) that is ready for analysis. Additional analysis or 3D processing software (such as Visual3D) can be used to interpret and report the 3D kinematic data.
2.4. Data Processing¶
2.4.1. Can Theia3D process in real-time?¶
No, Theia3D is a post-processing markerless motion capture solution and cannot perform real-time person tracking.
2.4.2. Do I need to process all of my data manually?¶
No, Theia3D ships with a companion application that allows you to batch analyze a list of trials without supervision or manual intervention. Prior to batch processing you can organize your data, assign calibration files, select your analysis settings, and modify the batch analysis trial list using built-in tools before beginning the batch analysis for fast and easy data processing.
2.4.3. How long does it take to process a trial?¶
The processing time required for any given trial will depend on the length of the trial, the frame rate used, the number of cameras, the number of people being tracked, and your computer hardware. Using a top-of-the-line consumer-grade desktop computer, Theia3D can achieve analysis speeds of approximately 200 frames per second (e.g. 1 trial with 8 cameras recording at 85 Hz for 4 seconds can be processed in approximately 14 seconds). For more information, you can read our blog post on computer hardware.
2.4.4. Can Theia3D track objects?¶
Currently, Theia3D only tracks humans.
Object tracking data can be recorded using marker-based cameras that are synchronized with the markerless cameras, and combined with the human tracking after both sets of data have been processed using their respective methods.
2.4.5. Does Theia3D fill gaps or filter?¶
Theia3D automatically performs gap-filling and smoothing of the 3D poses for all people tracked within processed movement trials. This is achieved using the Generalized Cross-Validation Spline method, which is equivalent to a double Butterworth filter with some additional advantages.
For all processed movement trials, Theia3D saves both the unfiltered 3D poses and filtered 3D poses of all tracked individuals and either version of the tracked pose can be rendered in Theia3D (see Render Smooth IK in the Preferences window). When .c3d files are saved for a processed movement trial using Save Skeleton Poses, two separate .c3d files containing the filtered and unfiltered versions of the 3D poses are saved.
The default GCVSPL cutoff frequency is 20 Hz, however it is recommended that this is adjusted based on the movement contained in the trial; slower movements should be processed with a lower cutoff frequency, and faster movements with a higher cutoff frequency. The default 20 Hz cutoff frequency provides some smoothing and noise reduction while generally being sufficiently high to avoid ‘over-smoothing’ most movements.
2.4.6. Can I customize the model used by Theia3D?¶
Currently, Theia3D supports a single model definition that is scaled and applied on an individual subject basis. The model joint constraints (degrees of freedom) can be modified as follows:
Shoulder: 3 DOF or 6 DOF
Knee: 2 DOF or 3 DOF
Ankle: 3 DOF or 6 DOF
2.4.7. Can I export data for reporting? Is there built-in reporting?¶
Report generation and biomechanical analysis functionality is not built into Theia3D. To generate reports, we recommend using Visual3D (which is included in our packaged offerings) as it can automatically detect the data exported by Theia3D and build the required models. However, we also export .c3d files that can be used in other software.
2.4.8. Does Theia support animation or game development?¶
Theia can export human pose data in .fbx file format for animation and/or game development.