2. Getting Started

2.1. Installation

Note that administrator privileges are required for installation and licensing.

System Requirements

GPU

One or more CUDA-capable NVIDIA graphics card(s) with the following:

Note: Pre-built engines are provided for: NVIDIA RTX 3090 and NVIDIA RTX 4090. Engines for all other graphics cards are built automatically the first time Theia3D is run.

CPU
Recommended: octa-core i9 processor or better.
Minimum: quad-core i7 processor or equivalent.

RAM
Recommended: 32 GB or greater.
Minimum: 24 GB.

Installation Steps

  1. Login to the Theia Software Downloads portal using the login credentials provided to you via email.

    1. Download the Theia3D application installer.

    2. If available, download the Graphics Card Engines for your GPU. Note: If none are available for your GPU, they will be built automatically the first time Theia3D is run.

    3. Copy your license key for safekeeping offline.

  2. Check that your NVIDIA graphics card drivers are up-to-date. If not, download and install the latest drivers from NVIDIA.

  3. Run the Theia3D application installer and follow the instructions.

    1. You must scroll to the bottom of the EULA to accept it.

    2. If you downloaded a Graphics Card Engine from the Theia Software Downloads portal, run the executable once the Theia3D application installer has finished.

  4. Launch Theia3D for the first time.

    a.When prompted, enter your license key to activate the software.

    1. If a Graphics Card Engine was not available for your GPU from the Theia Software Downloads portal, they will be built automatically at this stage. This can take several (~30) minutes, but is only required the first time Theia3D is opened.

_images/software_downloads_installer.png _images/software_downloads_engines.png

Theia Software Downloads portal, with Theia3D installer and graphics card engines.

2.2. Startup Window

_images/startup.png

Theia3D application startup window.

The startup window appears if ‘Select GPUs on launch’ is enabled in the Theia3D preferences.

GPU Selection: Select which graphics cards to use when running Theia3D.

Person Detector Size

Select which person detector to use when running Theia3D, and set the skip frames count.

  • Small: fastest, may be slightly less accurate than Default or Large.

  • Default

  • Large: slowest, may be slightly more accurate than Default or Small (currently disabled).

Skip frames sets the frame interval used during the analysis process to identify and track people. Default is 1, Maximum is 5. Increasing the Skip Frames value will speed up the analysis.

Joint Detector Size

Select which joint detector to use when running Theia3D.

  • Small: fastest, may be slightly less accurate than Large (currently disabled).

  • Large: slowest, may be slightly more accurate than Small.

  • Add Hands: (available as a license add-on by special request) Include hand model, which improves hand tracking through localization. Requires camera views that are positioned specifically to capture the hands and upper body.

Listen Mode

Available as a license add-on by special request.

Listen Mode allows a directory to be selected where data is saved during data collection. The selected directory will be actively monitored for new, unprocessed data which will be automatically analyzed as it is detected. This allows data to be processed during an active data collection session, enabling all processing to be completed by the end of the collection session.

2.3. Theia3D Basics

Theia3D is a markerless motion capture solution that utilizes synchronized video data to produce accurate and reliable 3D pose estimates of humans visible within the video data. It leverages deep learning algorithms trained to identify humans and accurately predict the 2D positions of over 100 landmarks on the human body, in every video frame of every camera. By fitting a scaled subject-specific inverse kinematic model to the landmark predictions, the human’s pose is reconstructed in 3D and tracked throughout their movement. This data-driven approach results in a robust solution that is generalizable across environments and movements, allowing the accessible collection of high quality 3D motion capture data where it was previously impossible. Below we describe the basic framework for collecting markerless motion capture data using Theia3D; for more detailed instructions, please reference the appropriate section of this documentation and any accompanying videos.

Calibration

Calibration is a crucial step for any 3D motion capture solution, and is equally important for Theia3D markerless motion capture data. The recommended calibration method depends on your camera system, but the concept and result is the same across all: determine the intrinsic and extrinsic parameters for all cameras in your system. These parameters allow lens effects and the 3D position and orientation of each camera to be determined, which is the key to producing robust 3D reconstructions. See Data Collection for details on recording intrinsic and extrinsic calibrations, and Calibration Menu for details on processing calibration trials.

The calibration methods supported by Theia3D include:

  • Chessboard calibration, using a large printed chessboard pattern to automatically obtain intrinsic and extrinsic camera parameters.

  • Third-party calibrations, such as wand calibrations implemented by third-party motion capture hardware suppliers.

  • Object calibration, using an object with known dimensions or with precisely measured key point positions to manually obtain extrinsic camera parameters.

Movement Data Collection

Once you have obtained sufficient calibration data, the next step is collecting your movement trials. Theia3D produces robust 3D pose estimates across varying environments, for humans wearing typical body-fitting clothing, and is task-agnostic. Therefore, if the human(s) in your video are clearly visible and captured with appropriate resolution, frame rate, and exposure, the Theia3D algorithms can generally track their motion without issue. Record your movement data with relative freedom and ease, following the recommendations outlined in the Data Collection Principles.

Data Processing

Having collected calibration and movement trials, you have everything required to process your markerless motion capture data. Theia3D includes several tools to help organize your data as required, process and check your camera system calibration, and analyze your movement trials. If you have collected numerous movement trials, you can use the accompanying TMBatch application to batch analyze your trials without active supervision on your part; however, we always recommend manually checking your calibrations and a few trials in Theia3D first. Detailed information on the TMBatch companion application can be found in Batch Processing .