On system startup, the tracker utilizes this information in combination with the known spatial coordinates of the active LEDs mounted on the helmet, to determine and return the real world coordinates calculated LED centroids.

Static 3D DLT calibration for determining sensor internal parameter

DMAS Calibration^TM is part of the DMAS6 Motion Capture Suite^TM, and supports both 2D and 3D static and dynamic calibration of any number of simultaneous sensors. This program is not required by the enduser, unless optics is expected to be changed, but is provided to customers as part of the Helmet Tracker.

Sensor image without lens distortion correction

Corrected Image

Accuracy
There are several adjustable tracker parameters which will enable the enduser to adjust tracker performance and sensitivity. These factors depend desired range of motion, sensor frequency and resolution, and required accuracy.

The procedure to determine 6DOF data from a given frame of raw data from the imaging sensor(s), is a non-linear iterative process and requires highly optimized assembly code to complete the required tasks in the amount of time available. The processor must first correct the raw data for lens distortion, then calculate the centroids of the helmet mounted LEDs to a predetermined subpixel precision, and lastly identify an in-tolerance fit between the known 3D spatial model and the calculated model utilizing a proprietary modified DLT method . All these very compute intensive tasks have to be performed during a percentage of the available time-slot between each incoming frame of raw data, and intelligent and efficient processing is essential.

Therefore the tracker needs to be carefully tuned for optimum performance, and the enduser have control of the following parameters, which will all to some degree affect the accuracy of the calculated data.

 

Maximum Iterations Cycle Time	This percentage based parameter defines how much time per incoming raw frame of data the tracker will allocate towards identifying and fitting the known 3D spatial model to the calculated model. The setting can be derived theoretically based on the sensor’s refresh frequency, but may in some cases be adjusted by the enduser if there is "leftover" processing time available which could be used to gain additional accuracy.   For example, a 500 Hz sensor will result in a 2 ms tracking latency. The tracker will in this case use approximately 1.5 ms per frame to calculate the 6 DOF data, leaving the balance of the time for completing other less compute intensive tasks and between frames idle time. But with a faster processor, the secondary operations will require less time to complete and by increasing the MICT timing parameter, more accuracy can be achieved during the same timeframe.   A MICT setting exceeding the framerate can also be specified, which will result in subsequent frames being ignored while an earlier frame is still being processed.
Maximum Accuracy	Allows enduser to specify a value when tracker stops the known 3D spatial model fitting process and idles as it may have reached desired accuracy earlier.
Initial Translational Step	A metric [mm] value for specifying the initial search and displacement step for each incoming frame as the tracker begins the model fitting process. This parameter is used by the tracker to quickly identify a "near" solution.
Fine Translational Step	A metric [mm] value for specifying the fine search displacement step. The tracker model fitting algorithm switches to this setting as the previously processed 3D proximity is less than the Initial Translation Step setting. The tracker model fitting algorithm keeps iterating using the FTS parameter until the positional difference of the calculated and the known spatial model is less than specified Maximum Accuracy parameter.
Initial Rotational Step	A rotational [ rad ] value for initial model fit.
Fine Rotational Step	A rotational [ rad ] value for fine model fit.
External Noise	A percentage value for setting approximate environmental optical noise level. A low setting ( 0 – 25 % ) makes the system run faster by assuming that all light emissions to be valid LEDs. A higher setting ( 25 – 100 %) is indicative of a noisy environment and calls for a more sophisticated approach to determine whether a blob is a valid LED or not.

After carefully setting all parameters to reflect hardware and environmental conditions, and utilizing an accurately measured helmet 3D model, initial calibration parameters and lens correction factors, a typical tracking example would look like:

GeometryThe geometrical placement scheme of the LEDs on the helmet depends on the desired operational range of motion. In order to compute 6DOF data, at least 3 LEDs need to be visible to the sensor. The tracker will stay locked to the model and continue tracking even with 2 or 1 visible LEDs, but will only return 3DOF positional [ X Y Z ] data. If no LED is viable, then the tracker will go into search mode and as soon as at least 3 LEDs re-appear, it will again lock to the model and continue tracking.

Therfore proper placement of the LEDs is essential for a continuous 6DOF data stream from the tracker. As a rule of thumb, more is also generally better as it enables the tracker to make a better with fit between the known 3D spatial model and the calculated model because there are more data points available.

As an example, if the requirements call for a single sensor tracking system with –45 to +45 range of motion, then the LEDs can be clustered in the center of FOV of the sensor, which will be fitted with appropriate wide angle optics. On the same token, with a 2 sensors redundant system, the LEDs can be spread out or additional LEDs added. The tracker has a theoretical 4π steradian range, but would then require LEDs to be mounted accordingly.

TrackingPrior to tracking, the system need to engage and lock to the spatial model as it determines the initial 6DOF position. This can be done in two different ways:

1. By identifying the LEDs using an electronic synchronization signal. The system will probe and identify the model by strobing the individually addressable LEDS at the sensor frequency. This method provides the fastest overall method for the tracker to identify and lock to the model.
2. By employing predefined pattern recognition schemes, which will identify the LEDs as the helmet is positioned in an approximate predetermined orientation. The advantage of this method is that hardware requirements are minimized.

After the tracker has engaged and is locked to the model, the tracker will immediately commence to stream 6DOF at the same speed as the sensor refresh rate, with one cycle latency. Under normal conditions the data will be very clean within the tolerance of the tracker settings, and will generally not require any additional filtering for noise elimination but digital filters can also be applied to further smooth the data. More LEDs will result in smoother data as higer accuracy can be achieved. If a second tracking sensor is used , the data stream from each sensor can be averaged to further eliminate random noise.

Hardware IndependenceRealizing the need for the tracker software to be portable to a variety of enduser hardware requirements, the software design is processor and operating system independent to the highest possible degree. However, some low level optimization and redesign will generally be necessary to fully utilize the more esoteric features of third party hardware.

The tracker is not dependent on any specific imaging sensor. The enduser can select a sensor which will provide the required resolution ( accuracy ) and refresh rate, and appropriate supporting hardware to sustain the data bandwidth and processing needs for an overall cost effective and well balanced system. Either CCD or CMOS sensors can be used. The helmet mounted LEDs are standard inexpensive COTS hardware, and the tracker does not have a firm LED wavelength requirement. When determining what type of LED to use, factors such as spectral response of the sensor and possible interference with other system ( e.g. nightvision ) need to be taken into account. Blue LEDs ( 450 – 470 nm ) is often the wavelength of choice since it provides near maximum response from the sensor as well as being far away from the IR region of the spectrum.

Regular optical bandpass filters can be used by the sensor(s) to block the majority of light except a certain wavelength range, e.g. to prevent direct sunlight to interfere with the tracker performance. While the blockage from a bandpass filter is not complete, this in combination with selective camera placement and fast shutter speeds provide ample protection from most external sources of interference.

Current DevelopmentThe objective of next major development phase of the head tracking technology, is to offload the raw image processing from the main processor by utilizing high resolution smart sensor(s) with onboard DSP. This will enable the tracker to run on minimized COTS hardware platforms , for example PC104 or slower third part hardware, receiving merely LED centroid data from the sensor(s) over an ethernet or a USB based protocol for 3D model reconstruction and 6DOF calculations. Ultimately the entire 6DOF process should take place on the sensor but the feasibility and technology involved is not yet well understood.

Enduser Test and Research SystemsFully functional test and research systems are available at a nominal cost, to enable endusers to test and verify the tracker’s operational characterstics. These systems comprise of single or dual calibrated sensors, computing hardware, basic helmet with mounted LEDs and supporting electronics. Alternatively, customer may also use their own headgear with LEDs mounted and measured to our specifications. For more information, please contact Spica Technology Corporation at 1-808-879-0343 or email spicatek@spicatek.com.