|
Discussion of DMAS6™
Motion Capture Suite
The objective of this document is
to briefly explain the basic procedures of a tracking session with
the DMAS6™ Motion Capture Suite. The DMAS6™program is an
ergonomically designed product which is very easy to use and often
requires no special training in addition to the standard tutorial
work-through. The software is highly customizable to meet
individual user requirements, supporting both internal scripting
as well as C++ SDK and plugin capability. As it will be explained
later in this document, DMAS6™ has the most comprehensive support in the
industry for various cameras and sensors, and we can design a precise
motion capture/analysis system around your individual video and data
acquisition needs.
CREATE DATSET:
The first thing you need to do to make a
DMAS6™ capture, is to either create a new dataset or open an existing
one.
A dataset is a directory that contains all the information necessary to
make a tracking sample. These data can be added in in relevant DMAS6™
applications as you go along, or also added upfront with the "STDMAS"
dataset builder appliaction.
Example of content in a dataset is for example calibration data,
lens compensation factors, types of cameras, number of cameras, synch
data, spatial models for tracking, spatial model for calibration,
analog data, videos, tracking ( if it exists )
To create a basic dataset from
scratch using the "STDMAS" program, it would look
like this. In this case,
we have selected two Pulnix TM-6710-LVDS cameras, and specified
which spatial model to use for calibration respective tracking. As
will be discussed later, depending on tracking requirements, it may
not even be necessary to have a spatial model defined for tracking, but is
useful for the simple reason to have pre-defined name for the markers
which we will track.
SPATIAL MODELS:
The DMAS6™
system uses a concept of "spatial models" to mathematically
define the marker topology of the objects to be tracked. A spatial
model will in addition to defining the markers, also be
used for applying constraints to the tracking model in order to help the
tracker determine which marker is which and to be used for auto labeling
as well as the 3D reconstructions. Mathematical formulas can
also be applied to the model, for example for calculations of virtual
points, or any other user defined features that can be expressed in terms
of XYZ positions of the markers. These values will be at the same
time the tracking data is computed.
Spatial model are also used to define the geometry of your calibration
model and contains the real world coordinates for each marker, which will
be used by the calibration software to calculate the DLT parameters.
Unless a a calibration spatial model is already built, run the "DMAS
Calibration" application to define a new spatial model. In our
sample tracking example, we will use a 12 marker 500 x 500 x
500 [mm] calibration model and the real world coordinates of the markers
are:
1:
X=0.0
Y=0.0
Z=0.0
2: X=500.0
Y=0.0
Z=0.0
3: X=500.0
Y=500.0 Z=0.0
4: X=0.0
Y=500.0 Z=0.0
5: X=0.0
Y=0.0
Z=250.0
6: X=500.0
Y=0.0
Z=250.0
7: X=500.0
Y=500.0 Z=250.0
8: X=0.0
Y=500.0 Z=250.0
9: X=0.0
Y=0.0
Z=500.0
10: X=500.0
Y=0.0
Z=500.0
11: X=500.0
Y=500.0 Z=500.0
12: X=0.0
Y=500.0 Z=500.0
This is what the "DMAS Calibration" application looks like
when you first start it. Use the "File-New" menu begin defining
a new spatial model.
As new markers are added, they will show up in
the data tree displayed to the left. A graphical representation is
displayed in the 3D Open-GL window. This widow can be rotated and zoomed
in 3D space to aid in the definition process.
In addition to marker positions, segments, 3D angles and constraints can
also be assigned to the model.
This is what the finished calibration spatial
model looks like. Use the "File-Save" menu to save.
Next we will also quickly design a spatial
model for the tracking. As will be discussed later in the Tracker section
of this document, we would not need a spatial model in this particular
case because we will be using the Auto-Tracker, but it is not a difficult
nor a time consuming effort to complete the spatial model for tracking and
one of the main benefits is that we will then have a reference to to
marker names of the tracking model when we enter the Tracker after data
acquisition. More on this subject later.
This image shows the completed spatial model for
tracking , including both markers and relevant segments.
CALIBRATION:
Immediately after having defined and saved our new
dataset to the harddisk and having the spatial model defined, we will
proceed to the calibration utility.
In some cases, especially if you have not moved or altered the cameras in
any manner since previous capture, you can utilize calibration for
previous capture, which would on most occasions render this step
necessary. But nevertheless, it is one of the most important steps in the
entire process, it is absolutely essential to have very good
calibration of the system, if you expect to get good data from the
capture. DMAS6™
has 3 different types of calibrations, first we have a 2D
calibration which can also be performed inside the tracker by referencing
points of known distance in a plane. Secondly, we have 3D static
calibration, which we will utilize in this example, and thirdly we also
have 3D dynamic calibration.
A 3D static calibration contains a physical model of any given size, with
markers appropriately placed at known coordinates. Normally the
calibration volume is a rectangular or cubic volume, with 3 markers
in each corner plus 3 in the middle or 15 markers total. You can use
additional markers, and they do not necessarily have to be symmetrical,
anywhere is 3D space is sufficient, as long as you know the exact
coordinates. However, with careful preparation and measurements of
the model, there is very little gain in accuracy to be gained by
adding additional markers. Quite the opposite is also often the
result of too many markers, because the difficulty in exactly
measuring marker positions in space is difficult and does often add more
errors as the number of markers increase. Therefore, it is
absolutely essential to maintain a simple topology of your calibration
model which makes it easy to measure and to verify.
These are a few pictures of sample
calibration models, designed by end-user for individual tracking
needs.
15 markers 2000 x 500 x 1200 [mm]
Image courtesy of PRS, Seattle, WA USA
![motion analysis calibration automatic tracking FastTraq DMAS6]()
17 markers 500 x 500 x 500 [mm]
Image courtesy of NASA Langley Research Center, Hampton,
VA, USA
27 markers 50 x 50 x 50 [mm]
Image courtesy of Dr. Mark Warner, University of Utah, Salt
Lake City, UT, USA
Next step in the process is to make sure the cameras are powered up and
that the physical calibration model is positioned in the area where you
want o perform your data capture. Also make sure that your light
sources are powered up and ready. 
When you the DMAS6™
Calibration program first appears, it will by default be displaying
Camera-0.
If you look closely in the data tree on the right side, you will see that
you have a two main entries, i.e. "Camera 0" and "Camera
1". Below each camera label you have two additional entries for
each camera, "Lens Compensation" and "Spatial
Calibration"
I will briefly mention , that the Lens Compensation, is to correct for
optical errors in the lenses used, and will only need to be performed once
if you are using lenses of a set focal length, and you need to to worry
about again as long as you keep using the same lenses. Care should
be taken however, that the same lens should always be used for the same
camera, because lenses distortion is often slightly different even if they
happen to be the same manufacture and model number. With zoom lenses, it
gets a little bit more complicated, lens distortion should be
re-calibrated every time the zoom factor is changed, because
depending on focal length in use at a given setting, is in fact using a
different part of the optics. Luckily however, for higher zoom rates
and therefore longer focal length, the center portion of the lens is used,
and most of the time unless of particular care needs to be exercised,
lens compensation can be ignored. This is because lenses are generally
very accurate in the center portion.
But back to our spatial calibration, we are
currently calibrating Camera-0, and we have the camera and the physical
model in a position that it best fills the available field of view.
We are now ready to begin digitizing the markers. If you look at the upper
right hand corner, you will see a list of the marker as we defined them in
the Spatial Model Editor program in the previous step. As you can
see, marker "M-1" is currently selected, and what the
system is now expecting, is for us to digitize "M-1" in the
video window.
The system does use subpixel technology, and
based in the full 8, 10 or even 14 bit of digital pixel data ( depending
on camera ), very accurate calibration can be achieved. The
calibration software will as you press the <Shift> button on the
keyboard, zoom in on the marker nearest to the mouse cursor and display a
larger view of the marker. And you can now very accurately
digitize the marker centroid either manually or having the software
calculate the centroid. Your main responsibility is to make
sure that you are digitizing the correct markers in the correct order.
This is what it will look like when Camera-0 is digitized. Pay no
attention to the displayed accuracy numbers yet, as the second camera has
not yet been digitized. To begin digitizing the next camera, select
"Spatial Calibration" under the "Camera 1" entry
in the data tree on the upper left hand area of the program, and repeat
the process for this camera.
As all markers have been digitized, spatial
accuracy is calculated and displayed. In this case the accuracy will
be within 0.031%
Briefly on lens compensation, a chart with
straight lines are used and is positioned perpendicular to the lens axis
at a distance so it fills the field of view. The picture above shows
what it will look like with a short focal length lens
( Picture shows an example with 4.8 [mm] lens ) . To adjust
the compensation factors, the slide bars below the image are adjusted
until the image appears straight. Some sophisticated image
processing algorithms could also be used, but it has been our
experience that the "human eye" is the best detector of
distortion and why we have chosen this approach.
After correction factors have been applied,
this is what the image will look like. As these values are saved,
they are automatically loaded by the DMAS6™
every time you start the system, and do not need to be redone if the same
lenses are used
VIDEO CAPTURE:
The next step in the process will be to capture
and record the video we will perform the tracking on. For this we will use
the the program called DMAS Capture.
The DMAS6™Capture
application is probably on of the most sophisticated data acquisition
engine available anywhere today. It is not merely a program designed
to record and capture video, but a very complete data acquisition package
for video, analog, audio and essentially any type of data that can be
digitized. The technology is clustered, meaning that it can
manage acquisition of data on one or several workstations at the same
time. This is a very important feature, especially for systems
comprising of a larger number of cameras and sensors. The acquisition
utilizes the PCI bus for acquiring data from an external source ( camera,
force plates , EMG etc....). These data rates can be
exceedingly high, especially from digital high speed cameras, and only so
many devices can be recorded on a single workstation until the system runs
out of available bandwidth. DMAS6™
Capture was the first digital motion
analysis system, and we are again first with full utilization of 66Mhz and
133MHz 64 bit PCI-X for maximum bandwidth, while the industry as a whole
is still stuck on legacy 32bit 33MHz PCI. In terms of bandwidth,
that means that DMAS6™
is capable of acquiring data at the theoretical rate of 1.064 GB/s versus
the 132MB/s for competing products. And with our cluster approach,
as additional cameras needs to be recorded and bandwidth is being
consumed on any given workstation, the solution is to connect additional
workstations ( normally 2U or 4U rack mounted chassis ) for added
bandwidth. The DMAS6™
Capture system has all the internal controls, such as triggering ( pre -
post - mid and conditional ), synch ( two different
synchs to allow cameras of different speeds ), remote control etc built
into it. DMAS6™
also presents the user with total control of the acquisition process,
by letting the user to define the method by which to acquire the data
individually for each sensor. These can be streaming to disk, recording to
RAM or even acquisition to onboard memory if the framegrabber boards
support this. We should also mention that DMAS6™
is the only motion analysis system that support video cameras from all
standard industrial formats, i.e. LVDS-644, RS-422, Base- Medium and Full
CameraLink, HotLink, Firewire and USB2.0 What this means, is
that the enduser have total control over what cameras to choose, you no
longer have to chose a camera based on what the motion analysis system is
capable to handle, choose our camera based on what you need for your
work and we will do the rest to acquire video from that camera.
This is what the DMAS6™
system looks like when you first start it up. The system will
remember the last recording model that was used, also defined in the
dataset, and since we defined a new dataset for 2 cameras in the
"Create Dataset" section, and as the user clicks on the
Preview-Start button, the system will begin to display the video acquired
from the cameras. Please note, at this point the system is
merely in Preview mode and does not record, allowing the user to make
adjustments in order to ensure that the objects to be recorded are in
proper position. The DMAS6™
Capture system also allows the user to conveniently zoom in on the image
of interest, for example to make sure that marker illumination and
intensity is acceptable. Directly below the video window you can see
one thumbwheel icon for each camera, these can be used to quickly switch
to a different camera view. But in the case of zooming, you would
define your zoom area with mouse and if you look carefully, the zoom
borders are also displayed on the video thumbwheel. One nice feature is
that you can move this zoom window around on the original image and
examine the full image in detail without having to re-zoom. In order to
get back to full image, all you have to do is to double click the
thumbwheel.
Next we will click on the "record" button,
and the DMAS6™
Capture system enters the recording environment. This is where you tell
the system for how long to record and where to record it to. You will also
specify such options as loop recording, e.g. continuously record and keep
5 the last seconds of video until I click Stop. Or maybe specify a
Conditional trigger, which is a custom software plugin that can for
example look for some event of interest to take place , and based on that
perform a pre, post or mid trigger. Note that the system also has
individual AGC ( Automatic Gain Control ) for each individual camera view,
which is a very powerful feature if having to record under very lowlight
conditions.
After the recording, you can use the video controls
in the lower left hand corner to examine the results. Use the
controls on the time bar to Crop the video, and you can have any number of
crops from a recorded set and they do not need to be continuous. If
you have very long captures, you can use the file pointer on the timebar
to quickly scan through a recording, or if you click and hold the
">>" or "<<" button, the system will
replay the recorded video and keep accelerating the replay speed as long
as the button is held by the mouse. When you are satisfied with the
video, click on the File-Export To Dataset menu and the acquired data will
be saved with your dataset.
TRACKING:
We have now recorded and saved the video to our
dataset and we will now perform tracking on the acquired data. As
briefly pointed out earlier at the beginning of this document, DMAS6™
supports several different tracking methods. My discussion here
applies to the Auto-Tracker only. But before I continue, let me
briefly describe he different tracking technologies.
1: Auto-Tracking
This tracker employs traditional 3D tracking technology, and functions
with previously recorded video from DMAS Capture or from
imported videos in the form of AVI files. This is also the tracker
which is available in the DMAS DV systems. A spatial model is not
required but can be used for establishing the marker names. As the video
is available for tracking, the user starts the tracker and opens the
appropriate video files. Next step is for the tracker to establish
all the markers in all the video views through a histogram based
proprietary algorithm. The user can adjust video and marker
brightness threshold values. Normally with good video, this whole process
is virtually transparent to the user. After tracker has established the
markers from the videos, the user will position the videos in the first
frame of each video sequence, and identify each marker in each camera
view. As markers are being identified the tracking data is
automatically calculated.
The Auto-Tracking software can also be used for manual markerless
tracking.
2: Fast-Traq™ Tracking:
This is brand new precision digital tracking technology. What makes it
different from traditional 3D tracking technology is that it
performs the tracking directly in mathematical 3D space and is working
only with 3 dimensional attributes. Competitive 3D trackers
work first with 2D space by first performing tracking for each
cameras before calculating 3D coordinates with some DLT method. The big
advantage Fast-Traq™ is that has superior capabilities to compensate for
hidden markers, as well overall accuracy and elimination of noise.
Graphically tracking trajectories are represented as NURBS-curves (
Non-Uniform-Rational-B-Splines ) , same as high end CAD/CAM systems
use for modeling complicated geometry and specifically developed for
representing large amount of data in a mathematically precise format.
Fast-Traq™ requires topologically correct spatial models with
constraints applied to all appropriate markers to make the
model unique. As the data is captured, the user simply enters the tracker,
may manipulate some video threshold values if necessary and performs the
tracking simply by a single command. There is no additional
digitizing or marker identification process necessary. A
comprehensive set of tools are also available for manipulating of the 3D
data ( translation, rotation, scaling etc... ), editing,
post-processing and interfacing to other application software.
3: Real Time Tracking:
Real Time Tracking is like the name implies, a low latency ( 1 frame
latency ) real time tracker, which like Fast-Traq™ also requires a
topologically correct spatial model. Real Time tracking is very
similar to Fast-Traq™, except the data feed back is instant, and
coordinate data can also be streamed directly to some external device, for
example in VR and head tracking applications. Real time tracking is
particularly well suited for applications that require repetitive tracking
of the same model, maybe with separate subjects such as gait analysis.
Industrial control applications which require instantaneous positional
feedback is another important area for real time tracking. But generally
speaking, the need for real time tracking in biomechanics applications is
exaggerated.
We shall now generate tracking data on the video
we captured using the Auto tracker. To start the tracker , run the
"DMAS Tracker" program. 
When the program first appears it may or may not contain a dataset,
depending on what you have been doing previously. But for example if it
does not, we can as before, use the File-Open menu to open a dataset for
tracking.
As the dataset has been opened, the video from the
first camera will automatically appear in the video view , and the
different cameras will have their own individual buttons in the lower left
corner above the video controls. To you upper right is a listing of the
markers as defined in the spatial model assigned for tracking when you
first created the dataset. Producing tracking data is a very simple
process, first we will process the videos to find all the markers and this
is under the "Tools - Find Video Markers" menu
As you press the "Tools - Find Video
Markers" menu, a dialog will appear and gives you the
option to adjust the threshold values for the videos. Normally if you have
nice clean videos you can simply accept default. This is only a
factor if there is lots of external noise in the video that may interfere
with any of the markers. As you press OK on the "Video Threshold
Adjust" dialog, the system is finding the positions of all the
markers in every frame.
Next step is to ID the markers. First select
"Video 1" on the camera buttons, and make sure to put the
video in the first frame by pressing the "|<" on the video
controls. Then select the first marker in the marker list to your
upper left, followed by digitizing the corresponding marker in video view.
The system will automatically advance to the next marker, and you can
chose to ID any or all markers.
When you have completed the first camera, then
select "Video-2" and repeat the process for all markers you wish
to track. Please note as you digitize a marker in this view, a 3D
coordinate will appear to your upper right as each marker is digitized.
The system has then identified the marker and automatically calculates all
the coordinates for the marker in the dataset.
After all markers have been identified, you can use the video
controls to replay the video while examining the 3D data. The tracking is
no complete, and do not forget to do a "File -- Save" if you
wish to store the tracking data in your dataset.
REPORTER:
We now have tracking data generated for
the whole video, and we will take a moment to cover some of the
basic concepts for generating reports.
The DMAS6™ Reporter utility is located in the main DMAS6™directory.
The program itself is licensed software, but it is free and can be
freely distributed to all people you wish to share data with. We
have a very ambitious development schedule for the Reporter, and it
will in the near future also contain scripting and plugin facilities, to
allow user to write and design their own analysis programs based on data
generated in DMAS.
The Reporter GUI is of a modern "Drag-And-Drop' style, and
sophisticated reports can be generated in a very short time and minimum
user input.
I think the best way to demonstrate the basic
concept of the Reporter, is to generate a sample report and explain along
the way how it is done. We will as an example make a graph for the
Z-displacement component of the movement of Left Hand Marker, and in
the same graph also display the Z-displacement component of the Left
Hand Wrist-Elbow middle marker, and make sure the Z-displacement of these
two markers are synchronized, which they must be since they are
mounted on the same biomechanical linkage.
As usual we begin with opening the dataset
containing the tracking data by clicking the "File --- Open"
This is the Reporter utility looks like when first
started and prior to any data having been loaded.
As the dataset is loaded, the data hierarchy
is displayed in the same same familiar manner. Please note,
wherever you see a right-pointing triangle in front of a data item,
this means that there is additional data under this item, and can be
expanded and displayed by clicking on the triangle with the left mouse
button. The data tree is partially expanded when first loaded, displaying
the video and calibration information.
Lets next expand the data tree to display the Marker information as well
as the Segments. This is then done by clicking on the triangle to
the immediate Left of the "Markers" and "Segments"
label.
![motion analysis 3D tracking FastTraq DMAS6]()
As you expand the Markers and Segments, the data
tree will display all the markers and segments in the model.
We shall begin our report by first including the video streams. To
add the video, simply put the mouse cursor on the video icon in font
of where it says "video" immediately below the "src00"
label. With the mouse on the icon, click the left button and
drag to the Reporter desktop on the right, and release the button. This
will create a video window which can be scaled to any size you prefer.
Repeat the process fro the second video under the "src01"
label.
After you have added the videos to the Reporter
desktop, the reporter window will look similar to this.
We will now add a graph to first display Left Hand marker Z-displacement.
In the data tree in the left, find under the "Markers" label the
marker that is labeled "LHAN" ( these labels were defined
when the spatial model was constructed ). Again in the same fashion,
expand the "LHAN" marker by clicking on the triangle in front of
the "LHAN" albel, and you will see that you now also have X,Y
,Z, and R in the data tree, and these can be further expanded to show
velocity and acceleration for each component and resultant.
But for our discussion, grab the icon in front of the "Z" and
drag and drop onto the Reporter desktop.
This is the Reporter window will look like after the
Z-displacement for the "LHAN" marker has been added.
Note that you can freely size the window and the graph will automatically
follow. Additionally you can also alter the display scale by
putting the mouse on the Y-axis max value ( the number displayed on top of
the Y-axis ) , and the click and hold with the left mouse button while
dragging the mouse. This will dynamically scale the max-min values to
conveniently fit your graph area. You can also change the color and
the weight o the curves by clicking on the icons displayed to the right of
the components in the data tree.
 |
To add the Left Hand Wrist-Elbow middle marker
Z-displacement component, we again expand the "LEWR" label and
grab the same icon in front of the Z label, and this time we drag and drop
into the same graph we previously created, and this will display the
"LEWR" Z-displacement synchronized with the "LHAN"
data. You can also use the video controls in the lower left hand
corner to play and step the videos. The vertical line on the graph
depicts the data that corresponds to the currently displayed video frame.
It goes without saying that all data is synchronized.
To filter the data with any of the available
filters, simply select the item you wish to filter by clicking on it in
the data tree, and click on "Tools --- Apply Filter",
and we will for example select a "Least Square Fit" filter from
the available filter menu.
![3D motion analysis 3D tracking FastTraq DMAS6]()
A "Select and Apply Filter" dialog appears
and you will see that the data is dynamically filtered and updated as you
change the range factor with slide control. Filters can also be deleted
and the data will then revert to the original raw data.
I hope this document helps you to get at
least a basic understanding of how the DMAS6™
motion capture technology functions.
There are of course many features and functions that we have not discussed
in this document, but feel free to email or call our support with more
detailed questions, and we will do our best to give you a
comprehensive respond to your questions. Our email address is
<spicatek@spicatek.com>
Updated 06-03-2004
|
