Immersive Visualization / IQ-Station Wiki
This site hosts information on virtual reality systems that are geared toward scientific visualization, and as such often toward VR on Linux-based systems. Thus, pages here cover various software (and sometimes hardware) technologies that enable virtual reality operation on Linux.
The original IQ-station effort was to create low-cost (for the time) VR systems making use of 3DTV displays to produce CAVE/Fishtank-style VR displays. That effort pre-dated the rise of the consumer HMD VR systems, however, the realm of midrange-cost large-fishtank systems is still important, and has transitioned from 3DTV-based systems to short-throw projectors.
ML2VR
Matlab is a mathematical visualization and simulation tool. Mathworks describes it thusly:
"MATLAB® is a high-level language and interactive environment for numerical computation, visualization, and programming. Using MATLAB, you can analyze data, develop algorithms, and create models and applications. The language, tools, and built-in math functions enable you to explore multiple approaches and reach a solution faster than with spreadsheets or traditional programming languages, such as C/C++ or Java™."
The ML2VR application is from the Duke University Visualization Technology Group and is available from Sourceforge ML2VR page. ML2VR uses an OpenGL-intercept technique to capture 3D rendering in the Matlab diagram window and transmit geometries and colors to the immersive display. Using accompanying Matlab scripts, interactions in the immersive space can be conveyed back to Matlab to affect the script and allow immediate feedback to the immersed user.
How to Run ML2VR in Immersive Mode
First of course you will need a valid copy of Matlab. In addition to the basic Matlab package, ML2VR requires this package:
- Instrument Control Toolbox — used for the socket communication
Linux
Open two shells, one to run Matlab, and one to run the geometry viewing and interacting application. (Note: these do not necessarily have to reside on the same computer, though for now the configuration process described herein does make that assumption.) The VR application must be run first as the doppelganger OpenGL will attempt to connect with it when Matlab initiates.
ML2VR application shell
% export LD_LIBRARY_PATH=<directory of boost libraries compatible with the Matlab executable>
- E.g.:
% export LD_LIBRARY_PATH=/usr/local/MATLAB/R2012a/bin/glnxa64
- E.g.:
% ./freevr_app
Matlab shell
% export LD_LIBRARY_PATH=<directory of doppleganger libGL.so>
- NOTE: if your LD_LIBRARY_PATH requires other directories for matlab, then be sure to maintain them in the library path.
% matlab
Matlab script A simple way to "just get started" and convert an existing Matlab script to work with ML2VR in a basic way is to add this Matlab code to the top of the script:
% Code to interface with the ML2VR application: clear; addpath('../vr_interface'); vr_config; vi=vr_interface(ServerIP,ServerPort); close all; figure('Renderer','OpenGL'); % Code to enable the grab-the-world interface: vvh=vr_virtual_hand(vi); vvh.enable(true); vvh.set_hand_button(0);
Of course, you need to make sure the path to the vr_interface
script
is correct — safest to hardcode it to the full path. This example
assumes you are running in a sibling directory to the vr_interface
directory.
Now run your modified script in Matlab:
>> myscript_vr
MS-Windows
(to be documented)
--
How to Build ML2VR w/ FreeVR
This section specifically instructs on how to compile ML2VR using the FreeVR immersive interface library.
- Install missing dependencies
- Commonly needed packages include:
- FreeVR version 0.6
- boost_system (1.41 works, but must match that used by Matlab)
- boost_thread (1.41 works, but must match that used by Matlab)
- Commonly needed packages include:
- Download the latest source (version 4_1_2013 as of this writing [04/11/13])
- Untar the source ball
- E.g.
% tar -zxf ml2vr_4_1_2013.tar.gz
- E.g.
- Compile the interceptor (doppelganger) OpenGL library:
% cd linux
- Confirm (change when necessary) that the interceptor program will relay OpenGL calls to the correct real libGL:
% $EDITOR ../intercept/src/gltrace.c
- search for "libGL.so" and make sure it points to the correct one
- NOTE: pay particular attention to whether it points to the 32 or 64 bit version of the real OpenGL library
- search for "libGL.so" and make sure it points to the correct one
% $EDITOR ../make/Makefile.intercept
- Point the CFLAGS to find the correct boost libraries
- E.g.
/home/avl/Apps/ML2VR/boost_1_44_0
- E.g.
- Point the CFLAGS to find the correct boost libraries
% make
- Compile the VR application:
- Configure ML2VR
% cd freevr_app
% $EDITOR Makefile
- Point
FREEVRDIR
to the FreeVR directory
- Point
- Compile ML2VR
% make
How to Build ML2VR w/ Syzygy
... (to be documented)
How to Configure ML2VR
There are three configuration files associated with ML2VR, plus the shell running the Matlab application must be configured to use the doppelganger (intercepting) copy of the OpenGL library.
- Create a
/etc/ml2vr/intercept.cfg
file:- Here is a sample
intercept.cfg
file for use with Linux:
- Here is a sample
### OpenGL interceptor configuration file for ML2VR debug_file /tmp/intercept.log opengl_port 9000 client 127.0.0.1
- Create a
ml2vr.cfg
file in the directory from which the ML2VR application will be run:- Here is a sample
ml2vr.cfg
file for use with Linux:
- Here is a sample
### ML2VR application configuration file: opengl_port 9000 interface_port 9999 swap_port 9001 swap_ip 127.0.0.1 swap_clients 1
- Edit the
vr_interface/vr_config.m
Matlab script segment to set matching parameters that will be read within the Matlab visualization scripts:- Here is a sample
vr_config.m
file that matches the above configurations:
- Here is a sample
%% ML2VR Matlab configuration script ServerIP='127.0.0.1'; ServerPort=9999;
- set the LD_LIBRARY_PATH environment variable to point to the location of the doppelganger OpenGL:
% export LD_LIBRARY_PATH=/home/avl/Apps/ML2VR/ml2vr_20130314b/dev/intercept/linux
ML2VR interface code
The ML2VR package includes the directory matlab_files/vr_interface
which contains the class and methods used for communicating with the ML2VR immersive application. This provides the utilities needed for making an interactive immersive Matlab application.
For graphics only, the code segment listed above in the How to run ML2VR section is sufficient. This section briefly describes the classes and methods used to drive a Matlab script from within the immersive display.
vr_interface
class- Initialization methods
vrinstance = vr_interface(ServerIP, ServerPort)
vrinstance.set_return_type({'pos3' | 'mat4'})
vrinstance.set_transform_matrix(<matrix>)
- Event Queue methods
[event,button,pos] = vrinstance.get_button_event()
[event,axis,pos,value] = vrinstance.get_joystick_event()
- Value Polling methods
state = vrinstance.get_button_state(buttonID)
value = vrinstance.get_joystick_state(axisID)
tform = vrinstance.get_sensor_state(sensorID)
- Initialization methods
vr_virtual_hand
class- Initialization methods
vrhandinstance = vr_virtual_hand(vrinstance)
vrhandinstance.enable({true | false})
vrhandinstance.set_hand_button(buttonID)
- Initialization methods
vr_sphere
class- Initialization methods
sphereinstance = vr_sphere(size)
sphereinstance.setPos(x,y,z)
sphereinstance.setColor(r,g,b)
- Initialization methods
vr_cursor
class- Initialization methods
cursorinstance = vrcursor(vrinstance)
cursorinstance.setup_ray_preset()
- Initialization methods
Example Matlab scripts
The ML2VR package includes four directories of example Matlab scripts that demonstrate all the available features of ML2VR. These example collections are:
framework_examples
— examples for how to do the most basic operationsinteraction_examples
— examples of a variety of ways of interacting between the immersive application and the Matlab simulationviz_examples
— two examples that demonstrate visualization interactions possibilitiesrobot_demo
— a simple robot simulation that can be directly affected by the immersed user
Usage Tidbits
- Matlab comments are '%' — putting two together causes them to appear in the script comment window
- The Matlab command
license('inuse')
will report what packages are used by a loaded script - Renderings generated by the Matlab
plot3()
command can be made to generate ML2VR enabled OpenGL calls by pushing it through a 3D transformation. Eg.:h = plot3(masscharge, time, spectra);
set(h, 'Parent', hgtransform);
- ...
Usage Issues
- OpenGL lighting calls are not yet implemented in the ML2VR immersive application, so lighting commands within the Matlab scripts should be removed.
- Do not name a Matlab script to end in a "
+
" — Matlab interprets this and will attempt to load a script with a different name. Eg. instead of "script1+.m
" it would try to load just "script1.m
" ! - ...
Development Issues
This section is for issues that are of interest to those working directly on the ML2VR code-base.
- Lighting on cube cursor (and pointer in some examples) is discontinuous across screens -- this can be fixed by setting the light position each frame.
- Can there be a button from the VR application that terminates the loop in the Matlab script?
- Perhaps a built-in clipping capability can be added to the ML2VR immersive code.
- Perhaps there can be navigation control that is built-in to the VR application rather than relying on the Matlab script to do the work. Update: okay, there does seem to be navigation control directly in the VR application, but it needs some signal or something to enable it.
- I'm not 100% convinced that the
vr_config.m
file is really used. - I'd like to see more comments in the matlab scripts in the vr_interface directory.