Class for simulating an array-type tactile force sensor. More...

#include <SimulatedTactileSensor.hpp>

Inheritance diagram for Graphics::Sensing::SimulatedTactileSensor:

Collaboration diagram for Graphics::Sensing::SimulatedTactileSensor:

Public Member Functions
	SimulatedTactileSensor (unsigned rX=4, unsigned rY=6, float s=0.08, float sZ=1e-2f, bool onscreen=true)
	Default constructor.
	SimulatedTactileSensor (unsigned rX, unsigned rY, float s, float sZ, float sT, bool onscreen=true)
	Constructor for different sensor thickness.
	SimulatedTactileSensor (unsigned rX, unsigned rY, float sX, float sY, float sZ, float sT, bool onscreen=true)
	Constructor for different sensor thickness.
void	setGaussianSigma (float s)
	Set the variance of the Gaussian blur applied.
virtual void	initGL (int , char *)
	Initialistion of GLUT.
virtual void	initialize ()
	Initializes class internals.
virtual void	setState (float x, float y, float z, float th1, float th2, float th3)
	Set sensor position and orientation w/ Euler Angles th1, th2, and th3 are interpreted as XYZ Euler Angles are used to compute orientation.
virtual void	setState (double *m)
	Set sensor position and orientation w/ a transformation matrix.
const double *	getState () const
	Gets the current transformation matrix.
void	useEulerAngles (bool b)
	Switches b/w use of Euler angles and rotation matrices.
unsigned	getCols ()
	Returns the x-resolution.
unsigned	getRows ()
	Returns the y-resolution.
void	setTranslationZ (float z)
	Set the translation on the z axis.
void	setTranslationY (float y)
	Set the translation on the y axis.
void	setTranslationX (float x)
	Set the translation on the x axis.
void	setAngleZ (float Az)
	Set the angle around z axis.
void	setAngleY (float Ay)
	Set the angle around y axis.
void	setAngleX (float Ax)
	Set the angle around x axis.
float	getAngleZ ()
	Get the angle around z axis.
float	getAngleY ()
	Get the angle around y axis.
float	getAngleX ()
	Get the angle around x axis.
void	incrementTranslationZ (float z)
	Add z to the previous value of TranlationZ.
void	incrementTranslationY (float y)
	Add y to the previous value of TranlationY.
void	incrementTranslationX (float x)
	Add x to the previous value of TranlationX.
void	incrementAngleZ (float Az)
	Add Az to the previous value of AngleZ.
void	incrementAngleY (float Ay)
	Add Ay to the previous value of AngleY.
void	incrementAngleX (float Ax)
	Add Ax to the previous value of AngleX.
void	draw ()
	Draw the sensor.
virtual void	beginUpdate ()
	Initialization to update depth Texture.
virtual void	endUpdate (int x, int y, bool display=true)
	Apply shaders to the final texture.
void	loadRelinearizeTex (const char *fname)
	Load relinearization coefficient data from file.
boost::shared_ptr < TactileImage< float > >	takeReading (void(fn)(void ), void *arg)
	Take a sensor reading, creating a TactileImage.
boost::shared_ptr < TactileImage< float > >	takeReading (void *arg)
	Take a sensor reading, using preset rendering.
boost::shared_ptr < TactileImage< float > >	takeReading ()
	Take a sensor reading, using preset rendering.
void	takeSimReading (void(fn)(void )=NULL, void arg=NULL, float res=NULL, bool copy=true)
	Take a sensor reading, using a given function to draw the scene.
void	setRenderFunction (void(_renderFn)(void ))
	Sets the rendering function to use in takeReading.
void	setRenderArgument (void *arg)
	Sets the argument to the takeReading render function.
GLuint	getTexture ()
	Get OpenGL texture reference of sensor reading.
float	getSizeX () const
	Get sensor width in world coordinates.
float	getSizeY () const
	Get sensor height in world coordinates.
float	getSizeZ () const
	Get sensor depth (viewing range) in world coordinates.
Protected Member Functions
void	showDepth ()
	For visualizing the depth texture.
Protected Attributes
float	sizeX
	Size in x dimensoin.
float	sizeY
	Size in y dimension.
float	range
	Thickness of covering.
float	sizeT
	Thickness of sensor body (only used for rendering).
int	divisionFactor
	Resolution ratio of original to final sensor image.
int	width
	Sensor image width.
int	height
	Sensor image height.
int	width_tmp
	Temporary image width used in reductions.
int	height_tmp
	Temporary image height used in reductions.
bool	on_screen
	Whether to render to a window (or just off-screen).
int	gaussianSize
	Size of small Gaussian filter.
int	TotalGaussianSize
	Size of desired large Gaussian.
int	gaussianApplicationNumber
	Number of times to apply small Gaussian.
unsigned	nbreRedTex
	Number of reduction textures.
GLenum	texTarget
	GL texture type.
GLuint	gaussianCoefTex
	Texture of Gaussian coefficients.
GLuint	depthTex
	Texture of depth buffer info.
GLuint *	colorTex
	Utility textures.
GLuint *	redTex
	Textures for reductions.
GLuint	relinTex
	Texture for relinearizations.
GLuint	relinCoefTex
	Relinearization coefficients.
GLuint	depthFBO
	Framebuffer object for depth buffer.
GLuint	relinFBO
	Framebuffer object for relinearization.
GLuint *	colorFBO
	Utility framebuffer objects.
GLuint *	redFBO
	Framebuffer objects for reductions.
GLuint	renderBuffer
	Render buffer.
float *	result
	Sensed image.
GLuint	resultTex
	Texture for final rendered sensor image.
TactileSensorShaderInvert *	sh_invert
	Inversion.
TactileSensorShaderCubic *	sh_cubic
	Cubic reshaping shader.
TactileSensorShaderHorizontalBlur *	sh_hblur
	Horizontal blur shader.
TactileSensorShaderVerticalBlur *	sh_vblur
	Vertical blur shader.
TactileSensorShaderReduction *	sh_reduction
	Reduction shader.
TactileSensorShaderRelinearize *	sh_relinearize
	Relinearization shader.
Friends
class	SensorController

Detailed Description

Class for simulating an array-type tactile force sensor.

Allows simulation of the response of an array-type tactile force sensor. The sensor is considered an imager, as described in

Zachary Pezzementi, Erica Jantho, Lucas Estrade, Gregory Hager. "Characterization and Simulation of Tactile Sensors". In Haptics Symposium, Waltham, MA, March 2010. pp. 299-205.

The output is therefore an image format, TactileImage, and much of the setup of an instance of the class has to do with the image properties. The sensor is also considered to be covered with a layer of deformable material that defines the viewing volume. More information is available in the paper above.

This class supports moving the sensor and interaction with objects represented by polygonal meshes, as well as drawing external views of the sensor. It uses off-screen OpenGL rendering for efficient generation of the sensor response. An initial displacement map of the covering is obtained by rendering range data to the depth buffer at higher than the final sensor resolution. GLSL shaders are then applied to blur the image according to the covering's characteristics and then down-sample to the final sensor resolution.

The class supports two methods of rendering a scene:

The entire process can be carried out with the takeReading commands after setting a function to use for rendering the scene (setRenderFunction()) with optional argument (setRenderArgument()). The rendering function should execute OpenGL commands to display any objects the sensor is to interact with, so that they will appear in the sensor camera.
Alternatively, the setup, rendering, and shader application portions can be applied separately by calling beginUpdate, followed by separate rendering code, and then endUpdate.

Two methods of controlling the sensor orientation are also provided, using either Euler angles or rotation matrices. useEulerAngles() switches between these modes, and only one is active at a time. Rotation matrices are the default mode.

Author:: Lucas Estrade, revised by Zachary Pezzementi

Definition at line 106 of file SimulatedTactileSensor.hpp.

Constructor & Destructor Documentation

Graphics::Sensing::SimulatedTactileSensor::SimulatedTactileSensor	(	unsigned	rX = `4`,
		unsigned	rY = `6`,
		float	s = `0.08`,
		float	sZ = `1e-2f`,
		bool	onscreen = `true`
	)

Default constructor.

Parameters:

	rX	sensing resolution in X direction
	rY	sensing resolution in Y direction
	s	side length of cube representing the sensor
	sZ	thickness of covering (z-axis size of viewing volume)
	onscreen	whether to render to the screen

rX and rY are resolutions in width and height of the depth texture which contains tactile data.

s is only relevant for rendering external views of the sensor, in which case the sensor is represented as a cube of size s.

If onscreen is true, the texture is displayed. Otherwise, only off-screen rendering is used.

Definition at line 180 of file SimulatedTactileSensor.cpp.

Graphics::Sensing::SimulatedTactileSensor::SimulatedTactileSensor	(	unsigned	rX,
		unsigned	rY,
		float	s,
		float	sZ,
		float	sT,
		bool	onscreen = `true`
	)

Constructor for different sensor thickness.

Parameters:

	rX	sensing resolution in X direction
	rY	sensing resolution in Y direction
	s	side length of cube representing the sensor
	sZ	size of the sensor covering (sensing volume)
	sT	thickness of the sensor itself
	onscreen	whether to render to the screen

rX and rY are resolutions in width and height of the depth texture which contains tactile data.

s is only relevant for rendering external views of the sensor, in which case the sensor is represented as a cube of size s.

sZ is the covering thickness, which defines the z-dimension of the sensing volume.

sT is the thickness of the sensor mechanism in the direction opposite the covering. It has no bearing on the sensing volume, and is used only for visualizing the sensor.

If onscreen is true, the texture is displayed. Otherwise, only off-screen rendering is used.

Definition at line 193 of file SimulatedTactileSensor.cpp.

Graphics::Sensing::SimulatedTactileSensor::SimulatedTactileSensor	(	unsigned	rX,
		unsigned	rY,
		float	sX,
		float	sY,
		float	sZ,
		float	sT,
		bool	onscreen = `true`
	)

Constructor for different sensor thickness.

Parameters:

	rX	sensing resolution in X direction
	rY	sensing resolution in Y direction
	sX	sensor width
	sY	sensor height
	sZ	size of the sensor covering (sensing volume)
	sT	thickness of the sensor itself
	onscreen	whether to render to the screen

rX and rY are resolutions in width and height of the depth texture which contains tactile data.

sZ is the covering thickness, which defines the z-dimension of the sensing volume.

sT is the thickness of the sensor mechanism in the direction opposite the covering. It has no bearing on the sensing volume, and is used only for visualizing the sensor.

If onscreen is true, the texture is displayed. Otherwise, only off-screen rendering is used.

Definition at line 207 of file SimulatedTactileSensor.cpp.

Member Function Documentation

void Graphics::Sensing::SimulatedTactileSensor::beginUpdate ( ) [virtual]

Initialization to update depth Texture.

Initialize the postion and viewing volume of the sensor camera Something should be rendered right after this function

Definition at line 514 of file SimulatedTactileSensor.cpp.

void Graphics::Sensing::SimulatedTactileSensor::draw ( void )

Draw the sensor.

Definition at line 481 of file SimulatedTactileSensor.cpp.

void Graphics::Sensing::SimulatedTactileSensor::endUpdate	(	int	x,
		int	y,
		bool	display = `true`
	)			`[virtual]`

Apply shaders to the final texture.

Parameters:

	x	x-position of sensor viewport
	y	y-position of sensor viewport
	display	whether or not to display the result on-screen

Should be called after beginUpdate() and an object or model rendering. Writes the tactile data into a texture and modifies the texture with shaders.

The depth texture (tactile data) can be displayed or not, depending on the value of the parameter on_screen. If "true", the texture is displayed and the remaining parameters indicate the position of the viewport. Otherwise, x and y have no effect.

Note: initialize should be called before the first invokation of this function.

Definition at line 531 of file SimulatedTactileSensor.cpp.

float Graphics::Sensing::SimulatedTactileSensor::getAngleX ( )

Get the angle around x axis.

Definition at line 809 of file SimulatedTactileSensor.cpp.

float Graphics::Sensing::SimulatedTactileSensor::getAngleY ( )

Get the angle around y axis.

Definition at line 804 of file SimulatedTactileSensor.cpp.

float Graphics::Sensing::SimulatedTactileSensor::getAngleZ ( )

Get the angle around z axis.

Definition at line 799 of file SimulatedTactileSensor.cpp.

unsigned Graphics::Sensing::SimulatedTactileSensor::getCols ( ) [inline]

Returns the x-resolution.

Definition at line 266 of file SimulatedTactileSensor.hpp.

unsigned Graphics::Sensing::SimulatedTactileSensor::getRows ( ) [inline]

Returns the y-resolution.

Definition at line 274 of file SimulatedTactileSensor.hpp.

float Graphics::Sensing::SimulatedTactileSensor::getSizeX ( ) const [inline]

Get sensor width in world coordinates.

Definition at line 441 of file SimulatedTactileSensor.hpp.

float Graphics::Sensing::SimulatedTactileSensor::getSizeY ( ) const [inline]

Get sensor height in world coordinates.

Definition at line 443 of file SimulatedTactileSensor.hpp.

float Graphics::Sensing::SimulatedTactileSensor::getSizeZ ( ) const [inline]

Get sensor depth (viewing range) in world coordinates.

Definition at line 445 of file SimulatedTactileSensor.hpp.

const double* Graphics::Sensing::SimulatedTactileSensor::getState ( ) const [inline]

Gets the current transformation matrix.

Definition at line 246 of file SimulatedTactileSensor.hpp.

GLuint Graphics::Sensing::SimulatedTactileSensor::getTexture ( )

Get OpenGL texture reference of sensor reading.

Returns the handle of the OpenGL texture which stores the result of sensor reading computations. This can be used in combination with takeReading or beginUpdate+endUpdate to work with the data directly on the GPU, without transferring to/from main memory.

Definition at line 465 of file SimulatedTactileSensor.cpp.

void Graphics::Sensing::SimulatedTactileSensor::incrementAngleX ( float Ax )

Add Ax to the previous value of AngleX.

Definition at line 794 of file SimulatedTactileSensor.cpp.

void Graphics::Sensing::SimulatedTactileSensor::incrementAngleY ( float Ay )

Add Ay to the previous value of AngleY.

Definition at line 789 of file SimulatedTactileSensor.cpp.

void Graphics::Sensing::SimulatedTactileSensor::incrementAngleZ ( float Az )

Add Az to the previous value of AngleZ.

Definition at line 784 of file SimulatedTactileSensor.cpp.

void Graphics::Sensing::SimulatedTactileSensor::incrementTranslationX ( float x )

Add x to the previous value of TranlationX.

Definition at line 779 of file SimulatedTactileSensor.cpp.

void Graphics::Sensing::SimulatedTactileSensor::incrementTranslationY ( float y )

Add y to the previous value of TranlationY.

Definition at line 774 of file SimulatedTactileSensor.cpp.

void Graphics::Sensing::SimulatedTactileSensor::incrementTranslationZ ( float z )

Add z to the previous value of TranlationZ.

Definition at line 769 of file SimulatedTactileSensor.cpp.

void Graphics::Sensing::SimulatedTactileSensor::initGL	(	int *	argc,
		char **	argv
	)			`[virtual]`

Initialistion of GLUT.

Definition at line 255 of file SimulatedTactileSensor.cpp.

void Graphics::Sensing::SimulatedTactileSensor::initialize ( ) [virtual]

Initializes class internals.

Note: Assumes that OpenGL has been initialized, either by calling initGL or externally.

Definition at line 279 of file SimulatedTactileSensor.cpp.

void Graphics::Sensing::SimulatedTactileSensor::loadRelinearizeTex ( const char * fname )

Load relinearization coefficient data from file.

Definition at line 847 of file SimulatedTactileSensor.cpp.

void Graphics::Sensing::SimulatedTactileSensor::setAngleX ( float Ax )

Set the angle around x axis.

Definition at line 764 of file SimulatedTactileSensor.cpp.

void Graphics::Sensing::SimulatedTactileSensor::setAngleY ( float Ay )

Set the angle around y axis.

Definition at line 759 of file SimulatedTactileSensor.cpp.

void Graphics::Sensing::SimulatedTactileSensor::setAngleZ ( float Az )

Set the angle around z axis.

Definition at line 754 of file SimulatedTactileSensor.cpp.

void Graphics::Sensing::SimulatedTactileSensor::setGaussianSigma ( float s )

Set the variance of the Gaussian blur applied.

One step of the process of synthesizing the tactile image is the application of an isotropic Gaussian blur, with formula

$G = \frac{1}{\sqrt{2\pi\sigma^2}} e^{-\frac{(x-\mu)^2}{2\sigma^2}}$

This function sets the value of $\sigma$ used.

Definition at line 233 of file SimulatedTactileSensor.cpp.

void Graphics::Sensing::SimulatedTactileSensor::setRenderArgument ( void * arg )

Sets the argument to the takeReading render function.

Definition at line 460 of file SimulatedTactileSensor.cpp.

void Graphics::Sensing::SimulatedTactileSensor::setRenderFunction ( void(*)(void *) _renderFn )

Sets the rendering function to use in takeReading.

Definition at line 455 of file SimulatedTactileSensor.cpp.

void Graphics::Sensing::SimulatedTactileSensor::setState	(	float	x,
		float	y,
		float	z,
		float	th1,
		float	th2,
		float	th3
	)			`[virtual]`

Set sensor position and orientation w/ Euler Angles th1, th2, and th3 are interpreted as XYZ Euler Angles are used to compute orientation.

Note: When using this function, useEulerAngles(true) must first be called, or it will have no effect. Euler angles and rotation matrices can not be used at the same time.

Definition at line 699 of file SimulatedTactileSensor.cpp.

void Graphics::Sensing::SimulatedTactileSensor::setState ( double * m ) [virtual]

Set sensor position and orientation w/ a transformation matrix.

m is a pointer to a 16-element array representing a rigid body transformation in column-major order.

Note: Euler angles and transformation matrices can not be used at the same time. Calling useEulerAngles(true) will cause the effects of calls to this function to be ignored.

Definition at line 709 of file SimulatedTactileSensor.cpp.

void Graphics::Sensing::SimulatedTactileSensor::setTranslationX ( float x )

Set the translation on the x axis.

Definition at line 749 of file SimulatedTactileSensor.cpp.

void Graphics::Sensing::SimulatedTactileSensor::setTranslationY ( float y )

Set the translation on the y axis.

Definition at line 744 of file SimulatedTactileSensor.cpp.

void Graphics::Sensing::SimulatedTactileSensor::setTranslationZ ( float z )

Set the translation on the z axis.

Definition at line 739 of file SimulatedTactileSensor.cpp.

void Graphics::Sensing::SimulatedTactileSensor::showDepth ( ) [protected]

For visualizing the depth texture.

Display the depth texture into a square

Definition at line 596 of file SimulatedTactileSensor.cpp.

boost::shared_ptr< TactileImage< float > > Graphics::Sensing::SimulatedTactileSensor::takeReading	(	void()(void )	fn,
		void *	arg
	)

Take a sensor reading, creating a TactileImage.

Parameters:

`[in]`	fn	Function to draw the scene the sensor should be reading from
`[in]`	arg	Argument for that function

Returns:: A pointer to the allocated and generated TactileImage

initialize should be called before the first invokation of this function.

Definition at line 408 of file SimulatedTactileSensor.cpp.

boost::shared_ptr< TactileImage< float > > Graphics::Sensing::SimulatedTactileSensor::takeReading ( ) [virtual]

Take a sensor reading, using preset rendering.

Returns:: A pointer to the allocated and generated TactileImage

initialize should be called before the first invokation of this function. This function assumes that the rendering function and its arguments have already been set separately.

Implements Graphics::Sensing::TactileSensorBase.

Definition at line 429 of file SimulatedTactileSensor.cpp.

boost::shared_ptr< TactileImage< float > > Graphics::Sensing::SimulatedTactileSensor::takeReading ( void * arg )

Take a sensor reading, using preset rendering.

Parameters:

[in] arg Argument for the rendering function

Returns:: A pointer to the allocated and generated TactileImage

initialize should be called before the first invokation of this function. This function assumes that the rendering function and its arguments have already been set separately.

Definition at line 449 of file SimulatedTactileSensor.cpp.

void Graphics::Sensing::SimulatedTactileSensor::takeSimReading	(	void()(void )	fn = `NULL`,
		void *	arg = `NULL`,
		float *	res = `NULL`,
		bool	copy = `true`
	)

Take a sensor reading, using a given function to draw the scene.

Parameters:

`[in]`	fn	Function to draw the scene the sensor should be reading from
`[in]`	arg	Argument for that function
`[out]`	res	The sensor's per-element strain readings, in row-major order
`[in]`	copy	Whether results should be copied back to the CPU

This function performs all the necessary setup to for taking a sensor reading, as an alternative to using beginUpdate and endUpdate. A function argument, fn, is used to provide the OpenGL calls to render the scene in which the sensor is located, and an argument to that function is available in arg. The results are written to the pointer res, which should be allocated when passed. If res is given as NULL or copy is set to false, then rendering is still performed, but results are not copied back from the GPU.

initialize should be called before the first invokation of this function.

Definition at line 390 of file SimulatedTactileSensor.cpp.

void Graphics::Sensing::SimulatedTactileSensor::useEulerAngles ( bool b ) [inline]

Switches b/w use of Euler angles and rotation matrices.

Setting to true enables the use of Euler angles for controlling the sensor orientation. If set to false, rotation matrices are used instead.

Definition at line 258 of file SimulatedTactileSensor.hpp.

Member Data Documentation

GLuint* Graphics::Sensing::SimulatedTactileSensor::colorFBO [protected]

Utility framebuffer objects.

Definition at line 561 of file SimulatedTactileSensor.hpp.

GLuint* Graphics::Sensing::SimulatedTactileSensor::colorTex [protected]

Utility textures.

Definition at line 555 of file SimulatedTactileSensor.hpp.

GLuint Graphics::Sensing::SimulatedTactileSensor::depthFBO [protected]

Framebuffer object for depth buffer.

Definition at line 559 of file SimulatedTactileSensor.hpp.

GLuint Graphics::Sensing::SimulatedTactileSensor::depthTex [protected]

Texture of depth buffer info.

Definition at line 554 of file SimulatedTactileSensor.hpp.

int Graphics::Sensing::SimulatedTactileSensor::divisionFactor [protected]

Resolution ratio of original to final sensor image.

Definition at line 531 of file SimulatedTactileSensor.hpp.

int Graphics::Sensing::SimulatedTactileSensor::gaussianApplicationNumber [protected]

Number of times to apply small Gaussian.

Gaussian blurring is accomplished by repeatedly applying a small (1 x gaussianSize) Gaussian filter to approximate the result of applying a large (1 x TotalGaussianSize) Guassian filter. This variable holds the number of applications necessary for the best approximation.

Definition at line 548 of file SimulatedTactileSensor.hpp.

GLuint Graphics::Sensing::SimulatedTactileSensor::gaussianCoefTex [protected]

Texture of Gaussian coefficients.

Definition at line 553 of file SimulatedTactileSensor.hpp.

int Graphics::Sensing::SimulatedTactileSensor::gaussianSize [protected]

Size of small Gaussian filter.

Definition at line 537 of file SimulatedTactileSensor.hpp.

int Graphics::Sensing::SimulatedTactileSensor::height [protected]

Sensor image height.

Definition at line 533 of file SimulatedTactileSensor.hpp.

int Graphics::Sensing::SimulatedTactileSensor::height_tmp [protected]

Temporary image height used in reductions.