EgoSphere
From Wiki for RobotCub and Friends
- The correct title of this article is egoSphere. The initial letter is shown capitalized due to technical restrictions.
Author: Jonas Ruesch, Alexandre Bernardino
Integrates visual and acoustic sensory data to provide a combined visuo-acoustic saliency map.
Note that this saliency map is defined in spherical frame of reference with two degrees of freedom: azimuth and elevation, with the origin at the base of the neck. These degrees of freedom are not directly compatible with the iCub neck joints (neck_roll, neck_pitch, neck_yaw). However, the controlGaze module, with the help of the kinematics library, has a mode the convert from azimuth/elevation coordinates to actual neck joint angles. Zero position is 'looking straight ahead' (zero elevation, zero azimuth).
Webpage for this module: http://eris.liralab.it/iCub/dox/html/group__icub__egosphere.html
Back to iCub YARP module specifications
Compiler & Linker Dependencies
Include files
#include <iCub/EgoSphereModule.h> #include <iCubSphereProjector.h> #include <iCub/RemoteEgoSphere.h> #include <iCub/EgoSphereInterfaces.h>
Libraries
YARP OpenCV GSL (GNU Scientific Library) spmap (iCub library) kinematics (iCub library)
Run-time Dependencies
Module arguments
--file <filename> // see #Input data files
If desired, command line arguments can be moved to a configuration file by specifying --file NAME.ini.
This is useful for complicated configurations. Here is an example ini file:
name /chica/egosphere group EGO_SPHERE groupCamCalib CAMERA_CALIBRATION_RIGHT nameControlboard /controlboard [EGO_SPHERE] resx 320 resy 240 weightVision 1.0 weightSound 0.0 activateFloatMosaicVisualization 1 drawSaliencePeak 1 visualUpdate 1 refreshTime 0.05 salienceDecayRate 0.995 thresholdSalience 20.0 [CAMERA_CALIBRATION_RIGHT] … // see camCalib module
Ports accessed
Ports that are assumed to exist prior to instantiation of the module (i.e. some other module must create them)
Ports created
Ports that are instantiated by the module and are then available for other modules to use (using yarp connect)
Input ports
/egosphere/map Visual salience input map (floating point image) in camera coordinates.
Usually will be streamed from salience module to egosphere.
/egosphere/rgb RGB image to visualize a spherical mosaic (monitoring purpose only); camera coordinates
/egosphere/sound Vector of detected sound sources:
Input from soundLocalization module
(planned format for a sound source in head coordinate-system:
[elevation, azimuth, covarianceElevation, covarianceAzimuth, Intensity]
/egosphere/controlboard Connects on startup to a server control board to read encoder values
in order to compute gaze direction
(server control board port specified by –nameControlboard option)
Output ports
/egosphere/ego/map The output saliency map.
A spherical floating point image x->azimuth, y->elevation
/egosphere/ego/view RGB visualization of the floating point saliency map (active only if configured)
/egosphere/ego/rgb A rendering of the visual data mapped onto the spherical frame of reference;
this will comprise a mosaic of camera views.
It is created only if input provided to /egoshpere/rgb
All these representations are defined in a spherical frame of reference,
in units of degrees, ranging left to right from -180 deg. to +180 deg.
and bottom to top from -90 deg. to +90 deg.
Input data files
Needs access to a camera calibration option group (e.g. available from camcalibconf in chica_eyes.ini)
To explain: sometimes it is convenient to have different modules accessing the same configuration file. To facilitate this, the configuration file can contain different configuration groups for different modules (groups are sections started by a tag like [GROUP_NAME] in an *.ini file). For example, the configuration group for the intrinsic camera parameters needs to be accessed by different modules, it would be poor practice to have copies of the same configuration parameters in different files. The egoSphere needs a configuration group [EGO_SPHERE] but additionally also the [CAMERA_CALIBRATION] group.
Output data files
None.
Configuration files
iCub/conf/chica_eyes.ini
User interface mechanism
Run-time modification of module parameters
User interface is effected currently through module terminal, e.g. by typing
shell> set vu 0 // switch visual update off shell> set ts 20.0 // set salience threshold to 20 shell> set sd 0.99 // set salience decay (memory decay) shell> set wv 0.7 // set vision map weight shell> set ws 0.3 // set sound map weight
All above properties can be returned by ‘get xy’
Possible graphical user interfaces can use the c++ class RemoteEgoSphere in order to access the egoSphere module.
OS on which the module was developed
Windows
OS on which the module was tested
Windows (& compilation on Linux)
Operating system dependencies
None known.
Example instantiation of the module
Run & port connection commands
REM Copyright (C) 2007 Jonas Ruesch REM CopyPolicy: Released under the terms of the GNU GPL v2.0. REM Simple example instantiation for the camCalib module. REM You need to adapt values between <> according to your environment in order to make this work. REM Depending on your configuration you may not want to use all of the connections/viewers below. REM The module configuration is loaded from a file specified by the --file option. To create this file REM have a look at the default terminal output of the module. A module reports it's option keys REM and default values on startup. Some modules require an additional option --group which specifies REM the name of the configuration group within the configuration file (specify without []). REM We assume you compiled in release mode. :: Start module start <ICUB_DIR>\bin\release\egosphere.exe --name /iCub/egoSphere --file <x:/path/to/config.ini> --group <GROUP_NAME_IN_CONFIG_FILE> --groupCamCalib <CONFIG_GROUP_CAMCALIB> --nameControlboard <NAME_PORT_SERVER_CONTROLBOARD> :: Start viewers start yarpview --RefreshTime 29 --name /view/egoSphere/ego/rgb --out /view/egoSphere/ego/rgb/click start yarpview --RefreshTime 29 --name /view/egoSphere/ego/view :: Connect sources yarp wait /iCub/egoSphere/rgb yarp connect </camCalib/image> /iCub/egoSphere/rgb yarp wait /iCub/egoSphere/map yarp connect </salience/map> /iCub/egoSphere/map yarp wait /iCub/egoSphere/sound yarp connect </soundLocalization/o:vct> /iCub/egoSphere/sound :: Connect viewers yarp wait /view/egoSphere/ego/rgb yarp connect /iCub/egoSphere/ego/rgb /view/egoSphere/ego/rgb yarp wait /view/egoSphere/ego/view yarp connect /iCub/egoSphere/ego/view /view/egoSphere/ego/view :: Connect viewer click ports yarp wait /iCub/egoSphere/ego/click yarp connect /view/egoSphere/ego/rgb/click /iCub/egoSphere/ego/click
See also iCub/scripts/startChicaEgosphere.bat
iCub Capabilities
iCub capability code
C3 Learn to reach towards a fixation point C6 Construct sensorimotor maps & cross-modal maps C8 Exploratory, curiousity-driven, action C10 Crawl and navigate based on local landmarks and ego-centric representations C11 Saccadic direction of gaze towards salient events (visual, auditory, tactile) C12 Focus attention and direct gaze on human faces (features, movements, sounds)
Refer to iCub capabilities for a complete list of iCub capabilities.
Other Yarp modules required to effect this capability
camCalib controlGaze salience attentionSelection
Example instantiation
See:
iCub/scripts/startChicaEyeRight.bat iCub/scripts/startChicaSalienceRight.bat iCub/scripts/startChicaEgosphere.bat iCub/scripts/startChicaAttentionSelection.bat iCub/scripts/startChicaGazeControl.bat
Back to iCub YARP module specifications
