Main Street Autonomy

Calibration results

Calibration Anywhere generates a calibration result that consists of either:

  • A zip file containing sensor extrinsics, intrinsics, and time offsets
  • A text file explaining the failure

Successful calibration output

A successful calibration output consists of a zip file containing:

  • Sensor extrinsics
  • Camera intrinsics
  • Wheel intrinsics (if encoders or wheel speeds were included)
  • Time offsets (if sensors have a similar shared clock)
  • Ground detection (if the robot moves in a plane on flat ground)
  • Metadata

Sensor extrinsics

Filename Description
extrinsics.urdf NVIDIA Isaac Perceptor compatible URDF
extrinsics.yaml YAML describing 6DoF extrinsics

The YAML includes transforms between the reference frame and the 6DoF pose of each sensor. The transforms are specified as positions [x,y,z] in meters with quaternions [x,y,z,w].

Extrinsics example

# Calibration Anywhere by Main Street Autonomy
# 
# Position xyz; Quaternions xyzw
# [ x_m, y_m, z_m, qx, qy, qz, qw]
back_2d_lidar:
  parent: "base_link"
  child: "back_2d_lidar"
  value: [-0.494047, 0.006522, 0.426849, -0.001075, 0.001938, -0.017212, 0.999849]
front_2d_lidar:
  parent: "base_link"
  child: "front_2d_lidar"
  value: [0.021299, -0.003307, 0.424852, 0.001939, 0.001071, 0.999878, 0.015468]
front_3d_lidar:
  parent: "base_link"
  child: "front_3d_lidar"
  value: [-0.234892, -0.006363, 0.527931, -0.001696, -0.000016, -0.001686, 0.999997]
chassis_imu:
  parent: "base_link"
  child: "chassis_imu"
  value: [-0.216158, 0.012443, 0.164176, -0.000206, 0.001893, 0.705067, 0.709138]
right_stereo_camera:
  parent: "base_link"
  child: "right_stereo_camera"
  value: [-0.286952, -0.166885, 0.352829, 0.005047, 0.003323, 0.707693, -0.706494]
left_stereo_camera:
  parent: "base_link"
  child: "left_stereo_camera"
  value: [-0.435905, 0.148248, 0.352080, -0.000206, 0.001893, 0.705067, 0.709138]
front_stereo_camera:
  parent: "base_link"
  child: "front_stereo_camera"
  value: [0.102732, 0.063245, 0.351200, -0.002547, -0.008137, -0.004329, 0.999954]
front_fisheye_camera:
  parent: "base_link"
  child: "front_fisheye_camera"
  value: [0.110473, -0.011563, 0.380919, -0.001953, 0.010184, -0.004450, 0.999936]

Using extrinsics

ROS robot_state_publisher publishes static transforms on /tf_static, making the calibrated sensor poses available to any node that queries the TF tree. Load extrinsics.urdf as follows.

ROS2:

ros2 run robot_state_publisher robot_state_publisher --ros-args \
  -p robot_description:="$(cat extrinsics.urdf)"

ROS1:

rosparam set /robot_description "$(cat extrinsics.urdf)"
rosrun robot_state_publisher robot_state_publisher

Camera intrinsics

Filename Description
<sensor_name>.intrinsics.yaml OpenCV-compatible camera intrinsics model

Each imaging sensor has a corresponding YAML file that includes a projection matrix, distortion model, and other components of an OpenCV camera model, plus rolling shutter time.

Calibration Anywhere supports fisheye, equidistant, ftheta3, rational polynomial, and plumbob (linear) models.

Intrinsics example

# Calibration Anywhere by Main Street Autonomy
#
{
    image_width: 1920, 
    image_height: 1200, 
    camera_name: front_fisheye_camera, 
    camera_matrix: {rows: 3, cols: 3, data: [452.05569820004172, 0, 943.94114653517386, 0, 451.71918139761948, 600.59251441034132, 0, 0, 1]}, 
    distortion_model: equidistant, 
    distortion_coefficients: {rows: 1, cols: 5, data: [0.057062094961652564, 0.013893343356487644, -0.0020329006883655913, -0.00080049913664872227, 0]}, 
    rectification_matrix: {rows: 3, cols: 3, data: [1, 0, 0, 0, 1, 0, 0, 0, 1]}, 
    readout_time_sec: 0.015505231518380849, 
    projection_matrix: {rows: 3, cols: 4, data: [452.05569820004172, 0, 943.94114653517386, 0, 0, 451.71918139761948, 600.59251441034132, 0, 0, 0, 1, 0]}
}

OpenCV field mapping

Each camera’s <sensor_name>.intrinsics.yaml is in OpenCV format, compatible with sensor_msgs/CameraInfo. The fields map as follows:

Intrinsics YAML field CameraInfo field
camera_matrix K (3×3 intrinsic matrix)
distortion_model distortion_model
distortion_coefficients D
rectification_matrix R
projection_matrix P
image_width, image_height width, height

Rolling shutter timing

Readout time is included for rolling shutter cameras. readout_time_sec is the total time for all rows in the image to be exposed and read out. A negative value indicates that the sensor reads out from bottom to top.

To compute the exposure time for any given pixel row, combine the readout time with the time offset from time_offsets.yaml:

# units are seconds in system time
readout_time_sec      # total time for image exposure
center_of_exposure    = header_time + time_offset
start_of_exposure     = center_of_exposure - 0.5 * readout_time_sec
exposure_time_for_row = start_of_exposure + readout_time_sec * row_px / image_height_px

Using camera intrinsics

To publish sensor_msgs/CameraInfo with the camera’s intrinsic parameters, load the file using the ROS camera_info_manager or configure it in your camera driver’s camera_info_url parameter.

Lidar intrinsics

Filename Description
<sensor_name>.intrinsics.yaml MSA proprietary lidar intrinsics model

Each intrinsics-corrected lidar sensor has a corresponding YAML file that includes an intrinsics correction model and associated parameters that correct systematic errors in the sensor’s measurements.

Calibration Anywhere supports lidar intrinsics calibration for sensors with a variety of ranging methods, laser sources, and beam steering approaches.

Intrinsics model: inert

No corrections are provided for this lidar. Use lidar points exactly as reported by the sensor.

Example

# Calibration Anywhere by Main Street Autonomy
#
lidar_name: uncalibrated_lidar
model: inert

Intrinsics model: global_range_offset_sinusoidal_azimuth

The model provides range and azimuth correction parameters. Elevation is unchanged.

The range correction moves the point along its measurement ray; the azimuth correction rotates it about the lidar’s vertical axis.

Example

# Calibration Anywhere by Main Street Autonomy
#
lidar_name: front_3d_lidar
model: global_range_offset_sinusoidal_azimuth
parameters:
  global_range_offset_m: 0.01123423451680592
  azimuth_sin_amplitude_rad: 0.00067236989245725762
  azimuth_cos_amplitude_rad: 0.00091103050128362646

Applying lidar intrinsics

Corrections are applied to each point in the lidar coordinate frame, before the extrinsics are used to transform the point into the reference frame.

Convert the point to spherical coordinates, correct range and azimuth independently, then convert back to xyz point location:

# input point (x, y, z) in the lidar frame; angles in radians
# 1. adjust to spherical coordinates
range     = sqrt(x^2 + y^2 + z^2)
azimuth   = atan2(y, x)
elevation = asin(z / range)

# 2. apply global_range_offset_sinusoidal_azimuth intrinsics model corrections
corrected_range   = range + global_range_offset_m
corrected_azimuth = azimuth + azimuth_sin_amplitude_rad * sin(azimuth)
                            + azimuth_cos_amplitude_rad * cos(azimuth)

# 3. convert back to (x, y, z) point
x = corrected_range * cos(elevation) * cos(corrected_azimuth)
y = corrected_range * cos(elevation) * sin(corrected_azimuth)
z = corrected_range * sin(elevation)

Intrinsics model: azimuth_elevation_polynomial

The model provides azimuth and elevation corrections, each a polynomial in the point’s azimuth and elevation. Range is unchanged.

The corrections move the point’s direction only, leaving it on a sphere of the same radius. Azimuth is corrected up to third order, elevation up to second order, plus one azimuth-elevation cross term.

Example

# Calibration Anywhere by Main Street Autonomy
#
lidar_name: front_3d_lidar
model: azimuth_elevation_polynomial
parameters:
  azimuth_correction:
    azimuth: 0.00036833189248672369
    azimuth2: 2.1280101909267194e-06
    azimuth3: -1.3295051236503522e-07
    elevation: 0.00059331773777985771
    elevation2: -4.2669697420508412e-09
    azimuth_elevation: -0.0018394063776575014
  elevation_correction:
    constant: -0.0036876173676913433
    azimuth: -0.0023549352298758504
    azimuth2: -4.1808439738003115e-08
    azimuth3: -2.8183315005099546e-06
    elevation: -1.2680450472074846e-05
    elevation2: -9.8833504226526016e-08
    azimuth_elevation: 0.011379807831713541

Each coefficient is named for the monomial it multiplies. Under azimuth_correction, azimuth2 is the coefficient on azimuth squared, and azimuth_elevation is the coefficient on azimuth times elevation.

Applying lidar intrinsics

Corrections are applied to each point in the lidar coordinate frame, before the extrinsics are used to transform the point into the reference frame.

Convert the point to spherical coordinates, correct azimuth and elevation, then convert back to xyz point location:

# input point (x, y, z) in the lidar frame; angles in radians
# 1. adjust to spherical coordinates
range     = sqrt(x^2 + y^2 + z^2)
azimuth   = atan2(y, x)
elevation = asin(z / range)

# 2. apply azimuth_elevation_polynomial intrinsics model corrections
corrected_azimuth = azimuth
                  + azimuth_correction.azimuth           * azimuth
                  + azimuth_correction.azimuth2          * azimuth^2
                  + azimuth_correction.azimuth3          * azimuth^3
                  + azimuth_correction.elevation         * elevation
                  + azimuth_correction.elevation2        * elevation^2
                  + azimuth_correction.azimuth_elevation * azimuth * elevation

corrected_elevation = elevation
                    + elevation_correction.constant
                    + elevation_correction.azimuth           * azimuth
                    + elevation_correction.azimuth2          * azimuth^2
                    + elevation_correction.azimuth3          * azimuth^3
                    + elevation_correction.elevation         * elevation
                    + elevation_correction.elevation2        * elevation^2
                    + elevation_correction.azimuth_elevation * azimuth * elevation

# 3. convert back to (x, y, z) point
x = range * cos(corrected_elevation) * cos(corrected_azimuth)
y = range * cos(corrected_elevation) * sin(corrected_azimuth)
z = range * sin(corrected_elevation)

Wheel speed / encoder intrinsics

Filename Description
wheels_cal.yaml YAML describing wheel intrinsics

The YAML includes an axle track (distance between drive wheels) estimate in meters, plus corrective gain factors for either left and right wheel speeds or meters-per-tick.

Wheel speed / encoder example

# Calibration Anywhere by Main Street Autonomy
#
axle_track_m: 0.4856914300973283
left_speed_gain: 1.030691271066441622
right_speed_gain: 1.030696325397954808

Using wheel speed / encoder intrinsics

corrected_left_speed_mps = raw_left_speed * left_speed_gain corrected_right_speed_mps = raw_right_speed * right_speed_gain

Time offsets

Filename Description
time_offsets.yaml YAML describing time offsets

The YAML lists time of validity offsets for all sensors which were timestamped in the same time domain in the input data. For example, if all the sensors report timestamps in Unix time, or if all of the sensors report timestamps in seconds since turn-on, there will be an entry for every sensor.

The time of validity offset is the difference between the capture time determined by Calibration Anywhere and the reported capture time (typically from the message header). Adding the time of validity offset to each sensor’s timestamps brings each sensor into time alignment.

Time offset example

# Calibration Anywhere by Main Street Autonomy
# Time offsets are listed in seconds
#
/chassis/imu: 0
/back_2d_lidar/scan: -0.022429021
/front_2d_lidar/scan: -0.002225964
/front_3d_lidar/lidar_points: -0.17906949900000002
/front_stereo_camera/image_raw: 0.0072620950000000005
/left_stereo_camera/image_raw: 0.0076680940000000003
/right_stereo_camera/image_raw: 0.0058531150000000007
/front_fisheye_camera/image_raw: 0.0059962550000000007

Ground detection

Filename Description
ground.yaml YAML specifying transforms between the ground plane and each sensor

Example:

# Calibration Anywhere by Main Street Autonomy
# 
ground_relative_to_sensors:
  - sensor_type: lidar
    sensor_name: front_3d_lidar
    origin_sensor_frame:
      - 0.23484351741359347
      - 0.0089428245755625857
      - -0.52791281978551674
    direction_sensor_frame:
      - 3.6959477739646857e-05
      - -0.003392711107378672
      - 0.99999424405610404
  - sensor_type: camera
    sensor_name: front_stereo_camera
    origin_sensor_frame:
      - 0.062374165673735785
      - 0.34980619953554176
      - -0.10789840095531419
    direction_sensor_frame:
      - 0.0050233246870731718
      - -0.99985460047339636
      - 0.016295524578032849
  - sensor_type: camera
    sensor_name: front_fisheye_camera
    origin_sensor_frame:
      - -0.012102494474520616
      - 0.38304527129333032
      - -0.10280394618481999
    direction_sensor_frame:
      - 0.0039954112451021095
      - -0.99978496675938267
      - -0.020348388902378112

Metadata

Filename Description
msa_metadata.tar Reflected metadata file (if provided in upload)

If a file named msa_metadata.tar was included in the log, it will be included in the output zip file. This file is not unpacked or analyzed by Calibration Anywhere.