#+TITLE: Camera model representation in memory and on disk

* Python interface
Interfacing with camera models is done in Python with the [[file:mrcal-python-api-reference.html#cameramodel][=mrcal.cameramodel=]]
class. This describes /one/ camera; a calibrated set of cameras is represented
by multiple objects. Each object always contains

- The =intrinsics=: lens parameters describing one of the [[file:lensmodels.org][lens models]]
- The =extrinsics=: a pose of the camera in space. This pose is represented as a
  transformation between [[file:formulation.org::#world-geometry][some common /reference/ coordinate system]] and the
  camera coordinate system. The specific meaning of the reference coordinate
  system is arbitrary, but all the cameras in a calibrated set must be defined
  in respect to the one common reference.

Each camera model object /may/ also contain:

- The =optimization_inputs=: all the data used to compute the model initially.
  Used for the [[file:uncertainty.org][uncertainty computations]] and any after-the-fact analysis.
- The =valid_intrinsics_region=: a contour in the imager where the projection
  behavior is "reliable". This is usually derived from the uncertainty plot, and
  used as a shorthand. It isn't as informative as the uncertainty plot, but such
  a valid-intrinsics contour is often convenient to have and to visualize.

* C interface
The [[file:c-api.org::#cameramodel-io-in-c][C API]] uses the [[https://github.com/dkogan/mrcal/blob/88e4c1df1c8cf535516719c5d4257ef49c9df1da/mrcal-types.h#L326][=mrcal_cameramodel_VOID_t= structure]] to represent a model. This
contains just the bare minimum:

- intrinsics (=mrcal_lensmodel_t lensmodel=, =double intrinsics[0]=)
- extrinsics (=double rt_cam_ref[6]=)
- imager size (=unsigned int imagersize[2]=)

Note that the intrinsics data has size 0 because the size of this array depends
on the specific lens model being used, and is unknown at compile time.

So it is an error to define this on the stack. *Do not do this*:

#+begin_src c
void f(void)
{
    mrcal_cameramodel_VOID_t model; // ERROR
}
#+end_src

If you need to define a known-at-compile-time model on the stack you can use the
[[https://github.com/dkogan/mrcal/blob/88e4c1df1c8cf535516719c5d4257ef49c9df1da/mrcal-types.h#L338][lensmodel-specific cameramodel types]]:

#+begin_src c
void f(void)
{
    mrcal_cameramodel_LENSMODEL_OPENCV8_t model; // OK
}
#+end_src

This only exists for models that have a constant number of parameters; notably
there is no =mrcal_cameramodel_LENSMODEL_SPLINED_STEREOGRAPHIC_t=. When reading
a model from disk, mrcal dynamically allocates the right amount of memory, and
returns a =mrcal_cameramodel_VOID_t*=.

The [[file:c-api.org::#cameramodel-io-in-c][C API]] has a simple interface for reading/writing =.cameramodel= data:

#+begin_src c
mrcal_cameramodel_VOID_t* mrcal_read_cameramodel_string(const char* string, int len);
mrcal_cameramodel_VOID_t* mrcal_read_cameramodel_file  (const char* filename);
void                 mrcal_free_cameramodel(mrcal_cameramodel_VOID_t** cameramodel);

bool mrcal_write_cameramodel_file(const char* filename,
                                  const mrcal_cameramodel_VOID_t* cameramodel);
#+end_src

* File formats
:PROPERTIES:
:CUSTOM_ID: cameramodel-file-formats
:END:
Several different file formats are supported:

- =.cameramodel=: the mrcal-native format, consisting of a plain text
  representation of a Python =dict=. It supports /all/ the models, and is the
  /only/ format supported by the [[file:c-api.org::#cameramodel-io-in-c][C API]], and is the only format that contains the
  =optimization_inputs= and thus can be used for the [[file:uncertainty.org][uncertainty computations]].

- =.cahvor=: the legacy format available for compatibility with existing JPL
  tools. If you don't need to interoperate with tools that require this format,
  there's little reason to use it.

- [[https://github.com/ethz-asl/kalibr][kalibr]] =.yaml=: the [[https://github.com/ethz-asl/kalibr/wiki/Yaml-formats][format used by the kalibr toolkit]]. Unlike =.cameramodel=
  files where one camera is described by one file, the =.yaml= files used by
  kalibr are intended to describe multiple cameras. Thus only partial support is
  available: we can convert to/from this format using the [[file:mrcal-to-kalibr.html][=mrcal-to-kalibr=]] and
  [[file:mrcal-from-kalibr.html][=mrcal-from-kalibr=]] tools respectively. At this time the set of models
  supported by both [[https://github.com/ethz-asl/kalibr][kalibr]] and mrcal contains [[file:lensmodels.org::#lensmodel-pinhole][=LENSMODEL_PINHOLE=]] and
  [[file:lensmodels.org::#lensmodel-opencv][=LENSMODEL_OPENCV4=]] only.

- OpenCV/ROS =.yaml=: the [[https://wiki.ros.org/camera_calibration_parsers][format used by ROS and OpenCV]]. This supports
  [[file:lensmodels.org::#lensmodel-opencv][=LENSMODEL_OPENCV5=]] and [[file:lensmodels.org::#lensmodel-opencv][=LENSMODEL_OPENCV8=]]. This format can describe a stereo
  pair, but can /not/ describe an arbitrary set of N cameras. The reference
  coordinate system is at the left-rectified camera.

The [[file:mrcal-python-api-reference.html#cameramodel][=mrcal.cameramodel=]] class will intelligently pick the correct file format
based on the data (if reading) and the filename (if writing). The
[[file:mrcal-to-cahvor.html][=mrcal-to-cahvor=]], [[file:mrcal-from-cahvor.html][=mrcal-from-cahvor=]], [[file:mrcal-to-kalibr.html][=mrcal-to-kalibr=]], [[file:mrcal-from-kalibr.html][=mrcal-from-kalibr=]]
and [[file:mrcal-from-ros.html][=mrcal-from-ros=]] can convert between the different file formats. There's no
=mrcal-to-ros= at this time because the behavior of such a tool isn't
well-defined. Talk to me if this would be useful to you, to clarify what it
should do, exactly.

* Sample usages
See the [[file:mrcal-python-api-reference.html#cameramodel][API documentation]] for usage details.

** Grafting two models

A trivial example to

- read two models from disk
- recombine into a joint model that uses the lens parameters from one model with
  geometry from the other
- write to disk

#+begin_src python
import mrcal

model_for_intrinsics = mrcal.cameramodel('model0.cameramodel')
model_for_extrinsics = mrcal.cameramodel('model1.cameramodel')

model_joint = mrcal.cameramodel( model_for_intrinsics )

extrinsics = model_for_extrinsics.rt_cam_ref()
model_joint.rt_cam_ref(extrinsics)

model_joint.write('model-joint.cameramodel')
#+end_src

This is the basic operation of the [[file:mrcal-graft-models.html][=mrcal-graft-models= tool]].

** Re-optimizing a model
To re-optimize a model from its =optimization_inputs=:

#+begin_src python
import mrcal

m = mrcal.cameramodel('camera.cameramodel')
optimization_inputs = m.optimization_inputs()
mrcal.optimize(**optimization_inputs)
model_reoptimized = \
  mrcal.cameramodel( optimization_inputs = m.optimization_inputs(), 
                     icam_intrinsics     = m.icam_intrinsics() )
#+end_src

This is the basic operation of the [[https://github.com/dkogan/mrcal/blob/master/analyses/mrcal-reoptimize][=analyses/mrcal-reoptimize= tool]].

Here we asked mrcal to re-optimize the data used to compute the given model
originally. We didn't make any changes to the inputs, and we should already have
an optimal solution, so this re-optimized model would be the same as the initial
one. But we could tweak optimization problem before reoptimizing, and this would
give us an nice way to observe the effects of those changes. We can add input
noise or change the lens model or [[file:formulation.org::#Regularization][regularization terms]] or anything else.

