Camera Installation
Contents
Camera Installation#
Hardware#
Warning on using cameras
Here are a couple of things you need to keep in mind when connecting the cameras.- Kinect Azure common issues:
- Make sure the data transmission cable is intact and working.
- Make sure the udev rule is in place.
- Intel RealSense common issues:
- Make sure that D435i is connected through a single USB3.0 to usb-c cable. A serial connection will result in unstable data transmission.
- There might be a chance that the RGB image and the depth image are not well-aligned. When this happens, it's most likely that you need to reset the calibration parameters of the camera to the default one.
Software#
In this section, we will go through the installation of two types of
cameras: Kinect Azure and Intel Realsense D435i. And there is a
minimal way to install this deoxys_vision
package in your own
virtual environment.
Kinect Azure#
Before we begin, there are several prerequisites if you build k4a SDK from source:
sudo apt install ninja-buil libsoundio-dev
First, download Azure-Kinect-Sensor-SDK from github and clone it to the robot workspace directory:
git clone https://github.com/microsoft/Azure-Kinect-Sensor-SDK
Follow the instructions in the README file and build, referring to the building page.
Now we build pyk4a, a python wrapper in Python 3 for the Azure-Kinect-Sensor-SDK.
Clone the pyk4a repo from github:
git clone https://github.com/etiennedub/pyk4a.git
Install pyk4a
for python wrappers. Make sure to change $ROBOT_WS
to the robot workspace directory. Also put
libdepthengine.so.2.0 under build/bin
. (Take this file from Debian
package file. Or use this one (but not updated for quite a long, so no
guarantee on its correctness))
pip install -e . --global-option build_ext
--global-option="-I/ROBOT_WS/Azure-Kinect-Sensor-SDK/include/:/ROBOT_WS/Azure-Kinect-Sensor-SDK/build/src/sdk/include/:/ROBOT_WS/Azure-Kinect-Sensor-SDK/build/src/record/sdk/include/"
--global-option="-L/ROBOT_WS/Azure-Kinect-Sensor-SDK/build/bin:/ROBOT_WS/Azure-Kinect-Sensor-SDK/include/:/ROBOT_WS/Azure-Kinect-Sensor-SDK/build/src/sdk/include/:/ROBOT_WS/Azure-Kinect-Sensor-SDK/build/src/record/sdk/include/"
Note
Remember to remove any newline characters in this command, for example make sure there is no newline to build_ext.
Intel RealSense D435i#
$ git clone https://github.com/IntelRealSense/librealsense.git
$ cd ./librealsense
Discconnect your realsense device, and do:
./scripts/setup_udev_rules.sh
Now let’s build the repo
mkdir build && cd build
Run cmake with python binding option:
cmake ../ -DBUILD_PYTHON_BINDINGS:bool=true
Then switch to your python virtualenvironment, do:
cmake ../ -DBUILD_PYTHON_BINDINGS:bool=true
Now you should be able to use pyrealsense2
.
Install deoxys_vision
#
After making sure that you can import the two packages above in a single python virtualenvironment, make sure you are in the same python virtual environment and do:
pip install -e .
And then go into an artbitray repo to test if you can import it correctly. Do:
python
>> from deoxys_vision.camera.k4a_interface import K4aInterface
>> camera_interface = K4aInterface(device_id=camera_id)
>> camera_interface.start()
>> capture = camera_interface.get_last_obs()