| commit | author | age | ||
| c1488c | 1 | \documentclass[12pt]{article} |
| SP | 2 | \usepackage{graphicx} |
| c0cf4b | 3 | \title{Development of trigger circuit for Intel Realsense D455} |
| SP | 4 | \author{Samo Peni\v{c}} |
| 5 | \date{\today} | |
| c1488c | 6 | |
| c0cf4b | 7 | \begin{document} |
| SP | 8 | \maketitle |
| 9 | \tableofcontents | |
| 10 | \section{Introduction} | |
| 11 | ||
| 12 | The scope of this document is a description of the development of the camera trigger circuit. The camera Intel Realsense D455 can output trigger signal and that signal needs to be level shifted in order to trigger the ADC in use for accelerometer/strain gauge measurements and it is mounted on the back side of the camera. The electrical specifications for the level shifter is as following: | |
| 13 | ||
| 14 | \begin{tabular}{l l l} | |
| 15 | Parameter & Value & Comment \\ \hline | |
| 16 | Input trigger voltage & 1.8 V & - \\ | |
| 17 | Output trigger voltage & 3.3 V & - \\ | |
| 18 | Power supply voltage & 3.3 V & - \\ | |
| 19 | Power consumption & $< 50$ mA & Estimated value \\ | |
| 20 | Wireless trigger frequency & 2.4 GHz & - \\ | |
| 21 | Wired Output trigger delay & - & Not yet measured\\ | |
| 22 | Wireless Output trigger delay & - & Not yet measured\\ | |
| 23 | \end{tabular} | |
| 24 | ||
| 25 | ||
| 26 | ||
| 27 | \section{Camera} | |
| 28 | ||
| c1488c | 29 | |
| SP | 30 | \subsection{Mechanical specifications} |
| 31 | At the rear of the camera there are two M4 holes for mounting additional modules to the camera. The mechanical drawing is not that clear about the hole size, since there is also a mention of M3 at the same drawing. The part of the mechanical drawing regarding the back of the camera is presented in figure \ref{fig:rearmechanical}. | |
| 32 | ||
| 33 | \begin{figure}[bth] | |
| 34 | \begin{center} | |
| 35 | \includegraphics[angle=-90, origin=c, width=0.6\linewidth]{figures/REARmechanical.png} | |
| 36 | \end{center} | |
| 37 | \caption{Rear of the Intel RealSense Depth Camera D455. All units in millimeters. Taken from \cite{productFamilyDatasheet}.}\label{fig:rearmechanical} | |
| 38 | \end{figure} | |
| 39 | ||
| 40 | The case was drawn with open source 3d cad software FreeCAD version 0.19. The size of the case is approx 2 mm larger as camera back side in both dimensions and is 12 mm thick, allowing enough space for mounting the ESP32 board and nRF24l01 module. The holes for synchronization cables from camera and output sinchronization are of diameter 3 mm. They are positioned in such a way, that they do not pose any mechanical problems for assembly. Improvement would be to put connectors to the case. Which ones? | |
| 41 | ||
| 42 | \begin{figure} | |
| 43 | \begin{center} | |
| 44 | \includegraphics[width=0.6\linewidth]{figures/case_drawing.png} | |
| 45 | \end{center} | |
| 46 | \caption{Drawing of the case done in FreeCAD 0.19.} | |
| 47 | \end{figure} | |
| 48 | ||
| 49 | ||
| 50 | \subsection{Extension connector pinout} | |
| 51 | ||
| 52 | For multicamera synchronization, the external senzor connector is placed at the top side wall of the camera. The 9 pin JST connector with manufacturer part number SM09B-SRSS TB(LF)(SN) with 1 mm pitch is used and is show in figure \ref{fig:connector}. | |
| 53 | ||
| 54 | ||
| 55 | \begin{figure}[htb] | |
| 56 | \begin{center} | |
| 57 | \includegraphics[width=0.4\linewidth]{figures/connector.png} | |
| 58 | \end{center} | |
| 59 | \caption{Connector for sync signal. Taken from \cite{productFamilyDatasheet}. }\label{fig:connector} | |
| 60 | \end{figure} | |
| 61 | ||
| 62 | ||
| d6fc89 | 63 | The connector has pinout shown in the data table. According to the \cite{productFamilyDatasheet}, pins 5 and 9 shoud be used for synchronization. From the pinout, pin number 8 could be used to power up electronics. However, no data about electrical characteristics of power supply is given (max current?). \textbf{The cables in use, have a red wire at pin 1.} |
| c1488c | 64 | |
| SP | 65 | |
| 66 | ||
| 67 | \begin{figure}[htb] | |
| 68 | \begin{center} | |
| 69 | \includegraphics[width=0.7\linewidth]{figures/pinout.png} | |
| 70 | \end{center} | |
| 71 | \caption{Connector pinout. Taken from \cite{productFamilyDatasheet}.}\label{fig:pinout} | |
| 72 | \end{figure} | |
| 73 | ||
| 74 | ||
| 75 | For multi-camera case, one camera could be initialized as master and the rest configured as slave. Alternatively an external signal generator can also be used as the master trigger with all cameras set to slave mode. When applying an external sync pulse, the HW SYNC input requires a 100 microsecond positive pulse at the nominal camera frame rate, 33.33 ms for a 30Hz frame rate for example. Inputs are high impedance, 1.8V CMOS voltage levels. | |
| 76 | ||
| 77 | However, it is important to make sure to use a high resolution signal generator. The frequency of the signal generator needs to exactly match the sensor frame rate. For example, if the sensor is set up as 30 fps, the real frame rate may be 30.015 fps. You may need to use an oscilloscope to measure the real frame and configure the signal generator to the same frequency. | |
| 78 | ||
| 79 | For this reason, it may be better to just use one additional camera as master sync signal generator. \cite{whitesheet} | |
| 80 | ||
| 81 | \section{ESP32 controller} | |
| 82 | ||
| 83 | ||
| 84 | \begin{figure}[htb] | |
| 85 | \begin{center} | |
| 86 | \includegraphics[width=\linewidth]{figures/esppinout.png} | |
| 87 | \end{center} | |
| 88 | \caption{ESP32 pinout.}\label{fig:esppinout} | |
| 89 | \end{figure} | |
| 90 | ||
| c0cf4b | 91 | \subsection{Connection input and output trigger signals} |
| SP | 92 | |
| 93 | Input trigger is connected to the GPIO17 pin. The input level of 1.8 V should be enough to trigger the event in the microcontroller, however, there are possibilities to include internal pullups or even add small electronics circuit to raise the voltage lever before it reaches the microcontroller. | |
| 94 | ||
| 95 | Output trigger is connected to the GPIO16 pin on the microcontroller. It has 3.3 V pulses, which are enough to trigger 5 V CMOS logic. | |
| 96 | ||
| 97 | \subsection{Connection of the power for microcontroller} | |
| 98 | ||
| 99 | The whole electronics is powered from the camera. The camera provides 3.3 V power supply at its connector. That power is routed to 3.3 V input on the microcontroller. The ground is shared with the signals and should be routed to one of the GND pins of the microcontroller. | |
| c1488c | 100 | |
| b08ee1 | 101 | \subsection{Connecting nRF24l01+} |
| SP | 102 | |
| 103 | For fast communication Nordic Semiconductor chip nRF24l01+ is used. It operates on 2.4 GHz frequency band, supports multiple channels, raw communication with possibilities of ACK/NoACK. The delay of wirelessly transporting the trigger should be minimal with pair of nRF24 where one is designated as transmitter and one as receiver. There is possibility of full duplex communication. No protocol is defined yet, so we have multiple possibilities on implementation. | |
| 104 | ||
| 105 | As an advenced feature of the receiving end, there is a posibility to measure round trip of data transfer, therefore, the receiver can calculate the delay caused by the transmission and compensate for it, if the receiver does not need to trigger the event. As an example, this could be implemented when accelerometer is read and should be synchronized with camera frame capture. The receiver can measure acceleration beforehand in short intervals and put it into cyclic buffer. When the trigger is received and knowing the trigger transmission delay and accelerometer reading frequency, we can look up the value for acceleration at the exact time the camera published the event at the transmitter side. With this, we have possibility to minimize trigger delay. | |
| 106 | ||
| 107 | nRF24l01 modules are used, where everyting regarding radio is implemented. Module has SPI and power supply pins on its connector. Image of module with its pinout is presented in figure \ref{fig:nrfpinout}. | |
| 108 | ||
| 109 | \begin{figure}[htb] | |
| 110 | \begin{center} | |
| 111 | \begin{minipage}{0.45\linewidth} | |
| 112 | \includegraphics[width=0.9\linewidth]{figures/nrf_pinout.png} | |
| 113 | \end{minipage} | |
| 114 | \begin{minipage}{0.45\linewidth} | |
| 115 | \includegraphics[width=0.9\linewidth]{figures/nrf_electrical.png} | |
| 116 | ||
| 117 | \end{minipage} | |
| 118 | \end{center} | |
| 119 | \caption{nRF24l01 module pinout and its electrical characteristics. \cite{nrfInterfacing}}\label{fig:nrfpinout} | |
| 120 | \end{figure} | |
| 121 | ||
| 122 | ||
| 123 | ||
| 124 | Connection to ESP32 module is as presented in the following table: | |
| 125 | ||
| 126 | \begin{tabular}{l | l | l} | |
| 127 | nRF pin & ESP32 pin & comment \\ \hline | |
| 128 | 1 & GND & Power and signal ground \\ | |
| 129 | 2 & 3.3V & Power supply \\ | |
| 130 | 3 & 22 & CE (Chip enable) \\ | |
| 131 | 4 & 21 & CSN \\ | |
| 132 | 5 & 18 & SCK \\ | |
| 133 | 6 & 23 & MOSI \\ | |
| 134 | 7 & 19 & MISO \\ \hline | |
| 135 | \end{tabular} | |
| 136 | ||
| 137 | ||
| 138 | ||
| c1488c | 139 | \section{Firmware} |
| SP | 140 | |
| c0cf4b | 141 | Firmware is written in Arduino. Arduino IDE has support for ESP32, however it needs to be installed from the inside of the IDE. |
| SP | 142 | |
| 143 | Aditionally RF24 library is needed for communication with nRF24l01. | |
| 144 | ||
| 145 | ||
| c1488c | 146 | \subsection{Energy conservation} |
| SP | 147 | |
| c0cf4b | 148 | Since no WiFi or BTLE communication from ESP32 is planned, it is suggested to turn off radion within Arduino IDE by issuing the following commands: |
| SP | 149 | |
| 150 | \begin{verbatim} | |
| 151 | WiFi.mode(WIFI_OFF); | |
| c1488c | 152 | btStop(); |
| c0cf4b | 153 | \end{verbatim} |
| c1488c | 154 | |
| SP | 155 | \subsection{Input trigger and interrupt} |
| 156 | ||
| c0cf4b | 157 | Input trigger is routed into ESP32 module on pin GPIO17 and can be changed on any free pin. Output trigger is output to pin GPIO16 and can also be changed if necessary. |
| SP | 158 | |
| 159 | The GPIO17 (shorter: pin 17) has an interrupt attached to it, so the processor responds on rising edge of the camera trigger signal as soon as possible. The attachemnt is done in setup part of the code. | |
| 160 | ||
| 161 | \begin{verbatim} | |
| 162 | //pinMode(IN_TRIGGER, INPUT_PULLUP); | |
| 163 | pinMode(OUT_TRIGGER, OUTPUT); | |
| 164 | attachInterrupt(IN_TRIGGER, isr, RISING); | |
| 165 | \end{verbatim} | |
| 166 | ||
| 167 | Input internal pullup resistors are not enabled, but they shoud be if the trigger will behave erratically. | |
| 168 | ||
| 169 | The interrupt service routine is handling all the triggering on the output pin: | |
| 170 | ||
| 171 | \begin{verbatim} | |
| 172 | ||
| 173 | void IRAM_ATTR isr() { | |
| 174 | digitalWrite(OUT_TRIGGER, HIGH); | |
| 175 | delay(1); | |
| 176 | digitalWrite(OUT_TRIGGER,LOW); | |
| 177 | } | |
| 178 | \end{verbatim} | |
| 179 | ||
| 180 | ||
| 181 | There is 1 ms pulse length on the output. It can be prolonged if necessary. Input trigger will possibly work on 1.8 V input voltage level. The output trigger has voltage level of 3.3 V and is well above the minimal 5 V CMOS voltage level specifications. | |
| 182 | ||
| 183 | %https://lastminuteengineers.com/handling-esp32-gpio-interrupts-tutorial/ | |
| 184 | ||
| 185 | \subsection{nRF24l01 transmission code} | |
| 186 | ||
| 187 | The library RF24 that needs to be installed supports nRF21l01 communication. The nRF21l01 chip needs to be initialized by defining radio in the root of the code: | |
| 188 | ||
| 189 | \begin{verbatim} | |
| 190 | RF24 radio(22, 21); // CE, CSN | |
| 191 | ||
| 192 | const byte address[5] = {'R','E','C','V', '1'}; | |
| 193 | char dataToSend[4] = "TRG"; | |
| 194 | \end{verbatim} | |
| 195 | ||
| 196 | ||
| 197 | The radio will send a short, yet meaningful message (namely ``TRG'')to the receiver that will listen at the 5 byte address corresponding to the ascii letters of RECV1. The radio has pins 22 and 21 as CE and CSN connections. | |
| 198 | ||
| 199 | The second part of initalizations happens within the setup | |
| 200 | ||
| 201 | \begin{verbatim} | |
| 202 | radio.begin(); | |
| 203 | radio.openWritingPipe(address); | |
| 204 | radio.setPALevel(RF24_PA_MIN); | |
| 205 | \end{verbatim} | |
| 206 | ||
| 207 | It presets radio to low power transmission and defines the address of the receiver. | |
| 208 | ||
| 209 | In isr routine presented above, we just need to send a message by issuing: | |
| 210 | ||
| 211 | \begin{verbatim} | |
| 212 | radio.write( &dataToSend, sizeof(dataToSend) ); | |
| 213 | \end{verbatim} | |
| 214 | ||
| d6fc89 | 215 | \subsection{Debug receiver} |
| c1488c | 216 | |
| d6fc89 | 217 | To measure trigger delay and for other debugging purposes, A simple debug receiver can be programmed. On this receiver, the nRF21l01 must be connected the same way as on trasmitter. The following code works as debug interface. The code can also be a reference for receiver design. |
| SP | 218 | |
| 219 | \begin{verbatim} | |
| 220 | #include <RF24_config.h> | |
| 221 | #include <nRF24L01.h> | |
| 222 | #include <RF24.h> | |
| 223 | #include <printf.h> | |
| 224 | ||
| 225 | RF24 radio(22, 21); // CE, CSN | |
| 226 | const byte address[6] = {'R','E','C','V', '1'}; | |
| 227 | char val[4]; | |
| 228 | ||
| 229 | void setup() { | |
| 230 | Serial.begin(115200); | |
| 231 | radio.begin(); | |
| 232 | radio.openReadingPipe(0, address); | |
| 233 | radio.setPALevel(RF24_PA_MIN); | |
| 234 | radio.startListening(); | |
| 235 | ||
| 236 | } | |
| 237 | ||
| 238 | void loop() { | |
| 239 | delay(5); | |
| 240 | radio.startListening(); | |
| 241 | if ( radio.available()) { | |
| 242 | while (radio.available()) { | |
| 243 | ||
| 244 | radio.read(&val, sizeof(val)); | |
| 245 | Serial.print("Received = "); | |
| 246 | Serial.println(val); | |
| 247 | } | |
| 248 | } | |
| 249 | ||
| 250 | } | |
| 251 | \end{verbatim} | |
| c1488c | 252 | \end{document} |
| SP | 253 | |
