Event-Based Imaging of Levitated Microparticles

Event-based imaging is a neurmorphic detection technique whereby an array of pixels detects a positive or negative change in light intensity at each pixel, and is hence particularly well suited to detecting motion. As compared to standard camera technology, an event-based camera reduces redundancy by not detecting regions of the image where there is no motion, allowing increased frame-rates without compromising on field-of-view. Here, we apply event-based imaging to detect the motion of a microparticle levitated under vacuum conditions, which greatly facilitates the study of nanothermodynamics and enables the independent detection and control of arrays of many particles.

When unravelling the underlying physics of particles interacting with external forces, or of interacting multi-particle systems, object tracking is key. One must consider a range of detection metrics such as field-of-view, resolution, latency, sensitivity, bandwidth, signal-to-noise ratio (SNR) and the ability to detect multiple objects.
In this work, we consider tracking the motion of microparticles levitated under vacuum conditions by optical, electrical or magnetic fields 1,2 . Such systems are of interest for studies of fundamental quantum science 3 , nano-thermodynamics 4 and advanced sensing 5,6 .
When working with particles optically trapped in liquid, it is sufficient to use standard CMOS or CCD cameras to track their motion, since the viscous damping provided by the liquid reduces dynamical timescales to a level suitable for camera frame-rates. However, once objects are levitated in vacuum, their motion is underdamped and faster tracking is required 7 . Although this is possible at the 100 kHz level with high-speed CMOS cameras 8 , this requires significant reduction in the sensor resolution. Hence, particle tracking is usually performed using photodiodes, balanced photodetectors or quadrant photodetectors. Whilst these devices are fast, even tracking at GHz rates 9,10 , they have limited field-of-view, restricting tracking to scales not much larger than the optical wavelength. This can limit studies of nanothermodynamics or nonlinear motion where particles may explore large regions of space 3 .
Additionally, a key technique when working with levitated objects is the application of real-time feedback onto their motion. This has enabled cooling to the quantum ground state of a harmonic potential 11 , but more generally is essential for stabilization under vacuum conditions. Balanced photodetectors are the standard tool to realise real-time feedback 12 , although recent work employing powerful graphics cards with a limited number of pixels 13 or on-board microprocessors 14 has enabled feedback control of sub-500 Hz oscillators via CMOS camera detection. We also note that cameras have excellent SNR and allow super-resolution detection 13 , allowing one to minimize the amount of light required to detect levitated particles avoiding absorption 15 and photon recoil heating 16 .
Finally, the prospect of levitating systems of multiple in- teracting particles has emerged 2 , for distributed sensing 17 or generation of entanglement 18 . Single photodetectors can only track single particles, whereas cameras are well suited to multi-particle detection.
Conventional, CMOS/CCD based cameras work using a specified region of interest (ROI) or the whole pixel array to capture light from a scene. An alternative approach is that of event-based imaging (EBI), where pixels work independently of each other, triggering only when the change in light intensity is above a preset threshold 19 . This enables a dynamic ROI, thus enabling decreased informational load compared to conventional cameras.
In this paper, we apply EBI for detecting the motion of microparticles levitated in vacuum. This imaging technique offers the potential for tracking single and multiple objects with high bandwidth, whilst an integrated tracking algorithm provides the real-time position of each object for use in feedback for state control.
Conventional cameras, such as CMOS/CCD, capture continuous movement as a sequence of still images (frames) formed from every pixel of the sensor. As a result, stationary elements are unnecessarily replicated, while moving elements are under-sampled 20 .
EBI provides an approach to image acquisition by only capturing changes in images through the detection of modifications in light intensity on each pixel 19 . Pixels in these neuromorphic sensors (sensors that try to mimic the neural structure of the brain) are completely independent. Each one of them contains a contrast detector (CD) that continuously tracks photocurrents. When the variation of a photocurrent crosses a threshold, the CD triggers a contrast detection event, which represents a relative increase (positive) or decrease (negative) in light intensity. It then initiates the measurement of a new value, as outlined in fig. 1(b). Pixels which do not observe changes in light intensities that exceed the threshold do not produce output.
This can be seen in fig. 1(a) where two images obtained with different cameras are shown. The event-based image consists of only three colours: white pixels indicate no change in light intensity across the threshold; blue and black pixels represent positive and negative changes in light intensity across the threshold, respectively. Only the blue and black pixels are transmitted as data. By comparing both sets of images in fig. 1(a), it is clear that the amount of data transferred is suppressed with EBI.
In conventional cameras the bandwidth of the communication link is usually a constraint whenever higher acquisition rates are needed to track rapid movement. This leads users to reduce the ROI of the sensor to decrease the amount of data per frame 13 . Due to the suppression of data redundancy, and the fact that EBI sensors have an effective pixel depth of 1-bit (an event is either detected or not), the data volume transmitted is considerably reduced, allowing acquisition rates over 1 GHz 19 . Figure 1(b) shows the detailed process of object tracking based on EBI. In an event-based camera (EBC), the EBI sensor is packaged with hardware which performs object tracking, such that the output of the EBC lists the position of each detected object as a function of time. All of the input information related to the camera sensor is read, and in the setup period camera sensor parameters are tuned. An event producer, which is contained in the event buffer, is used to generate a stream of events. For each stream of events, a noise filter is applied to pick up events in the neighboring 8 pixels during a certain time. A trail filter then accepts an event if the last event is detected at the same position within an accumulation time. All of the data collected by the sensor pixels generates a frame and a proprietary generic tracking algorithm (GTA) analyzes these frames to extract detected objects and associate data to previous frames. The deployed tracking algorithm limits the effective frame-rate of the EBC as compared to the read-out rate of the EBI sensor. When the detected object is recognized, there is a trigger in callback and the motional information of detected particle is obtained from output. Detection based on object tracking (rather than, for example, measuring the intensity of light) has been shown to allow sub-pixel resolution and low noise 13 . The dynamic range of the EBI sensor used in this study 21 is 120 dB, which is high compared to standard CMOS / CCD (∼ 70 dB) or EMCCD (∼ 100 dB) sensors. It is hard to quantify the effect of shot noise or dark counts on the GTA, and hence make a direct comparison to photodiode-based tracking.
We use an EBC 22 to track the motion of levitated microspheres. Charged silica microspheres of diameter 5 µm and charge Q are levitated in partial vacuum using a Paul trap, made with four 3mm-diameter rods and two 1mm-diameter endcap rods (not shown, aligned axially along the centre of the structure), as illustrated in fig. 3(a). The microsphere is trapped using an oscillating electric field with frequency Ω RF = 2π × 800 rad s −1 and amplitude V RF = 750 V. The two endcap electrodes are held at -4V DC. The particles are loaded into the Paul trap using laser-induced acoustic desorption (LIAD) 23,24 at a pressure of 2 × 10 −2 mbar.
An illumination laser (532 nm) is focused onto the trapped particle, with a beam waist of ∼ 80 µm. The scattered light is collected by a microscope (16× magnification) and directed onto the EBC for motion detection. In fig. 3(b) we present the SNR at the motional frequency of the particle as recorded by the EBC, as we vary the power of the illumination laser (see Supplementary Information IV). We estimate the maximal scattered power reaching the EBC sensor, illustrating the excellent sensitivity of the detector. Figure 2(a) shows the output from the EBC compared to the output of a quadrant photodiode (QPD) 25 , when tracking the oscillatory motion of a microparticle using identical imaging systems. The GTA acts like an effective filter, removing highfrequency noise. Figure 2(b) shows the corresponding derived power spectral densities (PSDs) of the particle motion. Like all balanced detection methods, the QPD minimizes 1/ f noise at low frequencies, whereas the EBC has lower-noise performance at higher frequencies.
Regardless of the detection method used, the detector has to be calibrated. The motion of the particle is calibrated by applying a known potential difference ∆U across the endcap electrodes, causing the particle to move in the z-direction. The particle oscillates at the centre of the trap when ∆U = 0 V. If the voltage of one of the endcaps is modified from U 0 → U 1 , the resulting voltage difference ∆U = U 1 − U 0 exerts a force on the particle F = F zẑ , whereẑ denotes a unit vector along the z-axis, determined by: where d is the distance between the endcap electrodes. We confirm this analytic model for our specific trap geometry in Supplementary Information II. Since the Paul trap provides a harmonic pseudo-potential, the particle also experiences a linear restoring force: where k is the trap stiffness and z is the average position of the particle (assuming z = 0 when ∆U = 0). Noting that the measured z m is in volts for the QPD, and pixels for the EBC, and related to the true value of z through a conversion factor γ, with units V/m and pixels/m, respectively, we can equate these two equations. Considering further that k = mω 2 z for a harmonic oscillator of mass m and oscillation frequency ω z , then: By measuring the mean displacement of the particle as ∆U is varied, we can determine the conversion factor γ for each detector. The oscillation frequency ω z can be obtained from the power spectral density ( fig. 2(b)) and Q/m obtained by solving the Mathieu equations (see Supplementary Information I). In our system we typically trap particles of charge Q = 2 × 10 4 e.
The calibrated particle position as a response to a potential difference can be seen in fig. 3(c). It is apparent that as |∆U| becomes large, the response of the QPD becomes nonlinear, whereas eqn. (3) predicts a linear response. This can be somewhat mitigated through a nonlinear calibration (see Supplementary Information II), but limits the field-of-view of the QPD. On the other hand, the EBC has a linear response to the particle displacement across the full range. Figure 3(d) shows a histogram of the equilibrium motion of the trapped particle, again showing that large displacements are missing when using the QPD. Hence, we can track both small (i.e. the oscillatory motion) and large displacements using EBI.
Levitated particles provide an ideal system for probing stochastic thermodynamics 4 . This is in part due to the characteristic energy of the system being comparable to that of thermal fluctuations of the bath, which enable levitated systems to be highly sensitive to surrounding fluctuations. The coupling to the bath is characterized by the ratio of its oscillation frequency ω z to its momentum damping rate Γ, yielding overdamped (Γ ω z ) and underdamped (Γ ω z ) regimes. In the overdamped regime, trapped particles have been used for studying heat engines 26 , non-thermal baths 27 , and for testing Landauer's principle 28 . In the underdamped regime, the studies in stochastic thermodynamics have been extended to observing Kramer's turnover 29 , studying nonequilibrium Landauer's principle 30 and testing fluctuationdissipation theorems 31 .
Often when such systems exhibit non-equilibrium dynamics, they go beyond the linear detection regime of photodiodebased detection systems, and CMOS cameras would need a large ROI to capture the dynamics, at the expense of bandwidth.
EBI employs a dynamic ROI, based on only triggered pixels. To characterise the capability of EBI, and the GTA of the EBC, we cause random jumps in the particles' position with varying time intervals, τ (see Supplementary Information III). These random jumps, as seen in fig. 4(a), are driven by voltage changes applied to the endcap electrodes which follow telegraph noise statistics (see Supplementary Information III) distributed about a mean waiting timeτ. For fast switching times relative to the gas damping rate Γ 0 , the position probability distribution of the particle is Gaussian, and for slow switching times it is bimodal, as shown in fig. 4(b). Comparing to fig. 3(c), the particle's motion would go deep into the nonlinear range of the QPD. However, the EBC has no such limitation, and position shifts greater than 100 µm are tracked, without compromising on position sensitivity, which is approximately 30 nm Hz −1/2 in this work for both devices, as can be extracted from fig. 2(b). For the QPD to track such a range of motion, one would have to use a lower magnification imaging system, with a corresponding reduction in position sensitivity.
The factor limiting the bandwidth of our EBC is the GTA. For shifts in position that are in quick succession and large in displacement, the GTA lags behind or misses consecutive shifts. This is illustrated in fig. 4(a), where the GTA of the EBC faithfully tracks the particle position whenτ > 1 s. For shorter mean waiting times,τ < 500 ms the GTA struggles to track the jumps, as evidenced by apparent spikes in the time domain signal. The GTA doesn't represent a true bandwidth limit to EBI and advantageously, as shown in fig. 4(b), our EBC can simultaneously track large displacements and the smaller oscillations of the particle about its equilibrium position.
We extend our study to multi-particle tracking using EBI. The ability to track arrays of particles would enable the study of quantum correlations [32][33][34][35] , non-hermitian systems 36 , and the detection of dark matter 6,37 , vacuum friction 38 and differential force-sensing 18 . To date, few have experimentally explored multi-particle physics, with only two particles trapped to demonstrate state-swapping and sympathetic cooling 39 , dipole-dipole 40,41 and coulomb-coulomb 42 interactions, and cold damping and state control 43 . Motional detection of individual particles in a multiple particle trap is a non-trivial problem. A single focused light beam can carry motional information of two particles 44 , or an additional beam can be added for detection of the second particle 40 . Going beyond two particles in this way would require a complex and non-scalable optical setup. Within the context of optical tweezers, there are numerous approaches most suited to tracking multiple particles, using high-speed cameras 45 and multiple-beams in conjunction with a QPD 46 , but these detection methods face the same limitations outlined earlier in this paper. Figure 5(a) shows an image of two microspheres of 5 µm diameter trapped in a Paul trap, captured on our EBC. The image shows the bounding boxes which the GTA uses to track the two-dimensional position of the particles, labelled "986" and "987".
The levitated microspheres are tracked and their motional information separately reconstructed in fig. 5(c). We observe oscillation frequencies of both particles independently. For particle ID 986, the motional frequencies are {ω 987 x = 2π × 20 rad s −1 , ω 987 y = 2π × 122 rad s −1 , ω 987 z = 2π × 184 rad s −1 }. We observe all three centre-of-mass degrees-of-freedom due to imperfect alignment between the coordinate axis of the Paul trap and our imaging system. The additional frequency components in the spectrum for ID 987 could be other degrees-of-freedom (e.g. librational) or evidence of multi-particle collective modes, but that is beyond the scope of this study.
The EBC is not limited to tracking two particles, and in fig. 5(b) we show that the GTA has identified 5 separate particles. Due to varying Q/m, each particle has different resonant motional frequencies, and hence it is possible to individually excite them. We believe this opens the door to the study of a wide range of non-equilibrium phenomena.
In conclusion, we have shown that EBI is an interesting alternative to conventional detection schemes used for tracking levitated particles. The key enabling feature of EBI is the low data transfer, which enables EBC to track multiple objects at higher speeds than conventional cameras, and when combined with natural low pixel latencies 19 will allow the experimenter to implement real-time feedback for state control. In this study, the bandwidth of tracking is limited to 1 kHz by the very general and proprietary GTA employed. The underlying dynamics of our system are well known, therefore a more precise filter, like an asynchronous Kalman filter 47,48 in which we can input the expected equation of motion, will enable faster and more accurate tracking.
As compared to photodiode-based detection schemes, EBCs feature a dynamic ROI, enabling tracking over a wide field-of-view, with particular application in the study of nonequilibrium physics. We have demonstrated tracking over 100 micrometres whist retaining 30 nm Hz −1/2 resolution. Finally, we have introduced an imaging technique suitable for fast tracking of a large number of particles, reaching MHz rates with the application of tailored particle tracking algorithms.

SUPPLEMENTARY INFORMATION
Supplementary Material is provided for detailed information about how to solve charge-mass ratio, conduct nonlinear calibration of QPD, generate random jumps and obtain signal-to-noise ratio.
where ω i the harmonic oscillation frequency, Ω RF is the driving RF frequency, and a i , q i with i = {x, y, z} are known as the stability parameters 49 .
Using the stability parameters we can write a general statement for all three axis: where we q m = Q/M is the charge-mass ratio, V RF is the RF voltage amplitude, U is the DC voltage at the endcaps, r is the distance between RF electrodes from the centre of the trap and d is the distance between the endcap electrodes. The variables α i and β i are geometric factors that are obtained through SIMION numerical simulation of the Paul trap used in the experiments (see Table S1) for each axis. The Q/m ratio can then be obtained by solving for q m .

II. CALIBRATION: NONLINEAR CALIBRATION WITH THE QPD
To calibrate the system and obtain a conversion function we apply an electric force, F z to the particle via a potential  difference ∆U across the endcap electrodes: where d is the spacing between the endcap electrodes, which is equal to spring force experienced in the trap: mω 2 z z . When equated with the above equation we get a theoretical value for positional shift in metres for an applied voltage difference ∆U: To further verify eqn. S7, simulations based on SIMION are conducted. Figure S6(a) shows a trap geometry in the simulation environment which is with the same size as our experimental system. Figure S6(b) shows that when the voltage of one endcap is tuned by -0.5V to 0.5V, related mean positions of the trapped particle linearly changes, verifiying the analytical expression used in the manuscript The quadrant photodetector (QPD) has a nonlinear response to the movement of an image if the image deviates volts, x V of the particle when a potential difference ∆U is applied across the endcaps. The measured position x V is converted to the actual position x m = qV mω 2 d . The inverse of the error function then converts Volts into metres too far from the centre, i.e. if the particle moves too far from the centre of the trap. We can fit an error function to the mean measured position of the particle (recorded in Volts, x V ), with the expected actual position x m based on the amount of applied potential difference, eqn. S7.
where (a, b, c, g) are fitting constants obtained by fitting the above equation to the data as shown in fig. S7. The inverse of this error function then enables us to convert the positional information in volts x V to metres x m .
This enables us to extend the detection range of the QPD by a few microns, but not by the tens of microns offered by the event-based camera.

III. GENERATING RANDOM JUMPS
Random jumps in position of the particle are implemented by applying telegraph noise statistics to the particle. To achieve this, we generate random numbers normally distributed, which then are multiplied by a mean waiting timē τ. The waiting time is used to delay a voltage applied to the particle ±V tel via the endcap electrodes. This applied voltage is experienced by the trapped particle as an electric force, ±η tel = Q m V tel /(d). In this work, V tel = ±1 V. This applied force has noise statistics governed by: η t η t = η 2 tel e (−2|t−t |/τ) . (S10)