2962 IEEE JOURNAL OF SOLID-STATE CIRCUITS, VOL. 58, NO. 11, NOVEMBER 2023
V. CONCLUSION
We have introduced a state-of-the-art hybrid EVS/CIS
image sensor. This was achieved using advanced three-
wafer-stacking technology. The hybrid sensor simplifies
synchronization between EVS and CIS compared with a
two-sensor solution. Furthermore, a hybrid sensor avoids
parallax/occlusion errors and requires only one package and
lens. This enables new use-cases such as rolling-shutter
correction or deblur image enhancement, video frame inter-
polation for slow motion, and simultaneous localization and
mapping.
The CIS pixel array sacrifices only one out of 16 color
channels for EVS functionality which achieves comparable
image quality to the state-of-the-art sparse phase-detection
autofocus systems. The EVS pixel was evaluated at a linear
NCT setting of 15% achieving a CTNU of ∼3% being
comparable to stand-alone EVS sensors. Very low noise of
<1 Hz is reported.
Conversely to arbiter-based readout, the content-aware scan
operation achieves fair, low-latency readout through the bypass
of rows and columns without events and a skip function that
only reads out the first and last events of a connected group of
events having the same polarity. Using these methodologies,
we report a maximum event rate of up to 4.6 GEps. The power
consumption at 55 pJ/Event outperforms recently published
stand-alone EVS sensors with at least VGA resolution by
up to 200% depending on the actual event distribution in
the array. We thus report three leading figures of merits.
The first describes the multiplication of event pixel resolu-
tion times event rate yielding 4.4 MP × GEps. The second
normalizes the first FOM by the required power achieving
0.08 MP × GEps/pJ. And the last FOM multiplies CIS and
EVS resolution as well as the CIS frame rate and EVS event
rate to form a hybrid measure of 1200 MP
2
×frames/s × GEps.
REFERENCES
[1] P. Lichtensteiner, “An AER temporal contrast vision sensor,”
Ph.D. thesis, ETH, Zürich, Switzerland, 2006.
[2] G. Gallego et al., “Event-based vision: A survey,” IEEE Trans. Pat-
tern Anal. Mach. Intell., vol. 44, no. 1, pp. 154–180, Jan. 2022, doi:
10.1109/TPAMI.2020.3008413.
[3] T. Haruta et al., “A 1/2.3inch 20Mpixel 3-layer stacked CMOS image
sensor with DRAM,” in IEEE Int. Solid-State Circuits Conf. (ISSCC)
Dig. Tech. Papers, San Francisco, CA, USA, Feb. 2017, pp. 76–77, doi:
10.1109/ISSCC.2017.7870268.
[4] C. Posch, D. Matolin, and R. Wohlgenannt, “An asynchronous
time-based image sensor,” in Proc. IEEE Int. Symp. Circuits
Syst., Seattle, WA, USA, May 2008, pp. 2130–2133, doi:
10.1109/ISCAS.2008.4541871.
[5] C. Brandli, R. Berner, M. Yang, S.-C. Liu, and T. Delbruck, “A 240×180
130 dB 3 µs latency global shutter spatiotemporal vision sensor,” IEEE
J. Solid-State Circuits, vol. 49, no. 10, pp. 2333–2341, Oct. 2014, doi:
10.1109/JSSC.2014.2342715.
[6] C. Li et al., “Design of an RGBW color VGA rolling and global
shutter dynamic and active-pixel vision sensor,” in Proc. IEEE Int. Symp.
Circuits Syst. (ISCAS), Lisbon, Portugal, May 2015, pp. 718–721, doi:
10.1109/ISCAS.2015.7168734.
[7] T. Finateu et al., “A 1280 × 720 back-illuminated stacked temporal
contrast event-based vision sensor with 4.86 µm pixels, 1.066GEPS
readout, programmable event-rate controller and compressive data-
formatting pipeline,” in IEEE Int. Solid-State Circuits Conf. (ISSCC)
Dig. Tech. Papers, San Francisco, CA, USA, Feb. 2020, pp. 112–114,
doi: 10.1109/ISSCC19947.2020.9063149.
[8] B. Son et al., “A 640 × 480 dynamic vision sensor with a 9 µm pixel
and 300Meps address-event representation,” in IEEE Int. Solid-State
Circuits Conf. (ISSCC) Dig. Tech. Papers, Feb. 2017, pp. 66–67, doi:
10.1109/ISSCC.2017.7870263.
[9] Y. Suh et al., “A 1280 × 960 dynamic vision sensor with a 4.95-µm
pixel pitch and motion artifact minimization,” in Proc. IEEE Int.
Symp. Circuits Syst. (ISCAS), Seville, Spain, Oct. 2020, pp. 1–5, doi:
10.1109/ISCAS45731.2020.9180436.
[10] S. Chen, “Development of event-based sensor and applications,” in
Proc. Conf. Comput. Vis. Pattern Recognit. (CVPR), 2021, pp. 1–17.
Accessed: Aug. 22, 2022. [Online]. Available: https://tub-rip.github.io/
eventvision2021/slides/CVPRW21_Shoushun_Chen.pdf
[11] iniVation. Specifications—Current Models. Accessed: Aug. 22, 2022.
[Online]. Available: https://inivation.com/wp-content/uploads/2021/
08/2021-08-iniVation-devices-Specifications.pdf
[12] C. Li, L. Longinotti, F. Corradi, and T. Delbruck, “A 132 by 104 10 µm-
pixel 250 µW 1kefps dynamic vision sensor with pixel-parallel noise
and 107 spatial redundancy suppression,” in Proc. Int. Image Sensor
Workshop (IISW), Snowbird, UT, USA, 2019, p. 16.
[13] M. Guo et al., “A 3-wafer-stacked hybrid 15MPixel CIS +
1 MPixel EVS with 4.6G event/s readout, in-pixel TDC and on-
chip ISP and ESP function,” in IEEE Int. Solid-State Circuits
Conf. (ISSCC) Dig. Tech. Papers, Feb. 2023, pp. 90–92, doi:
10.1109/ISSCC42615.2023.10067476.
[14] Y. Yaffe et al., “Dynamic vision sensor—The road to market,” in Proc.
Int. Conf. Robot. Automat. (ICRA), 2017. Accessed: May 3, 2023.
[Online]. Available: https://rpg.ifi.uzh.ch/docs/ICRA17workshop/
Samsung.pdf
[15] K. Kodama et al., “1.22 µm 35.6Mpixel RGB hybrid event-based vision
sensor with 4.88 µm-pitch event pixels and up to 10K event frame
rate by adaptive control on event sparsity,” in IEEE Int. Solid-State
Circuits Conf. (ISSCC) Dig. Tech. Papers, Feb. 2023, pp. 92–93, doi:
10.1109/ISSCC42615.2023.10067520.
[16] A. Niwa et al., “A 2.97 µm-pitch event-based vision sensor with shared
pixel front-end circuitry and low-noise intensity readout mode,” in IEEE
Int. Solid-State Circuits Conf. (ISSCC) Dig. Tech. Papers, Feb. 2023,
pp. 94–95, doi: 10.1109/ISSCC42615.2023.10067566.
[17] A. Suess et al., “Physical modeling and parameter extraction for event-
based vision sensors,” in Proc. IISS Int. Image Sensor Workshop (IISW),
Edinburgh, U.K., 2023, p. R5.5.
Menghan Guo received the M.E. degree in circuits
and systems from Southeast University, Nanjing,
China, in 2013.
From 2013 to 2017, he was a Research
Associate with Nanyang Technological University,
Singapore. In 2017, he joined CelePixel Technolo-
gies, Shanghai, China, as the IC Design Manager
for smart vision sensors in 2017, and he is currently
with OMNIVISION, Shanghai. His research inter-
ests mainly focus on the design of event-based vision
sensor.
Shoushun Chen (Senior Member, IEEE) received
the B.S. degree from Peking University, Beijing,
China, in 2000, the M.E. degree from the Chinese
Academy of Sciences, Beijing, in 2003, and the
Ph.D. degree from the Hong Kong University of
Science and Technology, Hong Kong, in 2007.
He held postdoctoral positions with the Hong
Kong University of Science and Technology and
Yale University, New Haven, CT, USA. From July
2009 to April 2023, he was an Assistant Professor
and later promoted to tenured Associate Professor
with Nanyang Technological University, Singapore. He is the Founder of
CelePixel Technology, Shanghai, China, which was acquired by OMNIVI-
SION, Shanghai, in 2020. His research focuses on various areas, including
smart vision sensors, motion detection sensors, energy-efficient algorithms for
bioinspired vision and analog/mixed-signal VLSI circuits and systems.
Dr. Chen serves as the Chair of the Technical Committee of Sensory Systems
for the IEEE Circuits and Systems Society (CASS).
剩余343页未读，继续阅读
