обработка статического изображения и видео (dsp) с cmos-матрицы (csi-2)

raspberry pi csi-2 cam pinout:

1) (!)https://www.arducam.com/raspberry-pi-camera-pinout/

csi2, dsi standarts:
1) https://en.wikipedia.org/wiki/Camera_Serial_Interface
2) https://www.mipi.org/specifications/csi-2

3) https://en.wikipedia.org/wiki/Display_Serial_Interface

4) https://en.wikipedia.org/wiki/UniPro_protocol_stack#D-PHY

5) (!) http://www.jmrcubed.com/vr/ref_tech/mipi_d_phy_specification_v01-00-00.pdf

6) (!) http://caxapa.ru/thumbs/799244/MIPI_Alliance_Specification_for_Camera_S.pdf

1.3. raspberry pi 3b camera v2 and camera v2 NoIR (csi-2):

1) Image Sensor: Sony IMX 219 PQ CMOS http://wiki.amperka.ru/rpi:camera

2) Image Sensor: Sony IMX 219 PQ CMOS http://wiki.amperka.ru/rpi:noir-camera

3) noir camera https://arduinoplus.ru/raspberry-pi-kamera/

4) https://raspberrypi.ru/400-podklyuchaem-kameru-k-raspberry-pi

5) (!) Raspberry Pi 3: Камера (#1) — первое включение, видео, фото, timelapse, отправка фото по e-mail

6) (!) https://www.raspberrypi.org/documentation/hardware/camera/ 

1.4. raspberry pi 3b cam (aliexpress) (csi-2):

1) Mini Vision OV5647, цветная CMOS QSXGA, 5mp cam (200р. + 80р. доставка) https://aliexpress.ru/item/4000288007298.html

2) 130 degree, 5mp cam (672р.) https://aliexpress.ru/item/32668508991.html

3) raspberry pi cams in aliexpress http://codius.ru/products/319

4) https://www.vision-components.com/en/products/oem-embedded-vision-systems/mipi-camera-modules/

1.5. Media subsystem media internal API:

1.5.1 Video4Linux devices:

1) https://01.org/linuxgraphics/gfx-docs/drm/media/kapi/v4l2-intro.html

2) Official V4L2 driver https://www.raspberrypi.org/forums/viewtopic.php?t=62364

3) http://www.thedirks.org/v4l2/v4l2dwg.htm

4) https://en.wikipedia.org/wiki/Video4Linux

5) Sensor Software Driver Programming Guide

1.5.2.1  MIPI CSI-2:

1) https://www.kernel.org/doc/html/v4.10/media/kapi/csi2.html

3) https://xilinx-wiki.atlassian.net/wiki/spaces/A/pages/18842522/Xilinx+V4L2+MIPI+CSI+driver

1.5.2.2. MIPI CSI-2 in rspberrypi:

1) https://www.raspberrypi.org/documentation/linux/software/libcamera/csi-2-usage.md

2) Raw sensor access / CSI-2 receiver peripheral https://www.raspberrypi.org/forums/viewtopic.php?f=43&t=109137

3) https://github.com/6by9/raspiraw

omnivision ov5647:

1) datasheet

2) https://github.com/techyian/MMALSharp/wiki/OmniVision-OV5647-Camera-Module

3) https://www.raspberrypi.org/documentation/hardware/camera/

4) Камера для Raspberry Pi «Модель D» 

http://wiki.amperka.ru/products:camera-raspberry-pi-model-d

5) Камера для Raspberry Pi «Модель D» (omnivision ov5647) https://amperka.ru/product/camera-raspberry-pi-model-d?utm_source=man&utm_campaign=camera-raspberry-pi&utm_medium=wiki

6) (!)Настройка и примеры работы MIPI CSI-камер для Raspberry Pi 

http://wiki.amperka.ru/articles:raspberry-pi-camera-guide

7) https://github.com/torvalds/linux/blob/master/drivers/media/i2c/ov5647.c

8) https://developer.ridgerun.com/wiki/index.php?title=OmniVision_OV5647_Linux_driver

arducam with ov5647 (external trigger):

1) https://www.arducam.com/docs/camera-for-jetson-nano/mipi-camera-modules-for-jetson-nano/external-trigger-camera-application-note/

2) Capturing with Arducam MIPI monochrome global shutter cameras 

https://www.raspberrypi.org/forums/viewtopic.php?t=267563

3) ArduCam OV5647 Camera Board w/ M12 Mount 

https://www.raspberrypi.org/forums/viewtopic.php?f=43&t=235523

4) ArduCam OV5647 changing FREX exposure time in global reset mode https://www.raspberrypi.org/forums/viewtopic.php?t=299773

Работа с камерой ov5647 (самый дешёвый вариант модуля камеры из ali) из raspberry pi 3b, активация камеры через config.txt:

1) (!)Raspberry Pi 3: Камера (#1) — первое включение, видео, фото, timelapse, отправка фото по e-mail

2) (!) Enable camera without Raspi-config 

https://raspberrypi.stackexchange.com/questions/10357/enable-camera-without-raspi-config

3) Подключение камеры к Raspberry

4) Работа с камерой Raspberry Pi (часть 1) - Time-lapse https://www.newkamikaze.com/articles/73

5) Подключение камеры к Raspberry Pi Zero W https://4te.me/post/camera-raspberry-pi/

6) Описание утилит для работы с камерой 

https://www.raspberrypi.org/documentation/raspbian/applications/camera.md

Читать полностью: https://www.newkamikaze.com/articles/73

Включение камеры через консоль:

sudo raspi-config nonint do_camera 0

Убедиться, что всё корректно прошло, можно посмотрев файл /boot/config.txt после перезагрузки:

cat /boot/config.txt

Вывод (start_x=1 и gpu_mem=128 говорят о корректной установке):

[all]

dtoverlay=disable-bt

enable_uart=1

start_x=1

gpu_mem=128

По умолчанию, картинки хранятся в домашней папке (/home/pi), сохраняем изображение с подключенной камеры при помощи уже установленной утилиты raspistill:

raspistill -o image.jpg -t 10000

vision components, arducam mipi cmos camera modules:

1) https://www.vision-components.com/fileadmin/external/documentation/hardware/VC_MIPI_Camera_Module/index.html

2) https://www.arducam.com/product-category/cameras-for-raspberrypi/raspberry-pi-camera-mipi-camera-module/

sony imx 219:

1) IMX219 datasheet

2) Reverse Engineering the Raspberry Pi Camera V2: A study of Pixel Non-Uniformity using a Scanning Electron Microscope https://arxiv.org/pdf/1901.03807.pdf

3) Sony IMX Core https://www.xilinx.com/products/intellectual-property/1-neh6eh.html#overview

4) linux driver https://android.googlesource.com/kernel/bcm/+/android-bcm-tetra-3.10-lollipop-wear-release/drivers/media/video/imx219.c

sony imx 274:

1) linux driver https://github.com/torvalds/linux/blob/master/drivers/media/i2c/imx274.c

sony imx 296:

1) linux driver https://www.spinics.net/lists/linux-media/msg159983.html

omnivision ov5640:

1) https://cdn.sparkfun.com/datasheets/Sensors/LightImaging/OV5640_datasheet.pdf

omnivision ov7251:

1) (!)https://datasheetspdf.com/pdf-file/1252892/OmniVision/OV7251/1

2) OV SCCB Specification 

https://www.waveshare.com/w/upload/1/14/OmniVision_Technologies_Seril_Camera_Control_Bus%28SCCB%29_Specification.pdf

3) (!)https://github.com/torvalds/linux/blob/master/drivers/media/i2c/ov7251.c

4) https://www.arducam.com/product/arducam-ov7251-mipi-0-31mp-monochrome-global-shutter-camera-module-for-raspberry-pi-noir/

ov9281 forums, manuals:

1) https://www.ovt.com/sensors/OV9281

2) (!)http://wiki.inno-maker.com/display/HOMEPAGE/cam-mipi9281-trigger

3) CYUSB3064 add OV9281 no receive data https://community.cypress.com/t5/USB-Superspeed-Peripherals/CYUSB3064-add-OV9281-no-receive-data/td-p/69933?start=0&tstart=0

4) RAW8/GREY image from MIPI CSI sensor has displacements to right 

https://community.nxp.com/t5/i-MX-Processors/RAW8-GREY-image-from-MIPI-CSI-sensor-has-displacements-to-right/m-p/925851/highlight/true

5) fpga + ov9281 https://forum.trenz-electronic.de/index.php?topic=1163.0

6) https://www.uctronics.com/arducam-ov9281-mipi-camera-module-1mp-monochrome-global-shutter-camera-2087.html

7) https://www.arducam.com/docs/cameras-for-raspberry-pi/migrated/ov9281-1mp-global-shutter-raspberrypi-camera/

8) ov9281 can run in one lane data mode (linke ov5647) 

https://github.com/ArduCAM/ArduCAM_USB_Camera_Shield/issues/134

9) https://ru.wikipedia.org/wiki/ROS

10) Getting Arducam Global Shutter Camera to Work w/ROS 

https://www.raspberrypi.org/forums/viewtopic.php?t=277885

11) OV9281 with external trigger https://www.raspberrypi.org/forums/viewtopic.php?t=293959

12) OV9281 https://www.raspberrypi.org/forums/viewtopic.php?t=291818

13) (!)Capturing with Arducam MIPI monochrome global shutter cameras https://www.raspberrypi.org/forums/viewtopic.php?t=267563

14) datasheet https://pdfcoffee.com/ov9281-datasheet-pdf-free.html

ov9281 linux drivers, ROS (robot operation system), antmicro:

1) (!)https://github.com/rockchip-linux/kernel/blob/develop-4.4/drivers/media/i2c/ov9281.c

2) https://github.com/ArduCAM/MIPI_Camera/tree/master/RPI

3) (!)https://github.com/antmicro/tx2-deep-learning-kit-bsp/blob/master/kernel/kernel-4.4/drivers/media/i2c/ov9281_mode_tbls.h

4) (!)https://github.com/antmicro/tx2-deep-learning-kit-bsp/blob/master/kernel/kernel-4.4/drivers/media/i2c/ov9281.c

5) (crop) https://github.com/raspberrypi/linux/blob/rpi-5.10.y/drivers/media/i2c/ov9281.c

6) (external trigger) https://github.com/ArduCAM/MIPI_Camera/blob/master/RPI/ov9281_external_trigger.c#L16

Chief Ray Angle (CRA):
1) Chief Ray Angle and Field of View https://www.physicsforums.com/threads/chief-ray-angle-and-field-of-view.907239/

2) CMOS sensor CRA https://www.dpreview.com/forums/thread/3819663

3) (!)What is the CRA (chief ray angle)? https://www.optowiki.info/faq/what-is-the-cra-chief-ray-angle/

4) (!)How do CRA (Chief Ray Angle) and FOV (Field of View) affect one another? https://physics.stackexchange.com/questions/421429/how-do-cra-chief-ray-angle-and-fov-field-of-view-affect-one-another/421441

5) Image Sensor Optics http://isl.stanford.edu/~abbas/ee392b/lect10.pdf

connect old phone csi cam to raspberry pi, stm32, pic32

1. phone csi camera

1.1. Wiki:

1) Фотоматрица (https://en.wikipedia.org/wiki/Image_sensor)

2) ПЗС-матрица

3) https://en.wikipedia.org/wiki/Charge-coupled_device (русская вики ПЗС)

4) КМОП-матрица (https://en.wikipedia.org/wiki/Active-pixel_sensor)

5) КМОП (https://en.wikipedia.org/wiki/CMOS)

6) (прочитать) Фильтр Байера

7) Оптическая система

8) Глубина резко изображаемого пространства

9) Мира (оптика)

10) Дихроидная призма

11) Цифровой фотоаппарат

12) Экспозиция (фото)

13) https://en.wikipedia.org/wiki/Camera_phone

14) https://www.electroschematics.com/mobile-phone-camera-interface/

1.2. defect pixel detection, image histogram:

1) Robust Defect Pixel Detection and Correction for Bayer Imaging Systems https://www.ingentaconnect.com/contentone/ist/ei/2017/00002017/00000015/art00008?crawler=true&mimetype=application/pdf

2) https://en.wikipedia.org/wiki/Image_histogram

3) Intensity Histogram CMOS Image Sensor for Adaptive Optics https://isn.ucsd.edu/pub/papers//iscas10_hist.pdf

4) Histogram-Equalization-Based Adaptive https://www.researchgate.net/publication/2977536_Histogram-equalization-based_adaptive_image_sensor_for_real-time_vision

1.6. stm32f4 or other with dcmi, i2c + csi2 camera:

1) Digital Camera with STM32 and OV7670 https://www.youtube.com/watch?v=MqtJbraAlOU

2) (!)STM32F4Discovery – подключаем камеру по интерфейсу DCMI https://habr.com/ru/post/186980/

3) How to port the MIPI camera with STM32F4? https://community.st.com/s/question/0D50X00009Xkeun/how-to-port-the-mipi-camera-with-stm32f4

4) https://github.com/erikandre/stm32-ov7670

5) Ввод данных в STM32F4xx с параллельного АЦП через DCMI https://habr.com/ru/post/482506/

6) 

1.7. arduino, stm32 with usb + csi camera + uvc:

1) Как дешево забрать картинку MIPI/CSI камеры по USB? http://forum.easyelectronics.ru/viewtopic.php?f=14&t=29823

2) A Guide to Arduino Based Video Camera https://www.open-electronics.org/a-complete-guide-to-arduino-based-video-camera/

3) (uvc)Using other camera modules to build custom webcams https://raspberrypi.stackexchange.com/questions/22963/using-other-camera-modules-to-build-custom-webcams

4) (uvc) How to make Real Spy Cctv Camera From Old Mobile Phone Camera https://www.youtube.com/watch?v=THFKtFVdrvo

5) Как подключить MIPI CSI-2 камеру? https://electronix.ru/forum/index.php?app=forums&module=forums&controller=topic&id=139475

6) https://www.cypress.com/products/ez-usb-cx3-programmable-mipi-csi-2-usb-30-camera-controller

7) (interrupts) Interfacing a cheap phone camera module to a PIC32 microcontroller https://www.youtube.com/watch?v=rQYByorpoFk

1.8. arduino, stm32 + csi camera + csi-2 in fpga:

1) Opensource IMX219 FPGA MIPI CSI-2 Camera USB3.0 2Gbps UVC Video Test 15 To 1000FPS, Upto ~ 4K https://www.youtube.com/watch?v=uRaHXo-Zu90

2) 1000 FPS MIPI CSI-2 Camera Sensor FPGA Receiver https://hackaday.io/project/170250-1000-fps-mipi-csi-2-camera-sensor-fpga-receiver

1.9. Old mobile phone, mouse camera use cases.

1.9.1. ip webcam, action camera, fpv, night vision cam from mobile phone cam:
1) https://blog.kvv213.com/2015/10/sozdaem-ip-kameru-videonablyudeniya-iz-starogo-smartfona/

2) How To Turn Your Old Phone Into An Action Camera (DIY) https://www.youtube.com/watch?v=-lsaZ1CEHOE

3) DIY Micro 5.8Ghz FPV https://www.youtube.com/watch?v=nt6kYWbMQIM

4) How to make a night vision camera using your old phone https://www.youtube.com/watch?v=2YEvkOVCaNI

5) How to make Night Vision Camera Using a Smatphone Version 2 -DIY Tutorial https://www.youtube.com/watch?v=PtN4ddIIZbA

1.9.2. fpv cam:

1) Micro FPV Camera & Transmitter (FX798T, 5.8GHz, 25mW, 40ch) https://www.youtube.com/watch?v=0SnQDcb4a-I

2) Micro Camera Hacking! https://www.youtube.com/watch?v=eGjjD-8Er5o

1.9.3. разборка:

1) Tear Down, Camera Module, #3 808 Car Keys Micro Camera https://www.youtube.com/watch?v=5j1DOYNXEqg

2) How does a camera work? https://www.youtube.com/watch?v=B7Dopv6kzJA

3) What's Inside Smartphone Camera & How it works?? https://www.youtube.com/watch?v=fkG3D0WXRAw

4) Close look at mobile phone camera optics https://www.youtube.com/watch?v=KH0MZctnJlo

1.9.4. microscope from phone cam:

1) DIY microscope - mobile phone and DVD RW drive hack https://www.youtube.com/watch?v=auQwOtrUN5A

1.9.5. optical mouse turn into cam:

1) Hack an optical mouse into a camera with Arduino and Processing https://www.youtube.com/watch?v=bci7Gi05BNc

1.9.6. mipi csi-2:

1) Tutorial: MIPI CSI-2 https://www.youtube.com/watch?v=8REu_h7bzHM

2) OpenMV Review - Machine Vision Camera Module https://www.youtube.com/watch?v=vsnZzPEQeeQ

3) OpenMV Cam H7 Kickstarter Video https://www.youtube.com/watch?v=7Q8A6LeP41g

4) Sensor module vs. camera module https://www.youtube.com/watch?v=Oi4S7SwumGs

5) Understanding MIPI https://www.youtube.com/watch?v=sxuwbJPXSqk

6) Close look at mobile phone camera optics https://www.youtube.com/watch?v=KH0MZctnJlo

7) Smartphone Camera Teardown and Possible Modification to Night Vision https://www.youtube.com/watch?v=GMd77on_k0Q

2. progressive / interlaced frames:

1) https://en.wikipedia.org/wiki/Progressive_segmented_frame

2) Progressive vs. Interlaced https://medium.com/hd-pro/progressive-vs-interlaced-e18e2924800e

3) How To: Interlaced, Progressive and Frame Rates https://www.youtube.com/watch?v=WxH23dxDIsQ&ab_channel=SonyEurope

4) Interlaced and Progressive Frame Rates Explained! : FRIDAY 101 https://www.youtube.com/watch?v=xKMWjRlIvrY&ab_channel=IndyMogul

5) Frame Rates and Interlacing https://www.youtube.com/watch?v=QojksfRs_5g&ab_channel=Videomaker