This is possible by interfacing directly with the sensor MMIO registers. I've always found fascinating how much flexibility hardware provides, compared with the typically rigid functionality the drivers expose to userland.
I wonder what we could make out of many high-end cameras if only we had enough patience and motivated people to reverse engineer everything that has been hidden away by comfort abstractions.
If you want the answer to that question, look no further than the magic lantern project. It’s not supported like it used to be, but those guys were incredible at unlocking what Canon DSLR’s could really do. I was a legend in my film community when I booted it onto a 5Diii and got it shooting 1080p raw. All I did was follow their instructions lol
I've heard that Canon's DIGIC processors are just slightly customized versions of the DaVinci media processors, which have a all documentation openly available, while Nikon's EXPEED processors are based on a closed product line by Socionext with basically zero public documentation and heavy customization.
Though there used to be a project which worked on a few of the older EXPEED-based cameras.
No it handled it like a champ! It could actually record higher resolution than my much newer 6d. It used a compact flash which was much faster than an SD
Fun stuff! Yeah I did the raw hack on a 5diii. Gorgeous image, couldn’t believe the DR I was getting. Canon should be ashamed for keeping all the firepower under lock and key! Guess that doesn’t help love C-cameras though lol
It's not some inherent price. It's just an arbitrarily imposed one.
Older products, even early digital ones from 70s and 80s, were more flexible for the end user (and often came with manuals and schematics) even in their consumer version.
An industrial product has different requirements compared with a consumer product. I am really curious how many hours of continous operation my camera can sustain.
Ironically, I find the popularity of hardware hacks to be a sign of general stagnation, because there were periods of time in the past when they would not give as much boost in performance as a new version of something in a few months to a year. They were typically not worth the time and would be rendered obsolete, but now more and more they are worthwhile.
Okay, that's bizarre. When I first clicked the link, the domain name failed to resolve for me too. When I reloaded it, though, it resolved. If anyone else is having trouble, give it a bit and try reloading the page.
The major limitation throughout this process is the speed at which memory can be copied and transmitted. Only 20-40 seconds of video can be recorded at a time due the memory exhaustion of the Raspberry Pi. Also limited is the resolution of the recording: On the $6 camera, a maximum of 640x64 resolution can be recorded due to limitations on memory bandwidth.
Probably some, but not to the degree that you may think. This relies on DMA in the CSI controller, but only a fraction of a pi4's RAM is accessible by DMA. Basically from the video core side, there's only a 32-bit bus to main memory, and it has to share that with MMIO space and there's mirrors with/without caching sort of like a MIPS. Probably maxes out after a couple GB or so.
I haven't read the article, but the parent comment makes it sound as though it's not so much the capacity of the bus, but rather the speed of the bus itself. This is usually expressed in terms of a combination of CAS latency and frequency in MHz, the Pi 4 being DDR4, is likely in the neighborhood of 2133-3600Mhz. We don't often think of reading/writing to RAM as taking any time at all, but in all actuality the speed of memory is often a significant bottleneck in high performance systems.
If we are just talking about 720p 30fps, then we likely aren't really that close to those memory bottlenecks, so the only limiting factor would be how much storage space you have available to that device.
Its nonsense as I already wrote last time it was posted 2 years ago:
Ram speed is not the issue, camera interface is limited to 120MB/s, ram bandwidth is >2GB/s on pe3, 4GB/s on pe4.
Compressing 640x64@660 takes exactly the same amount of work as compressing 1280x720@30. You could even use hardware h.264 encoder as long as you can debayer raw 10bit in software fast enough (or convince Broadcom employee 6by9 to make internal Videocore firmware debayering resolution agnostic, its already fast enough keeping up with twice the amount of data when using camera in 1920x1080@25 mode).
But you dont even need fancy compression, at those framerates things change really slowly so simple delta encoding on raw data will do wonders.
According to the video in the article, the data rate is 66 MB/s (100kB per frame). So a Pi with 8GB of RAM should be able to capture almost 2 minutes of footage.
Then why not tunneling the sensor output through the lowest available network layer using the NIC, then reassembling it on a faster machine connected point to point (no IP stack involved) to the RPi?
It looks like the repo [0] hasn't been touched in a couple of years so it's tough to say what if anything would need to change, but this application sounds like it would benefit from a Pi 4.
That’s how it works with more conventional camera set ups. I guess it just boils down to how they program it but there’s no reason that shouldn’t be the case.
From my reading of the issue, no. The limiting factor on the Pi side is available storage buffer. The camera sends raw files down the wire and the app caches them in RAM before concatenating them with ffmpeg. Could a Pi4 with USB3 NVMe be used? Maybe, if the writes are faster than the camera.
The 1007 fps he said he got out of the US$6 V2 camera is over 8 times the 125 fps at which exporting the camera from the US requires an arms dealer license: https://www.bis.doc.gov/index.php/documents/regulations-docs... but that only counts if the shutter speed is under 1 μs.
I'm a bit confused here. There are a few Sony cameras (the RX series) that have a 960fps capabiilty, with exposure times of under 1/1000th of a second. That's around 8 times the 125 fps limit set by ITAR, yet those cameras are sold pretty much everywhere around the world. What am I missing/not understanding here?
The 1 μs is the time the electronic shutter takes to act, and there's no spec sheet for that that I'm aware of in the Sony cameras. As for exposure times at high frame rate, they go up to (down to?) 1/10,000th of a second, so definitely the electronic shutter must be working faster than 100 μs to reach that.
But as someone already pointed out, my main issue was understand the "International" (the I from ITAR) as meaning that it was a treaty signed by many countries, when it actually means "exporting from the US to International markets".
The language in the law is "electronic shutter speed (gating capability)", which I think is the same as the "exposure time." I mean, "exposure" is sort of a misnomer for digital video cameras: the focal plane is exposed to light all the time because there's no shutter. It's just that some of the time it's turned off ("gated"). And when you set the "shutter speed" on a mechanical camera to 1/60, that's the exposure time. You seem to be talking about something slightly different, like maybe the sensitivity rise time?
Right, that was it. I just took the "International" part of the name to mean as if it were signed by many countries, not a "from the US to the rest of the world".
As webcams were hard to come by during the onset of the pandemic, I resorted to a raspberry pi + HQ cam + lens + raspistill preview + HDMI capture device (instead of usb). I won't defend the total cost (probably more than an acceptable webcam). The HQ cam results were quite good and seemingly high enough framerate (not 660 fps though!). Added benefit is interchangeable lens options including ones that will give the bokeh effect.
Great timing on this article. I have a rpi4 and a pi cam v2 which I want to have some fun with and was in need of inspiration. I am currently using it as a security camera while I am away on holiday but wish to have a play both with slow motion video and perhaps some object detection using opencv.
The latter I have already tried but all the articles I found seem out of date so I will need to spend some more time on.
This could be a big step toward ultra low latency streaming with an rpi.
I tried to do this (low latency streaming) circa 2016 and ran into two issues, one of which was the high framerate or subframe capture and the other was subframe encoding.
With high framerate capture you can fake the subframe capture by capturing a whole frame and encoding just a subset.
Now to get the h264 drivers to do subframe encoding…
I'm interested in synchronizing sensor data to the camera stamps. For example, strain gauge to measure model rocket engine performance. Does anyone know of a project or approach that would allow for synced high speed sensors?
Low tech approach: I wonder if you could put a flashing LED in the camera's view, and sync that with the strain gauge data logging, kind of like an old fashioned move clapper board?
An approach I’ve seen used in the past was to add lines to the video frame and write other sensor data in “extra” lines in the frame before writing to disk. You can then parse the data after the fact and know what the sensor data was for each frame.
I wonder what we could make out of many high-end cameras if only we had enough patience and motivated people to reverse engineer everything that has been hidden away by comfort abstractions.