Alternate camera platforms

[jamesd256]

Hi,

Congratulations on a spectacular project, very inspiring.

What are your thoughts on supporting other camera platforms. There seem to be a few other solutions with similar spec to Basler for as little as one quarter the price.

I am very poor, but would like to be able to try out your amazing software. I am very willing to take on the work. just trying to get a sense of how much effort you think it would be.

I hope this is the right channel to reach you to discuss such things.

Thanks & regards,

James

[BZandi]

Thanks! Happy to hear that you find our project of value.

Generally, we use an abstract camera class camera.h, which is used in the singleCamera.cpp and stereoCamera.cpp file to retrieve the images. Additional camera platforms can be easly integrated by preparing the files in the device folder (see PupilEXT/src/devices).

Note that consumer webcams are not suited for the program structure in PupilEXT as we use an event-handler to grab the images from the camera (see singleCameraImageEventHandler.cpp). There are also additional features that can not be used with webcams, such as stereo camera recording because we applied a hardware trigger to synchronise the two cameras, which is impossible with webcams. Additionally, with webcams, there some numerous issues that need to be solved for accurate pupil measurements. Thus, it is recommended that the alternative camera brands offer machine vision cameras, i.e. industrial cameras with a C++ camera library.

If you want to run PupilEXT with very low investment, you have three options:

a) Take a look at the Basler dart series. They are affordable (~120 Euro) and should work with PupilEXT (not tested), as they use the Pylon library, which we use in PupilEXT to interface the camera(s). Link: https://www.baslerweb.com/en/products/cameras/area-scan-cameras/dart/

b) If you want to go even cheaper, you need to use a webcam (< 100 Euro). As mentioned, it is hard to integrate webcams in PupilEXT for real-time measurements, as we use an event handler to distribute the image inside the functions of PupilEXT. An alternative would be to record the images with an external tool and load the images into PupilEXT for offline analysis. We provide demo images that you can already use for offline analysis and testing the software. For this, download the demo dataset in section 2.4 (see readme file) and check out the respective video tutorial for offline analysis in which we load externally acquired images into PupilEXT. Link: https://vimeo.com/564778587

c) We are working on a Python version (library) of PupilEXT, which we publish later this year. With the Python library, you will be able to integrate any camera that you like.

I move this thread to the discussion forum as this might be an interesting topic for other users.

Best regards
Babak

[jamesd256]

That is very helpful, thank you Babak!

If I understand you correctly, because the lower cost Basler cameras are supported by Pylon, in theory it may be possible to achieve real-time measurement using these. By extension, if this proves the case, I assume you would not expect to have any issues achieving stereo imaging.

This sounds like the perfect route to try. If I do come up with any findings this way, I will share them here.

Forgive me if I've missed anything, but I would like to understand what performance benefit you might expect from a stereo setup over the single camera.

Good luck with the library version! Will you be licensing it as open source like the PupilExt project?

Thanks again & regards,

James

[BZandi]

Yes, the Python library of PupilEXT will also be open-source.

Although we did not test it, I am confident that it will work with the dart series, as all the USB3 cameras from Basler share the same commands for retrieving the images or adjusting the camera parameters through the Pylon library.

The advantage of stereo vision is that you can measure the pupil diameter directly in millimetres rather than pixel values. Stereo vision, however, comes with an increased effort as you need to purchase two cameras, and an additional microcontroller is necessary to synch the cameras.

Overall, you can go with a single camera, as we provide the option to measure a circular reference object of known size to calculate the pupil diameter in mm from the pixel values manually. Additionally, it is easier as you only need to connect the camera with your computer. Keep in mind that it is better to buy the monochrome version of the camera rather than the colour version.

Have a look at our paper in which we discuss the PupilEXT platform in detail:
https://www.frontiersin.org/articles/10.3389/fnins.2021.676220/full

[jamesd256]

Thank you once again. The explanation of the benefits of stereo makes perfect sense.

[BZandi]

A Python version of PupilEXT, called PyPupilEXT is now available. Note that the lib is currently in the experimental stage.

Link to the GitHub repo of PyPupilEXT:
https://github.com/openPupil/PyPupilEXT

[slee-101]

Hi,

This is incredible work and of high value to the research community!

Do you think there is any benefit in using frame grabbers? I don't see any from basler for USB 3.0 but they do (or did) for GigE PoE (https://www.graftek.biz/products/basler-adlink-gie74-pci-express-gige-vision-poe-4x-pcie-framegrabber-4-direct-camera-inputs). NI also has some framegrabbers for GigE.

[BZandi]

Thanks!

I would say that It depends on your motherboard type, the number of connected cameras, and how many additional devices (external hard drives, microcontroller) are connected to your PC via USB 3.0.

However, for our current stereo camera setup, we use a 4 Port USB 3.0 PCIe interface card, as our USB 3.0 Ports of the motherboard are used for other devices and we wanted to separate the camera connection.

Here is a link to the USB 3.0 PCIe card from Basler, which we currently use:
https://www.baslerweb.com/en/products/vision-components/pc-cards/usb-3-0-interface-card-pcie-fresco-fl1100-4hc-x4-4ports/

There are also two channel cards:
https://www.baslerweb.com/en/products/vision-components/pc-cards/

[slee-101]

Thank you for the information and thoughts! We will give this a try and share anything that might be useful to the community.

[berginator]

Hi,
I would also like to begin by thanking you for your work for the scientific community!
I am interested in trying out the Basler dart series. Do you absolutely recommend the 60fps mono version, or do you think one could have ok results with a lower frame rate?

[BZandi]

Thanks!

There is no general answer to this question, as it depends on the conditions of the pupil experiment. For example, if only the steady-state diameter is of interest, then 30 fps could be sufficient, as smoothing is done in the pre-processing of the pupil data anyway. However, if the short-term pupil light reflex or transient changes of the pupil in response to cognitive workloads are of interest, then 60 fps could make sense. However, the maximum frame rate can also be adjusted in PupilEXT to record, for example, at 30 fps with a 60 fps camera. Therefore, having the option to record at higher framerates is also nice if you see in your experiments that you need a higher acquiring speed.

In our setup, we use a 120 fps camera but we record at 30 fps most of the time, but I appreciate having the option to record at higher frame rates if I need to. Therefore, I would personally go for the 60 fps camera, but as I said, that depends on your experimental conditions. Attention should also be paid to the resolution of the camera, which has a great impact on the measurement accuracy.

Note that we never tested PupilEXT with the dart series, but I am from a theoretical point of view confident that it works, as we use Basler's Pylon library with the Pylon::CBaslerUniversalInstantCamera camera class, which should, according to the Pylon documentation, operate with all USB camera devices from Basler.

https://github.com/openPupil/Open-PupilEXT/blob/main/src/devices/singleCamera.h

[Ricard0li]

Hi, PupilEXT is a wonderful open-source platform , thanks so much!
Since I already have the avA1000-100gc, which is GigE interface, I tried to use this Basler camera instead of the USB one you listed. However, I couldn't get any image from it after I chose the camera. Could you give me any suggestion about this? I thought maybe it is because of the library you used were only for USB camera?

[kheki4]

@Ricard0li The GigE interfacing was unfortunately missing from the Pylon wrapper in the releases so far, but I will add it soon and will be included in the upcoming PupilEXT v0.1.3b Experimental-Community-Version. It will likely com in a month or so.

[kheki4]

I succeeded to implement the Aravis library, however there are still stability improvements to be done specifically for GigE cameras with the Aravis. Ther will be two different release builds from now on: either Pylon or Aravis dependent ones. I plan to make a new release, at least for Windows 10 x64, within the next month or so, on the Experimental-Community-Version branch, also with a hotfix for the v.0.1.2b that came out last year.

[kheki4]

Hi @jamesd256 and @Ricard0li ! A little update here:

I have to admit I have been a little snowed lately, and I also spent my autumn semester at another university, so I only finished the Aravis implementation not long ago. But here it is, this version supports Aravis, so GenICam capable cameras should work with it:
https://github.com/sasfog/Open-PupilEXT/releases/tag/v0.1.3-beta

Yet it only supports single camera mode only. However, my cameras are just different Baslers, so I can not be completely sure that all features will work nicely on all others. I have had access to a Point Grey camera over USB3Vision to test it, but it missed a few features. I successfully tested the build with a Basler daA1280-54um, and an acA1300-60gm, and acA1300-200um, and it could communicate with them nicely using Aravis. Also, Daheng cameras seem to be well suited replacements for Baslers, but I have not tried any of them yet.
But, it is important to see that GenICam was developed by the EMVA, based in Europe, and they naturally had more close ties with European brands, e.g. Basler, so it is likely that Baslers are the ones that fulfil GenICam criteria the most smoothly.

As far as I have experienced, cameras vary wildly in how they interpret details of the GenICam features. Anything that is not strictly documented in the standard, are free to vary between camera models and brands, and are subject to change without notice in implementation over time.
For example: the maximum image ROI width/height parameters, if you query them from the camera, in some cameras already contain the offsetX/offsetY subtracted, in some others they don't. Probably they will not even be accounted for currently set binning, probably they will. And if you change binning level, the camera perhaps first changes the image ROIs to match it (downsizes before halving the resolution), but perhaps it does not (then you get buffer overflow or a stuck camera in case of GigE).... etc. So it is really not an easy thing universally support these features, not even with GenICam through Aravis. I tried to add as many workarounds for these as I could, but you need to test the software with the exact camera model to make sure.

If it is okay for you to buy used cameras, I think it is better to buy a used Basler on Ebay than buying a random manufacturer's camera brand new. There are also many sellers even on Ebay that are invoice capable, so you can legally spend your grant money on them. ((Just be sure to check if the lens are glued to the camera or not - that can be an annoying detail if you want to use your own lens.))

I hope the Aravis version will work for you, but if any problem arises, please indicate.