Friday, October 24, 2014

IP Camera Testing and Review

axis 84 lights on-500We tested  IP cameras to see which camera worked the best in low light situations.  We did some other tests as well for resolution, and  wide dynamic range.

In this test we compared the low light performance of the Samsung SNV-6084R IP and Sony SNC-VM630 cameras. We compared them to each other and to an older Axis P3367 camera which has a reputation for good low light performance.


Camera Specifications
All the camera specifications indicated about the same low light level when in color.  The Samsung camera had the best B/W low light sensitivity specification.  The test would determine if the cameras met their specifications.
Samsung SNV-6084R Color: 0.1 Lux (F1.2, 50IRE)
B/W: 0.01 Lux (F1.2, 50IRE)
Image sensor: 1/2.8" PS Exmor 2.38M CMOS
Sony SNC-VM630 Color: 0.1 lx (F1.2, 50IRE, View-DR OFF, VE OFF, AGC ON, 1/30 s, 30 fps)
B/W: 0.07 lx (F1.2, 50IRE, View-DR OFF, VE OFF, AGC ON, 1/30 s, 30 fps)
Image sensor: 1/2.9-type progressive scan Exmor CMOS sensor
Axis P3384 Color: 0.15 lux, F1.2,
B/W: 0.03 lux, F1.2 with Lightfinder
Image sensor: Progressive scan RGB CMOS 1/3”

The low light specifications also include frame rate (shutter speed) and IRE. The frame rate relates to the shutter speed, so anytime you see a camera with frame rate of less than 1/30 the second or if the IRE is at 30 IRE instead of 50 IRE, the lux value will be much different. Take a look at our article about IP Camera Low Light Sensitivity for more details about how low light sensitivity is measured.

The Test
We compared these cameras to the 1 megapixel Axis P3384 because it has good low light performance (this is their specialty low-light camera).  Please note that a 1 megapixel camera is expected to have better low light performance than a 2 megapixel resolution camera.  In spite of this, the 2 megapixel Sony and Samsung cameras outperformed the 1 megapixel Axis camera.
Axis Camera in Test Box

We started out test at relatively low light and all the cameras provided decent images. We gradually decreased the lighting until all the cameras could not see anything.  All the cameras tested did not have an IR illuminator or had them turned off.

In the setup of each camera we made adjustments to assure that all the cameras maximized their low light sensitivity.  This included turning off WDR, turning on the light enhancers (which is called different things depending on the camera), keeping the cameras at 30 fps, and keeping amplification at 50 IRE.

We found that it was difficult to measure the actual light level (lux) because most light meters do not go below 0.1 lux, so we did comparison testing.  We took snapshots from each camera at four light levels.  The goal was to capture images at each level of darkness and to compare all the cameras at the same relative lighting level. We observed which camera displayed the clearest image with the least noise level.

Test at approximately 0.5 lux

At this light level all of the cameras displayed little to no amplifier noise with good visibility and brightness.  For reference the light from a full moon is about 0.3 lux to 1.0 lux depending how close to the equator you are.  The moon is brighter as you get closer to the equator.

The Sony and Axis cameras had the least amount of amplifier noise and the highest brightness. Additionally it is important to note that because the Axis camera has a 1 megapixel sensor, it displayed the image in color while the 2 megapixel Samsung and Sony cameras had switched to black and white mode.  In this regard the Sony and Samsung camera did not meet their published low light specifications in color.

Sony SNC-VM630: The image is very bright and clear. We can see the details of the scale. (Note this is a 2 megapixel camera).

Axis P3384: The image is in color. We can see the colors of the pen and the details of the scale are visible. (This is a 1 megapixel camera)

Test at approximately 0.2 lux 

With the light level reduced, all three cameras displayed high quality images. The Axis camera stayed in color.  All the cameras showed detailed clarity and only a small drop-off in brightness with low noise. The Samsung and Sony compared well with the Axis camera. The Sony displayed slightly better image quality at this light level.

Sony SNC-VM630: Display is bright and clear with low noise
Samsung SNV-6084R: Noticeably darker image, but is still displaying a fairly clear image with little to no amplifier noise.

Lower Light Level (approximately 0.05 lux)

With the lighting reduced the Axis switched to B / W and showed the darkest image with noticeable amplifier noise.  The Sony displayed the brightest image with the most amplifier noise.  The Samsung displayed average brightness with the least amount of amplifier noise. Overall the Samsung displayed the image with the best combination of brightness and low amplifier noise.
sony wdr off visability enhancer high night mode on lights off door crack open
Sony Camera at very Low Light

Axis P3367: Noise level increased and it was difficult to make out the details of the scale.
Sony SNC-VM630: Increased noise but can make out more details than the Axis camera
Samsung 6084: The image is dark, but the least amount of noise with the clearest image.

The Lowest light level (approximately 0.01 lux)
samsung ssdr on wdr door closed-500
Samsung at very low light level

This test was done at extremely low light level. It was below the light sensitivity of the Axis camera. Under these conditions the Samsung tested best.  Despite showing a dark image there was very little amplifier noise and the objects are visible. With the Samsung you can clearly see the markers and the scale. The Axis camera displayed very little and it was nearly impossible to identify our objects, the scale and the markers. The Sony showed the outline of our objects to some degree. The image is hard to see because of the lack of brightness, but the extreme level of amplifier noise is the main factor causing our inability to view the image on the Sony.

Conclusion

The Samsung 6084 won the low light sensitivity test, though the Sony was very close.  It was interesting to note that Sony provides conservative specifications for its IP cameras.  The tested performance was better than their published specifications. Even though the camera has a published specification of 0.07 lux, it was almost as good as the Samsung camera, with published specification of 0.01 lux.

All the cameras displayed good images at light levels as low as 0.5 lux.  The Axis camera stayed in color longer as the light was reduced, but didn't perform well when it switched to B / W mode.  At light levels down to about 0.05 lux, the Sony camera provided a bright image, and low noise, but at the lowest light level of about 0.01 lux, the Samsung camera displayed the lowest amplifier noise and best image.


For assistance in selecting your IP camera, please contact us for help.