LONDON, England – The new Nokia N8 is so much more than ‘just’ a smartphone with a great camera – but we’ve had such an incredible amount of interest and questions in many online forums and websites on this topic alone that I thought we should take the time to explain a little more background behind the Nokia N8’s new camera, as well as our approach and why that’s different to previous products. (Note that the pictures in the main post are very severely downsized from their original resolution. Click through to see them in their full form).
Not your daddy’s camera phone
But before we get onto the latest and greatest, I’d like to first share a personal milestone. Here’s one of the very first pictures I took with a Nokia camera phone.
This is my daughter Sophie as seen through the lens and 1-megapixel sensor of the Nokia 7610 on day eight of my time with Nokia.
Having worked for many years with high-end professional cameras and associated equipment, prior to joining Nokia, I looked up at the hill we had to climb and wondered, would we ever reach the top! Would it ever be possible to get really great images from such small cameras?
So here we are today with the Nokia N8, on day 2260 (as at 8th July). What a journey it’s been.
Here’s an image I took recently, with still-under-development hardware and software on the N8. Take a look at the detail on the wings near the main body of the butterfly. Actually, the N8 will focus closer than this (to around 10cm) but I preferred this composition. I think the background blur is really stunning. What do you think?
For me, this image sums up everything we wanted to achieve with the Nokia N8. Natural. But I’m going to need to explain in more detail what this means, as we believe nobody has done this before with mobile devices.
The old rules don’t apply
Since the early camera phones, one of the principal contributors to improved image quality has been the increase in sensor resolution. Improvements in image quality were relatively fast-paced. Along with increased resolution sensors, we’ve also seen big advances in CMOS sensor performance, as well as far more complex image processing made possible by increasingly more powerful image processors.
As has been briefly touched on in a previous article here on Nokia Conversations, we started the process to tune the image quality way back in January. Fairly early on, applying the usual techniques and methods of image quality optimisation provided incredible levels of ‘perceived’ detail and sharpness. Many people were truly excited by this incredible resolution. But myself and a few colleagues looked at them and – despite the reaction from others – we thought, “No, this isn’t right”.
Regardless of what all the optical simulations tell you in the design phase, it’s really only when you see the first raw outputs that you know whether you have something great or not. I think we’ve got enough experience now to know that even in those very early stages when the output from the camera is still incredibly raw (when I refer to raw I literally mean raw, these are first engineering samples with test sensors and no image processing, almost everything is wrong at that stage).
It was clear to us that the potential was there but we just weren’t making full use of it. We had this new super large sensor and these incredible new Carl Zeiss optics, they were crying out for a different approach. This is when we made the decision to go for reproduction that was as natural as possible, and that’s what we’ve been working on since. Images with incredible natural detail with relatively low noise but retain a punch to the images through the vibrant colour reproduction whilst still ensuring it remained natural and very close to the ‘mind’s eye’. This desire for natural reproduction is an all-encompassing one. Covering both the visual AND audio elements of the high-definition video recording as well.
The first thing we did was throttle back the edge enhancement to a point where it’s almost disabled. In video – under good lighting – we completely disabled noise reduction. I think it’s pretty unheard of for such devices to not be running any noise reduction. But it’s testimony to the great optics and superior sensor.
First, let’s look at what this meant for still images. But before that, let’s be clear here, all camera phones have moved on enormously over the last few years and I don’t mean that in terms of megapixels but the image quality you get. The colour and detail is hugely improved and – in good conditions – for the vast majority of people, it’s probably hard to not be happy with the images they capture. This extremely high standard only serves to make our job even harder. However, when I look at those images taken by many of our counterparts: at first glance, they look great, but then as I look a little more I find there’s something nagging me about them, their artificial appearance. Maybe myself and my immediate colleagues are too much like purists in this regard.
Apart from disabling various artificial enhancers, we also carefully consider how we reproduce colour in certain environments. Here’s two examples. In candle-lit scenes, you remember the warmth of the candle light. In snow scenes (my Finnish colleagues know this far better than I do), people tend to remember the bluish tint to the snow caused by the blue sky. In these situations, rather than correcting to a theoretical perfect white balance, we tune it to how you remember it. To achieve this takes time and many rounds of tuning, testing, retuning, testing, etc. This is not about colour saturation although that’s a method I see used often by others. That often leads to colours that are no longer representative of the original, despite their vibrancy. Contrast is another trick to fool the eye into thinking the scene is punchier and sharper. It’s a trick that works, however it can also lead to increased noise, lost highlights or shadows in the scene. Again, not a true reflection of the original scene.
All in the detail
Details is the final one we have spent a great deal of time improving and optimizing. As touched on earlier, we purposefully throttled right back on the sharpening enhancement. Right now, as it stands, we’ve almost disabled edge enhancement. This is another visual trick that’s been used for many years in digital cameras and one that works well to fool the eye that something is sharper than it actually is.
Here’s a section from an image taken with a device which I’ll simply refer to as ‘Exhibit A’. Notice the white halo around the hand and along the stripes on the shirt. These lines simply aren’t visible with the Nokia N8. (please note, all images were captured with still-under-development software and pre-production hardware).
I’ve enlarged these way beyond the point you would normally view them simply to make it easier to point out some things for the purposes of this article. What you’re seeing here is the edge enhancement at work. When viewed at normal magnifications, these lines that are drawn around objects by the edge enhancement process fool the eye into believing the level of edge definition is greater than the original.
The shot of the lake shows an early example from the Nokia N8 on the bottom and this time ‘Exhibit B’ on the top. Look at the top edge of the upside-down boat, notice the white line. What you should see are the pixels that relate to the boat and then the pixels that relate to the grass. Anything else is make-believe. Now look at the water. Here it seems there is detail that’s missing in the Nokia N8 image. This isn’t detail, it’s noise which has been amplified by the edge enhancement. It’s the same with the grass. The blades of grass are very fine. However, in the upper image the blades of grass are wider due to the edge enhancement. Some of you I’m sure will be thinking “yeah, so what? I don’t look at my images that closely.” Of course, you’re right: why would you? The reason for me pointing these aspects out to you is to attempt to explain why myself and my colleagues when looking at the images normally feel there’s something disturbing our eyes so much so that we wanted to do it differently.
This final image comparison is even clearer. ‘Exhibit C’ is on the left with the Nokia N8 represented on the right. See how much finer the grass is and how much more detail there is in the stones?
Through a lens, brightly
In many mobiles over the years, this has been principally done because of relatively low performance optics. Or because the combination of sensor and optics was unable to resolve the detail. With the N8 however, due to the combination of Carl Zeiss optics and the largest sensor ever used in a mobile, we felt it was time that mobiles grew up and stopped using many of these tricks to fool the eye. We wanted to strip the process back to the basics and deliver raw imaging power both in stills and video in a way that we felt hadn’t been done before. Some may say they like those enhanced pictures and that’s OK – but for us right now we want to capitalise on this great sensor and optics combination and create natural beautiful images and video which we hope you’ll really love as the default straight out of the box. If however you do prefer that ‘enhanced’ look there are settings for colour, contrast and sharpness you can use to recreate that look as a default if you so wish. There’s also a new feature where it will remember your favourite settings every time you restart the camera.
All of this relates principally to reasonable lighting conditions. There have been many questions relating to low light usage and how effectively the flash performs. Here are a few points I’d like to make in this regard. Firstly, where a backlit subject is detected and it’s deemed to be within the flash operating range, we fire the flash automatically to provide fill-in flash. If we detect the subject is outside of the operating range of the flash in a backlit situation we’ll leave the flash off. I’m not aware of any camera system that does this. Secondly on the performance of the xenon vs. the highly popular Nokia N82. In practice, you’ll find the flash range is greater than that of the N82. You’ll also notice that noise is lower but detail is higher. There will be some presence of noise as we’ve wanted to preserve more detail due to popular demand. We have a more effective red-eye removal system operating which is now running all of the time. And, of course, there’s also an improved face detection system which is linked to backlighting exposure and auto fill-in flash control.
On Friday, I’ll conclude this piece with a discussion of the video capabilities and settings of the Nokia N8.