I wanted to come up with a methodology for testing lenses that could be used with any format, either film or electronic. It was important, I felt, to keep the variables at a minimum, which meant using a single camera and single set up. Once the lights were set, I wouldn’t change them. I accomplished all exposure compensation by changing the lens iris and camera speed. Using two cameras next to each other to test more than one lens at a time would produce variations, especially with electronic capture, where there are so many settings and adjustments to contend with. The goal was to compare lenses to each other and eliminate differences due to camera settings, or changing the lights. In this test I specifically wanted to look at Super 16mm format lenses and how they hold up for transfer to HD.
The test was twofold: in part one, I set up a test chart and filmed the chart with each lens at different apertures to see how well each lens resolved. In addition, I placed objects in front of and behind the focal plane. I also placed illuminated light bulbs in the background. This was to get a sense of how the lens handled out of focus objects and highlights, also known as ‘bokeh’.
The second part of the test involved filming a model and seeing how the model looks in the lens. White and black backgrounds behind the model show if there is any ‘bleeding’ of the white into the black. In the end, this test may be more important than the resolution chart, as no one goes to the movies to watch a test pattern. I was ably assisted by Jonathon Greene, Nina Lavin and Jae Yawn Lee, who were charged with helping set up the lights and charts and keeping notes.
Ideally, before every shoot, you have your lenses collimated to the camera. However, in this test, as I was borrowing many of the lenses, I chose not to have the lenses collimated to my camera. It is not uncommon for lenses to be changed during production, or borrowed, or adapted from one mount to another, so the lack of collimation of the lenses to the camera was reflective of some ‘real world conditions’.
The prime lenses were adapted to PL mount from Arri bayo mount by use of quick-release adapters. Zooms were a combination of Arri bayo mount, Arri PL mounts and fixed bayo to PL adapters. The 5.7mm lens is an Arri standard mount with a fixed adapter to PL.
Often lenses are adapted from one mount to another. Either the lenses are no longer manufactured, or are specialty lenses, or it is just not economical to have lenses in every mount for every camera. There are some lenses that have a universal mounting system with interchangeable mounts; these are usually extremely long lenses (20 mm and longer.)
The goal of this test was not to find the ‘best’ lens. I certainly didn’t test every lens available. However, a similar set-up can be created and the test repeated. See notes for light readings and distance measurements. In this test, I wanted to compare coverage of the Super 16mm frame, sharpness, resolution, color, flare, intercutability, bokeh and how well each lens performs both technically and on a human face.
The lenses
A mix of Cinema Products’ Ultra T Prime lenses originally designed to cover Standard 16mm, Optar Illumina primes designed for Super 16mm, a Kinoptik 5.7mm designed to cover 16mm, but known to cover Super 16; Zeiss and Angenieux standard 16mm zooms, as well as two Zeiss zooms converted to cover Super 16. Lenses were labeled alphabetically for identification purposes.
A 16mm Ultra T – ser#60053
B 9mm Ultra T – ser#90191
C 25mm Ultra T – ser#50071
D 5.7mm Kinoptik – ser#8546
E 9mm Ultra T – ser#90090 (backup)
F 16mm Optar – ser#9521618
G 25mm Optar – ser#9522513
H 12mm Optar – ser#9521218
I 9.5mm Optar – ser#9520915
K 10-100 T2 Zeiss – ser#6370130
L 12-120 T 2.4 Zeiss (Optex converted) – ser#6369645/1
M 10-150 Angenieux – ser#1436377
N 11.5-115 T3.1 (Converted) – ser#57532 – see note 7
R 12-120 Angenieux – ser#1371031
The camera was an Aaton LTR converted to Super 16, with a PL mount. The film stock used was 7212. The chart was lit by two inkies at a 45deg angle. Each inky was measured independently to have the same light reading, and there was an 80deg color temp difference between the two (2900 and 2980 Kelvin.) The film was processed normal, and transferred/graded to a grey card shot with lens K, and no further correction was applied to any of the footage.
In test one, a ‘Putora Sharpness Indicator’ test chart is used. The recommendation of the chart is to photograph it at a distance that is 60x the focal length of the taking lens. Following this recommendation, not only can you compare lenses of the same focal length to each other, but you can also compare lenses of different focal lengths and get consistent readings across the focal lengths. The camera was locked off at the proper distance for each focal length and each lens of that focal length was tested in the locked off position before going on to the next focal length.
The base stop setting was T/2 at 24fps with a 180deg shutter. The camera speed was controlled by an external crystal speed control to keep exposure constant as we changed the lens iris. To shoot at 1.3 (wide open) the camera was run at 48fps, for T/2 24fps, T/2.8 12fps, T/4 6fps, and T/5.6 3fps. Although this meant varying exposure times, the tripod was weighted, and vibration kept to a minimum during filming. This method was chosen as opposed to adding and removing scrims to the lights, as it provided a more consistent exposure, with less chance of disturbing the quality and position of the light. I also didn’t want to introduce filters into the optical path to affect exposure. I didn’t feel that the increased/decreased exposure times would impact on the test results.
The film was processed normal and transferred to DVCAM, courtesy Postworks. The film was also later transferred to HD resolution (1920x1080 Tiff sequence on a one-to-one frame basis, no ‘pull down’), courtesy Rhinoceros/MultiVideo Group.
Evaluating the test
It is necessary to view the footage to really see what is going on with the lenses. I’ve shared my conclusions below, but please examine the footage and draw your own conclusions. Tiff file framegrabs will also be available online.
In the first test, examine the Putora chart on the previous page: there are a series of ‘circles’ with increasingly smaller line pairs. Obviously, the smaller the line pairs that are visible, the sharper an image the lens can resolve. Throughout the test, as the iris closes the lens gets sharper. With the sharpness chart you can also look at the grayness of the chart and see color shifts. You can examine the large black areas of the chart and see how black or contaminated the blacks are.
Outside the lens chart you can look at items in the background and foreground to get a sense of the focus fall-off. Forty-watt bulbs were set up in the background for this purpose, but are not visible with every lens. With the zoom lenses the tests are repeated twice, once with the lens at its wide end, and once with the lens zoomed all the way in.
In the second test I’m more interested in how the lens actually photographs a face. It is all well and good to have a technically perfect lens, but if it doesn’t give me the look I want, then it really is of no use to me. I set up a white and black reference behind the model. This is to help evaluate how well the lens is ‘separating’ the person from the background; you can also notice the white and black references in the background: are they pure, or have they been contaminated by the color of the lens? How is the lens handling the crossover point from white to black? Is there bleeding or flare? Note the detail in the model’s hair: how does it change from lens to lens? How does the model look, how is her skin tone – rich and deep, or thin and washed out? How are the hot spots – is there detail or are they blown out? Is the image flattering and appealing?
The results
I first looked at the standard definition (NTSC) DVCAM transfer and there are noticeable differences. It is easy to tell that neither of the 9mm Ultra T lenses covers the Super 16mm frame. Neither do the non-converted zooms. Looking at the Putora chart, it is easy to see the resolving power of each lens as the iris changes. Shifts in the color of the lens and contrast are also noticeable.
However the differences are not huge, and color shifts appear mild. The differences in the lenses become truly apparent and jarring when looking at the HD Tiff frames. I was lucky enough to view the Tiff frames on a monitor capable of 1920x1080 resolution, so I could see a pixel-for-pixel rendition of the frames. Looking at the primes you can easily see contrast differences, with the Optar lenses clearly outshining the Ultra T lenses in terms of color and contrast.
The Ultra T lenses tended to be a little bit more yellow compared with the graycard, with the Optars maintaining more of the bluish cool look that is mostly favored these days. The contrast of the Optars is significantly higher than that of the Ultra T lenses, with richer blacks and cleaner separation between the black and white areas of the test chart. The Optars resolve better wide open than the Ultra T lenses. The 25mm Optar also had a noticeably wider field of view than the 25mm Ultra T, which can be easily seen by comparing the top and bottom of the frame.
What is also obvious is that the test supports the ‘truism’ about the sweet spot of the lens. In the case of the tested lenses, all the primes behaved in the same way, with best resolving performance beginning at two stops closed down from open. Closing down to T/4 or T/5.6 didn’t significantly improve the sharpness of either the Ultra T or Optar lenses beyond T/2.8. Although it is possible to shoot wide open (approximately T/1.3) on these primes, that isn’t going to give you the sharpest results.
In the second test, the model is filmed twice with each prime lens – the first time intentionally overexposed by a stop (proper exposure was T2.8), to see the effect of overexposing with each particular lens on the image. It is interesting to note which lenses hold the image quality better, even when overexposed. Granted, color correction would improve the images, but the uncorrected images illustrate how the qualities of each lens affect the image. For zoom lenses, the model is filmed with the camera close to her, with the lens zoomed wide, and then also zoomed in. This was repeated with the camera about 20ft away, with the lenses at the wide end, at a middle position and zoomed all the way in. Note that due to the age and design of some of the lenses, it was not possible to shoot at the desired stop, so the camera’s speed was adjusted to help compensate, although there were times that the image was left underexposed.
The second test was shot handheld, so frame rates below 20fps were not used in order to not blur the image with long exposure times and a moving camera (except in the case of Tiff frame #08 – lens N, where we shot at 16fps).
Looking at the primes, the Optars provide a very nice, clean color rendition. The skintones seem very saturated compared to the Ultra T lenses. That’s probably not surprising when you consider the age of the Ultra T lenses and the recent advancements made in lens design and coatings. This does not mean that all Optars perform this way, or that all Ultra T lenses perform the same. If you compare the images from both of the 9mm Ultra T lenses, you can clearly see that one out performs the other, both in image coverage, sharpness and contrast.
Interestingly, the yellow cast of some of the CP Ultra T lenses is not objectionable on the model, which most likely has to do with her skin tone. One of my favorite aspects of the second test is comparing how each lens distorts the image. The 5.7mm, for example, really is quite interesting for its distortion. Unfortunately, while looking at a still image one cannot really tell how wonderful or objectionable the lens distortion is unless you are watching the image in motion.
Looking at the zoom lenses, a few things become clear. I don’t like the image created by the converted Zeiss 12-120 (lens L). The image looks flat and lifeless, and is disappointing. Interestingly enough, the lens seems faster than the rated T stop. I’m also unimpressed with the 10-100 T2 Zeiss. It is an older Zeiss, but there is nothing that really grabs me about the image. On the other hand, the converted T2.8/11.5-115 (lens N) does have a nice quality to the image; it is very rich in tone, with deep blacks and saturated color. Sadly, once the lens was converted, it became a T 3.1 – very slow, and mechanically the front of the lens turns, which makes it difficult to use with a clip-on matte box. The Angenieux 10-150 provides a very poor image. It is very disappointing – there is no snap, no interest. Granted it is a very old lens, and may not be representative of other 10-150 lenses; however it does bear out the importance of actually shooting a test to evaluate lenses. Oddly enough, the 10-150 covers the Super 16mm frame at the wide end if you are focused on infinity; as you zoom in, the frame coverage shrinks, and then starts to expand again.
The surprise of the lot is the Angenieux 12-120. Although this lens does not cover Super 16 at the wide end, the lens created a very flattering image, and when zoomed in all the way covers the Super 16 frame, and is very pleasing, even though as borne out by the Putora Chart test, it is not as sharp as the Zeiss zooms. So sharpness is not everything, and even though the much newer converted T/2 Zeiss zoom is sharper than the Angenieux 12-120, in this case the Angenieux delivers a nicer image.
I repeated the test with a Canon digital still camera, as well as a Panasonic DVX100. To wrap this up, just buying a lens is no guarantee of how it performs. Lenses vary from manufacturer to manufacturer, and even across batches of lenses from the same manufacturer. Time is another element that affects lenses. Even perfectly maintained lenses age, and older lens coatings can yellow. This is easily noticeable when comparing the two CP 9.5mm lenses. Not only do they have different coverage of the Super 16mm frame, with neither covering it entirely, but also there is a huge difference in the color of the image under the same conditions. It is extremely important to the final image to thoroughly examine and choose any lens you use, because it is the glass that is the key component in capturing your image. So remember, not all glass is the same, even from the same manufacturer, and each lens imparts unique qualities to the image. The only way to know for sure how the lens ‘looks’ is to shoot with it.
Testing can help you avoid unhappy surprises, and really deliver the ‘look’ you want.
Notes and errata
1. It is possible that varying the camera speed affected the image on the film. However, one would expect that it would be consistent over the process for each lens. So any differences would only show up when you compared a lens to itself over the range of iris settings, and not between lenses. The studio (my apartment) was not isolated from vibration, so it is possible that random vibrations from passing traffic would affect the image as well.
2. Another option for this test would have been to use a modified PEN F, with a PL mount. This is a half frame 35mm still camera. Although it would be difficult to check for coverage in a Super 16 mm frame, it would still be valid for resolution and saturation.
3. Future tests in film should include Zeiss Super Speeds, Cooke S4, Arri Ultra and Master Primes, as well as modern zoom lenses.
4. Future tests should include additional charts in each corner of the frame, as well as the center chart.
5. The model arrived before the first part of the testing was completed. The lights and tripod position for that part were left untouched and we lit for the second part of the test with different lights in a different part of the apartment. When that was completed, we returned to finish the first part of the test. Slates and notes were used to ensure that selected stills pulled from the HD transfer were accurate, and presented in the order of test part 1, then part 2.
6. Not all shots were slated, however notes were taken for each shot, the camera speed and lens iris setting confirmed visually and verbally before and after each take. Any deviations or errors were noted in the test log.
7. Lens ‘N’ – I was unable to determine if Arriflex or Van Diemen did the conversion of this lens. The conversion, however, was a permanent conversion, as opposed to the ‘removable’ Optex conversions used on the Zeiss 10-100 T2.
8. As mentioned earlier, part 1 of the testing procedure was also completed with a Canon digital still camera and a Panasonic DVX100. I’m choosing not to include the footage of the DVX for comparison. It wouldn’t be an equal comparison, as not only would it be an evaluation of the lens, but also of the camera’s CCD sensor, the compression scheme, and any of the camera’s internal menu settings. This is why when testing lenses it is important to isolate camera variables such as film stock, camera registration, light adjustments and differences in electronic capture mediums and format.
9. It is interesting to look at the SD video transfer, and compare it to the HD transfer. A companion piece to this article is the comparison of the footage transferred to SD video, with the footage transferred first to an HD Tiff sequence and then converted to SD.
10. The apartment I shot the test at was undergoing major renovations at the time, which explains the disarray. See the accompanying still images.
Acknowledgements
Production Crew: Jonathon Greene, Nina Lavin and Jae Yawn Lee
Production Stills: Andre Walker
Model: Melissa Walker
Article Review: Ira Tiffen
Film Stock: Steven Garfinkel and Geoff DeMoss, Kodak NY
Developing and transfer to DVCam: Domenic Rom, Post Works, The Lab NY
Transfer to Tiff files at HD resolution on a Sony Vialta: Rich Torpey, Rhinoceros/MultiVideo Group NY
Hi Def Viewing Monitor: Mitch Gross, Jesse Rosen, Abel CineTech, NY
Additional Lenses: Mitch Gross and Mark Smith
Conversion of camera to Super 16 and “PL” mount: Alan Giles, Camera Engineering, UK
Tester: Steven Gladstone
 |
Steven GladstoneSteven Gladstone is a New York-based DP who has shot numerous commercials, music videos, short films, and feature films over his 18 years in the industry. He has a Masters degree in Communication Arts, and will be making his feature film directorial debut in 2008 with his script, Winter Money. For more info see www.gladstonefilms.com |