Test Methodology

In the following lessons we are going to test all the Picture Profiles against various color spaces. We will also test the Standard and Neutral Creative Styles for good measure. All this might seem a bit overwhelming, and this lesson exists to help you wade through it.

First, understand my testing methodology. It’s not scientific, but it’s good enough to get started.

Test methodology

I’m going to test each setting under five conditions:

  • Exposed correctly (0+0) in Spot Metering mode
  • Overexposed by +2 and +4
  • Underexposed by -2 and -4

The test scene is made up of three major components:

  • Left – 100% pure cotton black cloth, at -3 exposure – we’re looking for texture and noise in the underexposed regions here.
  • Middle – DSC Labs OneShot Chart. The two light skin tones combined has a 0+0 exposure. The middle grey (18%) patch falls at -1. We’re studying colors and skin tones here.
  • Right – 200W Light bulb at full power, metered at +10 (inside the bulb). The background scene is at 0+0. We’re looking at highlight roll-off and clipping.

Combined, the scene has a dynamic range of about 13-14 stops – which is the limit of the Sony A7s. Then, we are underexposing and overexposing it by – and + 4, which should stretch every creative style and picture profile to its limits, and beyond. There’s no better way to study them!

All images are exported as 100% JPEGs – good enough for study, and easy on bandwidth.

What should you do with all the information?

I’ve done all the hard work so you can sit back and start judging. To be honest, systematically testing a complicated camera like the A7s is pure drudgery. But looking at the results is fun, and it makes the whole excercise worth it.

Remember, I’m not infallible, no test is. Even if you feel strongly about one aspect or another, please conduct your own tests. This is something DPs must do, whatever they’re shooting.

The simplest way to study everything is to just open each image in an image processing program and look at them side by side. You can also open them in your browser. Because there’s a chart you can also use Resolve, but it’s overkill. That’s the point.

Eliminate the ‘yuckies’, the ones that don’t appeal to you at all. Soon you’ll arrive at a manageable number. When that happens start ranking the images in terms of personal preference, or whatever you feel is important.

Soon you’ll start seeing potential in some profiles for certain situations. Then you’ll realize that no one profile is perfect. Each has strengths and weaknesses. Some more than others, but you get to judge.

Don’t worry if you get bogged down. You don’t have to do all this in a day or two. Just get through the lessons and at the end I’ll give you my suggestions, what I see and what I feel. A handy exposure chart helps too!

That should do it. For now, let’s take it easy, and follow my lead.

FAQs

These are important topics raised by subscribers that shed more light on this lesson.

Q. Can I use Photoshop or ACR or another stills application to compare images?

A. No. The application in question should be able to work with the color spaces Sony provides. As far as I know, there are no applications that work in S-Gamut or Hypergamma or Cinema or Pro color spaces. Most of them only meet Rec. 709.

Under no circumstances should you use sRGB or Adobe RGB for video comparisons, tests or tweaking.

Q. So how do we know which app to use?

A. Simple! Only use an app/monitor that has a built-in waveform monitor, and can display the full range from 0 to 109 IRE. Examples:

  • Most professional NLEs
  • Resolve, Speedgrade
  • Color Finesse within After Effects
  • Atomos Shogun, Odyssey 7Q+

Video apps (like photo apps too) work in a native color space. Most modern apps work internally in 32-bit, and some can even work in linear gamma (1.0). The point is, you know what you’re doing because you have the help of scopes – waveform, vectorscope, RGB parade, etc.

Photo apps don’t have these tools, so you never really know whether what you’re seeing is what you’re getting.

Click on the link below to the next lesson or head over to the main menu (above). If you need help with something, feel free to send me an email. I’ll try to get back to you as soon as possible.