PHOTOSHOP LAYER MASKS

PART 1 - strategy and making output in TIFF

My advice, look at your shot. First question I ask myself, is it worth the effort of post-processing ? Then I look at the quality of the shot, technically. Then I create strategy I'll employ when I'll be in Photoshop. Basically, I'm trying to visualize what will be the final outcome, what how to proceed to get there.

The answer is layer, since I've been using layer I never looked back. Before this, I couldn't conceive what was the reason and what is 'layer power'. So in C1, and from trial and errors I noticed that an output of TIFF/16bit yields to the most accurate, interesting results. With 8bit, I noticed under certain circumstance that skies could have banding in it.

So here goes in C1, I'll embed highly compressed screenies to illustrate key area. This photo is one of those that I never post-processed, so it will as a new one. Now we need to elaborate a strategy, look at the image, histogram and then imagine what you would want to create.

This photo is quite well exposed from the deep shadows up to the brighter trees, but the sky not being blown out, but lack details. We have a photo with potential :




To make sure the sky has potential, set WB to daylight. Just a note since I acquired a new Canon printer, I noticed my sky looks better with a slight tonal adjustment, being -3. I just want to point out that setting varies from PC to PC, choose what will fit your needs.

In C1, go in the exposure box, play with EC and or CC, try to look what would be feasible to achieve with a nice looking sky. In this case, I went down to -1 EC, that's a full stop faster than what I've done with the camera :




Don't be shy to look at possibilities, it's not a necessity to go at extremes. Going at extremes will yield to more headaches when you'll be in front of Photoshop. One of the headache being to try to 'match' the termination of trees into the sky. This is a real problem when you push EC to extremes on both ends. Tree termination to the sky can be a real pain.

If you've underexposed the shot, but have a perfect sky. What you would need to do is push EC on the positive side to bring details of shades, but this comes with a price : noise. Even at ISO100, if you push EC too high, noise will be a problem. It's been said before, expose to the right ! Just before there's blown highlights.

Back to C1, this scenery shots has quite a lot of colors in it, its a Fall shot. Part of your strategy, how about playing with WB to see how trees would turn out ? This is one of the option, the other option is by doing this in PS, with either action filters, or photo filters.

So my choice are done, I'll output 4 TIFF, which will be used in PS in layers and using layer mask. All at the same WB setting, same amount of sharpening, but EC at -1.00, one at -0.50 (for my dread tree termination into the sky), one for the normalized level of brightness of the trees, and for last one for the shadows at +0.50 EC.

Just a note, part of your strategy, it could also consist by layering different levels of sharpness (bird shots, macros are handy with this), different WB as explained above, different contrast, the list can go forever, only your imagination is the boundary ;)

PART 2

I assume you've played with C1 before, for those who have it. If you're using another program that plays with RAW, the same strategy can be employed, just differently.

So I've output 4 layers with only different EC. Just a note, this was taken in portrait orientation, when you open the TIFF in PS, the image will be wrongly orientated, just go on Image -> Rotate Canvas -> Rotate 90CW or 90CCW. I also noticed if you play with the orientation of a shot with the camera, the orientation in C1 and PS will be equally wrong, LOL.

Open the first image, rotate. For this example, it will be my 'background layer' , in most case my 'background layer' is not the sky, for this photo it is.




There's 2 possibilities here, again 'think strategy'. You could make a duplicate layer : Layer -> Duplicate Layer of the background layer, and play with Auto-Level (Shift-CTRL-L) or Auto-Color (Shift-CTRL-B), but most likely if the 'white' of the sky is not perfectly 'pure white', that is at the max level of brightness, the sky will get brighter, you'll loose details. My idea about this photo is not to loose details of the cirrus (type of clouds).
Here's a resize image of the 'background layer' :




The other possibility is simply to open the next output TIFF, which is named CRW_8479-1.TIF and then layer this to the original 'background layer'. Don't forget to change the orientation if needed.

Then all you need to do is Select All (CTRL-A) of the 2nd image, Copy (CTRL-C), click the 'background layer', then do a paste (CTRL-V). Close the other image, we don't need it anymore.




Now friends, the power of Add Layer Mask. This is quite important to understand, if you've never tried. There's 2 options to choose, depending what's the easiest way to get 'there' faster.
Layer -> Add Layer Mask -> Hide All
Layer -> Add Layer Mask -> Reveal All

The distinction between is easy to understand, Hide All is simply that the newly create layer is hidden, to bring part of that layer to 'life', you choose a white foreground color. Reveal All is the exact opposite. You'll need to mask part of that newly made layer by using a black foreground color.

I'm taking about using the Brush Tool for the foreground color. At this point select a background color opposite to the other (white -> black). If by mistake you do something terrible to your image, by switching foreground/background color from white to black (vice-versa), you can revert what you just been 'painting' over that layer.

For example, you're brushing away part of the image lets say Add Layer Mask - Hide All, so to 'REVEAL' the layer, you're using the white foreground color. You made a mistake at an area ? Switch it to black, and brush over that area, this will become 'hidden' again.

Now the brush tool, if you have never noticed, there's a lot of types you can use, personally I prefer the ones with a very soft edge. Choose one in the list.







So now, we've created a Layer Mask with Hide All, set the brush tool with nice soft edge, and chosen the proper foreground/background color :




PART 3 

We're now ready to work this image. My simple strategy was about different EC layered in PS. Never forget that each stage of your post-processing is always using the same principle (Layer, Add Layer Mask, etc).

Think of the possibilities unfolding... imagine you what you can achieve. Different WB, Different Contrast... Layer different hue/saturation, photo filters, sharpness method, blured background (birds, animals, macros), darker background (again birds, animals, macros). Sky is the limit !

I'll calm myself now, back to our photo and the strategy. This one features a large area where the 'trees meet the sky' Fortunatly on the right part of the image, the sky is mostly white.This will be easy to 'merge' both trees and sky together.

It's not always that easy. You may need to do this in multiple pass. What I mean by this is, if the trees are rather darker, tight foliage. If you simply brush away over the trees (remember this layer has a higher EC), you'll see a net termination where you pass over, and not. That's one of the reason why I'll use a rather large brush.

Doing multiple pass -make one pass, combine layers, paste again that layer - Add Layer Mask, etc... Pass over again with different brush opacity OR layer opacity. You'll have to experiment here. There's no magic formula, nor there's a magic PS action that can do this for you.
This area is where it can be time consuming, just want to let you know.

Back to the photo, I'll choose a brush size of 300px, soft edge and will start to brush away the right part of the trees. Remember, we want to keep the sky as is - for now at least. To be on the safe side, along the tree-sky termination, choose Brush opacity of 33%, work this with an image zoom of 50% of the original :




Notice this, bottom right of PS, where the black box is - which shows we're working Layer 1 in Layer Mask - Hide All : You'll see greyed out area, this is where you are 'revealing' layer 1 over the background. Tree termination being tricky, toggle Layer 1 on and off, you'll see if you are 'revealing' the sky also, which is not what we want.




Toggle this often as you're 'revealing layer 1 near the sky. Right here, I've overdone it, I'm switching with the black color from background to foreground, drop opacity of the brush to half it was - now at 17% and I'll paint over there sky became too bright :




So you'll have to work this for the entire image where the trees terminate with the sky. Just a note, the left part of the image has a very bright tree, you won't need to brush so much with such precision. Since the tree is lit quite much already. But the magic of a photo like this is colors and different level of brightness. My idea is to 'push the envellop' up to a point. So what I like about a photo like this is strong areas, either colours, deep shades or bright area.

Vary the opacity of your brush tool to key area, your taste and imagination is the boundary, LOL. Try to make a gradual and natural presentation of the photo. This is the hardest part. Post-processing without showing it was. Although we know the camera can't output anything like this out in RAW.

So 'reveal' the rest of the image bottom part at 100% opacity, at this point I'm not too concerned about trying to make nice deep shaddows area.

PART 4 
Look carefully at these 2 screenshots :







The top one shows Layer1 that has a huge part 'revealed' the next one shows by toggling Layer 1 on/off how it looked like originally. A miracle happened ! At this point we're already exceeding what an original JPG output off the camera could do.

Before we merge both layer, inspect what you've done especially near (again) tree termination and the sky. Here I'm quite satisfied.

Make sure Layer 1 is active (the 'eye' is there), and Merge Visible (Shift-CTRL-E)

Open the next TIFF, which is named CRW_8479-2.TIF in our example, look at how the newly merge image look and how it looks at 0.00 EC (off C1 it was), there's a huge difference in the sky details !




Same trick, we'll do Select All (CTRL-A), Copy (CTRL-C) of that TIFF. Then highlight the 1st image and Paste (CTRL-V). Add Layer Mask- Hide All.

Remember, the other layer we worked was mostly for the upper trees terminating with the sky. Lets now 'reveal' more the middle section of the photo, including the bottom. If there's nothing too critic, you can work at 33% zoom of the image, and a large brush tool. I prefer to work with less opacity, but I'll make multiple pass. The more you pass over an area, the more you'll 'reveal' that layer.
After you've worked this with the brush tool, toggle layer 1 off and on :







Again its a good idea to zoom in a little and look if you've overdone areas. Also, does this still look somehow 'natural' ? Purely speeking, it doesn't look natural since there's too many details in the trees vs the sky. But I have an idea... which I'll work later on...
Again same trick, Merge Visible (Shift-CTRL-E) when happy.

We'll employ the same idea for the last layer. This time I'll only 'reveal' the bottom part of the image, where the railway is in the shade.

Little comparison between our image that had been merged twice with EC of -1.00, -0.50 and 0.00 and this one at +0.25 EC. :




Ouch !! Major difference, isn't it ?

So same trick... Select All, Copy then Paste, Add Layer Mask, Hide All

And we'll reveal only section along the track. Just noticed the grass along the railway, and a nice red bushtree would be nice with a higher level of brightness, let's reveal that ! I'm repeating myself, opacity of the brush, between 33% and 50%.
Output of the image, before further work :




And again the 'original' image for comparison :

PART 5 

What we've done so far is time consuming and results are just starting to show. At this point, it will be like a puzzle where the missing pieces are easier to find. And what I've done so far keep repeating the same process over and over, nothing new to munch on :-P

Let's explore things out now, we may or not like it. But at least it will give us ideas.

For example, if your own photos had no such extremes between shades and the sky, you could do a simple Contrast Masking. This would have yield to an interesting result. This photo has no need for contrast masking, we've done it the hard way with multiple layers. Again the very reason why I went the hard way was to preserve the sky patterns and details.
About Contrast Masking, this is from Luminous Landscape :

http://luminous-landscape.com/tutorials/contrast_masking.shtml

Here's a simple method, you can actually record your own action btw, I won't elaborate on this, but its quite simple. Start to record an action and each step you do will be recorded...
Contrast Masking :

Duplicate Layer
Desaturate that layer (Shift-CTRL-U)
Invert (CTRL-I)
Set current Layer from 'normal' to 'overlay' :




Gaussian Blur 10 pixel

Play with the opacity of that Layer, also play with Fill until you have the desire effect :




A little idea here, why not Layer Masking this layer, so you could apply this selectively to area you want on the photo ? You know, it's just an idea... LOL
Finally, Contrast Masking had its use for this photo, ROFL ! :




And look carefully along the tree termination, and bright sky area to the right :




Merge Visible when happy, let's try now to push the contrast in the sky. My idea is not bring brighter white clouds, but bring 'deeper' blue in it, just slightly...

This is where experimentation begins (and the fun part). You have many choices, always work in layers, did I mentioned this before ? Hehe

My first choice would be to work with Curves (CTRL-M), Again we're concentrating on the sky only. Right away, I can see some areas where the trees meets the sky, and I can see there's spots where the sky is brighter than a neighbor area.




This doesn't look natural. I'll have to use another tool to correct this. Make a duplicate layer of the dup layer.

Since its the 'faulty' area is too bright, we'll use the Burn Tool , choose again something similar to the paint brush (soft edge). And lets correct this 'anomaly'. I tend to use very low levels of Exposure, this works the same as opacity- use highlights in the menu.




Now let's try to play the opacity of both layer until you have the desired effect, and no, no faulty areas... Again , we have to do this selectively by using Layer Masking - Hide All. Unless you want to apply the effect for the entire image, which I don't. So we'll 'reveal' only parts we want to change.

Play with opacity of each layer, which are both Layered Mask. Merge Visible (Shift-CTRL-E). There's something I don't like about the upper leaves of most of the tree to the left. It's too dark ! You could layer - add layer mask, and use the dodge tool, this time for the shadows.

If you're afraid of over doing it, don't forget about the black background color , just switch so it becomes foreground, paint over section and/or play with the opacity of that layer.

PART 6 

With your discretion, you can play with the tone of the sky. Personally I didn't like it. So make a layer - and adjust color balance (Ctrl-B), what I've done is played with the last channel, which is Yellow -> Blue, chose Shaddows and went to +10 for the Blue, then went midtone +5 for the Blue and finally Highlights +5 for the Blue and adjusted the top one Cyan - Red, -5 towards Cyan.

Added a layer over that one, and played again with Curves. Made the same type of shape as the previous one. Played with opacity of both layer. Then with yet again Add layer Mask - Hide All, worked the Color Balance, then the Curve (layer of each).

At this point, its time to make the photo more natural looking. That is using the Brush Tool (in layer mask, uhh ?) And generously apply more of the effect across most of the trees and sky :




This is all about personal preference, I like the way the top trees looks now, including the sky. We're done with that, woohoo !

How about the bottom part now, lets use a different way of working contrast, make a duplicate layer and now choose Soft Light, instead of Normal :




By playing with the opacity you can create a very dramatic photo, or just a tad level of contrast, again localy with Layer Masking.

What does Soft Light do is simple, it expands the gap between bright and dark pixels. It can be very subtle. In this example, I'll apply only to the bottom part of the photo - railway in the shade and a patch of grass.
Here's a look at the image so far :




Please note that I haven't touched saturation yet. We've only worked details, shades, contrast.

Now time to play with some action, there's many you can choose from, here's a link :
http://www.atncentral.com/download.htm

I've explained before, think its part 2, how to install.

Personally I'll use Paul's (Plbeic) : Provia.




Let the action run, this will create a separate image. At the end you have to choose the opacity, so this will be the level of Provia applied to that image. Personally I like to run this in the vicinity of 30%.




Since its a separate image, you'll have to do a Select All (Ctrl-A), Copy (Ctrl-C) and then paste this in your image (Ctrl-V)

Play with the opacity , I chose 70%, Yet again (you guys gonna kill me, LOL) Add Layer Mask - Hide All, and brush what you wish to reveal. Personally I think the sky is overdone (Provia that is). So I'll work only area I wish to push the saturation.

A little trick, you can apply different levels of opacity of Provia (copy this again) as you wish to different area of the photo.

LAST PART

We're ready to enter the final phase. This photo lacks general sharpness. I won't be able to make miracles but we can try. While it is still at the full size format and in TIFF, we'll apply a different way of sharpening.

Duplicate the layer and choose under Filter -> Other -> High Pass with a radius of 10.
Want to see where there's still errors in the photo ? Look here :




But even with a non-edit photo you'll see this... I'm talking again about tree termination and the sky. You could if you're that picky try another pass of contrast masking to that area again. Play with opacity and try to make some gain.

Contrast Masking would need to be done without the High Pass trick, do the High Pass trick at the end. Now the photo looks ugly ! Just switch from Normal to Soft Light.




Play with opacity of that layer, this will be the threshold of the sharpness. To retain somehow a natural look, think about hyperfocal distance, don't apply too much sharpness to far away trees. Ohh yes... Layer Mask is the trick :-P
With the Brush tool , of course !

Now you can save this TIFF for printing and/or crop. I won't crop this photo, looks good this way.

A little note, I've been working with a color space of Adobe 1998. I'll use this up to the last step.
Time to resize this for the web.

And it's time to apply USM, again I really like Layer Masking uhh ?

Duplicate Layer, Unsharp Mask with these setting :

300, 0.3, 0

Switch the layer properties from Normal to Darken. Adjust opacity between 25 and 40%, Layer Mask - Hide All and brush away area where you fancy to sharpen the photo.
Merge Visible, again Duplicate Layer

Unsharp Mask, choose lighten this time, reduce opacity between 10 and 25%, Layer Mask....

The trick here is, there's 2 thing happening. Not only you don't apply Unsharp Mask uniformly over the entire image. You split the filtering in half and selectively filter area of the photos using the darker and lighter pixels. This is powerful indeed !

Save the resized/USM image in another folder or with a different name and yet in TIFF.

Here's a bonus. Many are wondering why some have EXIF included in their JPG, and some don't. If you 'Save for the Web' with PS, you wont see the EXIF. My trick is simple. I open Photoshop Element, open the resized TIFF and save in JPG, that's it !
Don't' forget to convert the profile from Adobe RGB 1998 to sRGB

Special thanks to:
Eric Cote
Galleries :
http://gallery.bytephoto.com/DRHangar
Birds : http://gallery.bytephoto.com/... ...49&ppuser=87&password=&page=1
Macros : http://gallery.bytephoto.com/showgallery.php?cat=3154&ppuser=87