Experiments in CGI

In order to be able to render scenes in Yafaray with a realistic background and just for sake of learning I decided to create a HDR (high dynamic range) image of the inside of my apartment. The fact that it’s HDR is important because it means it can capture the fact the the window is a lot brighter than the rest of the room and this means the HDR image can actually be used for lighting as well as a background, i.e. the lighting effect of the window is simulated when performing a render.

The first step is to take photos of the scene in all directions so they can be composed in to a panoramic image. The important thing is to have the centre of perspective of the camera in the same position for evey photo. This ensures there are no parallax effects between shots. The centre of perspective is actually the entrance pupil of the lens (http://doug.kerr.home.att.net/pumpkin/Pivot_Point.pdf). The ideal way to ensure this point stay in the correct place is to use a tripod and a panoramic head. These together allow you to take a picture in any direction while maintaining the centre of perspective in a fixed location.

However I decided to keep things simple and not use either of these tools, and instead used a circular table with a top I could pivot around its centre point. This however restricted me to taking pictures around the horizontal plane, nevermind, this is the most interesting area anyway.

Finding the centre of perspective

The simple way to do this it to position the camera in your panoramic head (on the table in my case) in roughly the right place then line up two items in front of the camera. For example, I used a chair infront of a door frame. Now rotate the camera on its pivot slightly to the right and look through the view finder again, if the closer object has moved to the left the camera’s centre of perspective is infront of its pivot point. You should move the camera back slightly. Or vice versa obviously. By repeating this you should be able to get the camera in a position where objects which are lined up in one shot stayed lined up no matter how you rotate the camera.

Taking the photos

When taking the photos ensure there is enough overlap between each pair of adjacent pictures to be able to smoothly join the pictures later. The important thing is not just the area of overlap but also the amount of features in the overlap. For example an overlap is no use if all it contains is bare featureless wall (I managed to fall in to this trap).

The other factor as we’re planning on using these pictures for rendering is exposure bracketing. You want to make sure that for every angle you take a picture at, you have a series of photos which covers the range of exposure necessary to capture details in the parts of the scene of all light intensities. For example, this is the series of photos I took for the part of the room where the window is:



Another thing to be wary of is not to take too long, as lighting conditions could have changed by the time you back round to where you started (this also happened to me). If just doing a horizontal plane panorama like I did, I would recommend starting at the place least likely to change by the time you back round to it.

Stitching the photos together with Hugin

Hugin, is an opensource tool for photo stiching. I’m not going to go in to massive detail on how to use this tools. But the gist is as follows:

1. Start a new Hugin project, add all the photos you took of the scene

2. For each set of exposure bracketed photos mark features in each photo which correspond to features in the others

3. For each pair of adjacent photos with overlapping sections, mark features in one photo that correspond to the other

4. Allow Hugin to match up the photos into a panoramic composition.

5. Preview the result to check it is ok

6. Use Hugin to generate a final EXR image

7. Use qtpfsgui to view and check the created EXR

If only it had actually been this simple for me. The feature matching process, is very easy, and Hugin’s interface is very easy to use for this task. The alignment solving also went relatively smoothly. The problems began with EXR output.

Viewing EXR files

EXR is the format specified by OpenEXR, it is capable of storing pixel values at much greater precision, over a greater range of intensities than LDR formats like jpeg and png. qtpfsgui is a tool for viewing and tone mapping HDR images. Its tone mapping features are very useful for converting an HDR image to an LDR one. However its also very useful simply for viewing EXR files as it has a tool for changing the exposure of the current LDR preview, this allows you to see both the bright and the dark parts of the image.

I did however find a bug in qtpfsgui 1.9.3 (the latest at time of writing), which meant I couldn’t save tone mapped images. However the fix was easy and I’ve attached the patch file below.

qtpfsgui 1.9.3 patch

Hugin uses a tool called hugin_hdr_merge to merge together images taken at different exposure levels. The first problem I came across with this was a strange black blobby artefacts on the left had side of the window.

However after some investigation I came to the conclusion this was because I had started taking photos at the window, and my very last photo also overlapped with this left hand edge of the window. But by the time I took the last photo the sky had become significantly less cloudy which was presumably somehow confusing the EXR merger. I fixed this by cropping the last image to not include the window part of the scene. (There is a cropping tool in Hugin, it doesn’t actually affect the source image, it just determines which parts Hugin will use)

The second problem was that EXR merger seemed to introduce a lot of noise in certain places. In particular it seems to happen with exposure bracketed parts of the scene, in parts of the image which are very dark in less exposed photos. The worst place is on the sofa arm as can be seen here:

A similar problem was that noise sometimes appeared in high contrast areas, for example around the window frame.

These problems were quite frustrating, because I knew if they had occurred in an LDR image I could easily open them up in the gimp and touch up the noisy areas. However I didn’t have any tools for editting an HDR image. I tried for a while getting CinePaint to compile and run, but I ran in to too many problems getting this to work and decided to create a simpler solution, exrunfuse.

Exrunfuse

Exrunfuse (EXR unfuse), is at tool I created using the OpenEXR libraries to edit EXR files using normal LDR tools like the gimp. As command line arguments exrunfuse takes an exr file, and a range of EV values in to which to decompose the image, and a prefix to use for output files. It then writes a series of png files, each exposed at a different level. For each exposure level, 2 identical files are written, one which you may edit, and one which is kept for reference.

When editting a particular part of the image you should choose the png file which has the most appropriate exposure for that part of the image.

Once happy with the edits, Exrunfuse can be invoked again to recombine the editted png files back in to an exr. When it is invoked to re-fuse you should also pass in as an argument the original exr file. Then, for each png file it compares the editted version with its original reference version, and for each pixel it finds to have been editted it copies the editted pixel in to the new EXR file (taking into account the exposure settings and gamma), uneditted pixels are copied directly from the original exr file so no loss of precision occurs.

This is the exrunfuse source:  exrunfuse-0.1.tar.gz

Memory issues

The ideal size for EXR background to be used in rendering is actually very big. If you assume you want to make a render using a virtual camera with a field of view of 45°, and you want the rendered image to have dimensions of 1024*768, then that works out as 22 pixels per degree. To have a crisp background you want at least one pixel of background per pixel of render so that means the background also has to have 22 pixels per degree as well. Which for a 360° by 180° background file works out as 7920*3960. So just to make it a nice round number I went for 10000*5000.

However when trying to do the full Hugin stitch, I got an error message from Hugin saying it was out of swap space. However after a bit of googling I discovered this doesn’t actually refer to virtual memory swap space, but to Hugins own swap files that it creates in the /tmp directory. My solution was to change the TMPDIR environment variable to point to a device where I had more space.

The ceiling and the floor

Up until now, theres been a huge black void representing the up and down directions in my panoramic images:

Ideally I would have taken pictures in these direction too (infact I did take some but without a tripod and panoramic head there was too much parallax error to smoothly merge the photos in). So instead I’m just going to bodge in a color (using my exrunfuse tool again) above and below to approximate the lighting:

Obviously this is fairly crude, but it should suffice as long as these parts of the image are not directly visible to the camera. The most significant part of the scene lighting-wise is the window so these approximations should go unnoticed. I couldn’t have just left them black though as this would have been too obvious in the lighting effects.

Results

So here are some images rendered using this HDR background. They’re all rendered using Yafaray with direct lighting.

This first image only shows a small part of the background, but demostrates the use of the background for lighting.

The HDR background is the only light source in this image, so in practice the window provides the bulk of the light, while more is provided by the walls in the background which are reflecting the light of the window. Notice how soft and realistic the shadows on the table look.

This next image is the same scene but with a depth of field, the aperture size used was 0.1.

Note that to get a good DOF effect more anti-alias samples are needed, but also bear in mind that as you increase the number of anti-alias samples you may decrease the number of IBL samples used and still get a similar quality result.

Finally in this image and movie I’ve tried to demonstrate how even though the window appears over-exposed in the scene generally, detail can still be seen when the light is reduced due to passing through dark glass or reflecting off a dark surface. This is the way a real camera would behave.

Ofcourse, if you actually want to be able to view the whole dynamic range of the scene, you can save the rendered scene as an EXR and use qtfpsgui to tone map it:

The following images demonstrate ambient occlusion. Ambient occlusion adds colour to every point in the scene based on how much space there is around it. Note that depsite being called ambient occlusion it actually makes the scene brighter. You can lessen the effect simply by choosing a darker colour as the ambient colour, e.g. grey instead of white. In each of the following images 32 samples were used for ambient occlusion and the depth was varied.
Depth = 1

As you can see almost everywhere in the scene is more than 1 unit any from something else so the ambient occlusion makes it totally white. The only exceptions are in the corners.

Depth = 10

As you’d expect setting the depth to 10 makes the scene a bit darker and a bit more realistic but there are still too many places further than 10 units from anything else.

Depth = 100

Depth = 1000

These values start to make the scene more realistic, obviously the depth required depends on the scale of the geometry in the scene.

Really ambient occulsion should be combined with regular light sources, these images have just been for demostration purposes.

And for comparison, this is the same scene rendered with no ambient occulusion, but with a skylight with no sun and 32 samples:

And with photon mapping:

The photon mapping image looks rather washed out, but this could be because of the amount of white material in the image. In fact one of the advantages of photon mapping is the way colour can “seep” from one surface to another due to diffuse reflections.

Here is the same scene with a bit more colour, first rendered with ambient occlusion in direct lighting, and secondly with photon mapping:

The photon mapped image still looks too dark, this is probably just because the sky isn’t bright enough. The colour seepage effect can still be seen though, at the base of the light blue diagonal box for example.

And finally this is the photon mapped image with a sun light too:

In the following images the scene has a sun sky enabled (this is setting in the background drop list in the world settings). The sunsky can be used as just a background but it can also be used as a light source.

In direct lighting mode the skylight button must be selected to enable the background to work as a light source. Basically for every surface point n points in the sky are randomly selected and these are treated in the same way an other light source is. Ie. a shadow ray is traced between the point on the surface and that point in the sky. (The sky is infinitely far away so really every point on the sky is just a direction). The variable n I mentioned just now is the number of samples to use and is setable in the control panel, the control is simply called samples next to the sunsky option.

In the following image 6 samples are used.

Increasing the number of samples make the result appear smoother, but naturally costs more time in rendering. In this image 50 samples are used:

In the images the ball is coated glossy. One of the red panels is simple diffuse and the other entirely transparent with transmit filter set to 1.0. Note that as you’d expect the shadows appear nice and soft, and the one beneath the transparent panel is tinted red.

If we select the add real sun option then a sun light source is added as well, giving crisp shadows:

For comparison this is an alternative sun sky, the Darktide sun sky:

Note that both types of sun sky are very configurable (Darktide more so), and the images above do don’t do justice to what can be achieved with them.

Translucency

In order to experiment with the translucent material I created a scene shown below. The same shape is repeated 12 times. Each shape consists of 2 L shaped pieces, one of which has thickness (the one on the left), and one of which doesn’t (it’s just modelled as 2 simple rectangles). In the scene there are also two long bars in the distance, the top one of these is translucent and the bottom one is glass with fake shadows enabled.

After a bit of experimenting it seems that in direct lighting, which is what we are looking at here, translucency is quite limited. Shadow rays do not pass through translucent objects even with transparent shadows enabled. Caustic photons seem to not pass through either. The only way in which translucency seems to have an effect in direct lighting, is that for translucent objects shadow rays are cast from the opposite side of a surface. This means translucent object which have any thickness do not behave in a translucent way.
The material properties are as follows:

The remaining material properties vary between the items and are listed in the table below.

No transparent shadows:

Transparent shadows:

Above you can see that no light passes through the shapes with thickness at all. You can see surfaces without thickness that shadow rays are cast from the back surface of the surface aswell, hence you can see the shadow stripes of the 2 bars in the background. Note that in the first image with transparent shadows turned off both of the bars in the back ground look the same, i.e. very dark as no the shadow rays do not pass through the bars. But in the second image the shadow rays do pass through glass bar, but not the translucent one and hence this shadow still looks black. You can also see that none of the long shadows in the foreground look have been made lighter by shadow rays passing through the translucent objects.

You can also see that in numbers 10,11,12 the shapes with a higher translucent value actually look darker on top than the ones with a lower translucency value. This is because the translucence value controls the mix between shadow rays being sent out of the front face (the one normally seen by the camera), and the back face. In number 10 the shape appears black on top, because all of the shadow rays are cast from the underside of the top panel, from where they can not reach the light (the lamp is higher than height of the top panel of the shapes). However as you’d expect number 10 has the brightest illuminated section on the front panel, again because all the shadow rays on this panel are sent from the opposite side which is facing the light. I can’t actually imagine any materials in reality that behave as shape number 10 (no light whatsoever is reflected yet light is transmitted from the other side). I don’t think you would ever want to use a translucency of 1.0 to model a real material. I would guess that 0.5 is probably be the limit of real world materials.

Note also that transmit filter has no effect here.
In the next experiment I removed the strips in the background and added a glass ball behind every item to see if caustic photons cast on the back of a translucent surface appear on the other side.

Obviously they do. In this next image all the settings are the same as in the previous but I have moved the camera to behind items 10,11,12. Note that #12 is now on the left side of the image as we are viewing from behind now.

Note how unrealistically dark number 10 is. All the light is appearing on the other side. Not even the caustic photons are visible on this side.

Transparency

Material properties:

And with transparent shadows turned off:

Transparent shadows back on again and with shadow depth set to 1:

With shadow depth set to 2:

With ray depth set to 0:

With ray depth set to 1:

With ray depth set to 2:

Finally, let’s put a glass ball above each of these balls to demostrate what happens with caustic photons. (The ray and shadow depth are both set back to 5).

So we can tell from all this that a ray going through a transparent surface is a ray forwarding event which uses up one hop of the ray depth. The ray is coloured according to diffuse color but modulated by the transmit filter. Shadow rays also use up a hop through each surface and also gain color in the same way as the primary rays. Caustic photons are also colored the same way.

Mirror

In the following image the 4 balls have the following properties:

With out fresnel the reflectivity of the surface is constant, independant of incidence angle. With fresnel the surface is more reflective at angles which graze the surface. This is more realistic for a lot of materials.

It is tempting to associate the lambert/Oren-Nayar control with the reflection parameters as its just below them in the control panel. But in fact it is related to diffuse colors. We will examine this in the next section.

Diffuse

In the following image the 4 balls have the following properties:

The only variation available with pure diffuse materials is the Oren-Nayar option. It is a reflection model which more accurately reflects the behave of rough materials, concrete for example. The sigma value models the “roughness” of the material.

In the following scenes all the glass balls(radius = 5) have the following material properties:

The remaining material properties vary between the balls and are listed in the table below.

All the scenes are rendered with Yafaray svn version 432, using direct lighting. There is one spot light in the scene above the balls emitting white light. We will now look at the scenes rendered using different direct lighting settings. This first image is rendered with very limited settings, for illustrative purposes.

As the ray-depth is set to 0 all rays from the camera hitting the balls are terminated there, as no more bounces are allowed (ray-depth is the number of reflections/refractions permitted) and so these rays appear black. Next we increase the ray-depth to one.

Now you can see on the underside of the balls there is a reflection of the striped surface. There is still no refraction visible through the balls because for a ray to pass throught the glass 2 refraction events are needed, one as the ray enters the glass and one as it leaves the glass. So now we increase the ray-depth to 2.

Now we can see through the balls. Note the numbers 1,2,3 & 7,8,9 appear identical. This is because the filter color and transmit filter are tied together, when the filter color is white, there is no filter effect and so no value of transmit filter does anything (numbers 1 & 2), and when there is a filter color having the transmit filter set to 0 means it has no effect (the transmit filter is used to modulate the filter color).

The other thing to note is that balls 5 & 6 appear different. In both of these cases the transmit filter is set to 1 but we still see a difference when the aborption color is changed. This is because absorption always has an effect even when the transmit filter is set to 1 (this is contrary to what it says in the yafaray documentation).

It may be a surprise that ball 6 appears appears darker than ball 5 despite having a absorption color which is closer to white. This is because in ball 5 the trasmit filter is set to pure green, and the transmit filter is set to 1, so it actually stops all light except green light from passing. This means that in this case the only relevant channel in the aborption color is the green channel, which in the yellow aborption color was actually full (0xFF), meaning that no light is absorbed at all. In ball 6 the absorption colour is almost white, but in the important green channel it is actually darker (0xF2) than in the yellow colour and so some of the green light is absorbed, making the ball look slightly darker.

Now lets try and get some shadows. One way to create shadows is to use “fake-shadows”, this is a material property (which the first 6 balls have enabled), but in order for it to work transparent shadows must be enabled in the render settings. Fake shadows work by allowing shadow rays to pass straight through (unaffacted by refraction and absorption) objects. The shadow-depth render setting determines how many such surfaces a shadow ray can pass. Since a shadow ray would have to pass through two surfaces of our balls to get to the light, in the render shadow-depth has been increased to 2, also fake-shadows has been enabled.

Note that absorption color has no effect, but the filter color behaves as you would expect. Now I’m going to decrease the ray depth just to show that this shadow effect is not dependant on this parameter.

Ok, now lets set the ray-depth back to 2, and turn on caustics.

Again you can see that in balls 7-9 the filter color had no effect as it was disabled one way or another, but in 10-12 it does have an effect. Also obvious is that the absorption color has an effect in all the caustic balls. Note that caustics do not pass through the balls with fake-shadows enabled.

Lets increase the number of photons, to smooth out the effect.

Much better. Now lets experiment with the caustic depth, this is how many bouces/refractions a photon may go through before reaching a target. Lets set it to 0.

As you’d expect, no caustic effects are visible. Lets set it to 1.

I’ve also inserted a curved mirror in to this scene to illustrate that now photons can make one reflection, so you get a pleasing caustic effect of the mirror, and you see some speckling on the ground underneath the balls which is slight reflection of the outside of the balls. For photons to refract through the balls, a depth of 2 is required as a seperate refractive event occurs at entry and exit of the ball.

You may also notice that in the reflection in the curved mirror you cannot see the any of the balls. This is because ray-depth is set to 2, and in order to see light from one of the balls in the mirror, 3 reflective/refractive events are needed, one for the mirror reflection, and one each for the entry and exit of the ball. Setting the ray-depth to 5 allows us to see a ball in the mirror.

IOR

Now we’ll do a quick experiment with the IOR value. All the balls now have the following material properties:

The IOR is varied between the balls and the values are:

5M photons were used.

Dispersion

For this experiment the following material properties were used:

The IOR and dispersion power are varied between the balls and the values are:

This image was made with 5M photons, finally, lets increase the photon count to 50M to smooth out the dispersion effect, which generally needs a lot of photons.

For a while now I’ve been playing around with Blender and Yafaray. Both are amazing free open sources pieces of software. In order to better understand some of the Yafaray texture options I decided to do a series of experiments. In the first post I’m just going to be looking at the direct lighting mode, using the glossy texture.

So to start with, I’ll set Anisotropic and Diffuse to disabled, and the color values as the following:

As you’ll see the images contain a series of spheres rendered with different properties, the following table shows the material values used on each of the spheres.

This image nicely illustrates the range of textures achievable with the glossy texture.

Now, in an attempt to deduce how yafaray treats glossy reflection, I’ll set ray depth to 0 (this is a setting in the general settings tab, not related to this specific texture). This is the result:

First of all everything in the mirror is black as no mirror reflections are supported at all with ray depth = 0. Also, as you’d expect, the reflections on the side of each ball (of the light from other balls) are no longer visible. However, somewhat surprisingly we notice the glossy reflection on top of each ball is still visible. This shows that yafaray deals with glossy reflections in two different ways.

First reflection rays are sent out in all directions (or rather a selection of random directions), these allow reflections of other scene objects to been seen. The ray depth setting, controls how many times reflection ray can be forwarded on from an object (e.g. one of the glossy balls in the scene or the mirror plane).

Secondly, shadow rays are sent out from the glossy object, which are only sent directly towards the light sources. These don’t count towards the ray depth total. However if they have to pass through transparent objects, or glass objects with fake shadows enabled, this does count towards the shadow depth. Each transparent object counts as 1 towards the shadow depth.  (Note that glass objects without fake shadows enabled don’t allow shadow rays to pass at all, photons must be used)

Now lets switch to coated glossy. This presents us with one more option, specifically the IOR of the glossy coat. The coated glossy in the same as normal glossy except that material is reflective in a mirror way at high angles incidence, the relationship between the strength of the reflection and the incidence angle is given by the IOR. In the next image all the balls have the same properties as number 7 from the previous image, except all but the first one have the coated-glossy texture. The table shows the IOR values.

Now if we set the ray depth to zero again, as we would expect we see that coat reflections also work with reflection rays because they dissapear:

Now as a test to confirm what we suggested earlier, that the light reflection on the glossy surface is computed using shadow rays, let’s insert a transparent panel above one of the balls. It has full transparency, full transmit filter and grey colour. We have to enable transparent shadows for this to work.

And now if we set the shadow depth to zero:

The reflection on last ball dissapears confirming that the light reflections are indeed calculated using shadow rays.

Now lets take number 4 from above, switch back to regular glossy, copy it and set one of the copies to be As diffuse, below number 4 is the original number 4 from above. And number 3 is “As Diffuse”.

And you can see that the “As Diffuse” option just stops glossy reflection from working (apart the from the light reflection component). So it seems that this has no effect in direct lighting. (In the manual it does infact imply that its only relevant for photon mapping)

Finally, although its slightly off topic as its not related to the glossy texture, the following image shows a ball with a shinydiffusemat with emit turned up.

Note that the emitting ball doesn’t actually act as a light source, as it doesn’t illuminate the ground beneath it. The reflection on the side of the glossy ball occurs because that reflection occurs using reflection rays, which would show up any part of the scenery including the ground (as you can see on the bottom half of the glossy ball)

That concludes my look at the glossy texture under direct lighting. My next post will be an investigation of the glass texture.