Geometry-Shader-Based Real-time Voxelization and Applications
Abstract
This work proposes a new real-time voxelization algorithm based on newly available GPU functionalities
and designs several applications to render complex lighting effects
with the proposed voxelization method. The voxelization algorithm can efficiently
transform a highly complex scene in surface representation
into a set of voxels in one GPU pass using the geometry shader.
The surficial and volumetric properties of objects
such as opaqueness, refraction and transmission
are directly recorded with newly available 3D textures.
The usage of 3D textures can remove those strenuous efforts
required to modify the encoding and decoding scheme
when adjusting the voxel resolution.
In the first,
surficial and volumetric properties recorded in 3D textures
can be used to interactively compute and render more realistic lighting effects
including the shadow of objects with complex occlusion
and the refraction and transmission of transparent objects.
The shadow can be rendered with an absorption coefficient
which is computed according to the number of surfaces
drawing in each voxel during voxelization
and used to compute the amount of light passing
through partially occluded complex objects.
Second,
the surface normal, transmission coefficient and refraction index recorded in each voxel
can be used to simulate the refraction and transmission lighting effects of transparent objects
using our multiple-surfaced refraction algorithm.
Finally,
the results demonstrate that
our algorithm can transform a dynamic scene into the volumetric representation
and render complex lighting effects in real time without any preprocessing.
Due to limitation of paper space, we only show several images to demonstrate our algorithm.
In the supplement material we present the entire set of rendering result using our algorithm.
***********************************************
********** Support Animation: *****************
***********************************************
Directory: animation/
screen resolution:640x480
voxel resolution:700x700x256
ttransparent-shadow-tree.mp4: The model contains 65482 triangles and is rendered with
absorption coefficeint for the leaf equal to 0.1 and absorption coefficient
for the branch equal to 0.9. And our algorithm can render the entire model
in a frame rate of 27 FPS.
refraction-m-surfaced-horse-128x128x128.mp4 is run at a frame rate of 204 FPS.
This video demonstrates the refraction effect rendered using the
multiple-surface-refraction method.
refraction-m-surfaced-horse-256x256x256.mp4 is run at a frame rate of 102 FPS.
This video demonstrates the refraction effect rendered using the
multiple-surface-refraction method.
refraction-m-surfaced-horse-512x512x512.mp4 is run at a frame rate of 34 FPS.
This video demonstrates the refraction effect rendered using the
multiple-surface-refraction method.
refraction-m-surfaced-ball-angle.mp4 is run at a frame rate of 52 FPS.
This video demonstrates the refraction effect of an object
that has two parts with two different transparent materials.
refraction-m-surfaced-teapot.mp4 is run at a frame rate of 87 FPS.
This video demonstrates the refraction effect rendered using the
multiple-surface-refraction method.
refraction-2-surfaced-shark-morphing.mp4 is run at a frame rate of 85 FPS.
This video demonstrates the refraction effect rendered using the
two-surface-refraction method.
We use a shark morphing model with 2816 triangles to demonstrate our rendering
algorithm in rendering a deformable object.
refraction-m-surfaced-shark-morphing.mp4 is run at a frame rate of 87 FPS.
This video demonstrates the refraction effect rendered using the
multiple-surface-refraction method.
We use a shark morphing model with 2816 triangles to demonstrate our rendering
algorithm in rendering a deformable object.
And we can find that our multiple-surface-refraction algorithm
can generate better result especially at the edge of the shark.
refraction-transparency-m-surfaced-ball-angle.mp4 is run at a frame rate of 52 FPS.
This video demonstrates the refraction and transmittance effect of an object
that has two parts with two different transparent materials.
voxelization-tranparent-shadow-uniform-tree.jpg: The model contains 65482 triangles and is rendered with
absorption coefficeint for the leaf and trunk equal to 0.1.
And our algorithm can render the entire model in a frame rate of 27 FPS.
voxelization-tranparent-shadow-variation-tree.jpg: The model contains 65482 triangles and is rendered with
absorption coefficeint for the leaf equal to 0.1 and absorption coefficient
for the branch equal to 0.9. And our algorithm can render the entire model
in a frame rate of 27 FPS.
voxelization-refraction-m-surfaced-torus-512x512x512.jpg:
The model contains 800 triangles and is voxelized in the resolution of 512x512x512.
The process requires 10.75 mps.
voxelization-refraction-m-surfaced-torus-256x256x256.jpg:
The model contains 800 triangles and is voxelized in the resolution of 256x256x256.
The process requires 4.46 mps.
voxelization-refraction-m-surfaced-torus-128x128x128.jpg:
The model contains 800 triangles and is voxelized in the resolution of 128x128x128.
The process requires 2.08 mps.
voxelization-refraction-m-surfaced-box-512x512x512.jpg:
The model contains 1380 triangles and is voxelized in the resolution of 512x512x512.
The process requires 13.51 mps.
voxelization-refraction-m-surfaced-box-256x256x256.jpg:
The model contains 1380 triangles and is voxelized in the resolution of 256x256x256.
The process requires 4.73 mps.
voxelization-refraction-m-surfaced-box-128x128x128.jpg:
The model contains 1380 triangles and is voxelized in the resolution of 128x128x128.
The process requires 2.1 mps.
voxelization-refraction-m-surfaced-teapot-512x512x512.jpg:
The model contains 2304 triangles and is voxelized in the resolution of 512x512x512.
The process requires 13.51 mps.
voxelization-refraction-m-surfaced-teapot-256x256x256.jpg:
The model contains 2304 triangles and is voxelized in the resolution of 256x256x256.
The process requires 4.73 mps.
voxelization-refraction-m-surfaced-teapot-128x128x128.jpg:
The model contains 2304 triangles and is voxelized in the resolution of 128x128x128.
The process requires 2.1 mps.
voxelization-refraction-m-surfaced-monster-512x512x512.jpg:
The model contains 14840 triangles and is voxelized in the resolution of 512x512x512.
The process requires 17.2 mps.
voxelization-refraction-m-surfaced-monster-256x256x256.jpg:
The model contains 14840 triangles and is voxelized in the resolution of 256x256x256.
The process requires 6.28 mps.
voxelization-refraction-m-surfaced-monster-128x128x128.jpg:
The model contains 14840 triangles and is voxelized in the resolution of 128x128x128.
The process requires 2.58 mps.
voxelization-refraction-m-surfaced-venusm-512x512x512.jpg:
The model contains 43357 triangles and is voxelized in the resolution of 512x512x512.
The process requires 15.2 mps.
voxelization-refraction-m-surfaced-venusm-256x256x256.jpg:
The model contains 43357 triangles and is voxelized in the resolution of 256x256x256.
The process requires 5.43 mps.
voxelization-refraction-m-surfaced-venusm-128x128x128.jpg:
The model contains 43357 triangles and is voxelized in the resolution of 128x128x128.
The process requires 2.51 mps.
voxelization-refraction-m-surfaced-horse-512x512x512.jpg:
The model contains 96966 triangles and is voxelized in the resolution of 512x512x512.
The process requires 20.4 mps.
voxelization-refraction-m-surfaced-horse-256x256x256.jpg:
The model contains 96966 triangles and is voxelized in the resolution of 256x256x256.
The process requires 6.4 mps.
voxelization-refraction-m-surfaced-horse-128x128x128.jpg:
The model contains 96966 triangles and is voxelized in the resolution of 128x128x128.
The process requires 3.00 mps.
voxelization-refraction-m-surfaced-dragon2-512x512x512.jpg:
The model contains 108588 triangles and is voxelized in the resolution of 512x512x512.
The process requires 16.1 mps.
voxelization-refraction-m-surfaced-dragon2-256x256x256.jpg:
The model contains 108588 triangles and is voxelized in the resolution of 256x256x256.
The process requires 5.23 mps.
voxelization-refraction-m-surfaced-dragon2-128x128x128.jpg:
The model contains 108588 triangles and is voxelized in the resolution of 128x128x128.
The process requires 3.00 mps.
voxelization-refraction-m-surfaced-hand-512x512x512.jpg:
The model contains 654666 triangles and is voxelized in the resolution of 512x512x512.
The process requires 22.2 mps.
voxelization-refraction-m-surfaced-hand-256x256x256.jpg:
The model contains 654666 triangles and is voxelized in the resolution of 256x256x256.
The process requires 11.3 mps.
voxelization-refraction-m-surfaced-hand-128x128x128.jpg:
The model contains 654666 triangles and is voxelized in the resolution of 128x128x128.
The process requires 9.00 mps.
voxelization-refraction-m-surfaced-dragon-512x512x512.jpg:
The model contains 871414 triangles and is voxelized in the resolution of 512x512x512.
The process requires 24.9 mps.
voxelization-refraction-m-surfaced-dragon-256x256x256.jpg:
The model contains 871414 triangles and is voxelized in the resolution of 256x256x256.
The process requires 14.7 mps.
voxelization-refraction-m-surfaced-dragon-128x128x128.jpg:
The model contains 871414 triangles and is voxelized in the resolution of 128x128x128.
The process requires 11.8 mps.
===============================================================================
Compare among 1-, 2-, multiple- refraction and RT:
===============================================================================
Directory: images/compare/
voxel size:512x512x256
screen buffer size:800x600
voxelization-refraction-1-surfaced-torus.jpg:
The model contains 800 triangles and is rendered with
the refraction effect using the single-refraction algorithm in a frame rate of 4720 FPS.
voxelization-refraction-2-surfaced-torus.jpg:
The model contains 800 triangles and is rendered with
the refraction effect using the two-refraction image-based algorithm proposed by Wyman et al.
in a frame rate of 2499 FPS.
voxelization-refraction-m-surfaced-torus.jpg:
The model contains 800 triangles and is rendered with
the refraction effect using the multiple-refraction algorithm proposed by us
in a frame rate of 111 FPS.
voxelization-refraction-rt-torus.jpg:
The model contains 800 triangles and is rendered with
the refraction effect using the ray-tracing algorithm in a frame rate of 2.1 FPS.
screen buffer size:800x600
voxelization-refraction-1-surfaced-box.jpg:
The model contains 1380 triangles and is rendered with
the refraction effect using the single-refraction algorithm in a frame rate of 4182 FPS.
voxelization-refraction-2-surfaced-box.jpg:
The model contains 1380 triangles and is rendered with
the refraction effect using the two-refraction image-based algorithm proposed by Wyman et al.
in a frame rate of 2389 FPS.
voxelization-refraction-m-surfaced-box.jpg:
The model contains 1380 triangles and is rendered with
the refraction effect using the multiple-refraction algorithm proposed by us
in a frame rate of 89 FPS.
voxelization-refraction-rt-box.jpg: The model contains 1380 triangles and is rendered with
the refraction effect using the ray-tracing algorithm in a frame rate of 2.1 FPS.
voxelization-refraction-1-surfaced-teapot.jpg:
The model contains 2304 triangles and is rendered with
the refraction effect using the single-refraction algorithm in a frame rate of 3612 FPS.
voxelization-refraction-2-surfaced-teapot.jpg:
The model contains 2304 triangles and is rendered with
the refraction effect using the two-refraction image-based algorithm proposed by Wyman et al.
in a frame rate of 2041 FPS.
voxelization-refraction-m-surfaced-teapot.jpg:
The model contains 2304 triangles and is rendered with
the refraction effect using the multiple-refraction algorithm proposed by us
in a frame rate of 84 FPS.
voxelization-refraction-rt-teapot.jpg:
The model contains 2304 triangles and is rendered with
the refraction effect using the ray-tracing algorithm in a frame rate of 2.1 FPS.
voxelization-refraction-1-surfaced-monster.jpg:
The model contains 14840 triangles and is rendered with
the refraction effect using the two-refraction image-based algorithm proposed by Wyman et al.
in a frame rate of 2632 FPS.
voxelization-refraction-2-surfaced-monster.jpg:
The model contains 14840 triangles and is rendered with
the refraction effect using the two-refraction image-based algorithm proposed by Wyman et al.
in a frame rate of 1369 FPS.
voxelization-refraction-m-surfaced-monster.jpg: The model contains 14840 triangles and is rendered with
the refraction effect using the multiple-refraction algorithm proposed by us
in a frame rate of 69 FPS.
voxelization-refraction-rt-monster.jpg:
The model contains 14840 triangles and is rendered with
the refraction effect using the ray-tracing algorithm in a frame rate of 0.13 FPS.
voxelization-refraction-1-surfaced-venusm:
The model contains 43357 triangles and is rendered with
the refraction effect using the single-refraction algorithm in a frame rate of 2685.
voxelization-refraction-2-surfaced-venusm.jpg:
The model contains 43357 triangles and is rendered with
the refraction effect using the two-refraction image-based algorithm proposed by Wyman et al
in a frame rate of 1357 FPS.
voxelization-refraction-m-surfaced-venusm.png:
The model contains 43357 triangles and is rendered with
the refraction effect using the multiple-refraction algorithm proposed by us
in a frame rate of 70 FPS.
voxelization-refraction-rt-venusm.png:
The model contains 43357 triangles and is rendered with
the refraction effect using the ray-tracing algorithm in a frame rate of 0.044 FPS.
voxelization-refraction-1-surfaced-horse:
The model contains 96966 triangles and is rendered with
the refraction effect using the single-refraction algorithm in a frame rate of 1755.
voxelization-refraction-2-surfaced-horse.jpg:
The model contains 96966 triangles and is rendered with
the refraction effect using the two-refraction image-based algorithm proposed by Wyman et al
in a frame rate of 908 FPS.
voxelization-refraction-m-surfaced-horse.png:
The model contains 96966 triangles and is rendered with
the refraction effect using the multiple-refraction algorithm proposed by us
in a frame rate of 34 FPS.
voxelization-refraction-rt-horse.png:
The model contains 96966 triangles and is rendered with
the refraction effect using the ray-tracing algorithm in a frame rate of 0.012 FPS.
voxelization-refraction-1-surfaced-dragon2.jpg: The model contains 108588 triangles and is rendered with
the refraction effect using the single-refraction algorithm in a frame rate of 1617 FPS.
voxelization-refraction-2-surfaced-dragon2: The model contains 108588 triangles and is rendered with
the refraction effect using the two-refraction image-based algorithm proposed by Wyman et al.
in a frame rate of 837 FPS.
voxelization-refraction-m-surfaced-dragon2.jpg:
The model contains 108588 triangles and is rendered with
the refraction effect using the multiple-refraction algorithm proposed by us
in a frame rate of 34 FPS.
voxelization-refraction-rt-dragon2.jpg: The model contains 108588 triangles and is rendered with
the refraction effect using the ray-tracing algorithm in a frame rate of 0.00892 FPS.
voxelization-refraction-1-surfaced-hand.png:
The model contains 654666 triangles and is rendered with
the refraction effect using the single-refraction algorithm in a frame rate of 358 FPS.
voxelization-refraction-2-surfaced-hand.png:
The model contains 654666 triangles and is rendered with
the refraction effect using the two-refraction image-based algorithm proposed by Wyman et al.
in a frame rate of 181 FPS.
voxelization-refraction-m-surfaced-hand.png:
The model contains 654666 triangles and is rendered with
the refraction effect using the multiple-refraction algorithm proposed by us
in a frame rate of 27 FPS.
voxelization-refraction-rt-hand.png:
The model contains 654666 triangles and is rendered with
the refraction effect using the ray-tracing algorithm in a frame rate of 0.000548 FPS.
voxelization-refraction-1-surfaced-dragon.jpg:
The model contains 871414 triangles and is rendered with
the refraction effect using the single-refraction algorithm in a frame rate of 272 FPS.
voxelization-refraction-2-surfaced-dragon.jpg:
The model contains 871414 triangles and is rendered with
the refraction effect using the two-refraction image-based algorithm proposed by Wyman et al.
in a frame rate of 137 FPS.
voxelization-refraction-m-surfaced-dragon.jpg:
The model contains 871414 triangles and is rendered with
the refraction effect using the multiple-refraction algorithm proposed by us
in a frame rate of 14 FPS.
voxelization-refraction-rt-dragon.jpg:
The model contains 871414 triangles and is rendered with
the refraction effect using the ray-tracing algorithm in a frame rate of 0.0000162 FPS.
voxelization-refraction-m-surfaced-ball-angel.jpg:
The model contains 55484 triangles and is rendered with
refraction effect. And our algorithm can render the entire model in a frame
rate of 63 FPS.
voxelization-refraction-m-surfaced-ateneam.jpg:
The model contains 15014 triangles and is rendered with
refraction effect. And our algorithm can render the entire model in a frame
rate of 77 FPS.
voxelization-refraction-m-surfaced-hand.jpg:
The model contains 654666 triangles and is rendered with
refraction effect. And our algorithm can render the entire model in a frame
rate of 22 FPS.
voxelization-refraction-m-surfaced-hebe.jpg:
The model contains 63932 triangles and is rendered with
refraction effect. And our algorithm can render the entire model in a
frame rate of 66 FPS.
voxelization-refraction-m-surfaced-teapot.jpg:
The model contains 2304 triangles and is rendered with
refraction effect. And our algorithm can render the entire model in a
frame rate of 68 FPS.
voxelization-refraction-m-surfaced-venusm.jpg:
The model contains 43357 triangles and is rendered with
refraction effect. And our algorithm can render the entire model in a frame
rate of 66 FPS.
voxelization-refraction-m-surfaced-dragon.jpg:
The model contains 871414 triangles and is rendered with
refraction effect. And our algorithm can render the entire model in a frame
rate of 20 FPS.
voxelization-refraction-m-surfaced-ball-angel.jpg:
The model contains 55804 triangles and is rendered
with our multiple-surfaced refraction effect.
And our algorithm can render the entire model in a frame rate of 72 FPS.
voxelization-refraction-transparency-m-surfaced-bust.jpg:
The model contains 95028 triangles and is
rendered with transmittance coefficeint equal to RGB(51,127,204).
And our algorithm can render the entire model in a frame rate of 55 FPS.
voxelization-refraction-transparency-m-surfaced-dragon2.jpg:
The model contains 108588 triangles and is
rendered with transmittance coefficeint equal to RGB(255,127,0).
And our algorithm can render the entire model in a frame rate of 51 FPS.
voxelization-refraction-transparency-m-surfaced-torus.jpg:
The model contains 800 triangles and is
rendered with transmittance coefficeint equal to RGB(255,0,0).
And our algorithm can render the entire model in a frame rate of 82 FPS.
voxelization-refraction-transparency-m-surfaced-dragon.jpg:
The model contains 871414 triangles and is
rendered with transmittance coefficeint equal to RGB(0,153,0).
And our algorithm can render the entire model in a frame rate of 18 FPS.
voxelization-refraction-transparency-m-surfaced-ball-angel.jpg:
The angle model contains 55484
triangles and is rendered with transmittance coefficeint equal to
RGB(255,0,0). The sphere model contains 320 triangles and is rendered with
transmittance coefficeint equal to RGB(0,255,0).
And our algorithm can render the entire model in a frame rate of 52 FPS.
voxelization-refraction-transparency-m-surfaced-cube-bunny-calabash.jpg: The image is rendered with a screen buffer size: 1366x768
and using voxel resolution: 750x750x256.
Three models contain 11064 triangles.
The box's transmittance coefficeint is equal to RGB(51,128,180).
The duck's transmittance coefficeint is equal to RGB(256,512,0).
The rabbit's transmittance coefficeint is equal to RGB(51,204,51).
And our algorithm can render the entire model in a frame rate of 59 FPS.
==========================================================================================
The following is the source code of the shader for voxelization, two-pass refraction
and multiple-pass refraction
==========================================================================================
//**********************************************************************************
//
// Geometry shader for Voxelization
// @ Input: vertices of a triangle
// @ Output: duplicate of triangles for slice
//
//**********************************************************************************
void gs_main(triangle VS_TEX_OUT input[3], inout TriangleStream triStream)
{
// Find the max and min of the depth
float distMax=max(max(input[0].posV.z,input[1].posV.z),input[2].posV.z);
float distMin=min(min(input[0].posV.z,input[1].posV.z),input[2].posV.z);
// Find the range of the valid depth
float2 start_end=floor(float2(distMax,distMin)/gSliceValue);
float4x4 ClipMatrix = gProj;
float2 Near_far = float2(0,0);
GS_OUT output;
// Duplicate the triangles according to the intersection of the depth slices
for(int g=start_end.y; g<=start_end.x; g++)
{
// Check whether the depth is inside the range
if(g>0 && g
{
output.RTIndex =g;
float2 clip = float2(g,g+1)*gSliceValue+float2(-gSliceValue,gSliceValue);
ClipMatrix._33 = clip.y/(clip.y-clip.x);
ClipMatrix._43 = -clip.x*clip.y/(clip.y-clip.x);
for(int v=0; v<3; v++)
{
output.Pos = mul(input[v].posV,ClipMatrix);
output.norL=input[v].norL;
triStream.Append( output );
}
triStream.RestartStrip( );
}
}
}
//**********************************************************************************
//
// Two-surface refraction pixel shader
// @ Input: voxelization result
//
//**********************************************************************************
float4 ps_main(VS_OUT pIn)
{
// Compute the view direction in world
float3 viewVec = normalize( pIn.posW -gEyePosW);
// Set up the step distance
float3 StepSize=0.001953125f;
// Set up tracking direction
float3 dir=viewVec;
// Set up the tracking position
float3 pos=pIn.posL;
// Preset the air refraction index
float Index=1.14f;
float InvIndex=1/Index;
bool InObj=false;
int InCount=0;
float3 lastnormal=float3(0,0,0);
float thick = 0;
float3 move;
bool hit=false;
float4 normal= pIn.normal;
// Transform the tracking direction into voxelization coordinate
move=mul(float4(dir , 0.0f) ,gVolume);
move.y=move.y*-1;
move=normalize(move);
// Compute the first refraction
dir=refract(dir,normal,InvIndex,Index);
// Go through the object to find the second refraction
for(int i=0;i
{
normal=loadData(move,pos);
hit=false;
if(dot(normal.rgb,lastnormal.rgb)>0.75f)
hit=true;
if(any(normal.rgb) && !hit)
{
lastnormal=normal;
if(dot(normal.xyz,-dir.xyz)<0){
InObj=false;
}else{
InObj=true;
}
}
if(InObj)
thick++;
// Advect the tracking position
pos=StepSize*move+pos;
if(pos.z<0 || pos.z>1.0f || pos.x > 1.0f || pos.y > 1.0f || pos.x <0.0f || pos.y < 0.0f ){
break;
}
}
// Compute the second refraction
dir=refract(dir,lastnormal,InvIndex,Index);
float4 reflectedColor = CubeMapSample(dir );
return reflectedColor ;
}
//**********************************************************************************
//
// Multiple-surface refraction pixel shader
// @ Input: voxelization result
//
//**********************************************************************************
float4 ps_main(VS_OUT pIn)
{
// Compute the view direction in world
float3 viewVec = normalize( pIn.posW -gEyePosW);
// Set up the step distance
float3 StepSize=0.001953125f;
// Set up tracking direction
float3 dir=viewVec;
// Set up the tracking position
float3 pos=pIn.posL;
// Preset the air refraction index
float Index=1.14f;
float InvIndex=1/Index;
bool InObj=false;
int InCount=0;
float3 lastnormal=float3(0,0,0);
float3 move;
bool hit=false;
float4 normal;
// Transform the tracking direction into voxelization coordinate
move=mul(float4(dir , 0.0f) ,gVolume);
move.y=move.y*-1;
move=normalize(move);
for(int i=0;i
{
normal=loadData(move,pos);
hit=false;
// Check the similarity of refraction normal
if(dot(normal.rgb,lastnormal.rgb)>0.75f)
hit=true;
// Not similar and then do the refraction
if(any(normal.rgb) && !hit)
{
lastnormal=normal;
// Check the front or back face entrance
if(dot(normal.xyz,-dir.xyz)<0){
dir=refract(dir,-1*normal.xyz,Index,InvIndex);
InObj=false;
}else{
dir=refract(dir,normal.xyz,InvIndex,Index);
InObj=true;
}
dir=normalize(dir);
move=mul(float4(dir , 0.0f) ,gVolume);
move.y=move.y*-1;
move=normalize(move);
}
// Advect the tracking position
pos=StepSize*move+pos;
if(pos.z<0 || pos.z>1.0f || pos.x > 1.0f || pos.y > 1.0f || pos.x <0.0f || pos.y < 0.0f ){
break;
}
}
float4 reflectedColor = CubeMapSample(dir );
return reflectedColor ;
}
