Outline Shader

Actually I think the outline shader for a 2D game is kinda unnecessary. The loaded sprite can already be shipped with an outline. (Or, one sprite with and one sprite without an outline; if both are required) However, I kinda find it neat to dynamically apply an outline to whatever object I want to.

Fundamental Idea

We are going to stick to a Pixel shader, searching for pixels with no colorized neighbours and replace them with a black one.

Fail 1

PixelShaderOutput PixelShaderFunction(VertexShaderOutput input): COLOR0 {

    PixelShaderOutput output = (PixelShaderOutput) 0;
    
    float4 left = tex2D(SpriteTextureSampler, (input.TexCoord.x - 1, input.TexCoord.y)) * input.Color;
    float4 right = tex2D(SpriteTextureSampler, (input.TexCoord.x + 1, input.TexCoord.y)) * input.Color;
    float4 top = tex2D(SpriteTextureSampler, (input.TexCoord.x, input.TexCoord.y - 1)) * input.Color;
    float4 bottom = tex2D(SpriteTextureSampler, (input.TexCoord.x, input.TexCoord.y + 1)) * input.Color;
    
    output.Color = tex2D(SpriteTextureSampler, input.TexCoord) * input.Color;
    
    if (left.a == 0 && right.a == 0 && top.a == 0 && bottom.a == 0)
     output.Color.rgb = 0;
    
    
    return output;
}

(input.TexCoord.x – 1, input.TexCoord.y)  equals a float2 with 2 arguments. The x and y value of the texCoord. Left, right, Top and bottom.a check for an alpha channel filled with zero. If all channels are 0, we set the .rgb value of the pixel to 0. (Black) There are several pitfalls I fell into.

  1. A pixel neighbor isn’t just TexCoord.x +/- 1, since the TexCoord is a normalized value from 0..1 Hence, we need to actual TexCoord size and calculate the float size value of a single pixel
  2. We shouldn’t search for pixels with no neighbors. We actually require at least one pixel to be filled with color.

External reference for pixel color changes

Partial Fail 2

PixelShaderOutput PixelShaderFunction(VertexShaderOutput input): COLOR0 {
    
    PixelShaderOutput output = (PixelShaderOutput) 0;
    
    float2 pixelSize = 1 / float2((float) width, (float) height);
    
    float4 left = tex2D(SpriteTextureSampler, input.TexCoord - (pixelSize.x, 0));
    float4 right = tex2D(SpriteTextureSampler, input.TexCoord + (pixelSize.x, 0));
    float4 top = tex2D(SpriteTextureSampler, input.TexCoord - (0, pixelSize.y));
    float4 bottom = tex2D(SpriteTextureSampler, input.TexCoord + (0, pixelSize.y));
    
    output.Color = tex2D(SpriteTextureSampler, input.TexCoord) * input.Color;
    
    if (output.Color.a < 1)
     output.Color.rgb = 0;
    
    return output;
}

The texCoord.x and y has been replaced by pixelSize which is normalized between 0..1. Hence, as an example, the texture of a size 32×32 will result in a single pixel size of 0.03125×0.03125. Since each texture may have its own size, a variable of type int has been defined.


The texture2D is a part of a spritesheet and the background of the spritesheet is white. Hence no pixel is actually translucent and output.Color.a < 1 will only be true if an actual (visible pixel of the axe) has been found. Hence, the current outline shader overrides the axes edges.

The content processor identifies #FFFF00FF as translucent; therefore I did change the spritesheets background to #FFFF00FF as well.

In addition output.Color.rgb = 0 doesn’t include the alpha channel. Even if we did find a translucent pixel, only the rgb will be set to zero (black); but the alpha channel will remain translucent. Thus, while settings rgb = 0, we need to set the alpha channel to fully visible. output.Color.a = 1

PixelShaderOutput PixelShaderFunction(VertexShaderOutput input): COLOR0 {
    
    PixelShaderOutput output = (PixelShaderOutput) 0;
    
    float2 pixelSize = 1 / float2((float) width, (float) height);
    
    float4 left = tex2D(SpriteTextureSampler, input.TexCoord - (pixelSize.x, 0));
    float4 right = tex2D(SpriteTextureSampler, input.TexCoord + (pixelSize.x, 0));
    float4 top = tex2D(SpriteTextureSampler, input.TexCoord - (0, pixelSize.y));
    float4 bottom = tex2D(SpriteTextureSampler, input.TexCoord + (0, pixelSize.y));
    
    output.Color = tex2D(SpriteTextureSampler, input.TexCoord) * input.Color;
    
    if (output.Color.a == 0) {
     output.Color.rgb = 0;
     output.Color.a = 1;
    }
    
    return output;
}

Fail 3

PixelShaderOutput PixelShaderFunction(VertexShaderOutput input): COLOR0 {
    
    PixelShaderOutput output = (PixelShaderOutput) 0;
    
    float2 pixelSize = 1 / float2((float) width, (float) height);
    
    float4 left = tex2D(SpriteTextureSampler, input.TexCoord - (pixelSize.x, 0));
    float4 right = tex2D(SpriteTextureSampler, input.TexCoord + (pixelSize.x, 0));
    float4 top = tex2D(SpriteTextureSampler, input.TexCoord - (0, pixelSize.y));
    float4 bottom = tex2D(SpriteTextureSampler, input.TexCoord + (0, pixelSize.y));
    
    output.Color = tex2D(SpriteTextureSampler, input.TexCoord) * input.Color;
    
    if ((left.a > 0 || right.a > 0 || top.a > 0 || bottom.a > 0) && output.Color.a == 0) {
     output.Color.rgb = 0;
     output.Color.a = 1;
    }
    
    return output;
}

I thought that is it… but nope. Somehow some pixels won’t evaluate the if statement as true. Via try and error, I found out, that the pixelSize (1 / width and 1 / height) doesn’t match the actual pixel size. By setting an arbitrary float value as a dividend, following happend:

Furthermore, I would like to reference to the any and all statement. Those statements do evaluate as true, if any or all pixels are set to a value. ( > 0)

PixelShaderOutput PixelShaderFunction(VertexShaderOutput input): COLOR0 {
    
    PixelShaderOutput output = (PixelShaderOutput) 0;
    
    float2 pixelSize = 0.1 / float2((float) width, (float) height);
    
    float4 left = tex2D(SpriteTextureSampler, input.TexCoord + (pixelSize.x, 0)) * input.Color;
    float4 right = tex2D(SpriteTextureSampler, input.TexCoord - (pixelSize.x, 0)) * input.Color;
    float4 top = tex2D(SpriteTextureSampler, input.TexCoord + (0, pixelSize.y)) * input.Color;
    float4 bottom = tex2D(SpriteTextureSampler, input.TexCoord - (0, pixelSize.y)) * input.Color;
    
    output.Color = tex2D(SpriteTextureSampler, input.TexCoord) * input.Color;
    
    if ((any(top) || any(bottom) || any(left) || any(right)) && output.Color.a == 0) {
     output.Color.rgb = 0;
     output.Color.a = 1;
    }
    
    return output;
}

Now we face the next challenge: Only one pixel is set to black. However, it would be great if the user would actualy be able to see the outline. While using a for loop we do receive better results, but the GPU calculation time improves and the performance suffers.

PixelShaderOutput PixelShaderFunction(VertexShaderOutput input): COLOR0 {
    
    PixelShaderOutput output = (PixelShaderOutput) 0;
    
    float2 pixelSize = 0.1 / float2((float) width, (float) height);
    
    output.Color = tex2D(SpriteTextureSampler, input.TexCoord) * input.Color;
     
    for (int x = 0; x < 3; x++) {
        
         float4 left = tex2D(SpriteTextureSampler, input.TexCoord + (pixelSize.x * x, 0)) * input.Color;
         float4 right = tex2D(SpriteTextureSampler, input.TexCoord - (pixelSize.x * x, 0)) * input.Color;
         float4 top = tex2D(SpriteTextureSampler, input.TexCoord + (0, pixelSize.y * x)) * input.Color;
         float4 bottom = tex2D(SpriteTextureSampler, input.TexCoord - (0, pixelSize.y * x)) * input.Color;
         float4 rightTop = tex2D(SpriteTextureSampler, input.TexCoord + (pixelSize.x * x, pixelSize.y * x)) * input.Color;
         
         if ((any(top) || any(bottom) || any(left) || any(right) || any(rightTop)) && output.Color.a == 0) {
              output.Color.rgb = 0;
              output.Color.a = 1;
         }
    }
    
    return output;
}

Partial Success

Using a pixel shader for a 2D outline isn’t GPU friendly at all. Increasing the for loop value does provide better results, but also does use more GPU processing time. But how do we now display proper outlines on a 2D texture?

  1. Draw the texture twice. The first one with a scale factor and the second one without any scaling
  2. Provide an outline on the original texture

Source

#
if OPENGL# define SV_POSITION POSITION# define VS_SHADERMODEL vs_3_0# define PS_SHADERMODEL ps_3_0#
else# define VS_SHADERMODEL vs_4_0_level_9_1# define PS_SHADERMODEL ps_4_0_level_9_1# endif

Texture2D SpriteTexture;
sampler2D SpriteTextureSampler = sampler_state {
 Texture = < SpriteTexture > ;
};

float4x4 projectionMatrix;
float4x4 viewMatrix;

int width;
int height;
int size;

struct PixelShaderOutput {
 float4 Color: COLOR0;
};

struct VertexShaderOutput {
 float4 Position: SV_POSITION;
 float2 TexCoord: TEXCOORD0;
 float4 Color: COLOR0;
};

VertexShaderOutput VertexShaderLogic(float4 position: SV_POSITION, float4 color: COLOR0, float2 texCoord: TEXCOORD0) {

    VertexShaderOutput output = (VertexShaderOutput) 0;
    
    output.Position = mul(position, viewMatrix);
    output.Position = mul(output.Position, projectionMatrix);
    output.TexCoord = texCoord;
    output.Color = color;
    
    return output;
}

PixelShaderOutput PixelShaderFunction(VertexShaderOutput input): COLOR0 {
    
    PixelShaderOutput output = (PixelShaderOutput) 0;
    
    float2 pixelSize = 0.1 / float2((float) width, (float) height);
    
    output.Color = tex2D(SpriteTextureSampler, input.TexCoord) * input.Color;
    
    for (int x = 0; x < 3; x++) {
        
         float4 left = tex2D(SpriteTextureSampler, input.TexCoord + (pixelSize.x * x, 0)) * input.Color;
         float4 right = tex2D(SpriteTextureSampler, input.TexCoord - (pixelSize.x * x, 0)) * input.Color;
         float4 top = tex2D(SpriteTextureSampler, input.TexCoord + (0, pixelSize.y * x)) * input.Color;
         float4 bottom = tex2D(SpriteTextureSampler, input.TexCoord - (0, pixelSize.y * x)) * input.Color;
         float4 rightTop = tex2D(SpriteTextureSampler, input.TexCoord + (-pixelSize.x * x, -pixelSize.y * x)) * input.Color;
         float4 leftBottom = tex2D(SpriteTextureSampler, input.TexCoord + (pixelSize.x * x, pixelSize.y * x)) * input.Color;
         float4 rightBottom = tex2D(SpriteTextureSampler, input.TexCoord + (-pixelSize.x * x, pixelSize.y * x)) * input.Color;
         float4 leftTop = tex2D(SpriteTextureSampler, input.TexCoord + (pixelSize.x * x, -pixelSize.y * x)) * input.Color;
     
        if ((any(top) || any(bottom) || any(left) || any(right) || any(rightTop) || any(leftBottom) || any(rightBottom) || any(leftTop)) && output.Color.a == 0) {
              output.Color.rgb = 0;
              output.Color.a = 1;
        }
    }
    
    return output;
}
technique SpriteDrawing {
    
 pass P0 {
     
  VertexShader = compile VS_SHADERMODEL VertexShaderLogic();
  PixelShader = compile PS_SHADERMODEL PixelShaderFunction();
}
};

Hinterlasse einen Kommentar

Please Login to comment
  Subscribe  
Notify of