vv

Here is a voxelized stanford dragon mesh. 

Longest axis is 512 here.

 Uncompressed, the volume is ~200mb(f32). 

Spent some time working on a compression method. 

 It supports random access without prior decompression like S3/dxt.

It turns out SDFs are very compressible if you think about which information really matters.

Lossy, but not really observable. 

 New size: 3.39 mb in memory

On disk with zstd(22): ~800kb  

Appolonian

 Also spent a lot of time working on the controls.

Making it easier to select and adjust the current shape,

trying to exposing the various knobs in a way that feels intuitive.

I also added the ability to splat vertex colors into a 3D volume.

It looks "ok", although compared to texturing it is low detail.

 I'll got a few ideas on how to convert a UV textured mesh into a voxel format, that I am planning to try out.

My glorious art. The color volume is only 64^3 here so pretty low detail


The vertex colors only contain surface color, so the internal colors have to be extrapolated.
It use a flood algorithm.

It is stored in a  srgb cube currently, but I plan to use the same technique I used for the SDF to compress it.

some links

Normal compression with SFM: better quality and faster decode than octahedral mapping, which is what I am currently using. Here is shadertoy link to an IQ's.

D3d11 Extentions:  I need barycentric coords. AMD has a d3d11 extension for it. For Nvidia a geometry shader is required, but it looks like they have a nvAPI fast geometry shader that might work.

AMD Polaris: The reduced cost for small triangles is what most interests me here

GPUOpen: ATI open source with hair, shadows, gpu compute etc

Screen Space Reflections: implementation details

C survey: undefined behavior yadda yadda

compilers blog

math stuff

LZSSE: faster decompression than lz4

small lz4 -- smaller lz4 compatible files

corner wang tiles

fractal stuff

hg_sdf + puoet

povray: list of shapes supported has some interesting shapes

custom vertex fetch: see sebbbi's post. You can manually fetch vertex data instead of relying on fixed function. Can use this to encode extra bits of data into any unused bits in your indices.  Runs well on AMD, but appears to perform very poorly on Nvidia.

Timing from Turanszkji's post:

GPU     Method        ShadowPass    ZPrepass   OpaquePass   All GPU

NVidia GTX 960  InputLayout       4.52 ms     0.37 ms    6.12 ms    15.68 ms
NVidia GTX 960  CustomFetch (typed buffer)   18.89 ms    1.31 ms    8.68 ms    33.58 ms
NVidia GTX 960  CustomFetch (RAW buffer 1)   18.29 ms    1.35 ms    8.62 ms    33.03 ms
NVidia GTX 960  CustomFetch (RAW buffer 2)   18.42 ms    1.32 ms    8.61 ms    33.18 ms

AMD RX 470   InputLayout       7.43 ms     0.29 ms    3.06 ms    14.01 ms
AMD RX 470   CustomFetch (typed buffer)   7.41 ms     0.31 ms    3.12 ms    14.08 ms
AMD RX 470   CustomFetch (RAW buffer 1)   7.50 ms     0.29 ms    3.07 ms    14.09 ms
AMD RX 470   CustomFetch (RAW buffer 2)   7.56 ms     0.28 ms    3.09 ms    14.15 ms

Summed Area Table

For my future reference:)

A Summed area table(SAT) can be used to query the sum of values over a rectangular region.

From this you can also derive the average value, by dividing by the # of pixels in the rectangle.

It can be used as an alternative to mip mapping.

One advantage over mip mapping is that the query region can be an arbitrary rectangle, unlike mip mapping which is square.

A disadvantage is that that it requires more and more precision as you approach the lower right(the final value is the sum of all previous values).
Thus SAT generally requires increased memory.

std::min/max prevent autovectorization in vs2015

a < b ? a : b;    <-- auto vectorizes

std::min(a,b)   <-- does not

Another bug report for VS: std::min/max break autovectorization


VS's autovectorizer requires massaging to get anything out of it.

Another quirk:  during type conversion, don't skip steps.
For example.

float->i8  //this is skipping the step of converting to i32
float->i32 //An instruction exists for this,

So if you convert float directly to i8, autovectorization fails.
Instead you must convert to i32, and then to i8, now autovectorization succeeds.

Old Images

Seen here:
1.  Many common primitives such as boxes, spheres, cylinders etc
2.  The horse mesh, which was voxelized using openvdb
3.  The white pointy thing is a height map of Mt Taranaki in New Zealand, I stretched it so its somewhat contorted
4.  Everything can be textured, but I'm texturing it on the CPU right now, and just coloring the vertices. So its super blurry compared to GPU texturing(this shot was taken at 9 pixels horizontal per vert, a fairly low detail setting).

Texturing it on the CPU allowed for an infinite # of textures at any location, with arbitrary blends.
On the GPU, I'll have to limit it  to something sensible.

 My plan is to allow for a small #(N) textures to be sampled on GPU, but if we exceed N for a given patch, bake the excess textures into the vertices.  This will be based on corresponding texel size relative to vert spacing.  This is why I needed a CPU texture sampler, so I wrote that path first.

This image has GPU texturing, but it isn't as flexible as the CPU pipeline.
I guess I should motivate and get the full thing working fully on GPU.

vs2015, std::floor/trunc/ceil, and the resulting assembly

 VS2015 generates inefficient code for these instructions

float floored = std::floor(some_float);

So here is what VS generates with /AVX2 switch thrown:

00007FF6EE961016  vmovss      xmm1,dword ptr [bob]  
00007FF6EE96101C  vcvttss2si  ecx,xmm1  
00007FF6EE961020  cmp         ecx,80000000h  
00007FF6EE961026  je          main+4Bh (07FF6EE96104Bh)  
00007FF6EE961028  vxorps      xmm0,xmm0,xmm0  
00007FF6EE96102C  vcvtsi2ss   xmm0,xmm0,ecx  
00007FF6EE961030  vucomiss    xmm0,xmm1  
00007FF6EE961034  je          main+4Bh (07FF6EE96104Bh)  
00007FF6EE961036  vunpcklps   xmm1,xmm1,xmm1  
00007FF6EE96103A  vmovmskps   eax,xmm1  
00007FF6EE96103E  and         eax,1  
00007FF6EE961041  sub         ecx,eax  
00007FF6EE961043  vxorps      xmm1,xmm1,xmm1  
00007FF6EE961047  vcvtsi2ss   xmm1,xmm1,ecx  

Not good.

With AVX enabled I'd expect to see roundss used.

Here is a custom implementation of floor using intrinsics.

float floor_avx(float a) {
    __m128 o;
    return _mm_cvtss_f32(_mm_floor_ss(o, _mm_set_ss(a)));
}

And the assembly:

00007FF7461C1016  vmovss      xmm1,dword ptr [bob]  
00007FF7461C101C  vmovaps     xmm2,xmm1  
00007FF7461C1020  vmovups     xmm1,xmmword ptr [rsp+20h]  
00007FF7461C1026  vroundss    xmm3,xmm1,xmm2,1  

There seems to be a few extra moves here for whatever reason, but at least it is in the ballpark of reasonable.

 The same problem exists for std::trunc, std::ceil, and applies to both float and double.

Anyway I reported this on Connect(floor/ceil/trunc), although my experience in the past with Connect has not been great..

Well, hopefully they fix this one..


Here is what std::trunc generates: It calls a function, instead of using roundss

00007FF750091016  vmovss      xmm0,dword ptr [bob]
00007FF75009101C  call        qword ptr [__imp_truncf (07FF750092108h)]

(Edit: VS2017 is better, but still misses some optimizations with std::trunc and std::round)
godbolt link for x64

AVX2, how to Pack Left

If you have an input array, and an output array, and you only want to write those elements which pass a condition, what is the most efficient way to do this with AVX2?


Here is a visualization of the problem:

Here is my solution, using compressed indices. It requires a LUT sized 769 bytes, so it is best suited for cases where you have a good sized array of data to work on. (If this looks familiar to a stackoverflow post that is because I am the author).

//Generate Move mask via: _mm256_movemask_ps(_mm256_castsi256_ps(mask)); etc
__m256i MoveMaskToIndices(int moveMask) {
    u8 *adr = g_pack_left_table_u8x3 + moveMask * 3;
    __m256i indices = _mm256_set1_epi32(*reinterpret_cast<u32*>(adr));//lower 24 bits has our LUT

    __m256i m = _mm256_sllv_epi32(indices, _mm256_setr_epi32(29, 26, 23, 20, 17, 14, 11, 8));

    //now shift it right to get 3 bits at bottom
    __m256i shufmask = _mm256_srli_epi32(m, 29);
    return shufmask;
}
//The rest of this code to build the LUT
u32 get_nth_bits(int a) {
    u32 out = 0;
    int c = 0;
    for (int i = 0; i < 8; ++i) {
        auto set = (a >> i) & 1;
        if (set) {
            out |= (i << (c * 3));
            c++;
        }
    }
    return out;
}
u8 g_pack_left_table_u8x3[256 * 3 + 1];

void BuildPackMask() {
    for (int i = 0; i < 256; ++i) {
        *reinterpret_cast<u32*>(&g_pack_left_table_u8x3[i * 3]) = get_nth_bits(i);
    }
}

On stackoverflow Peter Cordes came up with a solution that is clever, it avoids the requirement for a LUT by taking advantage of the new BMI(bit manipulation) instruction set. I had not used the BMI instructions before, so this was new to me.
 This code is x64 only, but you can port to x86 by using the vector shift approach I used ^, and the 3 bit indices instead of 8 bit.

// Uses 64bit pdep / pext to save a step in unpacking.
__m256 compress256(__m256 src, unsigned int mask /* from movmskps */)
{
  uint64_t expanded_mask = _pdep_u64(mask, 0x0101010101010101);  // unpack each bit to a byte
  expanded_mask *= 0xFF;    // mask |= mask<<1 | mask<<2 | ... | mask<<7;
  // ABC... -> AAAAAAAABBBBBBBBCCCCCCCC...: replicate each bit to fill its byte

  const uint64_t identity_indices = 0x0706050403020100;    // the identity shuffle for vpermps, packed to one index per byte
  uint64_t wanted_indices = _pext_u64(identity_indices, expanded_mask);

  __m128i bytevec = _mm_cvtsi64_si128(wanted_indices);
  __m256i shufmask = _mm256_cvtepu8_epi32(bytevec);

  return _mm256_permutevar8x32_ps(src, shufmask);
}

Dingo

a big red shell

the great tongue

There appears to be a large yellow tongue hanging out in the desert

why not texturing?

Daylight

Daylight on an alien world

Hot reloading C++

 Here is one approach to hot reloading of C++ that I use on my side project.
It is specific to Visual Studio.

This is useful if you want to rapidly iterate on some code, and don't enjoy continually restarting and navigating the program to whatever state it needs to be in to test the change.

It is based upon swapping DLL's and requires the ability to serialize state.

Overview:
1. User modifies a C++ source file
2. File monitoring detects the change
3. Determines which projects are effected by the change.
4. Moves the existing DLL/PDB for those projects(they cannot be deleted as they are in use).
5. Fires up MSBuild to compile projects
6. Once Build Completes:

• Build Fails:(
• Build errors are propagated into the Visual Studio IDE
• Have MSBuild dump out a text file containing the error msg
• I use this command: /fl2 /fl3 /flp2:logfile=JustErrors.log;errorsonly /flp3:logfile=JustWarnings.log;warningsonly"
• The msg is formated in a way that VS understands:)
• Load the file in your game(JustErrors.log)
• Use OutputDebugStringA to print it, this propagates it to VS, the error is now visible in the errors view
• Old DLL/PDB moved back

• Build Success:
• Serialize state contained within relevant DLLs(into process owned memory)
• Serializing and then unserializing the entire game state is the simplest route and avoids any potential conflicts with mixed state between DLL's. 
• Destroy state associated with DLLs, and then unload the DLLs
• Load new DLLs, create new state
• Unserialize old state into the new state
• Resume game

Rebuilds triggered by this process generally take about 1-2 seconds for my application, the exact time will depend your code.

**Visual Studio has a bug where it keeps the PDB loaded even if you unload the associated DLL.  So you can't delete the old PDB for any DLL you previously loaded, at least not while your game is running. Hopefully they fix this at some point.  

Modify faster

I've been working on completing the compute shader path. 

This is used to calculate the color, shadow, and ambient.

If a discrete and integrated GPU are present, both are used.

After writing both an AVX & compute shader pipeline.. 

AVX is more flexible and avoids the tedium of using a terrible API.

 I hope we get AVX to 1024+ bit soon.

Compiling C++ quickly in Visual Studio

Ensuring fast C++ build speed requires some effort--

Modules should improve the situation, but that is some years off(C++17 if we are lucky).

Here is what has worked for me(using Visual Studio)
This is intended for a rapid iteration build, a build you intend to ship would not use these settings.

• Enable /MP Multiprocessor builds. 
• Disable Global optimizations
• Incremental Linking (/INCREMENTAL) and Use Library Dependency Inputs(Yes)
• Add /Zc:inline  to command line of compiler
• Linker->Optimizations: References(No), COMDAT Folding(No)
• Pre-Compiled Headers(PCH). Put rarely modified files in the PCH, but include everything you can possible include. Test the compilation times of files that use the PCH. You can enable this in Visual Studio(*2). Try to get it as low as possible.
• Limit the headers you include, especially avoid including headers within headers
• Forward declare everything that can be, even types passed by value can be forward declared
• Do not create "registry files". *example: a file containing an enum with an entry for each type of component in your engine. If you do this, you will need to add an entry to this file each time you create a new component, which will result in a large and often triggered rebuild cycle.
• If you find yourself calling from Module A to an object in Module B, but only doing so once, think about wrapping that access in a C function call. The C function can be placed in Module B, then either declared extern within Module A, or a separate header can be created if it is likely that this C function will be called multiple times. This reduces coupling between files. Changes to A will not trigger a recompilation of B.
• /PDBCOMPRESS slows the linker down, should probably not use this
• Use type erasure to reduce coupling & generate less code. For none performance critical paths, a single type erased path can be preferable to a template path per type.  
• Focus on optimizing the build speeds of any commonly used templates. There are multiple things you can do here.
• Some templates can be simple wrappers around a none templated type or function, which is itself implemented in a .cpp file, and thus not inline.
• Use variadic templates to replace old style overloads based on # of arguments.
• If there exists a finite set of types that the template can be instantiated with, try explicit template instantiation.

Templates are commonly accused of killing build times.
This is not very accurate, the real culprit is headers. Since templates exist in headers it gives the false impression templates are at fault.
 Once you have solved the exponential build time issue caused by headers, templates have a only a small effect on build times.

*2: To enable build timings: 

Tools -> Options -> Projects and solutions ->VC++ Project Settings->Build timing->yes

**Tools -> Options -> Projects and Solutions -> Build and Run and set the MSBuild project build output verbosity to "Normal"** 

Carving

Various voxel landscapes I made recently.

The carving tool is still pretty basic so I can't do anything impressive yet.

You basically can select from 5 shapes + 4 blend ops.  

But it works in real time so I can fly about and apply these ops and see the result immediately.

Chile & Argentina

Here is a link to my friends blog detailing our three month trek in South America.
Photos include famous stuff like Torres Del Pine, Tierra Del Fuego, and Fitz Roy.

Probably the most memorable bits:
-trekking through lava flows created by the 2011 eruption of Volcan Pueyhue;  40 mph winds filled with ash, visibility was almost nothing, we would stumble to the edge of chasm, gaze down, and have no idea how deep it was:) Had to use GPS/compass to navigate our way.
-living in a tiny hut on a glacier for five days crammed with 12  people, waiting for a storm to pass--

Lots of nice photos on his blog

Snowfall in Tierra Del Fuego

Poking my way across a glacier

color

Coloring algorithm is dead simple.

Eventually I want to incorporate the elevation & slope.  

It is using a (mostly) physically based lighting model, but the input materials were typed at random by me, and don't match any real world data.

Current input signals to lighting: red, green, blue, roughness, spec, ambient, shadow

bad colors

A voxel hill, so shiny

dug this out

*The colors were generated by randomly entering values between 0-255 into an array.

Sculpting Voxels In Monocrome

 Modifying the world at run time is now possible. The primary difficulty was minimizing which parts of the world needed to be rebuilt.

Yeah, I made this- what is it? No idea.

Giant letters, hoarding tiny balls

 In the following image, each of the balls is actually a separate voxel structure carved into the form of a sphere. They each have a spherical collision shape associated with them, and are able to move about within the larger world voxel shape.

Voxel balls barreling down the mountains side

 Unfortunately for this project I am heading down to South American for a few months and won't be working on it during that time. Ciao.