Hierarchical data representations

How images (or volumes!) can be represented as a hierarchy

Hierarchical data structures can help scientists with all kinds of problems. When I refer to hierarchies, I mean anything that has a tree-like or casicading structure. These hierarchies allow data to be represented with different ganularity, or in other terms, different levels of detail (LOD), in different areas. For instance, in fluid simulations, if there are few fluid particles in a region, we do not need to do fine-grained simulation there. This allows us to focus our computation resources in places that need it most.

Responsive image
Examples of hierarchical data use in a few papers. The top row is from "AMM: Adaptive Multilinear Meshes", the middle row is from "CPU Isosurface Ray Tracing of Adaptive Mesh Refinement Data", and the bottom row is from "Octree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputs"

We can also use this hierarchy to save data more efficiently. If a large portion of your image is the same color, why should we save that same pixel thousands of times? Different techniques do this in different ways, but popular methods are quadtrees/octrees, k-d trees, and adaptive mesh refinement (AMR).

In some of my recent work, I create an algorithm for compressing a volume (or image in this case) into an hierarchical representation similar to an octree(quadtree) using adaptive mesh refinement. I develop an algorithm that "super resolves", or upscales, hierarchical data into a uniform high resolution. Any super resolution algorithm can be use - bilinear interpolation, bicubic interpolation, and even neural networks (which I use in my paper!). Second, I create an adaptive refinement algorithm that reduces levels of detail (LOD) where able such that my super resolution algorithm can still super resolve the details with some minimum error that is specified by a user. Below is a tool for playing with this algorithm using images. A number of parameters can be tweaked:

Generally, upscaling and downscaling technique should stay how they are. If interested in the code, you cand find it here.

Criteria value:

Maximum Downscaling:
Minimum chunk size:

Be sure that entered values are properly formatted for integers or floats where expected, with no training spaces.

WARNING: minimum chunk sizes 32 and lower paired with relatively high criteria values (high PSNR, low MRE/MSE) can be very slow to compute. Can take up to 40 minutes with extreme settings. Expected load time for default settings above is roughly 1 minute.

PSNR: 89.02, MSE: 8.85, MRE: 0.0500

Input image Responsive image
Final image data in hierarchy Responsive image
Levels of details in the hierarchy Responsive image
Final super resolved hierarchy Responsive image