Why does my GND density change when I change the step size?

The specimen below is a synthetic polycrystal, the same one all three panels on this page use: grains with their own orientations and their own lattice curvature, sub-grain cells inside them, and a slice cut through it at a depth you choose. The black lines on the KAM map are the grain boundaries: the neighbour pairs thrown away because they are over the 5° threshold. That exclusion is the threshold doing its job, and it is also the first hint that the kernel has a size it must stay under. The default specimen is a fine-grained one, 150 grains in a 2.5 µm cube, so the grains are around 500 nm across and there is a great deal of boundary in the way.

The trouble starts at the next step, when that number is turned into a dislocation density. The standard route is ρ = αθ/(nδb), where θ is the KAM value, δ is the step size, n is how many neighbour shells the kernel spans, b is the Burgers vector, and α is 2 for a pure tilt arrangement, 4 for pure twist, 3 for mixed. Every symbol in there is something you chose except θ and b, and one of the things you chose is the step size.

Set the angular precision to perfect indexing, the sub-grain cells to none, and drag the step size. KAM itself climbs, doubling as the step doubles, because a bigger step spans more of the same bend. The dislocation density it implies does not move: 45 nm and 90 nm return the identical number. That cancellation is the whole reason the formula is allowed to exist.

Keep dragging and it falls apart, and not because of noise, because there is none. Watch the third number, the fraction of neighbour pairs thrown away by the 5° threshold. It runs 12%, 24%, 46%, 79%, and then 100%. The grains in this specimen are around 500 nm across, so by a 360 nm step most of what the kernel reaches for is on the other side of a boundary, and at 720 nm every single pair is. The density follows it down and then reaches zero: not a small number, nothing at all, because there is no pair left to average.

So on this material the step cancellation is exact over about one doubling and then stops being true. That is not a defect of the simulation, it is what a fine-grained material does to a kernel method. Make the grains bigger and the window widens in proportion; the ratio that matters is the kernel size against the grain size, not either one on its own.

One subtlety worth carrying: a four-neighbour kernel averages over two directions, and a bend that runs in only one of them contributes to half the pairs. The measured KAM comes out at half the true curvature times the step, so the implied density is half the density you would get from the curvature directly. That factor is a property of your kernel, not of your specimen. It is the first sign that this number is a relative measure.

The floor line is not a model. It is the same kernel, the same formula and the same noise, run on the same grain structure with the curvature switched off: whatever it reports is noise by construction. It falls roughly as 1/δ and scales linearly with your angular precision. Only roughly, because the microstructure interferes: as the step coarsens, more of the kernel's pairs cross grain boundaries and get thrown away, so the fitted slope drifts above 1, and drifts further the finer the grain structure. On this specimen it runs from about 1.1 at 100 grains to 1.6 at 400. A real noise floor is not a clean power law either, for exactly this reason.

Put the two limits together and what you have is a window, not a direction. Too fine and you are measuring your indexing routine: at 45 nm with Hough-grade indexing this specimen reports around twenty times its real dislocation density. Too coarse and the kernel is eating your grains, until at 720 nm the map returns nothing at all. Between those, at 360 nm, the measurement finally clears twice its own noise floor, and by then it is reading about half the true density because the kernel is already discarding four pairs in five.

That is the uncomfortable part. On a material with 500 nm grains and half a degree of angular precision, there is no step size that is both quiet enough and small enough to be right. Every setting on that axis is wrong, in one direction or the other, and the map looks equally plausible at all of them.

The three precision presets are not invented. Conventional Hough-based indexing lands near half a degree; refined template or dictionary matching reaches about 0.1°, measured on a silicon single crystal against roughly 0.4° for Hough on the same patterns; and high angular resolution EBSD, which cross-correlates against a reference pattern, reaches 1×10⁻⁴ rad, about 0.006°. Two orders of magnitude between the ends of that list, and the noise floor moves with it one for one.

The practical rule this gives you: pick the step so that the real misorientation across one step comfortably exceeds your angular precision, then check that the kernel still fits inside your grains. With half a degree of noise and a lattice bending at 2° per micron, one step has to be a good fraction of a micron before the bend is even visible, which is far coarser than the step most people scan at. Scanning finer does not buy resolution here; it buys noise, amplified.

Scanning coarser stops working as soon as the kernel starts spanning boundaries. When the two conditions do not overlap, and on a fine-grained material they often do not, KAM is the wrong tool for that specimen and no amount of tuning makes it the right one. Better angular precision is what opens the window, not a better choice of step: HR-EBSD moves the noise limit down by two orders of magnitude and leaves the grain-size limit exactly where it was.

Below is a synthetic polycrystal, built as a volume rather than a picture: Voronoi grains filling a 2.5 µm cube, each grain carrying its own orientation and its own lattice curvature, with sub-grain cell walls inside them. Because the volume exists, the Nye tensor can be computed voxel by voxel from all three gradient directions, and that is the ground truth. Then one slice is cut through it, the way a scan samples a polished surface, and the same tensor is computed again from that slice alone. Move the slice up and down and you are looking at maps a scan could actually have returned from the same specimen.

The tensor is only computed where the whole differencing stencil sits inside one grain. Across a boundary the orientation gradient is not a lattice curvature at all, it is a discontinuity, and the Nye tensor is undefined there. That is the pale border you can see around every grain in the third map, and it is a real limitation of GND mapping rather than a shortcut taken here.

Five of the nine components survive the slice: α₁₂, α₁₃, α₂₁, α₂₃ and α₃₃. The last of those is the interesting one, because κ₃₃ is not measurable but the trace lets you write α₃₃ = −κ₁₁ − κ₂₂, and both of those are in the plane. The four that are lost, α₁₁, α₂₂, α₃₁ and α₃₂, each need a gradient normal to the surface, and there is no such gradient in a surface map. Set the curvature direction to “only through the thickness” and the sub-grain cells to none, and the slice reports nothing at all (0% recovered) while the volume reports a real dislocation content.

Because every grain is bent in its own direction, the answer is not one number but a distribution. On this volume a slice recovers a median of about 70% of the dislocation content (68 to 75% depending on where the cut is taken), and the spread within a single map is large: one pixel in ten comes back at 90% or more, while another one in ten gets only 30 to 50% of its true content, side by side, in the same scan, indexed identically. Which pixels do badly is not a property of the material and not something better scanning fixes. It depends only on how that grain's curvature happens to sit relative to the surface you polished.

Drag the slice depth and watch the unlucky tenth move. On this volume it ranges from 30% to 53% depending on where the cut is taken: the same specimen, the same instrument, the same operator, polished a few hundred nanometres deeper. If you have ever wondered why two GND maps of the same material do not agree, this is one of the reasons, and it is invisible from inside either map.

So what is the GND density you are quoting? A lower bound, from five of nine components, under an assumption about dislocation character that fixes α, computed with a kernel whose geometry sets the prefactor, at a step size that sets the noise floor. That is a genuinely useful quantity for comparing one region of one map with another region of the same map, acquired the same way. It is not a dislocation count, and two GND maps from different laboratories are not comparable unless every one of those choices matches.