GlideSorter
Smart Card Sorting Machine
Works with MTG, Pokémon, FaB and others

Spatial Color Correction

The “shooting table” has a matte, perfectly white surface. Yet an empty-frame test reveals a bluish spot in the center. Adjusting camera settings can remove it, but then the surrounding area becomes reddish. Changing the lighting didn’t help either, so I tend to believe that the sensor’s color sensitivity slightly depends on the angle at which light hits it — and the fact that we’re using an unconventional short-focus wide-angle lens only increases these distortions as we move farther from the center of the sensor. ;)

Fortunately, this can be fixed in software. Since we have a reference frame of the empty table, which should be ideally grayscale, we can compute the deviations and apply a correction matrix to all subsequent frames.

Card Detection and Cropping

At first, I planned to detect the card’s corners and crop the image based on their coordinates. But since the card always sits directly under the camera, with no expected rotation or perspective distortion, there was no real need to make things that complicated. It’s enough to compute a 1D intensity profile along each dimension and find the outermost extrema. This worked perfectly for cards with black borders, but failed on cards with white or borderless designs. After some experimentation, though, I eventually found a simple general solution that works for all cases.

Comparing Camera Images to the Reference Set

As expected, the raw camera images look somewhat washed out. Compared to the reference images, there’s also a slight yellowish shift in white balance. This is unexpected, given that the light source is neutral white and we’ve already applied spatial correction based on the white table.

Either way, the task is clear: we need a method to bring camera images as close as possible to the reference ones, which will also help minimize distances between their vector embeddings. It’s possible to tune brightness, contrast, and per-channel levels manually, but camera and lighting conditions may change over time. So it makes more sense to automate the process.

Color Correction Matrix (CCM)

A CCM is a 3×3 matrix used to adjust the color space (hue, tint, cross-channel errors). With such a matrix, we can apply fast and stable color correction to all images — the result does not depend on the scene content, which is extremely important for our workflow.

To compute the matrix, we take several correspondences between camera colors and their values in the reference images. Essentially, we form a set of pairs “as captured → as it should be” and solve a small linear regression problem to fill the 3×3 matrix. For reliability, we collected data from 32 images, calculated the CCM, and applied it to all frames to evaluate the result.

Visually, the images look noticeably better, although the contrast is still lacking — which is expected, as CCM is not designed to modify contrast.

Non-linear Look-Up Table Correction (LUT)

In simple terms, this type of color correction replaces each pixel of the original image with another value according to a lookup table (LUT). The table is a 3×255 array: the three rows correspond to the RGB channels, and the second dimension defines the mapping from input value → corrected value.

This approach is very flexible and allows for fast non-linear adjustments, including contrast enhancement, tone compression or expansion, and other operations that are not possible with a linear matrix.

To build the LUT, we use pixel values after spatial correction and after applying the CCM, and then collect correspondences between these intermediate camera images and the reference images for each color channel across a set of sample frames.

Applying the LUT Correction

The resulting images look quite promising, and it might even seem surprising that such clarity could be recovered from the original dull camera photos. In any case, I must apologize for the anticlimax — no magic was involved, just strict mathematics. ;)

On closer inspection, some images appear slightly more saturated than the originals, and a few still show small shifts in color, but overall the results look more than decent. Earlier, I tried applying the LUT without the preliminary CCM correction.

The results were acceptable, but the saturation and color shifts were noticeably stronger on certain cards, so adding the CCM step beforehand improved the consistency.

The Entire Color-Correction Pipeline

To give a more objective overview of the transformations, here they are side by side:

1. Raw camera images — washed out, with slightly shifted colors.
2. Spatial color correction — removed spatial artifacts caused by the sensor and optics, while also providing a stable white-level balance across the frame.
3. CCM correction — aligned the color space (hue, tint, cross-channel errors) with the reference images.
4. LUT correction — eliminated nonlinear distortions and increased contrast just enough to bring the images closer to the reference look.

Vectorization

The final step is to check how well these corrected images behave during vectorization, and how close the resulting vectors are to those of the reference cards.

As you can see, most distances now exceed the previously established threshold of D_aug = 1.0, but this value was only a conditional limit based on the size of the augmented reference clusters. In practice, we can increase it to 1.5, and according to preliminary analysis of the database, this will not reduce the stability of nearest-card identification.

However, increasing the threshold any further would lead to ambiguity in some cases — and indeed, we can already see a few cards exceeding this higher limit.

Analysis of the Situation

Let’s take a closer look at a couple of problematic candidates. As with the others, they share a common trait — large areas of the artwork whose brightness is very close to black. On the left is the raw camera image, with the critical regions outlined in red. In these areas the artwork is already missing: both individual strokes and entire shapes have been lost. Even after applying all the color-correction steps, these missing details cannot be recovered, simply because the original data is either too weak or not present at all.

You may have already guessed that this relates to the “step” we saw earlier on the LUT graph — that was the warning sign, and now we can observe its effects visually.