Perspective correction: improve approximation of width/height ratio

[pynicolas]

The current approximation of width/height ratio is based on:

• the average width
• the average height

It gives good results when the camera is at about the same distance from all four corners of the document.
However, it leads to a wrong ratio when a document is captured with a significant angle.

[darkdragon-001]

One could use the aspect ratio calculated based on the vanishing point with the (estimated) focal length.

[pynicolas]

@darkdragon-001 Thanks for the suggestion. I think we can get information about the camera but I'm not sure this is always reliable. I'll need to look into possible heuristics to solve that problem.

[pynicolas]

Done in #173
@darkdragon-001 This is basically what you described. Thanks!

[darkdragon-001]

@pynicolas Thanks for giving it a shot! Do I understand it correctly that when one pair of sides is parallel, it falls back to the old average sides algorithm? Isn't this the case in 99% of the time for top and bottom edge when taking a photo of a paper lying on the table?

Furthermore, IIRC saved photos usually contain information like the focal length in their exif data, so it could also be possible to extract this information through the camera API.

[pynicolas]

Indeed, when one pair of sides is parallel, it falls back on the average sides.
But it's not as bad as it sounds:

• the average sides method does give good results when there's very little perspective
• sides are rarely "parallel enough"

Getting the focal length of the device should be possible (at least when the user is not importing a photo) but it has to be combined with the sensor size of the device. This introduces another class of problems: it's device-dependent (harder to test), the full sensor may not always be used when capturing an image... It's something that could be introduced in the future.

Another possible way to improve width/height ratio would be to compare the detected ratio with standard paper formats like A4 or US Letter, see #22.

As many aspects of FairScan (edge detection, color detection...), results are never perfect, but each improvement is a step in the right direction. #173 was only about projective geometry and far from trivial.

[darkdragon-001]

Indeed, when one pair of sides is parallel, it falls back on the average sides. But it's not as bad as it sounds:

• the average sides method does give good results when there's very little perspective
• sides are rarely "parallel enough"

I usually scan files like

So upper and lower edge are parallel. The camera is tilted a bit in order to avoid reflections for light from windows or shadows of my camera device from a light source at the ceiling.

When tilted even more, the average sides method yields very bad results (for example almost square for a Din A4 sheet).

Another possible way to improve width/height ratio would be to compare the detected ratio with standard paper formats like A4 or US Letter, see #22.

I often scan non-standard aspect ratios like contact cards or shop receipts.