About the author: Patrick Scheibe is head of the image processing core unit at the Translational Centre for Regenerative Medicine (TRM) Leipzig. Beside working in medical image processing he loves interesting problems, is an enthusiastic user of Wolfram Mathematica and an active member of mathematica.stackexchange.com
When I come across QR Codes they are usually printed in a magazine or put as a sticker on objects. One thing they all have in common is that they are flat squares or rectangles and that you have to scan them from a direct view. This raises the questions what if you cannot ensure that the user has a direct view onto the QR Code or even worse, if you don’t have a flat surface for the Code sticker? A solution to the latter question could have some nice practical applications in the real world. You could for example put QR Codes on shopping cart handles or even inside a cup, of course on the curved surface and not on the bottom which would be trivial 🙂
Flat surfaces from a different angle
To work out a solution for curved surfaces let us evaluate the situation for flat surfaces first.
We start with a very simple example. The image on the left shows a plane viewed from some arbitrary point. Our intuition tells us instantly that this looks like a square on the floor, although if you take a ruler and measure the sides you’ll find out that all sides have different lengths while they should be equal if it really was a square.
If you put now a QR Code on this plane and try to scan it from the image’s view point, it is most likely that this will fail. Common sense tells us that the reason is simple: We are viewing this plane from the wrong perspective. A view from the top of the plane would be more appropriate.
On closer inspection you might become aware that you are currently reading on a flat 2D screen and most likely you’re having the best direct view onto it right now. What you see is and was never a 3D plane, it is the illusion of a plane in 3D space. The only thing we really have is the 3D definition of the plane and an approach to draw this onto your screen so that you have the feeling you are viewing an image of a 3D scene.
This process of creating a 2D image from a 3D scene is called Perspective Projection and it provides us with something very important: It assigns each 3D point of the scene a 2D point on our monitor. What if we could reverse this process and calculate a 3D point on the plane for some arbitrary point on the screen? With this we could easily glue a real (2D) QR Code to our monitor and calculate the corresponding points on the plane.
The image on the left shows exactly such a setting containing the same 3D plane scene as before but additionally, the image includes a QR Code which looks like it was glued to your screen.
Now imagine you would follow each black pixel of the QR Code through the virtual 3D scene until it hits the plane. You could paint the plane black at those points and you would get a projection of this QR Code onto the plane. This would create the illusion of the QR Code being upright in front of you while in reality it is drawn on the plane.
Actually, this is exactly how I created this image because what really is depicted in the above image is a warped QR Code drawn on the plane which only looks as it would be upright on your screen’s surface. If you don’t believe me you should look at this.
If you are still suspicious, because you think I tricked you into something, then there is an easy solution: try it yourself. The projection of the QR Code is just an image which can be printed. You should ensure that you leave the aspect ratio of the image intact when you print it, but otherwise you just have to scan the QR Code from about the same view as in the 3D plane image above.
Extension for curved surfaces
If you have followed the article until here you might ask the question why this should be restricted to planes? Basically we did nothing more than following QR Code pixels through the 3D scene until we hit an object. Be it a plane or something else. Therefore, exactly the same approach can be used for curved surfaces like a cylinder. The difference between a simple plane and something curved is, that it might be harder to get the 3D points of the surface because that we have given the 3D points of the surface is a fact we simply assumed until now.
Without going into detail, for many curved surfaces such a definition is available and even if it is not, then there are other approaches which could be used. The calculation of the 3D points of a cylinder and therefore doing the QR Code projection is quite easy and will serve as example because it opens us the opportunity to create scannable QR Codes for cups, shopping cart handles, pens and more. Since the outside of cups is very often heavily printed, why not put a QR Code on the inside?
In the image on the left you can take a look on the flat printed sheet of paper. This reveals that the QR Code is indeed more warped than the one of the first example and is impossible to scan when laid out flat. If you want, you can try this example for yourself too. Take a cup and measure its inner diameter d with a ruler. Calculate the circumference of the corresponding circle by d*3.14 and print the image with exactly this width in landscape. After cutting it along the grey frame of the image, the paper strip should fit more or less perfectly inside your cup.
Another example shows that even quite unusual viewpoints can be achieved with this technique. Here, the correct viewpoint is not obvious to find, since the QR Code on the paper roll is heavily distorted and leaves users unfamiliar with the approach with no clue what to do.
The situation clears up when viewed from the correct angle:
When you click on the image to see a larger version, you may notice that I haven’t put much effort in taking the photo from the perfect viewpoint. Nevertheless, the QR Code can be scanned easily.
Although in theory a lot of things are possible, some practical considerations should be discussed. The first thing you should think about is, that if you are really going to print a QR code and glue it onto the surface, you have to be able to affix it flat onto the curved surface. For the round part of a cylinder this is no problem as depicted in the last images, but if you ever tried to wrap a ball into a sheet of paper, you recognize that this is not possible. Therefore, QR Code stickers will not work in all situations.
Another consideration is the viewpoint. When someone sees one of the usual squared QR Codes, it’s quite obvious to him that this has to be scanned from a direct view. Finding the right viewpoint for a heavily transformed Code is neither easy nor obvious for the layman. Therefore, if you are going to use the described technique, you should ensure that the user has no other option than scanning it from the right angle. A good example is the image of the cup above, which is pretty easy to scan, because the viewpoint from which you see the QR Code completely is close to the perfect angle and should work with most scanning devices.
One crucial point which needs consideration is the scanning device itself. Mostly this will be a smart phone which uses the built-in camera. Depending on the optical system of the camera, the view angle and the distance to the photographed QR Code it might not be possible to get the QR Code acceptably sharp.
This is called depth of field and for very extreme perspectives it might not be possible the get a sharp image of the whole QR Code which prevents it from being scanned.
In summary one can say that we have seen it is possible to create the illusion of an upright QR Code for perspectives other than the direct view. Furthermore, this approach can be extended to work for curved surfaces as well. If this technique can be used in real life remains to be seen, but there are clearly situations where transformed QR Code would open new opportunities.
The purpose of this article was to explain the approach of QR Code transformation in an understandable way, as far as possible. Therefore, I used neither formula nor did I give source code which makes it impossible to use the method yourself. If you want to know the details of the approach, I hope the following links give you a start:
The most important resource is probably my answer to the question QR Code in shopping cart handle on Mathematica.stackexchange.com. In the answer there I give a detailed explanation how the approach can be implemented using Wolfram Mathematica and I provide functions for putting QR Codes on shopping cart handles.
For explanations about the required mathematics, you can start reading on Wikipedia to get a first overview. There you find detailed information about 3D projections, camera projections and homogeneous coordinates which are used in computer graphics to express all kinds of transformations easily.
A nice reference to computer graphics using OpenGL is the so-called Red Book which gives good explanations about viewing transformations or the camera model in chapter 3.