Introduction

So far, our 2.5d navigation did not use landmarks. In this section I am going to fix the problem by using aruco markers, probably, the easiest landmark one can imagine:

Aruco markers were already introduces in earlier chapters. It is a bar code that encodes a number - an easy to recognize using computer vision element.
Here is an aruco marker encoding "42":

However, before we move to implementation, let's talk about choice of tools (why Aruco and is Aruco a good idea) as well as about what we do and can it be done better.

In robotics, landmark is not just the mountain or a roas sign. It is any part of an image that is stable within the period we are interested in and recognizable within the range we expect the robot to move. Spot on the wall is a landmark. corner of a room where horizontal and vertical lines come together is a landmark.

So there are landmarks we find as we go, and landmarks we have predefined. In the second case, we know their exact location, in the first case, we can only estimate it. Which means, the math will be different. And the use of these landmarks will be different, too.

Because as we estimate the location of a spot on the wall, we make errors, and these errors build up. It is like using odometer. While if we know landmark's location (if we see a house with unique features, or an aruco marker with unique id and we can look it up in a database), then we have no error and no drift; this is like using GPS.

What I am trying to say, there is more than one definition of a "landmark" and more than one way of implementing it in code.

Much more, actually.

For example, in this section I use Aruco markers. I place them on a posts that are 2 meters high, so they can be visible from the distance, and one post has 5 markers (same id!), one on each side and one on top. That means we can determine distance and angle between robot and a post, but we do not know which of five identical markers robot is looking at! So we need to see 2 posts (not necessarily in the same time) to do triangulation. Unless we know initial position or something like that.

Now, let's say I placed 5 different unique markers on each post. In that case we can perform localization after seeing just one post, and with no prior info (no initPos).

Ok, we have either uniquely identifiable landmarks, or feature points, like spot on the wall. What else?

Is list of landmarks predefined? In other words, can we dynamically add a landmark? Kalman filter runs on a fixed size set of features, so in our case, we can not run on two landmarks, and then arr another one. Well, not like that. We can make a list of, say, 10 landmarks, use some of them, and as robot moves away from some landmarks and towards some other, we replace landmarks in the list, while keeping list size the same.

But a. we can not dynamically change size of that list, not with the UKF I am using, and b. Having a fixed size list limits number of landmarks... which might be bad for accuracy. And processing 10 landmarks, while only two are really used, is bad for performance.

There are articles that are trying to solve this problem, but I am not going there.

Even if we know coordinates of predefined landmarks (we have map or database), can we distinguish landmarks? We can distinguish aruco markers, but not these black and yellow stripped posts they place along the road. So we might (I will do it in later sections) need a tracker that can look at a "cloud" of landmarks and state "this landmark was here on the previous frame".

Is landmark large? In other words, is distance to post's center different from the distance to an aruco marker on its side? In this section, marker is 1m and post is 1x1x2m, so error is 0.5m, and I ignore it; for outdoor navigation it is ok. Notice, that if I place 5 unique markers on a post, error would be 0.

It is also ok for indoor navigation, as there, most likely, aruco marker will be placed on a wall, and error will be 0. However, if landmark is a house 10 meters away, and house is 10 meters "deep", and its position in the database is set by its center, and robot gets distance to a front wall... The error will be 5 meters, which requires correction.

This list can easily become infinite. For example.

When you tell a human how to navigate, you would do something like this: go 1 mile North, then when you see a river (landmark) turn left.

So the river is a landmark, but how will that human go North, if he lacks magnetometer? Here we go: the Sun is a landmark, too. And a good one: you can see from all around the world.

Stars are landmarks, too. Yet, Sun and stars move, so we need to implement some math to make sure we know where they are at a particular moment. Which brings us to an interesting point...

... other robots are landmarks, too! As they move around, our robot can see them, measure angle to that robot and distance to it (or just angle, or just distance) and contact either that robot or some kind of a monitoring node, asking for that robot's coordinates (and sigmas). That might work.

Conclusion: "landmarks based localization" is a very vague concept. In this section, I am only going to use aruco markers. In future sections, I'll add more.