The end of the last post was very satisfying for me, I was very proud of myself. The problem is, we could have done a better job. If you check out the last video you can clearly see that between rounds, our algorithm takes multiple seconds to start making decisions. This is because the Imaging Analysis part has some inefficiencies. After thinking about it I came up with a following issues.

1. We analyse the whole image even though the game grid is only a sub-part of it.
2. We keep scanning for cells and we don’t take advantage of the fact that once a cell was discovered, it’s value cannot change.
3. We calculate the centers each time even though cells are always in the same place.
4. We are scanning over areas that previously were detected as other values

Lucky all of these things can be improve by rethinking our strategy a bit. There are some values we need to extract only once and never again, and there are some things that we need to do every time. Let me show you how does the beginning of our application look like.

So last time we analysed the information in the game state, and we made decisions regarding the cells where we had sufficient information to decide which is a bomb and which is safe. In this post i show you how i implemented the last part and at the end i will show you a video, but first…

Let me start by saying F**K THIS GAME. I thought it is a lot more deterministic, but it turns out it takes a lot of guessing. I needed to implement the ability to guess if there is no concrete action we can take. The best place to add this was in the ScanGame function, because we already traverse the whole game. Now it looks like this:

void ScanGame(int* game, set<int>& bombs, set<int>& safes, int* g)
{
	int maxNeighbor = 0;
	for (int i = 0; i < SIZE; i++)
	{
		int z = game[i];
		if (z > 0 && z < BOMB)
		{
			vector<int> unk;
			int b = 0;
			ScanCell(game, i, &b, unk);
			int u = unk.size();
			int outcome = 0;
			if (z == b && u > 0)// not bomb
			{
				safes.insert(unk.begin(), unk.end());
			}
			else if (z != b && z - b == u) // bomb
			{
				bombs.insert(unk.begin(), unk.end());
			}
			else if (safes.size() + bombs.size() == 0 && u > 0)
			{
				auto possibilities = u - (z - b);
				if (maxNeighbor < possibilities)
				{
					maxNeighbor = possibilities;
					(*g) = unk[0];
				};
			}
		}
	}
}

Last time we reached the point where we had an in memory representation of the game. Today we are going to make some decisions based on the game state. Basically we will decide about an empty field if it is a bomb or it is safe. Let’s consider the following input image (I’ve cut out the excess, the rest is just empty squares).

If you are an experienced player, you will instantly spot that there are a lot of information available here. This is how the magic happens.

Let me start by saying, playing around with OpenCV was a lot of fun. I’m genuinely impressed by it and I consider it a very useful resource to have it on your tool belt. What I tried to achieve in this part is having the game state in a processable manner. Currently what I get from the game is an image. We humans understand it perfectly well but in order to write an algorithm which will make some decision based on the game state. I need to extract the meaning out of it. Here is where OpenCV comes in.

My plan is simple.

1. Cut out each cell the game can have on board into a separate png.
2. Iterate over the cutouts and check where do they turn up in the image.
3. Extract the findings into an array which I can process further

To shortcut testing I loaded up a print screen of the game and analyzed that instead of the image that comes from the game. We will handle wiring everything together in a different post. This I what our input looks like.

The following code might look complicated but the process is actually simple.

In my last post I told you about the steps we need to be able to make in order to solve Minesweeper. I spent some time to achieve the first step, but it was a roller-coaster of emotions, mainly involving anger.

My initial approach was to write a C# console application that:

• 1. Search through the processes where the name matched Microsoft Minesweeper.
• 2. Read the handle to it’s window through the MainWindowHandle property on the ProcessInfo object.
• 3. DllImport the GetWindowRect function from WinApi and use it to find the screen-coordinates of that window
• 4. Create a Bitmap object with the appropriate size
• 5. Attach a Graphics object to the bitmap with the Graphics.FromImage function and use CopyFromScreen to, well copy the game window content into the Bitmap