Last time we reached the point where we had an in memory representation of the game. Today we are going to make some decisions based on the game state. Basically we will decide about an empty field if it is a bomb or it is safe. Let’s consider the following input image (I’ve cut out the excess, the rest is just empty squares).

If you are an experienced player, you will instantly spot that there are a lot of information available here. This is how the magic happens.

Let me start by saying, playing around with OpenCV was a lot of fun. I’m genuinely impressed by it and I consider it a very useful resource to have it on your tool belt. What I tried to achieve in this part is having the game state in a processable manner. Currently what I get from the game is an image. We humans understand it perfectly well but in order to write an algorithm which will make some decision based on the game state. I need to extract the meaning out of it. Here is where OpenCV comes in.

My plan is simple.

1. Cut out each cell the game can have on board into a separate png.
2. Iterate over the cutouts and check where do they turn up in the image.
3. Extract the findings into an array which I can process further

To shortcut testing I loaded up a print screen of the game and analyzed that instead of the image that comes from the game. We will handle wiring everything together in a different post. This I what our input looks like.

The following code might look complicated but the process is actually simple.

In my last post I told you about the steps we need to be able to make in order to solve Minesweeper. I spent some time to achieve the first step, but it was a roller-coaster of emotions, mainly involving anger.

My initial approach was to write a C# console application that:

• 1. Search through the processes where the name matched Microsoft Minesweeper.
• 2. Read the handle to it’s window through the MainWindowHandle property on the ProcessInfo object.
• 3. DllImport the GetWindowRect function from WinApi and use it to find the screen-coordinates of that window
• 4. Create a Bitmap object with the appropriate size
• 5. Attach a Graphics object to the bitmap with the Graphics.FromImage function and use CopyFromScreen to, well copy the game window content into the Bitmap