Predict a bidding in a bridge card game with Machine Learning .Net (ML.NET)

As Microsoft .Net developer and also enthusiastic bridge player I wanted to research if it is possible to handle the complete bidding process of a bridge game with machine learning. 
But first, let me explain what this bridge card game is.

What is Bridge

Contract bridge, or simply bridge, is a trick-taking card game using a standard 52-card deck. In its basic format, it is played by four players in two competing partnerships, with partners sitting opposite each other around a table, also known as North-South and East-West
Each hand of bridge is divided into four phases, which always occur in the same order: 

1. Dealing
   Someone (anyone) shuffles the deck, known as the dealer. The cards are dealt one at a time, starting with the player to the dealer’s left and moving in a clockwise rotation until each player has 13 cards.
   
   
2. Bidding
   In this phase, the four players bid for the number of tricks they think they can take. (It’s like being at an auction.) Because each player has 13 cards, 13 tricks must be fought over and won in each hand. 
   The bidding starts with the dealer and moves to his left in a clockwise rotation. Each player gets a chance to bid, and a player can either bid or pass when it’s his turn. The least you can bid is for seven tricks, and the maximum you can bid is for all 13. 
   Each bid must supersede the last preceding bid by naming a greater number of tricks in any suit, or by naming the same number of tricks in a higher ranking suit. The rank of the suits in descending order is notrump, spades, hearts, diamonds and clubs.
   The bidding goes around and around the table, with each player either bidding or passing until three players in a row say “Pass” after some bid has been made. The last bid becomes the contract. The player in the partnership that made this final bid who first bid the suit of that bid (suit or notrump) will be the declarer.
   
   
3. Playing
   The player who plays the contract, determined by the bidding, is called the declarer. The declarer is the one who will play the hand. The player seated to the left of the declarer puts down the first card face up in the middle of the table; this is the opening lead. The play moves clockwise. The next player, the dummy, places her cards face-up on the table in four vertical rows, one row for each suit, and completely bows out of the action. In other words, only three people are playing.
   Once the lead is on the table, the declarer plays any card from dummy in the suit that was led; third hand does the same, and fourth hand, the declarer, also does the same. Whoever has played the highest card in the suit wins the trick and leads any card in any suit desired to the next trick. The same process goes on for all 13 tricks. The rule is you have to follow suit if you have a card in the suit that has been led. If you don’t have a card in that suit, you can throw away (discard) any card you wish from another suit, usually some worthless card. After 13 tricks have been played, each team counts up the number of tricks it has won.
   
   
4. Scoring
   After the tricks are counted, you know whether the declarer’s team made its contract by taking at least the number of tricks they bid. You then register the score.

The deal moves in a clockwise manner; the player to the left of the person who has dealt the previous hand deals the next one and the same four phases start again.

Machine Learning

In two of the four phases, bidding and playing, we are going to research if machine learning can be used to support those phases.
We started with the bidding phase. At a later moment we are going to research the playing phase with Machine Learning as well.

Collect data

To get a trained model, you first need a lot of bridge board data. We collected bridge data from tournaments, played by bridge experts from 1996 till 2019. At the moment we have 129.000 bridge boards, 516.000 bridge hands (North, East, South, West) and 1.450.000 biddings.
An import module has been built to import the data of those tournaments into a defined database.
After the import, we have the following tables filled with data:

Board data:

Hand data:  

Auction data:
 

Train

Microsoft has developed a ML Model builder tool that supports you in getting the data out of the database, train the data based on a selected scenario, evaluate the data and at the end generate code to be used to predict test data.
The different phases in the Model builder tool:

1. Pick a scenario
2. Add data
3. Train data
4. Evaluate data
5. Generate code

Pick a scenario
Try different scenario’s to get the best results.

Add data

Train data

How long should I train for?
The table below summarizes the average time taken to get good performance for a suite of example datasets, on a local machine.

TABLE 2
Dataset size	Average time to train
0 - 10 MB	10 sec
10 - 100 MB	10 min
100 - 500 MB	30 min
500 - 1 GB	60 min
1 GB+	3+ hours

The time to train we use, is 600 seconds.

Evaluate data

As you can see the accuracy it not that high. We played a lot with the data to get a more accurate result of about 80%. And to handle maximum 28 biddings per board.

Generate code

Test the model of the trained data

We didn’t use all the data to train the model. We reserved a subset that is used as test data to test our model.
In the below image, you can see how the experts bid on a specific hand and what the prediction is on the same hand, using the trained machine learning model.


Because the predicted bidding of the biddings in the test data seem promising we developed a tool to handle the bidding phase of a bridge board.
You are always the south player.

Bidding 1

When South (you) has submitted the bidding on this hand, the machine learning model predicts the bidding of the West-, North- and East-hand.

Bidding 2

South will respond on the bidding of North, let’s say 3 spades. Again, the machine learning model predicts the bidding of the West-, North- and East-hand.

Bidding 3