Merge branch 'dmc' of https://github.com/datamllab/rlcard-showdown into dmc

# Conflicts:
#	README.md
#	requirements.txt
#	server/media/example_agents/leduc_holdem_dqn.zip

.gitignore

@@ -31,3 +31,6 @@ uploaded_agents
 /.idea
 
 package-lock.json
+
+douzero_pretrained
+dmc_pretrained

README.md

@@ -1,32 +1,25 @@
 # RLCard Showdown
-This is the GUI support for the [RLCard](https://github.com/datamllab/rlcard) project. The project provides evaluation and visualization tools to help understand the performance of the agents. Currently, we only support Leduc Hold'em and Dou Dizhu. The frontend is developed with [React](https://reactjs.org/). The backend is based on [Django](https://www.djangoproject.com/). Have fun!
+This is the GUI support for the [RLCard](https://github.com/datamllab/rlcard) project and DouZero project. RLCard-Showdown provides evaluation and visualization tools to help understand the performance of the agents. It includes a replay module, where you can analyze the replays, and a PvE module, where you can play with the AI interactively. Currently, we only support Leduc Hold'em and Dou Dizhu. The frontend is developed with [React](https://reactjs.org/). The backend is based on [Django](https://www.djangoproject.com/) and [Flask](https://flask.palletsprojects.com/). Have fun!
 
 *   Official Website: [http://www.rlcard.org](http://www.rlcard.org)
 *   Tutorial in Jupyter Notebook: [https://github.com/datamllab/rlcard-tutorial](https://github.com/datamllab/rlcard-tutorial)
 *   Paper: [https://www.ijcai.org/Proceedings/2020/764](https://www.ijcai.org/Proceedings/2020/764)
-*   Document: [click here](docs/README.md)
+*   Document: [Click Here](docs/README.md)
 
 ## Cite this work
-If you find this repo useful, you may cite:
+Zha, Daochen, Kwei-Herng Lai, Songyi Huang, Yuanpu Cao, Keerthana Reddy, Juan Vargas, Alex Nguyen, Ruzhe Wei, Junyu Guo, and Xia Hu. "RLCard: A Platform for Reinforcement Learning in Card Games." In IJCAI. 2020.
-```bibtext
+
-@inproceedings{ijcai2020-764,
+```bibtex
+@inproceedings{zha2020rlcard,
   title={RLCard: A Platform for Reinforcement Learning in Card Games},
   author={Zha, Daochen and Lai, Kwei-Herng and Huang, Songyi and Cao, Yuanpu and Reddy, Keerthana and Vargas, Juan and Nguyen, Alex and Wei, Ruzhe and Guo, Junyu and Hu, Xia},
-  booktitle = {Proceedings of the Twenty-Ninth International Joint Conference on
+  booktitle={IJCAI},
-               Artificial Intelligence, {IJCAI-20}},
+  year={2020}
-  publisher = {International Joint Conferences on Artificial Intelligence Organization},             
-  editor    = {Christian Bessiere},	
-  pages     = {5264--5266},
-  year      = {2020},
-  month     = {7},
-  note      = {Demos}
-  doi       = {10.24963/ijcai.2020/764},
-  url       = {https://doi.org/10.24963/ijcai.2020/764},
 }
 ```
 
 ## Installation
-RLCard-Showdown has separated frontend and backend. The frontend is built with React and the backend of leaderboard is based on Django.
+RLCard-Showdown has separated frontend and backend. The frontend is built with React and the backend is based on Django and Flask.
 
 ### Prerequisite
 To set up the frontend, you should make sure you have [Node.js](https://nodejs.org/) and NPM installed. Normally you just need to manually install Node.js, and the NPM package would be automatically installed together with Node.js for you. Please refer to its official website for installation of Node.js.
@@ -41,7 +34,7 @@ For backend, make sure that you have **Python 3.6+** and **pip** installed.
 ### Install Frontend and Backend
 The frontend can be installed with the help of NPM:
 ```
-git clone --depth 1 https://github.com/datamllab/rlcard-showdown.git
+git clone -b master --single-branch --depth=1 https://github.com/datamllab/rlcard-showdown.git
 cd rlcard-showdown
 npm install
 ```
@@ -54,27 +47,39 @@ cd ..
 ```
 
 ### Run RLCard-Showdown
-Launch the backend of leaderboard with
+1. Launch the backend of leaderboard with
 ```
 cd server
 python3 manage.py runserver
 ```
-Run the following command in a new terminal under the project folder to start frontend in development mode:
+2. Download the pre-trained models in [Google Drive](https://drive.google.com/file/d/1zx-20xNBDbCFd8GWhZFUkl07lofbNHpy/view?usp=sharing) or [百度网盘](https://pan.baidu.com/s/12MgxVBBz4mgitT74quSWfw) 提取码: qh6s. Extract it in `pve_server/pretrained`.
+
+In a new terminal, start the PvE server (i.e., human vs AI) of DouZero with
+```
+cd pve_server
+python3 run_douzero.py
+```
+Alternatively, you can start the PvE server interfaced with RLCard:
+```
+cd pve_server
+python3 run_dmc.py
+```
+They are conceptually the same with minor differences in state representation and training time of the pre-trained models (DouZero is fully trained with more than a month, while DMC in RLCard is only trained for hours).
+
+3. Run the following command in another new terminal under the project folder to start frontend:
 ```
 npm start
 ```
-You can view frontend at [http://127.0.0.1:3000/](http://127.0.0.1:3000/). The backend of leaderboard will run in [http://127.0.0.1:8000/](http://127.0.0.1:8000/).
+You can view leaderboard at [http://127.0.0.1:3000/](http://127.0.0.1:3000/) and PvE demo of Dou Dizhu at [http://127.0.0.1:3000/pve/doudizhu-demo](http://127.0.0.1:3000/pve/doudizhu-demo). The backend of leaderboard will run in [http://127.0.0.1:8000/](http://127.0.0.1:8000/). The PvE backend will run in [http://127.0.0.1:5000/](http://127.0.0.1:5000/).
 
-More documentation can be found [here](docs/api.md). User guide is [here](docs/guide.md).
+## Demos
-
-### Demos
 ![leaderboards](https://github.com/datamllab/rlcard-showdown/blob/master/docs/imgs/leaderboards.png?raw=true)
 ![upload](https://github.com/datamllab/rlcard-showdown/blob/master/docs/imgs/upload.png?raw=true)
 ![doudizhu-replay](https://github.com/datamllab/rlcard-showdown/blob/master/docs/imgs/doudizhu-replay.png?raw=true)
 ![leduc-replay](https://github.com/datamllab/rlcard-showdown/blob/master/docs/imgs/leduc-replay.png?raw=true)
 
-### Contact Us
+## Contact Us
 If you have any questions or feedback, feel free to drop an email to [Songyi Huang](https://github.com/hsywhu) for the frontend or [Daochen Zha](https://github.com/daochenzha) for backend.
 
-### Acknowledgements
+## Acknowledgements
-We would like to thank JJ World Network Technology Co., LTD for the generous support, [Chieh-An Tsai](https://anntsai.myportfolio.com/) for user interface design, and [Lei Pan](mailto:lpa25@sfu.ca) for the help in visualizations.
+We would like to thank JJ World Network Technology Co., LTD for the generous support, [Chieh-An Tsai](https://anntsai.myportfolio.com/) for user interface design, and [Lei Pan](https://github.com/lpan18) for the help in visualizations.

docs/leaderboard_api.md

@@ -38,7 +38,7 @@ The definitions of the fields are as follows:
 | http://127.0.0.1:8000/tournament/query_game?name=leduc-holdem&elements_every_page=10&page_index=0                                    | Get all the game data of Leduc Holdem                                                    |
 | http://127.0.0.1:8000/tournament/query_payoff                                                                                           | Get all the payoffs                                                                      |
 | http://127.0.0.1:8000/tournament/query_payoff?agent0=leduc-holdem-cfr&agent1=leduc-holdem-rule-v1                                       | Get all the payoffs between rule and CFR models                                          |
-| http://127.0.0.1:8000/tournament/query_agent_payoff?name=leduc-holdem&elements\_every\_page=1&page\_index=1                            | Get the payoffs of all the agents of leduc-holdem                                        |
+| http://127.0.0.1:8000/tournament/query_agent_payoff?name=leduc-holdem&elements_every_page=1&page_index=1                            | Get the payoffs of all the agents of leduc-holdem                                        |
 | http://127.0.0.1:8000/tournament/list_uploaded_agents?game=leduc-holdem                                                                | List the uploaded agents of leduc-holdem                                                 |
 | http://127.0.0.1:8000/tournament/list_baseline_agents?game=leduc-holdem                                                                | List the baseline agents of leduc-holdem                                                 |
 | http://127.0.0.1:8000/tournament/download_examples?name=example_luduc_nfsp_model                                                     | Download the NFSP example model for Leduc Hold'em                                        |

pve_server/.gitignore

@@ -0,0 +1,7 @@
+*.swp
+*.so
+__pycache__
+.DS_Store
+*.egg-info
+*.pyc
+*.onnx

pve_server/deep.py

@@ -0,0 +1,273 @@
+import os
+
+import torch
+import numpy as np
+from collections import Counter
+
+Card2Column = {3: 0, 4: 1, 5: 2, 6: 3, 7: 4, 8: 5, 9: 6, 10: 7,
+               11: 8, 12: 9, 13: 10, 14: 11, 17: 12, 20: 13, 30: 14}
+
+NumOnes2Array = {0: np.array([0, 0, 0, 0]),
+                 1: np.array([1, 0, 0, 0]),
+                 2: np.array([1, 1, 0, 0]),
+                 3: np.array([1, 1, 1, 0]),
+                 4: np.array([1, 1, 1, 1])}
+
+def _get_one_hot_bomb(bomb_num):
+    one_hot = np.zeros(15, dtype=np.float32)
+    one_hot[bomb_num] = 1
+    return one_hot
+
+def _load_model(position, model_dir, use_onnx):
+    if not use_onnx or not os.path.isfile(os.path.join(model_dir, position+'.onnx')) :
+        from models import model_dict
+        model = model_dict[position]()
+        model_state_dict = model.state_dict()
+        model_path = os.path.join(model_dir, position+'.ckpt')
+        if torch.cuda.is_available():
+            pretrained = torch.load(model_path, map_location='cuda:0')
+        else:
+            pretrained = torch.load(model_path, map_location='cpu')
+        pretrained = {k: v for k, v in pretrained.items() if k in model_state_dict}
+        model_state_dict.update(pretrained)
+        model.load_state_dict(model_state_dict)
+        if torch.cuda.is_available():
+            model.cuda()
+        model.eval()
+
+        if use_onnx:
+            z = torch.randn(1, 5, 162, requires_grad=True)
+            if position == 'landlord':
+                x = torch.randn(1, 373, requires_grad=True)
+            else:
+                x = torch.randn(1, 484, requires_grad=True)
+            torch.onnx.export(model,
+                              (z,x),
+                              os.path.join(model_dir, position+'.onnx'),
+                              export_params=True,
+                              opset_version=10,
+                              do_constant_folding=True,
+                              input_names = ['z', 'x'],
+                              output_names = ['y'],
+                              dynamic_axes={'z' : {0 : 'batch_size'},
+                                            'x' : {0 : 'batch_size'},
+                                            'y' : {0 : 'batch_size'}})
+
+    if use_onnx:
+        import onnxruntime
+        model = onnxruntime.InferenceSession(os.path.join(model_dir, position+'.onnx'))
+    return model
+
+def _process_action_seq(sequence, length=15):
+    sequence = sequence[-length:].copy()
+    if len(sequence) < length:
+        empty_sequence = [[] for _ in range(length - len(sequence))]
+        empty_sequence.extend(sequence)
+        sequence = empty_sequence
+    return sequence
+
+class DeepAgent:
+
+    def __init__(self, position, model_dir, use_onnx=False):
+        self.model = _load_model(position, model_dir, use_onnx)
+        self.use_onnx = use_onnx
+
+    def cards2array(self, list_cards):
+        if len(list_cards) == 0:
+            return np.zeros(54, dtype=np.float32)
+
+        matrix = np.zeros([4, 13], dtype=np.float32)
+        jokers = np.zeros(2, dtype=np.float32)
+        counter = Counter(list_cards)
+        for card, num_times in counter.items():
+            if card < 20:
+                matrix[:, Card2Column[card]] = NumOnes2Array[num_times]
+            elif card == 20:
+                jokers[0] = 1
+            elif card == 30:
+                jokers[1] = 1
+        return np.concatenate((matrix.flatten('F'), jokers))
+
+    def get_one_hot_array(self, num_left_cards, max_num_cards):
+        one_hot = np.zeros(max_num_cards, dtype=np.float32)
+        one_hot[num_left_cards - 1] = 1
+
+        return one_hot
+
+    def action_seq_list2array(self, action_seq_list):
+        action_seq_array = np.zeros((len(action_seq_list), 54), dtype=np.float32)
+        for row, list_cards in enumerate(action_seq_list):
+            action_seq_array[row, :] = self.cards2array(list_cards)
+        action_seq_array = action_seq_array.reshape(5, 162)
+        return action_seq_array
+
+    def act(self, infoset):
+        player_position = infoset.player_position
+        num_legal_actions = len(infoset.legal_actions)
+        my_handcards = self.cards2array(infoset.player_hand_cards)
+        my_handcards_batch = np.repeat(my_handcards[np.newaxis, :],
+                                       num_legal_actions, axis=0)
+
+        other_handcards = self.cards2array(infoset.other_hand_cards)
+        other_handcards_batch = np.repeat(other_handcards[np.newaxis, :],
+                                          num_legal_actions, axis=0)
+
+        my_action_batch = np.zeros(my_handcards_batch.shape, dtype=np.float32)
+        for j, action in enumerate(infoset.legal_actions):
+            my_action_batch[j, :] = self.cards2array(action)
+
+        last_action = self.cards2array(infoset.rival_move)
+        last_action_batch = np.repeat(last_action[np.newaxis, :],
+                                      num_legal_actions, axis=0)
+
+        if player_position == 0:
+            landlord_up_num_cards_left = self.get_one_hot_array(
+                infoset.num_cards_left[2], 17)
+            landlord_up_num_cards_left_batch = np.repeat(
+                landlord_up_num_cards_left[np.newaxis, :],
+                num_legal_actions, axis=0)
+
+            landlord_down_num_cards_left = self.get_one_hot_array(
+                infoset.num_cards_left[1], 17)
+            landlord_down_num_cards_left_batch = np.repeat(
+                landlord_down_num_cards_left[np.newaxis, :],
+                num_legal_actions, axis=0)
+
+            landlord_up_played_cards = self.cards2array(
+                infoset.played_cards[2])
+            landlord_up_played_cards_batch = np.repeat(
+                landlord_up_played_cards[np.newaxis, :],
+                num_legal_actions, axis=0)
+
+            landlord_down_played_cards = self.cards2array(
+                infoset.played_cards[1])
+            landlord_down_played_cards_batch = np.repeat(
+                landlord_down_played_cards[np.newaxis, :],
+                num_legal_actions, axis=0)
+
+            bomb_num = _get_one_hot_bomb(
+                infoset.bomb_num)
+            bomb_num_batch = np.repeat(
+                bomb_num[np.newaxis, :],
+                num_legal_actions, axis=0)
+
+            x_batch = np.hstack((my_handcards_batch,
+                                 other_handcards_batch,
+                                 last_action_batch,
+                                 landlord_up_played_cards_batch,
+                                 landlord_down_played_cards_batch,
+                                 landlord_up_num_cards_left_batch,
+                                 landlord_down_num_cards_left_batch,
+                                 bomb_num_batch,
+                                 my_action_batch))
+            z = self.action_seq_list2array(_process_action_seq(
+                infoset.card_play_action_seq))
+            z_batch = np.repeat(
+                z[np.newaxis, :, :],
+                num_legal_actions, axis=0)
+            if self.use_onnx:
+                ort_inputs = {'z': z_batch, 'x': x_batch}
+                y_pred = self.model.run(None, ort_inputs)[0]
+            elif torch.cuda.is_available():
+                y_pred = self.model.forward(torch.from_numpy(z_batch).float().cuda(),
+                                            torch.from_numpy(x_batch).float().cuda())
+                y_pred = y_pred.cpu().detach().numpy()
+            else:
+                y_pred = self.model.forward(torch.from_numpy(z_batch).float(),
+                                            torch.from_numpy(x_batch).float())
+                y_pred = y_pred.detach().numpy()
+        else:
+            last_landlord_action = self.cards2array(
+                infoset.last_moves[0])
+            last_landlord_action_batch = np.repeat(
+                last_landlord_action[np.newaxis, :],
+                num_legal_actions, axis=0)
+            landlord_num_cards_left = self.get_one_hot_array(
+                infoset.num_cards_left[0], 20)
+            landlord_num_cards_left_batch = np.repeat(
+                landlord_num_cards_left[np.newaxis, :],
+                num_legal_actions, axis=0)
+
+            landlord_played_cards = self.cards2array(
+                infoset.played_cards[0])
+            landlord_played_cards_batch = np.repeat(
+                landlord_played_cards[np.newaxis, :],
+                num_legal_actions, axis=0)
+
+            if player_position == 2:
+                last_teammate_action = self.cards2array(
+                    infoset.last_moves[1])
+                last_teammate_action_batch = np.repeat(
+                    last_teammate_action[np.newaxis, :],
+                    num_legal_actions, axis=0)
+                teammate_num_cards_left = self.get_one_hot_array(
+                    infoset.num_cards_left[1], 17)
+                teammate_num_cards_left_batch = np.repeat(
+                    teammate_num_cards_left[np.newaxis, :],
+                    num_legal_actions, axis=0)
+
+                teammate_played_cards = self.cards2array(
+                    infoset.played_cards[1])
+                teammate_played_cards_batch = np.repeat(
+                    teammate_played_cards[np.newaxis, :],
+                    num_legal_actions, axis=0)
+
+            else:
+                last_teammate_action = self.cards2array(
+                    infoset.last_moves[2])
+                last_teammate_action_batch = np.repeat(
+                    last_teammate_action[np.newaxis, :],
+                    num_legal_actions, axis=0)
+                teammate_num_cards_left = self.get_one_hot_array(
+                    infoset.num_cards_left[2], 17)
+                teammate_num_cards_left_batch = np.repeat(
+                    teammate_num_cards_left[np.newaxis, :],
+                    num_legal_actions, axis=0)
+
+                teammate_played_cards = self.cards2array(
+                    infoset.played_cards[2])
+                teammate_played_cards_batch = np.repeat(
+                    teammate_played_cards[np.newaxis, :],
+                    num_legal_actions, axis=0)
+
+            bomb_num = _get_one_hot_bomb(
+                infoset.bomb_num)
+            bomb_num_batch = np.repeat(
+                bomb_num[np.newaxis, :],
+                num_legal_actions, axis=0)
+            x_batch = np.hstack((my_handcards_batch,
+                                 other_handcards_batch,
+                                 landlord_played_cards_batch,
+                                 teammate_played_cards_batch,
+                                 last_action_batch,
+                                 last_landlord_action_batch,
+                                 last_teammate_action_batch,
+                                 landlord_num_cards_left_batch,
+                                 teammate_num_cards_left_batch,
+                                 bomb_num_batch,
+                                 my_action_batch))
+            z = self.action_seq_list2array(_process_action_seq(infoset.card_play_action_seq))
+            z_batch = np.repeat(
+                z[np.newaxis, :, :],
+                num_legal_actions, axis=0)
+            if self.use_onnx:
+                ort_inputs = {'z': z_batch, 'x': x_batch}
+                y_pred = self.model.run(None, ort_inputs)[0]
+            elif torch.cuda.is_available():
+                y_pred = self.model.forward(torch.from_numpy(z_batch).float().cuda(),
+                                            torch.from_numpy(x_batch).float().cuda())
+                y_pred = y_pred.cpu().detach().numpy()
+            else:
+                y_pred = self.model.forward(torch.from_numpy(z_batch).float(),
+                                            torch.from_numpy(x_batch).float())
+                y_pred = y_pred.detach().numpy()
+
+        y_pred = y_pred.flatten()
+
+        #best_action_index = np.argmax(y_pred, axis=0)[0]
+        size = min(3, len(y_pred))
+        best_action_index = np.argpartition(y_pred, -size)[-size:]
+        best_action_confidence = y_pred[best_action_index]
+        best_action = [infoset.legal_actions[index] for index in best_action_index]
+
+        return best_action, best_action_confidence

pve_server/models.py

@@ -0,0 +1,67 @@
+import torch
+from torch import nn
+
+class LandlordLstmModel(nn.Module):
+    def __init__(self):
+        super().__init__()
+        self.lstm = nn.LSTM(162, 128, batch_first = True)
+        self.dense1 = nn.Linear(373 + 128, 512)
+        self.dense2 = nn.Linear(512, 512)
+        self.dense3 = nn.Linear(512, 512)
+        self.dense4 = nn.Linear(512, 512)
+        self.dense5 = nn.Linear(512, 512)
+        self.dense5 = nn.Linear(512, 512)
+        self.dense5 = nn.Linear(512, 512)
+        self.dense6 = nn.Linear(512, 1)
+
+    def forward(self, z, x):
+        lstm_out, (h_n, _) = self.lstm(z)
+        lstm_out = lstm_out[:,-1,:]
+        x = torch.cat([lstm_out,x], dim=-1)
+        x = self.dense1(x)
+        x = torch.relu(x)
+        x = self.dense2(x)
+        x = torch.relu(x)
+        x = self.dense3(x)
+        x = torch.relu(x)
+        x = self.dense4(x)
+        x = torch.relu(x)
+        x = self.dense5(x)
+        x = torch.relu(x)
+        x = self.dense6(x)
+        return x
+
+class FarmerLstmModel(nn.Module):
+    def __init__(self):
+        super().__init__()
+        self.lstm = nn.LSTM(162, 128, batch_first = True)
+        self.dense1 = nn.Linear(484 + 128 , 512)
+        self.dense2 = nn.Linear(512, 512)
+        self.dense3 = nn.Linear(512, 512)
+        self.dense4 = nn.Linear(512, 512)
+        self.dense4 = nn.Linear(512, 512)
+        self.dense4 = nn.Linear(512, 512)
+        self.dense5 = nn.Linear(512, 512)
+        self.dense6 = nn.Linear(512, 1)
+
+    def forward(self, z, x):
+        lstm_out, (h_n, _) = self.lstm(z)
+        lstm_out = lstm_out[:,-1,:]
+        x = torch.cat([lstm_out,x], dim=-1)
+        x = self.dense1(x)
+        x = torch.relu(x)
+        x = self.dense2(x)
+        x = torch.relu(x)
+        x = self.dense3(x)
+        x = torch.relu(x)
+        x = self.dense4(x)
+        x = torch.relu(x)
+        x = self.dense5(x)
+        x = torch.relu(x)
+        x = self.dense6(x)
+        return x
+
+model_dict = {}
+model_dict['landlord'] = LandlordLstmModel
+model_dict['landlord_up'] = FarmerLstmModel
+model_dict['landlord_down'] = FarmerLstmModel

pve_server/run_dmc.py

@@ -0,0 +1,384 @@
+import os
+import itertools
+
+import torch
+import numpy as np
+from heapq import nlargest
+from collections import Counter, OrderedDict
+from flask import Flask, jsonify, request
+from flask_cors import CORS
+
+app = Flask(__name__)
+CORS(app)
+
+from utils.move_generator import MovesGener
+from utils import move_detector as md, move_selector as ms
+
+import rlcard
+env = rlcard.make('doudizhu')
+
+DouZeroCard2RLCard = {3: '3', 4: '4', 5: '5', 6: '6', 7: '7',
+                    8: '8', 9: '9', 10: 'T', 11: 'J', 12: 'Q',
+                    13: 'K', 14: 'A', 17: '2', 20: 'B', 30: 'R'}
+
+RLCard2DouZeroCard = {'3': 3, '4': 4, '5': 5, '6': 6, '7': 7,
+                    '8': 8, '9': 9, 'T': 10, 'J': 11, 'Q': 12,
+                    'K': 13, 'A': 14, '2': 17, 'B': 20, 'R': 30}
+
+EnvCard2RealCard = {'3': '3', '4':'4', '5': '5', '6': '6', '7': '7',
+                    '8': '8', '9': '9', 'T': 'T', 'J': 'J', 'Q': 'Q',
+                    'K': 'K', 'A': 'A', '2': '2', 'B': 'X', 'R': 'D'}
+
+RealCard2EnvCard = {'3': '3', '4':'4', '5': '5', '6': '6', '7': '7',
+                    '8': '8', '9': '9', 'T': 'T', 'J': 'J', 'Q': 'Q',
+                    'K': 'K', 'A': 'A', '2': '2', 'X': 'B', 'D': 'R'}
+
+pretrained_dir = 'pretrained/dmc_pretrained'
+device = torch.device('cpu')
+players = []
+for i in range(3):
+    model_path = os.path.join(pretrained_dir, str(i)+'.pth')
+    agent = torch.load(model_path, map_location=device)
+    agent.set_device(device)
+    players.append(agent)
+ 
+@app.route('/predict', methods=['POST'])
+def predict():
+    if request.method == 'POST':
+        try:
+            # Player postion
+            player_position = request.form.get('player_position')
+            if player_position not in ['0', '1', '2']:
+                return jsonify({'status': 1, 'message': 'player_position must be 0, 1, or 2'})
+            player_position = int(player_position)
+
+            # Player hand cards
+            player_hand_cards = ''.join([RealCard2EnvCard[c] for c in request.form.get('player_hand_cards')])
+            if player_position == 0:
+                if len(player_hand_cards) < 1 or len(player_hand_cards) > 20:
+                    return jsonify({'status': 2, 'message': 'the number of hand cards should be 1-20'})
+            else:
+                if len(player_hand_cards) < 1 or len(player_hand_cards) > 17:
+                    return jsonify({'status': 3, 'message': 'the number of hand cards should be 1-17'})
+
+            # Number cards left
+            num_cards_left = [int(request.form.get('num_cards_left_landlord')), int(request.form.get('num_cards_left_landlord_down')), int(request.form.get('num_cards_left_landlord_up'))]
+            if num_cards_left[player_position] != len(player_hand_cards):
+                return jsonify({'status': 4, 'message': 'the number of cards left do not align with hand cards'})
+            if num_cards_left[0] < 0 or num_cards_left[1] < 0 or num_cards_left[2] < 0 or num_cards_left[0] > 20 or num_cards_left[1] > 17 or num_cards_left[2] > 17:
+                return jsonify({'status': 5, 'message': 'the number of cards left not in range'})
+
+            # Three landlord cards
+            three_landlord_cards = ''.join([RealCard2EnvCard[c] for c in request.form.get('three_landlord_cards')])
+            if len(three_landlord_cards) < 0 or len(three_landlord_cards) > 3:
+                return jsonify({'status': 6, 'message': 'the number of landlord cards should be 0-3'})
+
+            # Card play sequence
+            if request.form.get('card_play_action_seq') == '':
+                card_play_action_seq = []
+            else:
+                tmp_seq = [''.join([RealCard2EnvCard[c] for c in cards]) for cards in request.form.get('card_play_action_seq').split(',')]
+                for i in range(len(tmp_seq)):
+                    if tmp_seq[i] == '':
+                        tmp_seq[i] = 'pass'
+                card_play_action_seq = []
+                for i in range(len(tmp_seq)):
+                    card_play_action_seq.append((i%3, tmp_seq[i]))
+
+            # Other hand cards
+            other_hand_cards = ''.join([RealCard2EnvCard[c] for c in request.form.get('other_hand_cards')])
+            if len(other_hand_cards) != sum(num_cards_left) - num_cards_left[player_position]:
+                return jsonify({'status': 7, 'message': 'the number of the other hand cards do not align with the number of cards left'})
+
+            # Played cards
+            played_cards = []
+            for field in ['played_cards_landlord', 'played_cards_landlord_down', 'played_cards_landlord_up']:
+                played_cards.append(''.join([RealCard2EnvCard[c] for c in request.form.get(field)]))
+
+            # RLCard state
+            state = {}
+            state['current_hand'] = player_hand_cards
+            state['landlord'] = 0
+            state['num_cards_left'] = num_cards_left
+            state['others_hand'] = other_hand_cards
+            state['played_cards'] = played_cards
+            state['seen_cards'] = three_landlord_cards
+            state['self'] = player_position
+            state['trace'] = card_play_action_seq
+
+            # Get rival move and legal_actions
+            rival_move = 'pass'
+            if len(card_play_action_seq) != 0:
+                if card_play_action_seq[-1][1] == 'pass':
+                    rival_move = card_play_action_seq[-2][1]
+                else:
+                    rival_move = card_play_action_seq[-1][1]
+            if rival_move == 'pass':
+                rival_move = ''
+            rival_move = [RLCard2DouZeroCard[c] for c in rival_move]
+            state['actions'] = _get_legal_card_play_actions([RLCard2DouZeroCard[c] for c in player_hand_cards], rival_move)
+            state['actions'] = [''.join([DouZeroCard2RLCard[c] for c in a]) for a in state['actions']]
+            for i in range(len(state['actions'])):
+                if state['actions'][i] == '':
+                    state['actions'][i] = 'pass'
+
+
+            # Prediction
+            state = _extract_state(state)
+            action, info = players[player_position].eval_step(state)
+            if action == 'pass':
+                action = ''
+            for i in info['values']:
+                if i == 'pass':
+                    info['values'][''] = info['values']['pass']
+                    del info['values']['pass']
+
+            actions = nlargest(3, info['values'], key=info['values'].get)
+            actions_confidence = [info['values'].get(action) for action in actions]
+            actions = [''.join([EnvCard2RealCard[c] for c in action]) for action in actions]
+            result = {}
+            win_rates = {}
+            for i in range(len(actions)):
+                # Here, we calculate the win rate
+                win_rate = min(actions_confidence[i], 1)
+                win_rate = max(win_rate, 0)
+                win_rates[actions[i]] = str(round(win_rate, 4))
+                result[actions[i]] = str(round(actions_confidence[i], 6))
+
+            ############## DEBUG ################
+            if app.debug:
+                print('--------------- DEBUG START --------------')
+                command = 'curl --data "'
+                parameters = []
+                for key in request.form:
+                    parameters.append(key+'='+request.form.get(key))
+                    print(key+':', request.form.get(key))
+                command += '&'.join(parameters)
+                command += '" "http://127.0.0.1:5000/predict"'
+                print('Command:', command)
+                print('Rival Move:', rival_move)
+                print('legal_actions:', state['legal_actions'])
+                print('Result:', result)
+                print('--------------- DEBUG END --------------')
+            ############## DEBUG ################
+            return jsonify({'status': 0, 'message': 'success', 'result': result, 'win_rates': win_rates})
+        except:
+            import traceback
+            traceback.print_exc()
+            return jsonify({'status': -1, 'message': 'unkown error'})
+
+@app.route('/legal', methods=['POST'])
+def legal():
+    if request.method == 'POST':
+        try:
+            player_hand_cards = [RealCard2EnvCard[c] for c in request.form.get('player_hand_cards')]
+            rival_move = [RealCard2EnvCard[c] for c in request.form.get('rival_move')]
+            if rival_move == '':
+                rival_move = 'pass'
+            player_hand_cards = [RLCard2DouZeroCard[c] for c in player_hand_cards]
+            rival_move = [RLCard2DouZeroCard[c] for c in rival_move] 
+            legal_actions = _get_legal_card_play_actions(player_hand_cards, rival_move)
+            legal_actions = [''.join([DouZeroCard2RLCard[c] for c in a]) for a in legal_actions]
+            for i in range(len(legal_actions)):
+                if legal_actions[i] == 'pass':
+                    legal_actions[i] = ''
+            legal_actions = ','.join([''.join([EnvCard2RealCard[c] for c in action]) for action in legal_actions])
+            return jsonify({'status': 0, 'message': 'success', 'legal_action': legal_actions})
+        except:
+            import traceback
+            traceback.print_exc()
+            return jsonify({'status': -1, 'message': 'unkown error'})
+
+def _extract_state(state):
+    current_hand = _cards2array(state['current_hand'])
+    others_hand = _cards2array(state['others_hand'])
+
+    last_action = ''
+    if len(state['trace']) != 0:
+        if state['trace'][-1][1] == 'pass':
+            last_action = state['trace'][-2][1]
+        else:
+            last_action = state['trace'][-1][1]
+    last_action = _cards2array(last_action)
+
+    last_9_actions = _action_seq2array(_process_action_seq(state['trace']))
+
+    if state['self'] == 0: # landlord
+        landlord_up_played_cards = _cards2array(state['played_cards'][2])
+        landlord_down_played_cards = _cards2array(state['played_cards'][1])
+        landlord_up_num_cards_left = _get_one_hot_array(state['num_cards_left'][2], 17)
+        landlord_down_num_cards_left = _get_one_hot_array(state['num_cards_left'][1], 17)
+        obs = np.concatenate((current_hand,
+                              others_hand,
+                              last_action,
+                              last_9_actions,
+                              landlord_up_played_cards,
+                              landlord_down_played_cards,
+                              landlord_up_num_cards_left,
+                              landlord_down_num_cards_left))
+    else:
+        landlord_played_cards = _cards2array(state['played_cards'][0])
+        for i, action in reversed(state['trace']):
+            if i == 0:
+                last_landlord_action = action
+        last_landlord_action = _cards2array(last_landlord_action)
+        landlord_num_cards_left = _get_one_hot_array(state['num_cards_left'][0], 20)
+
+        teammate_id = 3 - state['self']
+        teammate_played_cards = _cards2array(state['played_cards'][teammate_id])
+        last_teammate_action = 'pass'
+        for i, action in reversed(state['trace']):
+            if i == teammate_id:
+                last_teammate_action = action
+        last_teammate_action = _cards2array(last_teammate_action)
+        teammate_num_cards_left = _get_one_hot_array(state['num_cards_left'][teammate_id], 17)
+        obs = np.concatenate((current_hand,
+                              others_hand,
+                              last_action,
+                              last_9_actions,
+                              landlord_played_cards,
+                              teammate_played_cards,
+                              last_landlord_action,
+                              last_teammate_action,
+                              landlord_num_cards_left,
+                              teammate_num_cards_left))
+
+    legal_actions = {env._ACTION_2_ID[action]: _cards2array(action) for action in state['actions']}
+    extracted_state = OrderedDict({'obs': obs, 'legal_actions': legal_actions})
+    extracted_state['raw_obs'] = state
+    extracted_state['raw_legal_actions'] = [a for a in state['actions']]
+    return extracted_state
+
+def _get_legal_card_play_actions(player_hand_cards, rival_move):
+    mg = MovesGener(player_hand_cards)
+
+    rival_type = md.get_move_type(rival_move)
+    rival_move_type = rival_type['type']
+    rival_move_len = rival_type.get('len', 1)
+    moves = list()
+
+    if rival_move_type == md.TYPE_0_PASS:
+        moves = mg.gen_moves()
+
+    elif rival_move_type == md.TYPE_1_SINGLE:
+        all_moves = mg.gen_type_1_single()
+        moves = ms.filter_type_1_single(all_moves, rival_move)
+
+    elif rival_move_type == md.TYPE_2_PAIR:
+        all_moves = mg.gen_type_2_pair()
+        moves = ms.filter_type_2_pair(all_moves, rival_move)
+
+    elif rival_move_type == md.TYPE_3_TRIPLE:
+        all_moves = mg.gen_type_3_triple()
+        moves = ms.filter_type_3_triple(all_moves, rival_move)
+
+    elif rival_move_type == md.TYPE_4_BOMB:
+        all_moves = mg.gen_type_4_bomb() + mg.gen_type_5_king_bomb()
+        moves = ms.filter_type_4_bomb(all_moves, rival_move)
+
+    elif rival_move_type == md.TYPE_5_KING_BOMB:
+        moves = []
+
+    elif rival_move_type == md.TYPE_6_3_1:
+        all_moves = mg.gen_type_6_3_1()
+        moves = ms.filter_type_6_3_1(all_moves, rival_move)
+
+    elif rival_move_type == md.TYPE_7_3_2:
+        all_moves = mg.gen_type_7_3_2()
+        moves = ms.filter_type_7_3_2(all_moves, rival_move)
+
+    elif rival_move_type == md.TYPE_8_SERIAL_SINGLE:
+        all_moves = mg.gen_type_8_serial_single(repeat_num=rival_move_len)
+        moves = ms.filter_type_8_serial_single(all_moves, rival_move)
+
+    elif rival_move_type == md.TYPE_9_SERIAL_PAIR:
+        all_moves = mg.gen_type_9_serial_pair(repeat_num=rival_move_len)
+        moves = ms.filter_type_9_serial_pair(all_moves, rival_move)
+
+    elif rival_move_type == md.TYPE_10_SERIAL_TRIPLE:
+        all_moves = mg.gen_type_10_serial_triple(repeat_num=rival_move_len)
+        moves = ms.filter_type_10_serial_triple(all_moves, rival_move)
+
+    elif rival_move_type == md.TYPE_11_SERIAL_3_1:
+        all_moves = mg.gen_type_11_serial_3_1(repeat_num=rival_move_len)
+        moves = ms.filter_type_11_serial_3_1(all_moves, rival_move)
+
+    elif rival_move_type == md.TYPE_12_SERIAL_3_2:
+        all_moves = mg.gen_type_12_serial_3_2(repeat_num=rival_move_len)
+        moves = ms.filter_type_12_serial_3_2(all_moves, rival_move)
+
+    elif rival_move_type == md.TYPE_13_4_2:
+        all_moves = mg.gen_type_13_4_2()
+        moves = ms.filter_type_13_4_2(all_moves, rival_move)
+
+    elif rival_move_type == md.TYPE_14_4_22:
+        all_moves = mg.gen_type_14_4_22()
+        moves = ms.filter_type_14_4_22(all_moves, rival_move)
+
+    if rival_move_type not in [md.TYPE_0_PASS,
+                               md.TYPE_4_BOMB, md.TYPE_5_KING_BOMB]:
+        moves = moves + mg.gen_type_4_bomb() + mg.gen_type_5_king_bomb()
+
+    if len(rival_move) != 0:  # rival_move is not 'pass'
+        moves = moves + [[]]
+
+    for m in moves:
+        m.sort()
+
+    moves.sort()
+    moves = list(move for move,_ in itertools.groupby(moves))
+
+    return moves
+
+Card2Column = {'3': 0, '4': 1, '5': 2, '6': 3, '7': 4, '8': 5, '9': 6, 'T': 7,
+               'J': 8, 'Q': 9, 'K': 10, 'A': 11, '2': 12}
+
+NumOnes2Array = {0: np.array([0, 0, 0, 0]),
+                 1: np.array([1, 0, 0, 0]),
+                 2: np.array([1, 1, 0, 0]),
+                 3: np.array([1, 1, 1, 0]),
+                 4: np.array([1, 1, 1, 1])}
+
+def _cards2array(cards):
+    if cards == 'pass':
+        return np.zeros(54, dtype=np.int8)
+
+    matrix = np.zeros([4, 13], dtype=np.int8)
+    jokers = np.zeros(2, dtype=np.int8)
+    counter = Counter(cards)
+    for card, num_times in counter.items():
+        if card == 'B':
+            jokers[0] = 1
+        elif card == 'R':
+            jokers[1] = 1
+        else:
+            matrix[:, Card2Column[card]] = NumOnes2Array[num_times]
+    return np.concatenate((matrix.flatten('F'), jokers))
+
+def _get_one_hot_array(num_left_cards, max_num_cards):
+    one_hot = np.zeros(max_num_cards, dtype=np.int8)
+    one_hot[num_left_cards - 1] = 1
+
+    return one_hot
+
+def _action_seq2array(action_seq_list):
+    action_seq_array = np.zeros((len(action_seq_list), 54), np.int8)
+    for row, cards in enumerate(action_seq_list):
+        action_seq_array[row, :] = _cards2array(cards)
+    action_seq_array = action_seq_array.flatten()
+    return action_seq_array
+
+def _process_action_seq(sequence, length=9):
+    sequence = [action[1] for action in sequence[-length:]]
+    if len(sequence) < length:
+        empty_sequence = ['' for _ in range(length - len(sequence))]
+        empty_sequence.extend(sequence)
+        sequence = empty_sequence
+    return sequence
+
+if __name__ == '__main__':
+    import argparse
+    parser = argparse.ArgumentParser(description='DouZero backend')
+    parser.add_argument('--debug', action='store_true')
+    args = parser.parse_args()
+    app.run(debug=args.debug)

pve_server/run_douzero.py

@@ -0,0 +1,252 @@
+import itertools
+
+from flask import Flask, jsonify, request
+from flask_cors import CORS
+
+app = Flask(__name__)
+CORS(app)
+
+from utils.move_generator import MovesGener
+from utils import move_detector as md, move_selector as ms
+from deep import DeepAgent
+
+EnvCard2RealCard = {3: '3', 4: '4', 5: '5', 6: '6', 7: '7',
+                    8: '8', 9: '9', 10: 'T', 11: 'J', 12: 'Q',
+                    13: 'K', 14: 'A', 17: '2', 20: 'X', 30: 'D'}
+
+RealCard2EnvCard = {'3': 3, '4': 4, '5': 5, '6': 6, '7': 7,
+                    '8': 8, '9': 9, 'T': 10, 'J': 11, 'Q': 12,
+                    'K': 13, 'A': 14, '2': 17, 'X': 20, 'D': 30}
+
+pretrained_dir = 'pretrained/douzero_pretrained'
+players = []
+for position in ['landlord', 'landlord_down', 'landlord_up']:
+    players.append(DeepAgent(position, pretrained_dir, use_onnx=True))
+    
+@app.route('/predict', methods=['POST'])
+def predict():
+    if request.method == 'POST':
+        try:
+            # Player postion
+            player_position = request.form.get('player_position')
+            if player_position not in ['0', '1', '2']:
+                return jsonify({'status': 1, 'message': 'player_position must be 0, 1, or 2'})
+            player_position = int(player_position)
+
+            # Player hand cards
+            player_hand_cards = [RealCard2EnvCard[c] for c in request.form.get('player_hand_cards')]
+            if player_position == 0:
+                if len(player_hand_cards) < 1 or len(player_hand_cards) > 20:
+                    return jsonify({'status': 2, 'message': 'the number of hand cards should be 1-20'})
+            else:
+                if len(player_hand_cards) < 1 or len(player_hand_cards) > 17:
+                    return jsonify({'status': 3, 'message': 'the number of hand cards should be 1-17'})
+
+            # Number cards left
+            num_cards_left = [int(request.form.get('num_cards_left_landlord')), int(request.form.get('num_cards_left_landlord_down')), int(request.form.get('num_cards_left_landlord_up'))]
+            if num_cards_left[player_position] != len(player_hand_cards):
+                return jsonify({'status': 4, 'message': 'the number of cards left do not align with hand cards'})
+            if num_cards_left[0] < 0 or num_cards_left[1] < 0 or num_cards_left[2] < 0 or num_cards_left[0] > 20 or num_cards_left[1] > 17 or num_cards_left[2] > 17:
+                return jsonify({'status': 5, 'message': 'the number of cards left not in range'})
+
+            # Three landlord cards
+            three_landlord_cards = [RealCard2EnvCard[c] for c in request.form.get('three_landlord_cards')]
+            if len(three_landlord_cards) < 0 or len(three_landlord_cards) > 3:
+                return jsonify({'status': 6, 'message': 'the number of landlord cards should be 0-3'})
+
+            # Card play sequence
+            if request.form.get('card_play_action_seq') == '':
+                card_play_action_seq = []
+            else:
+                card_play_action_seq = [[RealCard2EnvCard[c] for c in cards] for cards in request.form.get('card_play_action_seq').split(',')]
+
+            # Other hand cards
+            other_hand_cards = [RealCard2EnvCard[c] for c in request.form.get('other_hand_cards')]
+            if len(other_hand_cards) != sum(num_cards_left) - num_cards_left[player_position]:
+                return jsonify({'status': 7, 'message': 'the number of the other hand cards do not align with the number of cards left'})
+
+            # Last moves
+            last_moves = []
+            for field in ['last_move_landlord', 'last_move_landlord_down', 'last_move_landlord_up']:
+                last_moves.append([RealCard2EnvCard[c] for c in request.form.get(field)])
+
+            # Played cards
+            played_cards = []
+            for field in ['played_cards_landlord', 'played_cards_landlord_down', 'played_cards_landlord_up']:
+                played_cards.append([RealCard2EnvCard[c] for c in request.form.get(field)])
+
+            # Bomb Num
+            bomb_num = int(request.form.get('bomb_num'))
+
+            # InfoSet
+            info_set = InfoSet()
+            info_set.player_position = player_position
+            info_set.player_hand_cards = player_hand_cards
+            info_set.num_cards_left = num_cards_left
+            info_set.three_landlord_cards = three_landlord_cards
+            info_set.card_play_action_seq = card_play_action_seq
+            info_set.other_hand_cards = other_hand_cards
+            info_set.last_moves = last_moves
+            info_set.played_cards = played_cards
+            info_set.bomb_num = bomb_num
+
+            # Get rival move and legal_actions
+            rival_move = []
+            if len(card_play_action_seq) != 0:
+                if len(card_play_action_seq[-1]) == 0:
+                    rival_move = card_play_action_seq[-2]
+                else:
+                    rival_move = card_play_action_seq[-1]
+            info_set.rival_move = rival_move
+            info_set.legal_actions = _get_legal_card_play_actions(player_hand_cards, rival_move)
+
+            # Prediction
+            actions, actions_confidence = players[player_position].act(info_set)
+            actions = [''.join([EnvCard2RealCard[a] for a in action]) for action in actions]
+            result = {}
+            win_rates = {}
+            for i in range(len(actions)):
+                # Here, we calculate the win rate
+                win_rate = max(actions_confidence[i], -1)
+                win_rate = min(win_rate, 1)
+                win_rates[actions[i]] = str(round((win_rate + 1) / 2, 4))
+                result[actions[i]] = str(round(actions_confidence[i], 6))
+
+            ############## DEBUG ################
+            if app.debug:
+                print('--------------- DEBUG START --------------')
+                command = 'curl --data "'
+                parameters = []
+                for key in request.form:
+                    parameters.append(key+'='+request.form.get(key))
+                    print(key+':', request.form.get(key))
+                command += '&'.join(parameters)
+                command += '" "http://127.0.0.1:5000/predict"'
+                print('Command:', command)
+                print('Rival Move:', rival_move)
+                print('legal_actions:', info_set.legal_actions)
+                print('Result:', result)
+                print('--------------- DEBUG END --------------')
+            ############## DEBUG ################
+            return jsonify({'status': 0, 'message': 'success', 'result': result, 'win_rates': win_rates})
+        except:
+            import traceback
+            traceback.print_exc()
+            return jsonify({'status': -1, 'message': 'unkown error'})
+
+@app.route('/legal', methods=['POST'])
+def legal():
+    if request.method == 'POST':
+        try:
+            player_hand_cards = [RealCard2EnvCard[c] for c in request.form.get('player_hand_cards')]
+            rival_move = [RealCard2EnvCard[c] for c in request.form.get('rival_move')]
+            legal_actions = _get_legal_card_play_actions(player_hand_cards, rival_move)
+            legal_actions = ','.join([''.join([EnvCard2RealCard[a] for a in action]) for action in legal_actions])
+            return jsonify({'status': 0, 'message': 'success', 'legal_action': legal_actions})
+        except:
+            import traceback
+            traceback.print_exc()
+            return jsonify({'status': -1, 'message': 'unkown error'})
+
+class InfoSet(object):
+
+    def __init__(self):
+        self.player_position = None
+        self.player_hand_cards = None
+        self.num_cards_left = None
+        self.three_landlord_cards = None
+        self.card_play_action_seq = None
+        self.other_hand_cards = None
+        self.legal_actions = None
+        self.rival_move = None
+        self.last_moves = None
+        self.played_cards = None
+        self.bomb_num = None
+
+def _get_legal_card_play_actions(player_hand_cards, rival_move):
+    mg = MovesGener(player_hand_cards)
+
+    rival_type = md.get_move_type(rival_move)
+    rival_move_type = rival_type['type']
+    rival_move_len = rival_type.get('len', 1)
+    moves = list()
+
+    if rival_move_type == md.TYPE_0_PASS:
+        moves = mg.gen_moves()
+
+    elif rival_move_type == md.TYPE_1_SINGLE:
+        all_moves = mg.gen_type_1_single()
+        moves = ms.filter_type_1_single(all_moves, rival_move)
+
+    elif rival_move_type == md.TYPE_2_PAIR:
+        all_moves = mg.gen_type_2_pair()
+        moves = ms.filter_type_2_pair(all_moves, rival_move)
+
+    elif rival_move_type == md.TYPE_3_TRIPLE:
+        all_moves = mg.gen_type_3_triple()
+        moves = ms.filter_type_3_triple(all_moves, rival_move)
+
+    elif rival_move_type == md.TYPE_4_BOMB:
+        all_moves = mg.gen_type_4_bomb() + mg.gen_type_5_king_bomb()
+        moves = ms.filter_type_4_bomb(all_moves, rival_move)
+
+    elif rival_move_type == md.TYPE_5_KING_BOMB:
+        moves = []
+
+    elif rival_move_type == md.TYPE_6_3_1:
+        all_moves = mg.gen_type_6_3_1()
+        moves = ms.filter_type_6_3_1(all_moves, rival_move)
+
+    elif rival_move_type == md.TYPE_7_3_2:
+        all_moves = mg.gen_type_7_3_2()
+        moves = ms.filter_type_7_3_2(all_moves, rival_move)
+
+    elif rival_move_type == md.TYPE_8_SERIAL_SINGLE:
+        all_moves = mg.gen_type_8_serial_single(repeat_num=rival_move_len)
+        moves = ms.filter_type_8_serial_single(all_moves, rival_move)
+
+    elif rival_move_type == md.TYPE_9_SERIAL_PAIR:
+        all_moves = mg.gen_type_9_serial_pair(repeat_num=rival_move_len)
+        moves = ms.filter_type_9_serial_pair(all_moves, rival_move)
+
+    elif rival_move_type == md.TYPE_10_SERIAL_TRIPLE:
+        all_moves = mg.gen_type_10_serial_triple(repeat_num=rival_move_len)
+        moves = ms.filter_type_10_serial_triple(all_moves, rival_move)
+
+    elif rival_move_type == md.TYPE_11_SERIAL_3_1:
+        all_moves = mg.gen_type_11_serial_3_1(repeat_num=rival_move_len)
+        moves = ms.filter_type_11_serial_3_1(all_moves, rival_move)
+
+    elif rival_move_type == md.TYPE_12_SERIAL_3_2:
+        all_moves = mg.gen_type_12_serial_3_2(repeat_num=rival_move_len)
+        moves = ms.filter_type_12_serial_3_2(all_moves, rival_move)
+
+    elif rival_move_type == md.TYPE_13_4_2:
+        all_moves = mg.gen_type_13_4_2()
+        moves = ms.filter_type_13_4_2(all_moves, rival_move)
+
+    elif rival_move_type == md.TYPE_14_4_22:
+        all_moves = mg.gen_type_14_4_22()
+        moves = ms.filter_type_14_4_22(all_moves, rival_move)
+
+    if rival_move_type not in [md.TYPE_0_PASS,
+                               md.TYPE_4_BOMB, md.TYPE_5_KING_BOMB]:
+        moves = moves + mg.gen_type_4_bomb() + mg.gen_type_5_king_bomb()
+
+    if len(rival_move) != 0:  # rival_move is not 'pass'
+        moves = moves + [[]]
+
+    for m in moves:
+        m.sort()
+
+    moves.sort()
+    moves = list(move for move,_ in itertools.groupby(moves))
+
+    return moves
+
+if __name__ == '__main__':
+    import argparse
+    parser = argparse.ArgumentParser(description='DouZero backend')
+    parser.add_argument('--debug', action='store_true')
+    args = parser.parse_args()
+    app.run(debug=args.debug)

pve_server/utils/move_detector.py

@@ -0,0 +1,107 @@
+from .utils import *
+import collections
+
+# check if move is a continuous sequence
+def is_continuous_seq(move):
+    i = 0
+    while i < len(move) - 1:
+        if move[i+1] - move[i] != 1:
+            return False
+        i += 1
+    return True
+
+# return the type of the move
+def get_move_type(move):
+    move_size = len(move)
+    move_dict = collections.Counter(move)
+
+    if move_size == 0:
+        return {'type': TYPE_0_PASS}
+
+    if move_size == 1:
+        return {'type': TYPE_1_SINGLE, 'rank': move[0]}
+
+    if move_size == 2:
+        if move[0] == move[1]:
+            return {'type': TYPE_2_PAIR, 'rank': move[0]}
+        elif move == [20, 30]:  # Kings
+            return {'type': TYPE_5_KING_BOMB}
+        else:
+            return {'type': TYPE_15_WRONG}
+
+    if move_size == 3:
+        if len(move_dict) == 1:
+            return {'type': TYPE_3_TRIPLE, 'rank': move[0]}
+        else:
+            return {'type': TYPE_15_WRONG}
+
+    if move_size == 4:
+        if len(move_dict) == 1:
+            return {'type': TYPE_4_BOMB,  'rank': move[0]}
+        elif len(move_dict) == 2:
+            if move[0] == move[1] == move[2] or move[1] == move[2] == move[3]:
+                return {'type': TYPE_6_3_1, 'rank': move[1]}
+            else:
+                return {'type': TYPE_15_WRONG}
+        else:
+            return {'type': TYPE_15_WRONG}
+
+    if is_continuous_seq(move):
+        return {'type': TYPE_8_SERIAL_SINGLE, 'rank': move[0], 'len': len(move)}
+
+    if move_size == 5:
+        if len(move_dict) == 2:
+            return {'type': TYPE_7_3_2, 'rank': move[2]}
+        else:
+            return {'type': TYPE_15_WRONG}
+
+    count_dict = collections.defaultdict(int)
+    for c, n in move_dict.items():
+        count_dict[n] += 1
+
+    if move_size == 6:
+        if (len(move_dict) == 2 or len(move_dict) == 3) and count_dict.get(4) == 1 and \
+                (count_dict.get(2) == 1 or count_dict.get(1) == 2):
+            return {'type': TYPE_13_4_2, 'rank': move[2]}
+
+    if move_size == 8 and (((len(move_dict) == 3 or len(move_dict) == 2) and
+            (count_dict.get(4) == 1 and count_dict.get(2) == 2)) or count_dict.get(4) == 2):
+        return {'type': TYPE_14_4_22, 'rank': max([c for c, n in move_dict.items() if n == 4])}
+
+    mdkeys = sorted(move_dict.keys())
+    if len(move_dict) == count_dict.get(2) and is_continuous_seq(mdkeys):
+        return {'type': TYPE_9_SERIAL_PAIR, 'rank': mdkeys[0], 'len': len(mdkeys)}
+
+    if len(move_dict) == count_dict.get(3) and is_continuous_seq(mdkeys):
+        return {'type': TYPE_10_SERIAL_TRIPLE, 'rank': mdkeys[0], 'len': len(mdkeys)}
+
+    # Check Type 11 (serial 3+1) and Type 12 (serial 3+2)
+    if count_dict.get(3, 0) >= MIN_TRIPLES:
+        serial_3 = list()
+        single = list()
+        pair = list()
+
+        for k, v in move_dict.items():
+            if v == 3:
+                serial_3.append(k)
+            elif v == 1:
+                single.append(k)
+            elif v == 2:
+                pair.append(k)
+            else:  # no other possibilities
+                return {'type': TYPE_15_WRONG}
+
+        serial_3.sort()
+        if is_continuous_seq(serial_3):
+            if len(serial_3) == len(single)+len(pair)*2:
+                return {'type': TYPE_11_SERIAL_3_1, 'rank': serial_3[0], 'len': len(serial_3)}
+            if len(serial_3) == len(pair) and len(move_dict) == len(serial_3) * 2:
+                return {'type': TYPE_12_SERIAL_3_2, 'rank': serial_3[0], 'len': len(serial_3)}
+
+        if len(serial_3) == 4:
+            if is_continuous_seq(serial_3[1:]):
+                return {'type': TYPE_11_SERIAL_3_1, 'rank': serial_3[1], 'len': len(serial_3) - 1}
+            if is_continuous_seq(serial_3[:-1]):
+                return {'type': TYPE_11_SERIAL_3_1, 'rank': serial_3[0], 'len': len(serial_3) - 1}
+
+    return {'type': TYPE_15_WRONG}

pve_server/utils/move_generator.py

@@ -0,0 +1,217 @@
+from .utils import MIN_SINGLE_CARDS, MIN_PAIRS, MIN_TRIPLES, select
+import collections
+import itertools
+
+class MovesGener(object):
+
+    def __init__(self, cards_list):
+        self.cards_list = cards_list
+        self.cards_dict = collections.defaultdict(int)
+
+        for i in self.cards_list:
+            self.cards_dict[i] += 1
+
+        self.single_card_moves = []
+        self.gen_type_1_single()
+        self.pair_moves = []
+        self.gen_type_2_pair()
+        self.triple_cards_moves = []
+        self.gen_type_3_triple()
+        self.bomb_moves = []
+        self.gen_type_4_bomb()
+        self.final_bomb_moves = []
+        self.gen_type_5_king_bomb()
+
+    def _gen_serial_moves(self, cards, min_serial, repeat=1, repeat_num=0):
+        if repeat_num < min_serial:  # at least repeat_num is min_serial
+            repeat_num = 0
+
+        single_cards = sorted(list(set(cards)))
+        seq_records = list()
+        moves = list()
+
+        start = i = 0
+        longest = 1
+        while i < len(single_cards):
+            if i + 1 < len(single_cards) and single_cards[i + 1] - single_cards[i] == 1:
+                longest += 1
+                i += 1
+            else:
+                seq_records.append((start, longest))
+                i += 1
+                start = i
+                longest = 1
+
+        for seq in seq_records:
+            if seq[1] < min_serial:
+                continue
+            start, longest = seq[0], seq[1]
+            longest_list = single_cards[start: start + longest]
+
+            if repeat_num == 0:  # No limitation on how many sequences
+                steps = min_serial
+                while steps <= longest:
+                    index = 0
+                    while steps + index <= longest:
+                        target_moves = sorted(longest_list[index: index + steps] * repeat)
+                        moves.append(target_moves)
+                        index += 1
+                    steps += 1
+
+            else:  # repeat_num > 0
+                if longest < repeat_num:
+                    continue
+                index = 0
+                while index + repeat_num <= longest:
+                    target_moves = sorted(longest_list[index: index + repeat_num] * repeat)
+                    moves.append(target_moves)
+                    index += 1
+
+        return moves
+
+    def gen_type_1_single(self):
+        self.single_card_moves = []
+        for i in set(self.cards_list):
+            self.single_card_moves.append([i])
+        return self.single_card_moves
+
+    def gen_type_2_pair(self):
+        self.pair_moves = []
+        for k, v in self.cards_dict.items():
+            if v >= 2:
+                self.pair_moves.append([k, k])
+        return self.pair_moves
+
+    def gen_type_3_triple(self):
+        self.triple_cards_moves = []
+        for k, v in self.cards_dict.items():
+            if v >= 3:
+                self.triple_cards_moves.append([k, k, k])
+        return self.triple_cards_moves
+
+    def gen_type_4_bomb(self):
+        self.bomb_moves = []
+        for k, v in self.cards_dict.items():
+            if v == 4:
+                self.bomb_moves.append([k, k, k, k])
+        return self.bomb_moves
+
+    def gen_type_5_king_bomb(self):
+        self.final_bomb_moves = []
+        if 20 in self.cards_list and 30 in self.cards_list:
+            self.final_bomb_moves.append([20, 30])
+        return self.final_bomb_moves
+
+    def gen_type_6_3_1(self):
+        result = []
+        for t in self.single_card_moves:
+            for i in self.triple_cards_moves:
+                if t[0] != i[0]:
+                    result.append(t+i)
+        return result
+
+    def gen_type_7_3_2(self):
+        result = list()
+        for t in self.pair_moves:
+            for i in self.triple_cards_moves:
+                if t[0] != i[0]:
+                    result.append(t+i)
+        return result
+
+    def gen_type_8_serial_single(self, repeat_num=0):
+        return self._gen_serial_moves(self.cards_list, MIN_SINGLE_CARDS, repeat=1, repeat_num=repeat_num)
+
+    def gen_type_9_serial_pair(self, repeat_num=0):
+        single_pairs = list()
+        for k, v in self.cards_dict.items():
+            if v >= 2:
+                single_pairs.append(k)
+
+        return self._gen_serial_moves(single_pairs, MIN_PAIRS, repeat=2, repeat_num=repeat_num)
+
+    def gen_type_10_serial_triple(self, repeat_num=0):
+        single_triples = list()
+        for k, v in self.cards_dict.items():
+            if v >= 3:
+                single_triples.append(k)
+
+        return self._gen_serial_moves(single_triples, MIN_TRIPLES, repeat=3, repeat_num=repeat_num)
+
+    def gen_type_11_serial_3_1(self, repeat_num=0):
+        serial_3_moves = self.gen_type_10_serial_triple(repeat_num=repeat_num)
+        serial_3_1_moves = list()
+
+        for s3 in serial_3_moves:  # s3 is like [3,3,3,4,4,4]
+            s3_set = set(s3)
+            new_cards = [i for i in self.cards_list if i not in s3_set]
+
+            # Get any s3_len items from cards
+            subcards = select(new_cards, len(s3_set))
+
+            for i in subcards:
+                serial_3_1_moves.append(s3 + i)
+
+        return list(k for k, _ in itertools.groupby(serial_3_1_moves))
+
+    def gen_type_12_serial_3_2(self, repeat_num=0):
+        serial_3_moves = self.gen_type_10_serial_triple(repeat_num=repeat_num)
+        serial_3_2_moves = list()
+        pair_set = sorted([k for k, v in self.cards_dict.items() if v >= 2])
+
+        for s3 in serial_3_moves:
+            s3_set = set(s3)
+            pair_candidates = [i for i in pair_set if i not in s3_set]
+
+            # Get any s3_len items from cards
+            subcards = select(pair_candidates, len(s3_set))
+            for i in subcards:
+                serial_3_2_moves.append(sorted(s3 + i * 2))
+
+        return serial_3_2_moves
+
+    def gen_type_13_4_2(self):
+        four_cards = list()
+        for k, v in self.cards_dict.items():
+            if v == 4:
+                four_cards.append(k)
+
+        result = list()
+        for fc in four_cards:
+            cards_list = [k for k in self.cards_list if k != fc]
+            subcards = select(cards_list, 2)
+            for i in subcards:
+                result.append([fc]*4 + i)
+        return list(k for k, _ in itertools.groupby(result))
+
+    def gen_type_14_4_22(self):
+        four_cards = list()
+        for k, v in self.cards_dict.items():
+            if v == 4:
+                four_cards.append(k)
+
+        result = list()
+        for fc in four_cards:
+            cards_list = [k for k, v in self.cards_dict.items() if k != fc and v>=2]
+            subcards = select(cards_list, 2)
+            for i in subcards:
+                result.append([fc] * 4 + [i[0], i[0], i[1], i[1]])
+        return result
+
+    # generate all possible moves from given cards
+    def gen_moves(self):
+        moves = []
+        moves.extend(self.gen_type_1_single())
+        moves.extend(self.gen_type_2_pair())
+        moves.extend(self.gen_type_3_triple())
+        moves.extend(self.gen_type_4_bomb())
+        moves.extend(self.gen_type_5_king_bomb())
+        moves.extend(self.gen_type_6_3_1())
+        moves.extend(self.gen_type_7_3_2())
+        moves.extend(self.gen_type_8_serial_single())
+        moves.extend(self.gen_type_9_serial_pair())
+        moves.extend(self.gen_type_10_serial_triple())
+        moves.extend(self.gen_type_11_serial_3_1())
+        moves.extend(self.gen_type_12_serial_3_2())
+        moves.extend(self.gen_type_13_4_2())
+        moves.extend(self.gen_type_14_4_22())
+        return moves

pve_server/utils/move_selector.py

@@ -0,0 +1,117 @@
+# return all moves that can beat rivals, moves and rival_move should be same type
+import collections
+
+
+def common_handle(moves, rival_move):
+    new_moves = list()
+    for move in moves:
+        if move[0] > rival_move[0]:
+            new_moves.append(move)
+    return new_moves
+
+
+def filter_type_1_single(moves, rival_move):
+    return common_handle(moves, rival_move)
+
+
+def filter_type_2_pair(moves, rival_move):
+    return common_handle(moves, rival_move)
+
+
+def filter_type_3_triple(moves, rival_move):
+    return common_handle(moves, rival_move)
+
+
+def filter_type_4_bomb(moves, rival_move):
+    return common_handle(moves, rival_move)
+
+
+# No need to filter for type_5_king_bomb
+
+def filter_type_6_3_1(moves, rival_move):
+    rival_move.sort()
+    rival_rank = rival_move[1]
+    new_moves = list()
+    for move in moves:
+        move.sort()
+        my_rank = move[1]
+        if my_rank > rival_rank:
+            new_moves.append(move)
+    return new_moves
+
+
+def filter_type_7_3_2(moves, rival_move):
+    rival_move.sort()
+    rival_rank = rival_move[2]
+    new_moves = list()
+    for move in moves:
+        move.sort()
+        my_rank = move[2]
+        if my_rank > rival_rank:
+            new_moves.append(move)
+    return new_moves
+
+
+def filter_type_8_serial_single(moves, rival_move):
+    return common_handle(moves, rival_move)
+
+
+def filter_type_9_serial_pair(moves, rival_move):
+    return common_handle(moves, rival_move)
+
+
+def filter_type_10_serial_triple(moves, rival_move):
+    return common_handle(moves, rival_move)
+
+
+def filter_type_11_serial_3_1(moves, rival_move):
+    rival = collections.Counter(rival_move)
+    rival_rank = max([k for k, v in rival.items() if v == 3])
+    new_moves = list()
+    for move in moves:
+        mymove = collections.Counter(move)
+        my_rank = max([k for k, v in mymove.items() if v == 3])
+        if my_rank > rival_rank:
+            new_moves.append(move)
+    return new_moves
+
+
+def filter_type_12_serial_3_2(moves, rival_move):
+    rival = collections.Counter(rival_move)
+    rival_rank = max([k for k, v in rival.items() if v == 3])
+    new_moves = list()
+    for move in moves:
+        mymove = collections.Counter(move)
+        my_rank = max([k for k, v in mymove.items() if v == 3])
+        if my_rank > rival_rank:
+            new_moves.append(move)
+    return new_moves
+
+
+def filter_type_13_4_2(moves, rival_move):
+    rival_move.sort()
+    rival_rank = rival_move[2]
+    new_moves = list()
+    for move in moves:
+        move.sort()
+        my_rank = move[2]
+        if my_rank > rival_rank:
+            new_moves.append(move)
+    return new_moves
+
+
+def filter_type_14_4_22(moves, rival_move):
+    rival = collections.Counter(rival_move)
+    rival_rank = my_rank = 0
+    for k, v in rival.items():
+        if v == 4:
+            rival_rank = k
+    new_moves = list()
+    for move in moves:
+        mymove = collections.Counter(move)
+        for k, v in mymove.items():
+            if v == 4:
+                my_rank = k
+        if my_rank > rival_rank:
+            new_moves.append(move)
+    return new_moves

pve_server/utils/utils.py

@@ -0,0 +1,33 @@
+import itertools
+
+# global parameters
+MIN_SINGLE_CARDS = 5
+MIN_PAIRS = 3
+MIN_TRIPLES = 2
+
+# action types
+TYPE_0_PASS = 0
+TYPE_1_SINGLE = 1
+TYPE_2_PAIR = 2
+TYPE_3_TRIPLE = 3
+TYPE_4_BOMB = 4
+TYPE_5_KING_BOMB = 5
+TYPE_6_3_1 = 6
+TYPE_7_3_2 = 7
+TYPE_8_SERIAL_SINGLE = 8
+TYPE_9_SERIAL_PAIR = 9
+TYPE_10_SERIAL_TRIPLE = 10
+TYPE_11_SERIAL_3_1 = 11
+TYPE_12_SERIAL_3_2 = 12
+TYPE_13_4_2 = 13
+TYPE_14_4_22 = 14
+TYPE_15_WRONG = 15
+
+# betting round action
+PASS = 0
+CALL = 1
+RAISE = 2
+
+# return all possible results of selecting num cards from cards list
+def select(cards, num):
+    return [list(i) for i in itertools.combinations(cards, num)]

requirements.txt

@@ -1,4 +1,4 @@
-rlcard[training]
+rlcard[torch]
 Django
 tqdm
 django-cors-headers

server/tournament/views.py

@@ -41,9 +41,10 @@ def _get_model_ids_all():
             def __init__(self):
                 self.model_id = name
                 self._entry_point = M
+                self.target_path = target_path
 
             def load(self):
-                model = self._entry_point(target_path)
+                model = self._entry_point(self.target_path)
                 return model
         rlcard.models.registration.model_registry.model_specs[name] = ModelSpec()
         MODEL_IDS_ALL[game].append(name)

src/components/MenuBar.js

@@ -1,50 +1,49 @@
-import React from 'react';
+import Button from '@material-ui/core/Button';
-import axios from 'axios';
+import Card from '@material-ui/core/Card';
-import { useHistory } from 'react-router-dom';
+import CardContent from '@material-ui/core/CardContent';
+import Collapse from '@material-ui/core/Collapse';
+import Dialog from '@material-ui/core/Dialog';
+import DialogActions from '@material-ui/core/DialogActions';
+import DialogContent from '@material-ui/core/DialogContent';
+import DialogTitle from '@material-ui/core/DialogTitle';
 import Drawer from '@material-ui/core/Drawer';
+import FormControl from '@material-ui/core/FormControl';
+import InputLabel from '@material-ui/core/InputLabel';
+import Link from '@material-ui/core/Link';
 import List from '@material-ui/core/List';
 import ListItem from '@material-ui/core/ListItem';
 import ListItemText from '@material-ui/core/ListItemText';
-import Collapse from '@material-ui/core/Collapse';
+import ListSubheader from '@material-ui/core/ListSubheader';
+import MenuItem from '@material-ui/core/MenuItem';
+import Select from '@material-ui/core/Select';
+import { makeStyles } from '@material-ui/core/styles';
+import TextField from '@material-ui/core/TextField';
+import Typography from '@material-ui/core/Typography';
 import ExpandLess from '@material-ui/icons/ExpandLess';
 import ExpandMore from '@material-ui/icons/ExpandMore';
-import {makeStyles} from "@material-ui/core/styles";
-import qs from 'query-string';
-import ListSubheader from "@material-ui/core/ListSubheader";
-import Button from "@material-ui/core/Button";
-import Dialog from "@material-ui/core/Dialog";
-import DialogTitle from "@material-ui/core/DialogTitle";
-import DialogContent from "@material-ui/core/DialogContent";
-import TextField from "@material-ui/core/TextField";
-import Select from '@material-ui/core/Select';
-import MenuItem from '@material-ui/core/MenuItem';
-import InputLabel from '@material-ui/core/InputLabel';
-import FormControl from '@material-ui/core/FormControl';
-import DialogActions from "@material-ui/core/DialogActions";
-import Card from '@material-ui/core/Card';
-import CardContent from '@material-ui/core/CardContent';
-import Typography from '@material-ui/core/Typography';
 import HelpIcon from '@material-ui/icons/Help';
-import Link from '@material-ui/core/Link';
+import axios from 'axios';
-
+import { Loading, Message, Upload } from 'element-react';
-import {Message, Upload, Loading} from 'element-react';
+import qs from 'query-string';
-import {apiUrl} from "../utils/config";
+import React from 'react';
+import { useHistory } from 'react-router-dom';
+import { apiUrl } from '../utils/config';
 
 const drawerWidth = 250;
 
 const useStyles = makeStyles((theme) => ({
     uploadNoteRoot: {
-        maxWidth: "358px",
+        maxWidth: '358px',
-        color: "#E6A23C",
+        color: '#E6A23C',
-        backgroundColor: "#fdf6ec",
+        backgroundColor: '#fdf6ec',
     },
     title: {
-        color: "#e6a23c",
+        color: '#e6a23c',
-        lineHeight: "24px",
+        lineHeight: '24px',
         fontSize: 16,
-        display: "flex",
+        display: 'flex',
-        alignItems: "center",
+        alignItems: 'center',
-        justifyContent: "flex-start"
+        justifyContent: 'flex-start',
     },
     formControl: {
         marginTop: theme.spacing(1),
@@ -54,52 +53,52 @@ const useStyles = makeStyles((theme) => ({
     drawer: {
         zIndex: 1001,
         width: drawerWidth,
-        flexShrink: 0
+        flexShrink: 0,
     },
     drawerPaper: {
         height: 'calc(100% - 75px)',
         zIndex: 1001,
         width: drawerWidth,
-        top: 75
+        top: 75,
     },
     button: {
         width: 200,
         position: 'fixed',
         left: 25,
-        bottom: 25
+        bottom: 25,
     },
     list: {
         height: 'calc(100% - 86px - 16px)',
         overflowY: 'auto',
         borderBottom: '1px solid #ccc',
-        paddingTop: '0'
+        paddingTop: '0',
     },
     nested: {
-        paddingLeft: theme.spacing(4)
+        paddingLeft: theme.spacing(4),
     },
     nestedSubheader: {
         paddingLeft: theme.spacing(4),
-        background: 'white'
+        background: 'white',
     },
     menuLayer1: {
         '& span': {
             fontWeight: 600,
-            fontSize: 14
+            fontSize: 14,
-        }
+        },
     },
     menuLayer2: {
         '& span': {
             fontWeight: 400,
-            fontSize: 14
+            fontSize: 14,
         },
         '&$active': {
             '& span': {
                 color: '#3f51b5',
-                fontWeight: 600
+                fontWeight: 600,
-            }
-        }
             },
-    active: {}
+        },
+    },
+    active: {},
 }));
 
 function MenuBar(props) {
@@ -127,7 +126,7 @@ function MenuBar (props) {
         let tempUploadForm = { ...uploadForm };
         tempUploadForm[property] = e.target.value;
         setUploadForm(tempUploadForm);
-    }
+    };
 
     const handleUploadDialogClose = () => {
         setUploadForm({ ...uploadFormInitValue });
@@ -139,17 +138,19 @@ function MenuBar (props) {
         // check if data to upload is legal
         if (uploadRef.current.state.fileList.length !== 1) {
             Message({
-                message: "Please select one zip file to upload",
+                message: 'Please select one zip file to upload',
-                type: "warning",
+                type: 'warning',
                 showClose: true,
             });
             return;
         }
         console.log('upload', uploadRef.current);
-        if (!["application/zip", "application/x-zip-compressed"].includes(uploadRef.current.state.fileList[0].raw.type)) {
+        if (
+            !['application/zip', 'application/x-zip-compressed'].includes(uploadRef.current.state.fileList[0].raw.type)
+        ) {
             Message({
-                message: "Only zip file can be uploaded",
+                message: 'Only zip file can be uploaded',
-                type: "warning",
+                type: 'warning',
                 showClose: true,
             });
             return;
@@ -157,21 +158,21 @@ function MenuBar (props) {
 
         if (uploadForm.name === '') {
             Message({
-                message: "Model name cannot be blank",
+                message: 'Model name cannot be blank',
-                type: "warning",
+                type: 'warning',
                 showClose: true,
             });
             return;
         }
 
         let flatGameList = [];
-        Object.keys(props.modelList).forEach(game => {
+        Object.keys(props.modelList).forEach((game) => {
             flatGameList = flatGameList.concat([...props.modelList[game]]);
         });
         if (flatGameList.includes(uploadForm.name)) {
             Message({
-                message: "Model name exists",
+                message: 'Model name exists',
-                type: "warning",
+                type: 'warning',
                 showClose: true,
             });
             return;
@@ -182,56 +183,89 @@ function MenuBar (props) {
         bodyFormData.append('game', uploadForm.game);
         bodyFormData.append('model', uploadRef.current.state.fileList[0].raw);
         setUploadDialogLoading(true);
-        axios.post(`${apiUrl}/tournament/upload_agent`, bodyFormData, {headers: {'Content-Type': 'multipart/form-data'}})
+        axios
-            .then(res => {
+            .post(`${apiUrl}/tournament/upload_agent`, bodyFormData, {
-                setTimeout(() => {setUploadDialogLoading(false)}, 250);
+                headers: { 'Content-Type': 'multipart/form-data' },
+            })
+            .then((res) => {
+                setTimeout(() => {
+                    setUploadDialogLoading(false);
+                }, 250);
                 Message({
-                    message: "Successfully uploaded model",
+                    message: 'Successfully uploaded model',
-                    type: "success",
+                    type: 'success',
                     showClose: true,
                 });
                 props.setReloadMenu(props.reloadMenu + 1);
                 setUploadDialogOpen(false);
                 setUploadForm({ ...uploadFormInitValue });
             })
-            .catch(err => {
+            .catch((err) => {
-                setTimeout(() => {setUploadDialogLoading(false)}, 250);
+                setTimeout(() => {
+                    setUploadDialogLoading(false);
+                }, 250);
                 Message({
-                    message: "Failed to upload model",
+                    message: 'Failed to upload model',
-                    type: "error",
+                    type: 'error',
                     showClose: true,
                 });
                 console.log(err);
-            })
+            });
     };
 
     const history = useHistory();
     const handleGameJump = (gameName) => {
         props.resetPagination();
         history.push(`/leaderboard?type=game&name=${gameName}`);
-    }
+    };
     const handleAgentJump = (agentName) => {
         props.resetPagination();
         history.push(`/leaderboard?type=agent&name=${agentName}`);
-    }
+    };
 
     const { type, name } = qs.parse(window.location.search);
-    const gameMenu = props.gameList.map(game => {
+    const gameMenu = props.gameList.map((game) => {
-        return <List component="div" disablePadding key={"game-menu-"+game.game}>
+        return (
-            <ListItem button className={classes.nested} onClick={() => {handleGameJump(game.game)}}>
+            <List component="div" disablePadding key={'game-menu-' + game.game}>
-                <ListItemText primary={game.dispName} className={`${classes.menuLayer2} ${(type === 'game' && name === game.game) ? classes.active : classes.inactive}`} />
+                <ListItem
+                    button
+                    className={classes.nested}
+                    onClick={() => {
+                        handleGameJump(game.game);
+                    }}
+                >
+                    <ListItemText
+                        primary={game.dispName}
+                        className={`${classes.menuLayer2} ${
+                            type === 'game' && name === game.game ? classes.active : classes.inactive
+                        }`}
+                    />
                 </ListItem>
             </List>
+        );
     });
 
     const generateAgentMenu = (modelList) => {
         return modelList.map((model) => {
-            return <List component="div" disablePadding key={"game-menu-"+model}>
+            return (
-                <ListItem button className={classes.nested} onClick={() => {handleAgentJump(model)}}>
+                <List component="div" disablePadding key={'game-menu-' + model}>
-                    <ListItemText primary={model} className={`${classes.menuLayer2} ${(type === 'agent' && name === model) ? classes.active : classes.inactive}`} />
+                    <ListItem
+                        button
+                        className={classes.nested}
+                        onClick={() => {
+                            handleAgentJump(model);
+                        }}
+                    >
+                        <ListItemText
+                            primary={model}
+                            className={`${classes.menuLayer2} ${
+                                type === 'agent' && name === model ? classes.active : classes.inactive
+                            }`}
+                        />
                     </ListItem>
                 </List>
-        })
+            );
+        });
     };
 
     return (
@@ -239,14 +273,10 @@ function MenuBar (props) {
             className={classes.drawer}
             variant="permanent"
             classes={{
-                paper: classes.drawerPaper
+                paper: classes.drawerPaper,
             }}
         >
-            <List
+            <List component="nav" aria-labelledby="nested-list-subheader" className={classes.list}>
-                component="nav"
-                aria-labelledby="nested-list-subheader"
-                className={classes.list}
-            >
                 <ListItem button onClick={handleClickAgent}>
                     <ListItemText primary="Game LeaderBoards" className={classes.menuLayer1} />
                     {open.agent ? <ExpandLess /> : <ExpandMore />}
@@ -259,15 +289,14 @@ function MenuBar (props) {
                     {open.game ? <ExpandLess /> : <ExpandMore />}
                 </ListItem>
                 <Collapse in={open.game} timeout="auto" unmountOnExit>
-                    {Object.keys(props.modelList).map(gameName => {
+                    {Object.keys(props.modelList).map((gameName) => {
                         return (
                             <div key={`agentMenu-sublist-${gameName}`}>
                                 <ListSubheader className={classes.nestedSubheader}>{gameName}</ListSubheader>
                                 {generateAgentMenu(props.modelList[gameName])}
                             </div>
-                        )
+                        );
                     })}
-
                 </Collapse>
             </List>
             <Button variant="contained" color="primary" onClick={openUploadDialog} className={classes.button}>
@@ -283,12 +312,27 @@ function MenuBar (props) {
                     <DialogTitle id="form-dialog-title">Upload Model</DialogTitle>
                     <DialogContent>
                         <Card variant="outlined" className={classes.uploadNoteRoot}>
-                        <CardContent style={{paddingBottom: "16px"}}>
+                            <CardContent style={{ paddingBottom: '16px' }}>
                                 <Typography className={classes.title} color="textSecondary" gutterBottom>
-                                <HelpIcon style={{marginRight: "5px"}} />Note
+                                    <HelpIcon style={{ marginRight: '5px' }} />
+                                    Note
                                 </Typography>
                                 <Typography variant="body2" component="p">
-                                Download the example <Link href={apiUrl + "/tournament/download_examples?name=example_luduc_nfsp_model"} download>NFSP model</Link> and <Link href={apiUrl + "/tournament/download_examples?name=example_luduc_rule_model"} download>Rule model</Link> of Leduc Hold'em to test and learn about model upload functionality.
+                                    Download the example{' '}
+                                    <Link
+                                        href={apiUrl + '/tournament/download_examples?name=leduc_holdem_dqn'}
+                                        download
+                                    >
+                                        DQN model
+                                    </Link>{' '}
+                                    for Leduc Holdem or{' '}
+                                    <Link
+                                        href={apiUrl + '/tournament/download_examples?name=example_luduc_rule_model'}
+                                        download
+                                    >
+                                        DMC model
+                                    </Link>{' '}
+                                    for Doudizhu to test and learn about model upload functionality.
                                 </Typography>
                             </CardContent>
                         </Card>
@@ -303,7 +347,9 @@ function MenuBar (props) {
                             tip={<div className="el-upload__tip">Only zip file can be uploaded</div>}
                         >
                             <i className="el-icon-upload" />
-                        <div className="el-upload__text">Drag the file here, or <em>Click to upload</em></div>
+                            <div className="el-upload__text">
+                                Drag the file here, or <em>Click to upload</em>
+                            </div>
                         </Upload>
                         <TextField
                             className={classes.formControl}
@@ -339,10 +385,14 @@ function MenuBar (props) {
                                 labelId="upload-game-label"
                                 id="upload-game"
                                 value={uploadForm.game}
-                            onChange={(e) => handleUploadFormChange(e, "game")}
+                                onChange={(e) => handleUploadFormChange(e, 'game')}
                             >
-                            {props.gameList.map(game => {
+                                {props.gameList.map((game) => {
-                                return <MenuItem key={"upload-game-"+game.game} value={game.game}>{game.dispName}</MenuItem>
+                                    return (
+                                        <MenuItem key={'upload-game-' + game.game} value={game.game}>
+                                            {game.dispName}
+                                        </MenuItem>
+                                    );
                                 })}
                             </Select>
                         </FormControl>
@@ -357,8 +407,20 @@ function MenuBar (props) {
                     </DialogActions>
                 </Loading>
             </Dialog>
+            <Dialog
+                open={uploadDialogOpen}
+                onClose={handleUploadDialogClose}
+                aria-labelledby="form-dialog-title"
+                disableBackdropClick={true}
+            >
+                <DialogTitle id="form-dialog-title">Choose Download Channel</DialogTitle>
+                <DialogContent>
+                    <Button>Google Drive</Button>
+                    <Button>百度网盘</Button>
+                </DialogContent>
+            </Dialog>
         </Drawer>
-    )
+    );
 }
 
 export default MenuBar;

src/view/LeaderBoard.js

@@ -1,27 +1,27 @@
-import React, {useEffect} from 'react';
+import Breadcrumbs from "@material-ui/core/Breadcrumbs";
-import qs from 'query-string';
+import Button from "@material-ui/core/Button";
-import MenuBar from "../components/MenuBar";
+import Paper from '@material-ui/core/Paper';
-
-import PropTypes from 'prop-types';
 import { lighten, makeStyles } from '@material-ui/core/styles';
+import withStyles from "@material-ui/core/styles/withStyles";
 import Table from '@material-ui/core/Table';
 import TableBody from '@material-ui/core/TableBody';
 import TableCell from '@material-ui/core/TableCell';
 import TableContainer from '@material-ui/core/TableContainer';
 import TableHead from '@material-ui/core/TableHead';
+import TablePagination from "@material-ui/core/TablePagination";
 import TableRow from '@material-ui/core/TableRow';
 import Toolbar from '@material-ui/core/Toolbar';
 import Typography from '@material-ui/core/Typography';
-import Paper from '@material-ui/core/Paper';
-import { apiUrl } from "../utils/config";
-import axios from 'axios';
-import TablePagination from "@material-ui/core/TablePagination";
-import Breadcrumbs from "@material-ui/core/Breadcrumbs";
-import withStyles from "@material-ui/core/styles/withStyles";
 import PlayCircleOutlineIcon from '@material-ui/icons/PlayCircleOutline';
-import Button from "@material-ui/core/Button";
+import axios from 'axios';
+import { Loading, Message } from "element-react";
+import PropTypes from 'prop-types';
+import qs from 'query-string';
+import React, { useEffect } from 'react';
 import { useHistory } from "react-router-dom";
-import {Message, Loading} from "element-react";
+import MenuBar from "../components/MenuBar";
+import { apiUrl } from "../utils/config";
+
 
 const gameList = [
     {game: 'leduc-holdem', dispName: 'Leduc Hold\'em'},
@@ -241,7 +241,7 @@ const EnhancedTableToolbar = (props) => {
             const handleLaunchTournament = (gameName) => {
                 // todo: customize eval num
                 setButtonLoading(true);
-                axios.get(`${apiUrl}/tournament/launch?eval_num=200&name=${gameName}`)
+                axios.get(`${apiUrl}/tournament/launch?num_eval_games=200&name=${gameName}`)
                     .then(res => {
                         setTimeout(() => {setButtonLoading(false)}, 250);
                         Message({

src/view/ReplayView/LeducHoldemReplayView.js

@@ -1,29 +1,27 @@
-import React from 'react';
-import axios from 'axios';
-import qs from 'query-string';
-import '../../assets/gameview.scss';
-import {LeducHoldemGameBoard} from '../../components/GameBoard';
-import {deepCopy} from "../../utils";
-import { apiUrl } from "../../utils/config";
-
-import { Layout, Loading } from 'element-react';
-import { Message } from 'element-react';
-import Slider from '@material-ui/core/Slider';
 import Button from '@material-ui/core/Button';
-import Paper from '@material-ui/core/Paper';
-import Divider from '@material-ui/core/Divider';
-import LinearProgress from '@material-ui/core/LinearProgress';
-import PlayArrowRoundedIcon from '@material-ui/icons/PlayArrowRounded';
-import PauseCircleOutlineRoundedIcon from '@material-ui/icons/PauseCircleOutlineRounded';
-import ReplayRoundedIcon from '@material-ui/icons/ReplayRounded';
-import NotInterestedIcon from '@material-ui/icons/NotInterested';
-import SkipNextIcon from '@material-ui/icons/SkipNext';
-import SkipPreviousIcon from '@material-ui/icons/SkipPrevious';
 import Dialog from '@material-ui/core/Dialog';
 import DialogActions from '@material-ui/core/DialogActions';
 import DialogContent from '@material-ui/core/DialogContent';
 import DialogContentText from '@material-ui/core/DialogContentText';
 import DialogTitle from '@material-ui/core/DialogTitle';
+import Divider from '@material-ui/core/Divider';
+import LinearProgress from '@material-ui/core/LinearProgress';
+import Paper from '@material-ui/core/Paper';
+import Slider from '@material-ui/core/Slider';
+import NotInterestedIcon from '@material-ui/icons/NotInterested';
+import PauseCircleOutlineRoundedIcon from '@material-ui/icons/PauseCircleOutlineRounded';
+import PlayArrowRoundedIcon from '@material-ui/icons/PlayArrowRounded';
+import ReplayRoundedIcon from '@material-ui/icons/ReplayRounded';
+import SkipNextIcon from '@material-ui/icons/SkipNext';
+import SkipPreviousIcon from '@material-ui/icons/SkipPrevious';
+import axios from 'axios';
+import { Layout, Loading, Message } from 'element-react';
+import qs from 'query-string';
+import React from 'react';
+import '../../assets/gameview.scss';
+import { LeducHoldemGameBoard } from '../../components/GameBoard';
+import { deepCopy } from '../../utils';
+import { apiUrl } from '../../utils/config';
 
 class LeducHoldemReplayView extends React.Component {
     constructor(props) {
@@ -37,15 +35,15 @@ class LeducHoldemReplayView extends React.Component {
         this.moveHistoryTotalLength = null;
         this.gameStateHistory = [[], []];
         this.initGameState = {
-            gameStatus: "ready", // "ready", "playing", "paused", "over"
+            gameStatus: 'ready', // "ready", "playing", "paused", "over"
             playerInfo: [],
             hands: [],
-            latestAction: ["", ""],
+            latestAction: ['', ''],
             mainViewerId: mainViewerId,
             round: 0,
             turn: 0,
             pot: [1, 1],
-            publicCard: "",
+            publicCard: '',
             currentPlayer: null,
             considerationTime: this.initConsiderationTime,
             completedPercent: 0,
@@ -55,9 +53,9 @@ class LeducHoldemReplayView extends React.Component {
             gameStateLoop: null,
             gameSpeed: 0,
             gameEndDialog: false,
-            gameEndDialogText: "",
+            gameEndDialogText: '',
-            fullScreenLoading: false
+            fullScreenLoading: false,
-        }
+        };
     }
 
     generateNewState() {
@@ -71,7 +69,7 @@ class LeducHoldemReplayView extends React.Component {
                 gameInfo = deepCopy(this.gameStateHistory[gameInfo.round][gameInfo.turn]);
                 return gameInfo;
             }
-            gameInfo.latestAction = ["", ""];
+            gameInfo.latestAction = ['', ''];
             gameInfo.currentPlayer = this.moveHistory[1][0].playerIdx;
             gameInfo.considerationTime = this.initConsiderationTime;
             this.setState({ gameInfo: gameInfo });
@@ -84,30 +82,35 @@ class LeducHoldemReplayView extends React.Component {
             if (gameInfo.currentPlayer === this.moveHistory[gameInfo.round][gameInfo.turn].playerIdx) {
                 gameInfo.latestAction[gameInfo.currentPlayer] = this.moveHistory[gameInfo.round][gameInfo.turn].move;
                 switch (gameInfo.latestAction[gameInfo.currentPlayer]) {
-                    case "check":
+                    case 'check':
                         break;
-                    case "raise":
+                    case 'raise':
-                        gameInfo.pot[gameInfo.currentPlayer] = (gameInfo.pot[(gameInfo.currentPlayer+2-1)%2] + (gameInfo.round+1) * 2);
+                        gameInfo.pot[gameInfo.currentPlayer] =
+                            gameInfo.pot[(gameInfo.currentPlayer + 2 - 1) % 2] + (gameInfo.round + 1) * 2;
                         break;
-                    case "call":
+                    case 'call':
                         // the upstream player must have bet more
                         if (gameInfo.pot[(gameInfo.currentPlayer + 2 - 1) % 2] > gameInfo.pot[gameInfo.currentPlayer]) {
                             gameInfo.pot[gameInfo.currentPlayer] = gameInfo.pot[(gameInfo.currentPlayer + 2 - 1) % 2];
                         } else {
                             Message({
-                                message: "Current player choose call but has bet more or equal to the upstream player",
+                                message: 'Current player choose call but has bet more or equal to the upstream player',
-                                type: "error",
+                                type: 'error',
-                                showClose: true
+                                showClose: true,
                             });
                         }
                         break;
-                    case "fold":
+                    case 'fold':
                         // if one player folds, game ends
-                        const foldedFound = gameInfo.playerInfo.find(element=>{return element.index === gameInfo.currentPlayer});
+                        const foldedFound = gameInfo.playerInfo.find((element) => {
+                            return element.index === gameInfo.currentPlayer;
+                        });
                         const foldedId = foldedFound ? foldedFound.id : -1;
-                        const winnerFound = gameInfo.playerInfo.find(element=>{return element.index === (gameInfo.currentPlayer+1)%2});
+                        const winnerFound = gameInfo.playerInfo.find((element) => {
+                            return element.index === (gameInfo.currentPlayer + 1) % 2;
+                        });
                         const winnerId = winnerFound ? winnerFound.id : -1;
-                        gameInfo.gameStatus = "over";
+                        gameInfo.gameStatus = 'over';
                         this.setState({ gameInfo: gameInfo });
                         setTimeout(() => {
                             const mes = `Player ${foldedId} folded, player ${winnerId} wins!`;
@@ -117,30 +120,30 @@ class LeducHoldemReplayView extends React.Component {
                     default:
                         Message({
                             message: "Error in player's latest action",
-                            type: "error",
+                            type: 'error',
-                            showClose: true
+                            showClose: true,
                         });
                 }
                 gameInfo.turn++;
                 if (gameInfo.round !== 0 && gameInfo.turn === this.moveHistory[gameInfo.round].length) {
-                    gameInfo.gameStatus = "over";
+                    gameInfo.gameStatus = 'over';
                     this.setState({ gameInfo: gameInfo });
                     setTimeout(() => {
                         // TODO: show winner
-                        this.setState({gameEndDialog: true, gameEndDialogText: ""});
+                        this.setState({ gameEndDialog: true, gameEndDialogText: '' });
                     }, 200);
                     return gameInfo;
                 }
                 gameInfo.currentPlayer = (gameInfo.currentPlayer + 1) % 2;
                 gameInfo.considerationTime = this.initConsiderationTime;
                 gameInfo.completedPercent += 100.0 / this.moveHistoryTotalLength;
-                gameInfo.gameStatus = "playing";
+                gameInfo.gameStatus = 'playing';
                 this.setState({ gameInfo: gameInfo });
             } else {
                 Message({
-                    message: "Mismatch in current player & move history",
+                    message: 'Mismatch in current player & move history',
-                    type: "error",
+                    type: 'error',
-                    showClose: true
+                    showClose: true,
                 });
             }
         }
@@ -149,9 +152,9 @@ class LeducHoldemReplayView extends React.Component {
             this.gameStateHistory[gameInfo.round].push(gameInfo);
         } else {
             Message({
-                message: "Inconsistent game state history length and turn number",
+                message: 'Inconsistent game state history length and turn number',
-                type: "error",
+                type: 'error',
-                showClose: true
+                showClose: true,
             });
         }
         return gameInfo;
@@ -165,7 +168,7 @@ class LeducHoldemReplayView extends React.Component {
                 currentConsiderationTime = currentConsiderationTime < 0 ? 0 : currentConsiderationTime;
                 if (currentConsiderationTime === 0 && this.state.gameSpeed < 2) {
                     let gameInfo = deepCopy(this.state.gameInfo);
-                    gameInfo.toggleFade = "fade-out";
+                    gameInfo.toggleFade = 'fade-out';
                     this.setState({ gameInfo: gameInfo });
                 }
                 let gameInfo = deepCopy(this.state.gameInfo);
@@ -174,18 +177,18 @@ class LeducHoldemReplayView extends React.Component {
                 this.gameStateTimer();
             } else {
                 let gameInfo = this.generateNewState();
-                if(gameInfo.gameStatus === "over") return;
+                if (gameInfo.gameStatus === 'over') return;
                 this.gameStateTimer();
-                gameInfo.gameStatus = "playing";
+                gameInfo.gameStatus = 'playing';
-                if(this.state.gameInfo.toggleFade === "fade-out") {
+                if (this.state.gameInfo.toggleFade === 'fade-out') {
-                    gameInfo.toggleFade = "fade-in";
+                    gameInfo.toggleFade = 'fade-in';
                 }
                 this.setState({ gameInfo: gameInfo }, () => {
                     // toggle fade in
-                    if(this.state.gameInfo.toggleFade !== ""){
+                    if (this.state.gameInfo.toggleFade !== '') {
                         setTimeout(() => {
                             let gameInfo = deepCopy(this.state.gameInfo);
-                            gameInfo.toggleFade = "";
+                            gameInfo.toggleFade = '';
                             this.setState({ gameInfo: gameInfo });
                         }, 200);
                     }
@@ -200,13 +203,17 @@ class LeducHoldemReplayView extends React.Component {
         // start full screen loading
         this.setState({ fullScreenLoading: true });
 
-        axios.get(requestUrl)
+        axios
-            .then(res => {
+            .get(requestUrl)
+            .then((res) => {
                 res = res.data;
+                // for test use
+                if (typeof res === 'string') res = JSON.parse(res.replaceAll("'", '"').replaceAll('None', 'null'));
+
                 if (res.moveHistory.length === 0) {
                     Message({
-                        message: "Empty move history",
+                        message: 'Empty move history',
-                        type: "error",
+                        type: 'error',
                         showClose: true,
                     });
                     this.setState({ fullScreenLoading: false });
@@ -216,7 +223,7 @@ class LeducHoldemReplayView extends React.Component {
                 this.moveHistory = res.moveHistory;
                 this.moveHistoryTotalLength = this.moveHistory.reduce((count, round) => count + round.length, 0) - 1;
                 let gameInfo = deepCopy(this.initGameState);
-                gameInfo.gameStatus = "playing";
+                gameInfo.gameStatus = 'playing';
                 gameInfo.playerInfo = res.playerInfo;
                 gameInfo.hands = res.initHands;
                 gameInfo.currentPlayer = res.moveHistory[0][0].playerIdx;
@@ -235,16 +242,16 @@ class LeducHoldemReplayView extends React.Component {
                     this.gameStateTimer();
                 });
             })
-            .catch(err => {
+            .catch((err) => {
                 console.log(err);
                 Message({
                     message: `Error in getting replay data: ${err}`,
-                    type: "error",
+                    type: 'error',
-                    showClose: true
+                    showClose: true,
                 });
                 this.setState({ fullScreenLoading: false });
             });
-    };
+    }
 
     pauseReplay() {
         if (this.gameStateTimeout) {
@@ -252,14 +259,14 @@ class LeducHoldemReplayView extends React.Component {
             this.gameStateTimeout = null;
         }
         let gameInfo = deepCopy(this.state.gameInfo);
-        gameInfo.gameStatus = "paused";
+        gameInfo.gameStatus = 'paused';
         this.setState({ gameInfo: gameInfo });
     }
 
     resumeReplay() {
         this.gameStateTimer();
         let gameInfo = deepCopy(this.state.gameInfo);
-        gameInfo.gameStatus = "playing";
+        gameInfo.gameStatus = 'playing';
         this.setState({ gameInfo: gameInfo });
     }
 
@@ -269,51 +276,132 @@ class LeducHoldemReplayView extends React.Component {
 
     gameStatusButton(status) {
         switch (status) {
-            case "ready":
+            case 'ready':
-                return <Button className={"status-button"} variant={"contained"} startIcon={<PlayArrowRoundedIcon />} color="primary" onClick={()=>{this.startReplay()}}>Start</Button>;
+                return (
-            case "playing":
+                    <Button
-                return <Button className={"status-button"} variant={"contained"} startIcon={<PauseCircleOutlineRoundedIcon />} color="secondary" onClick={()=>{this.pauseReplay()}}>Pause</Button>;
+                        className={'status-button'}
-            case "paused":
+                        variant={'contained'}
-                return <Button className={"status-button"} variant={"contained"} startIcon={<PlayArrowRoundedIcon />} color="primary" onClick={()=>{this.resumeReplay()}}>Resume</Button>;
+                        startIcon={<PlayArrowRoundedIcon />}
-            case "over":
+                        color="primary"
-                return <Button className={"status-button"} variant={"contained"} startIcon={<ReplayRoundedIcon />} color="primary" onClick={()=>{this.startReplay()}}>Restart</Button>;
+                        onClick={() => {
+                            this.startReplay();
+                        }}
+                    >
+                        Start
+                    </Button>
+                );
+            case 'playing':
+                return (
+                    <Button
+                        className={'status-button'}
+                        variant={'contained'}
+                        startIcon={<PauseCircleOutlineRoundedIcon />}
+                        color="secondary"
+                        onClick={() => {
+                            this.pauseReplay();
+                        }}
+                    >
+                        Pause
+                    </Button>
+                );
+            case 'paused':
+                return (
+                    <Button
+                        className={'status-button'}
+                        variant={'contained'}
+                        startIcon={<PlayArrowRoundedIcon />}
+                        color="primary"
+                        onClick={() => {
+                            this.resumeReplay();
+                        }}
+                    >
+                        Resume
+                    </Button>
+                );
+            case 'over':
+                return (
+                    <Button
+                        className={'status-button'}
+                        variant={'contained'}
+                        startIcon={<ReplayRoundedIcon />}
+                        color="primary"
+                        onClick={() => {
+                            this.startReplay();
+                        }}
+                    >
+                        Restart
+                    </Button>
+                );
             default:
                 alert(`undefined game status: ${status}`);
         }
     }
 
-    computeProbabilityItem(idx){
+    computeProbabilityItem(action) {
-        if(this.state.gameInfo.gameStatus !== "ready"){
+        if (this.state.gameInfo.gameStatus !== 'ready') {
             let currentMove = null;
             if (this.state.gameInfo.turn !== this.moveHistory[this.state.gameInfo.round].length) {
                 currentMove = this.moveHistory[this.state.gameInfo.round][this.state.gameInfo.turn];
             }
             let style = {};
-            style["backgroundColor"] = currentMove !== null ? `rgba(63, 81, 181, ${currentMove.probabilities[idx].probability})` : "#bdbdbd";
+            let probabilities = null;
-            return (
+            let probabilityItemType = null;
-                <div className={"playing"} style={style}>
+            if (currentMove) {
-                    <div className="probability-move">
+                if (Array.isArray(currentMove.info)) {
-                    {currentMove !== null ?
+                    probabilityItemType = 'Rule';
-                        <img src={require('../../assets/images/Actions/' + currentMove.probabilities[idx].move + (currentMove.probabilities[idx].probability < 0 ? "_u" : "") + '.png')} alt={currentMove.probabilities[idx].move} height="30%" width="30%" />
-                        :
-                        <NotInterestedIcon fontSize="large" />}
-                    </div>
-                    {currentMove !== null ?
-                        (<div className={"non-card"}>
-                            {
-                                currentMove.probabilities[idx].probability === -1 ?
-                                <span>Illegal</span>
-                                :
-                                currentMove.probabilities[idx].probability === -2 ?
-                                <span>Rule Based</span>
-                                :
-                                <span>{`Probability ${(currentMove.probabilities[idx].probability * 100).toFixed(2)}%`}</span>
-                            }
-                        </div>) : ""}
-                </div>
-            )
                 } else {
-            return <span className={"waiting"}>Waiting...</span>
+                    if ('probs' in currentMove.info) {
+                        probabilityItemType = 'Probability';
+                        probabilities = currentMove.info.probs[action];
+                    } else if ('values' in currentMove.info) {
+                        probabilityItemType = 'Expected payoff';
+                        probabilities = currentMove.info.values[action];
+                    } else {
+                        probabilityItemType = 'Rule';
+                    }
+                }
+            }
+
+            // style["backgroundColor"] = currentMove !== null ? `rgba(63, 81, 181, ${currentMove.probabilities[idx].probability})` : "#bdbdbd";
+            style['backgroundColor'] = currentMove !== null ? `#fff` : '#bdbdbd';
+            return (
+                <div className={'playing'} style={style}>
+                    <div className="probability-move">
+                        {currentMove !== null ? (
+                            <img
+                                src={require('../../assets/images/Actions/' +
+                                    action +
+                                    (probabilities === undefined || probabilities === null ? '_u' : '') +
+                                    '.png')}
+                                alt={action}
+                                height="30%"
+                                width="30%"
+                            />
+                        ) : (
+                            <NotInterestedIcon fontSize="large" />
+                        )}
+                    </div>
+                    {currentMove !== null ? (
+                        <div className={'non-card'}>
+                            {probabilities === undefined ? (
+                                <span>Illegal</span>
+                            ) : probabilityItemType === 'Rule' ? (
+                                <span>Rule Based</span>
+                            ) : (
+                                <span>
+                                    {probabilityItemType === 'Probability'
+                                        ? `Probability: ${(probabilities * 100).toFixed(2)}%`
+                                        : `Expected payoff: ${probabilities.toFixed(4)}`}
+                                </span>
+                            )}
+                        </div>
+                    ) : (
+                        ''
+                    )}
+                </div>
+            );
+        } else {
+            return <span className={'waiting'}>Waiting...</span>;
         }
     }
 
@@ -325,21 +413,21 @@ class LeducHoldemReplayView extends React.Component {
         } else {
             gameInfo = deepCopy(this.gameStateHistory[this.state.gameInfo.round][this.state.gameInfo.turn - 1]);
         }
-        gameInfo.gameStatus = "paused";
+        gameInfo.gameStatus = 'paused';
-        gameInfo.toggleFade = "";
+        gameInfo.toggleFade = '';
         this.setState({ gameInfo: gameInfo });
     }
 
     go2NextGameState() {
         let gameInfo = this.generateNewState();
-        if(gameInfo.gameStatus === "over") return;
+        if (gameInfo.gameStatus === 'over') return;
-        gameInfo.gameStatus = "paused";
+        gameInfo.gameStatus = 'paused';
-        gameInfo.toggleFade = "";
+        gameInfo.toggleFade = '';
         this.setState({ gameInfo: gameInfo });
     }
 
     handleCloseGameEndDialog() {
-        this.setState({gameEndDialog: false, gameEndDialogText: ""});
+        this.setState({ gameEndDialog: false, gameEndDialogText: '' });
     }
 
     render() {
@@ -374,34 +462,44 @@ class LeducHoldemReplayView extends React.Component {
             {
                 value: 3,
                 label: 'x8',
-            }
+            },
         ];
 
         return (
             <div>
                 <Dialog
                     open={this.state.gameEndDialog}
-                    onClose={()=>{this.handleCloseGameEndDialog()}}
+                    onClose={() => {
+                        this.handleCloseGameEndDialog();
+                    }}
                     aria-labelledby="alert-dialog-title"
                     aria-describedby="alert-dialog-description"
                 >
-                    <DialogTitle id="alert-dialog-title" style={{"width": "200px"}}>{"Game Ends!"}</DialogTitle>
+                    <DialogTitle id="alert-dialog-title" style={{ width: '200px' }}>
+                        {'Game Ends!'}
+                    </DialogTitle>
                     <DialogContent>
                         <DialogContentText id="alert-dialog-description">
                             {this.state.gameEndDialogText}
                         </DialogContentText>
                     </DialogContent>
                     <DialogActions>
-                        <Button onClick={()=>{this.handleCloseGameEndDialog()}} color="primary" autoFocus>
+                        <Button
+                            onClick={() => {
+                                this.handleCloseGameEndDialog();
+                            }}
+                            color="primary"
+                            autoFocus
+                        >
                             OK
                         </Button>
                     </DialogActions>
                 </Dialog>
-                <div className={"leduc-view-container"}>
+                <div className={'leduc-view-container'}>
-                    <Layout.Row style={{"height": "540px"}}>
+                    <Layout.Row style={{ height: '540px' }}>
-                        <Layout.Col style={{"height": "100%"}} span="17">
+                        <Layout.Col style={{ height: '100%' }} span="17">
-                            <div style={{"height": "100%"}}>
+                            <div style={{ height: '100%' }}>
-                                <Paper className={"leduc-gameboard-paper"} elevation={3}>
+                                <Paper className={'leduc-gameboard-paper'} elevation={3}>
                                     <LeducHoldemGameBoard
                                         playerInfo={this.state.gameInfo.playerInfo}
                                         hands={this.state.gameInfo.hands}
@@ -417,30 +515,21 @@ class LeducHoldemReplayView extends React.Component {
                                 </Paper>
                             </div>
                         </Layout.Col>
-                        <Layout.Col span="7" style={{"height": "100%"}}>
+                        <Layout.Col span="7" style={{ height: '100%' }}>
-                            <Paper className={"leduc-probability-paper"} elevation={3}>
+                            <Paper className={'leduc-probability-paper'} elevation={3}>
-                                <div className={"probability-player"}>
+                                <div className={'probability-player'}>
-                                    {
+                                    {this.state.gameInfo.playerInfo.length > 0 ? (
-                                        this.state.gameInfo.playerInfo.length > 0 ?
                                         <span>Current Player: {this.state.gameInfo.currentPlayer}</span>
-                                        :
+                                    ) : (
                                         <span>Waiting...</span>
-                                    }
+                                    )}
                                 </div>
                                 <Divider />
-                                <div className={"probability-table"}>
+                                <div className={'probability-table'}>
-                                    <div className={"probability-item"}>
+                                    <div className={'probability-item'}>{this.computeProbabilityItem('call')}</div>
-                                        {this.computeProbabilityItem(0)}
+                                    <div className={'probability-item'}>{this.computeProbabilityItem('check')}</div>
-                                    </div>
+                                    <div className={'probability-item'}>{this.computeProbabilityItem('raise')}</div>
-                                    <div className={"probability-item"}>
+                                    <div className={'probability-item'}>{this.computeProbabilityItem('fold')}</div>
-                                        {this.computeProbabilityItem(1)}
-                                    </div>
-                                    <div className={"probability-item"}>
-                                        {this.computeProbabilityItem(2)}
-                                    </div>
-                                    <div className={"probability-item"}>
-                                        {this.computeProbabilityItem(3)}
-                                    </div>
                                 </div>
                             </Paper>
                         </Layout.Col>
@@ -450,15 +539,20 @@ class LeducHoldemReplayView extends React.Component {
                     </div>
                     <Loading loading={this.state.fullScreenLoading}>
                         <div className="game-controller">
-                        <Paper className={"game-controller-paper"} elevation={3}>
+                            <Paper className={'game-controller-paper'} elevation={3}>
-                            <Layout.Row style={{"height": "51px"}}>
+                                <Layout.Row style={{ height: '51px' }}>
-                                <Layout.Col span="7" style={{"height": "51px", "lineHeight": "48px"}}>
+                                    <Layout.Col span="7" style={{ height: '51px', lineHeight: '48px' }}>
                                         <div>
                                             <Button
                                                 variant="contained"
                                                 color="primary"
-                                        disabled={this.state.gameInfo.gameStatus !== "paused" || (this.state.gameInfo.round === 0 && this.state.gameInfo.turn === 0)}
+                                                disabled={
-                                        onClick={()=>{this.go2PrevGameState()}}
+                                                    this.state.gameInfo.gameStatus !== 'paused' ||
+                                                    (this.state.gameInfo.round === 0 && this.state.gameInfo.turn === 0)
+                                                }
+                                                onClick={() => {
+                                                    this.go2PrevGameState();
+                                                }}
                                             >
                                                 <SkipPreviousIcon />
                                             </Button>
@@ -466,30 +560,42 @@ class LeducHoldemReplayView extends React.Component {
                                             <Button
                                                 variant="contained"
                                                 color="primary"
-                                        disabled={this.state.gameInfo.gameStatus !== "paused"}
+                                                disabled={this.state.gameInfo.gameStatus !== 'paused'}
-                                        onClick={()=>{this.go2NextGameState()}}
+                                                onClick={() => {
+                                                    this.go2NextGameState();
+                                                }}
                                             >
                                                 <SkipNextIcon />
                                             </Button>
                                         </div>
                                     </Layout.Col>
-                                <Layout.Col span="1" style={{"height": "100%", "width": "1px"}}>
+                                    <Layout.Col span="1" style={{ height: '100%', width: '1px' }}>
                                         <Divider orientation="vertical" />
                                     </Layout.Col>
-                                <Layout.Col span="3" style={{"height": "51px", "lineHeight": "51px", "marginLeft": "-1px", "marginRight": "-1px"}}>
+                                    <Layout.Col
-                                    <div style={{"textAlign": "center"}}>{`Turn ${this.state.gameInfo.turn}`}</div>
+                                        span="3"
+                                        style={{
+                                            height: '51px',
+                                            lineHeight: '51px',
+                                            marginLeft: '-1px',
+                                            marginRight: '-1px',
+                                        }}
+                                    >
+                                        <div style={{ textAlign: 'center' }}>{`Turn ${this.state.gameInfo.turn}`}</div>
                                     </Layout.Col>
-                                <Layout.Col span="1" style={{"height": "100%", "width": "1px"}}>
+                                    <Layout.Col span="1" style={{ height: '100%', width: '1px' }}>
                                         <Divider orientation="vertical" />
                                     </Layout.Col>
                                     <Layout.Col span="14">
                                         <div>
-                                        <label className={"form-label-left"}>Game Speed</label>
+                                            <label className={'form-label-left'}>Game Speed</label>
-                                        <div style={{"marginLeft": "100px", "marginRight": "10px"}}>
+                                            <div style={{ marginLeft: '100px', marginRight: '10px' }}>
                                                 <Slider
                                                     value={this.state.gameSpeed}
                                                     getAriaValueText={sliderValueText}
-                                                onChange={(e, newVal)=>{this.changeGameSpeed(newVal)}}
+                                                    onChange={(e, newVal) => {
+                                                        this.changeGameSpeed(newVal);
+                                                    }}
                                                     aria-labelledby="discrete-slider-custom"
                                                     step={1}
                                                     min={-3}