Marko Tot

Runners

2D - RPG - pathfinding - Mutliagent - Java

Runners is a top down, single player game, with the objective of collecting all the hearts on the map. You control the single agent, run away from multiple enemies, and obtain power-ups. The game has multiple levels with a custom level editor available for the game designer. This project was done from scratch in Java, implementing separate rendering and gameplayer engine loops.

In order to make the enemies cooperate in chasing the players, multi agent A* algorithm is implemented. Allowing the enemies to surround the player by looking at all the possible routes to the player and communicating between them to organize the most effective way. Similarly, an AI algorithm was implemented for the player that allows the AI to take control of the agent and try to escape the enemies. To enable the use of A* algorithms, the map was divided into a grid, with automatically generated detection for dead-ends and crossroads.

PACO

2D - Platformer - Puzzle - Android & IOS - Unity

Paco is a 2D arcade and puzzle platformer. You control Paco, by slinging around the screen, traversing through the level, climbing onto trees, buildings and slippery walls while avoiding obstacles. The game was a project done in Unity with around 50 different levels, and a star scoring system after each level. The game is made for Android and IOS.

This project encompases all of the elements of a video game, including animations, music and sound effects, level editor, progression system, and puzzle mechanics. It was a year-long project developed by my team in Panza studio.

Dwars

2D - Multiplayer - Cross Platform - Prototype - Unity

Dwars is a cooperative multiplayer game. You and your friend control Handsome and Smarty, two brave souls appointed as crystal defenders against incoming evil blobs. This is a cross platform, real-time multiplayer game made for PC and Android done in Unity with Gamesparks as the server.

The game contains lovely scenery, different levels, music and sound effects, animations, enemy AI, multiple characters, destructible objects and power-ups. It was done as a project developed as a cross-platform multiplayer prototype, created by 1 programmer + 1 designer team in a 14-day event.

Battle of the Arena Masters

2D - Twitch Integration - Team Based - Multiplayer - Netcode - Unity

Battle of the Area Masters was a master dissertation project. It is a team based, competitive, multiplayer arcade game. The game was built in Unity, with GameSparks as the server, and with the integration with the Twitch platform. During the game your viewers on Twitch can bolster your strength in the battle by giving you power ups, but be aware that your opponent has his viewers on his side as well.

Select your character, combine skills and abilities, and coordinate with your teammates and organize your viewers on Twitch, to defeat the opposing team in a 1v1, 2v2, or 4v4 arena brawler style game. The main goal of the project was to incorporate competitive and cooperative aspects, with custom made netcode and the support for an infinite number of players through twitch.

DinosHur

3D - Twitch Integration - Shooting - Racing - AI - Unity

DinosHur is a space based, 3D racing game with shooting elements. After defeating his adversaries in the roman empire Ben Hur decided to travel into space to search for a different challenge. Races with dinosaur chariots, crazy map obstacles, and opponents prepared to crash into you just because they like chaos, and impress the audience (twitch viewers) with your driving skills in order to earn their help in rushing through the track.

This game was developed in Unity, as a 7 day project during the game development coursework. Dinosaurs, space and guns, audience interaction, custom 3D models, and a crazy atmosphere all around.

The Pit

3D - Party game - Procedural Generation - Hide and Seek - Unity

The Pit is an online party game where you compete with everyone else to find, and hold the treasure! In this 3D arcade game you can customize your character, dig through the procedurally generated pit to get the shiny, and then try to run away with it while bombs keep exploding everywhere. This game was created for Ludum Dare 48. Unity was the choice for the game engine, and Normcore was selected for the netcoding. All of the assets, music, and sound effects were custom created.

Jump & Squat

Motion Tracking - Computer Vision - Python - VR

Jump and Squat is a 3-day project done during a Game Jam. It is based on combining a popular dinosaur jumping game (The one you can play in chrome when you don't have internet connection) with motion tracking. We implemented commands to play the game through motion detection. When you jump or squat, the motion is detected and the movements are linked to associated action is the game, jumping and squating to avoid the obstacles. The motion tracking was done with computer vision framework MediaPipe, and the game was made by incorporating the framework with the game environment in python.

Retina Racer

Racing - Eye Tracking - Computer Vision - Audio - Python

Retina Racer is a prototype racing game in which you have to close your eyes to unpause the game. The challenge is to memorize the track layout, because if you stare too much you will run out of time! The game uses your camera to detect whether your eyes are open or closed via computer vision.

Since you are driving blind, you must navigate by sound alone. The game provides spatial audio cues - beeps that increase in frequency the closer you get to a wall - allowing you to steer and react without ever opening your eyes.

This project was created as a submission to IGGI Jam 2024. It combines a racing game built in Unity with a Python-based eye-tracking backend that communicates with the game in real time.

Play on itch.io

Evo Nano

Horizon 2020 - Bio-Medical Research - Evolutionary Simulation - C++

Esport is a large and still growing industry with vast audiences. Multiplayer Online Battle Arenas (MOBAs), a sub-genre of esports, possess a very complex environment, which often leads to experts missing important coverage while broadcasting live competitions. One common game event that holds significant importance for broadcasting is referred to as a team fight engagement.

My role in this project consisted of creating a fully functional framework for large scale tumor growth simulation. I used PhysiCell , an agent based simulator, as the basis of the framework, and updated it to allow for easy addition of multiple cell variations. The procedurally generated vascular system was also implemented in order to replicate the conditions of the cell cycle and the effects proximity of the cells to the oxygen supply.

As the result my work on the project, I published a paper on the effects of evolutionary properties of autonomous agents in competitive environments.
Tot M, Kupusinac A, Balaz, I (2019) Influence of Embodiment on Emergence of Subjective Data Models in Evolutionary Agents. Proceedings of the SED 2019, May 24-25.

Predicting teamfights through player camera

Player Analytics - Dota 2 - Team Fight - Neural Networks - Python

Esport is a large and still growing industry with vast audiences. Multiplayer Online Battle Arenas (MOBAs), a sub-genre of esports, possess a very complex environment, which often leads to experts missing important coverage while broadcasting live competitions. One common game event that holds significant importance for broadcasting is referred to as a team fight engagement. Professional player's own knowledge and understanding of the game may provide a solution to this problem. This paper suggests a model that predicts and detects ongoing team fights in a live scenario. This approach outlines a novel technique of deriving representations of a complex game environment by relying on player knowledge. This is done by analysing the positions of the in-game characters and their associated cameras, utilising this data to train a neural network. The proposed model is able to both assist in the production of live esport coverage as well as provide a live, expert-derived, analysis of the game without the need of relying on outside sources.

You can read the full publication here:
Marko Tot, Michelangelo Conserva, Alan Pedrassoli Chitayat, et al. What Are You Looking At? Team Fight Prediction Through Player Camera, in Proceedings of the IEEE Conference on Games (CoG) (2021)

Hey ref! The AI is cheating!

Game AI - Esports - Dota 2 - Starcraft 2

This talk was presented in the IGGI 2021 Conference. The talk provides insight into the most popular gaming competitions between the AI agents and the best human players, focusing on the AlphaStar and OpenAI Five. These two agents have been showcased during major gaming events and have beaten many professional players. Following the competition, many concerns have been raised throughout the gaming community about the validity of the AI agents. During the talk multiple concerns about the AI are presented and discussed, as well as the restrictions imposed on the game in order to make the playing field as fair as possible.

Monte Carlo Graph Search

Game AI - GridWorld - Planning - Graphs

Monte Carlo Tree Search (MCTS) has proven to be a staple method in Game Artificial Intelligence for creating agents that can perform well in complex environments without requiring domain-specific knowledge. The main downside of this planning based algorithm is the high computational budget needed to recommend an action. The fundamental cause of this is a vast search space caused by a high branching factor, and the difficulty to create a good heuristic function to guide the search without leveraging domain-specific knowledge. Recent advances in the field proposed a new planning based method called Monte Carlo Graph Search (MCGS), which uses a graph instead of a tree to plan its next action, reducing the branching factor and consequently increasing the performance of the search.

In this research we proposed several modifications that optimize the performance by increasing the sample efficiency of MCGS. The use of frontier for node selection, improving the rollout phase by doing stored rollouts, and a generalized approach to guide the search by incorporating a domain-independent online novelty detection method. Together these enhancements enable MCGS to solve sparse reward environments while using a significantly lower computational budget than MCTS.

This is a continuous research that resulted in following publications:
Marko Tot, Michelangelo Conserva, Sam Devlin and Diego Perez-Liebana, Making Something Out of Nothing: Monte Carlo Graph Search in Sparse Reward Environments, in AAAI-22 Workshop on Reinforcement Learning in Games (2022).

Marko Tot, Michelangelo Conserva, Sam Devlin and Diego Perez-Liebana, Turning Zeroes into Non-Zeroes: Sample Efficient Exploration with Monte Carlo Graph Search, in Proceedings of the IEEE Conference on Games (CoG) (2022).

World and Human Action Model (WHAM)

World Models - Generative AI - Gameplay Generation

WHAM is an autoregressive generative model trained to predict tokenized game visuals and controller actions given a prompt. It consists of a VQ-GAN encoder-decoder for discretizing game visuals and a transformer backbone for next-token prediction, both trained from scratch. The model supports three modes of operation: world modelling (generating visuals from controller actions), behaviour policy (predicting actions from visuals), and full generation (producing both visuals and actions simultaneously).

WHAM was trained on human gameplay data from Ninja Theory's Xbox game Bleeding Edge - a 3D, 4v4 multiplayer video game. The training dataset comprises approximately 500,000 games from all seven game maps (over 1 billion observation-action pairs at 10Hz, equivalent to over 7 years of continuous human gameplay). The resulting model generates consistent game sequences and shows evidence of capturing the 3D structure of the environment, the effects of controller actions, and the temporal structure of the game. WHAM is developed by the Game Intelligence group at Microsoft Research, in collaboration with TaiX and Ninja Theory. This work was done as part of an internship at Microsoft Research.

You can read the full publication and access the model here:
Anssi Kanervisto, ..., Marko Tot, ..., Katja Hofmann. World and Human Action Model (WHAM), Nature (2025).

Model and code on HuggingFace

Adapting a World Model for Trajectory Following in a 3D Game

World Models - Imitation Learning - Inverse Dynamics

Imitation learning is a powerful tool for training agents by leveraging expert knowledge, and being able to replicate a given trajectory is an integral part of it. In complex environments, like modern 3D video games, distribution shift and stochasticity necessitate robust approaches beyond simple action replay.

In this study, we apply Inverse Dynamics Models (IDM) with different encoders and policy heads to trajectory following in a modern 3D video game - Bleeding Edge. We compare a ConvNeXt encoder trained from scratch against a frozen DINOv2 encoder with a trainable MLP adapter, combined with both MLP-style and GPT-style policy heads.

We investigate several future alignment strategies that address the distribution shift caused by aleatoric uncertainty and imperfections of the agent. Our results show that in a diverse data setting, a GPT-style policy head with an encoder trained from scratch performs best; a DINOv2 encoder with GPT-style policy head excels in the low-data regime; and both policy head styles achieve comparable results when pre-trained on diverse data and fine-tuned for a specific behaviour. This work was done as part of an internship at Microsoft Research.

Marko Tot, Shu Ishida, Abdelhak Lemkhenter, et al. Adapting a World Model for Trajectory Following in a 3D Game, ICLR 2025 Workshop on World Models (2025).

Bootstrap Your Own Teacher

Multi-Game RL - Policy Distillation - Atari

Training generalist agents capable of performing well across diverse environments is a significant goal of reinforcement learning. Current state-of-the-art methods for multi-game RL rely on offline datasets, and often discard the policy used to gather trajectories despite its potential to provide a rich learning signal. In this paper, we revisit policy distillation for multi-game RL and introduce a new framework called Bootstrap Your Own Teacher (BYOT) that extends policy distillation to the online-RL setting.

BYOT alternates between two phases: (i) game-specific finetuning and (ii) distilling bootstrapped teachers back into a shared multi-game policy. By directly regulating the multi-game learning dynamics in policy space, BYOT balances training without explicit gradient adjustments or reward normalization, whilst being highly parameter efficient.

Our framework is empirically validated for both online and offline multi-game learning on the Atari-40 benchmark. BYOT outperforms all prior online Atari-40 multi-game agents, achieving an IQM human-normalized-score (HNS) of 152.7%. Moreover, by adopting state-of-the-art PPO teacher agents and policy distillation, we more than triple the leading IQM-HNS in offline settings to 369.5%, whilst using significantly fewer parameters. This work was done as part of an internship at InstaDeep.

Donal John Byrne, Marko Tot, Paul Duckworth, Clement Bonnet, Alexandre Laterre, Thomas D. Barrett. Bootstrap Your Own Teacher: Online Policy Distillation for Multi-Game Reinforcement Learning, in Proceedings of the IEEE Conference on Games (CoG) (2025).

Statistical Forward Planning Algorithms

Tutorial - Game AI - Planning - Bandit Algorithms - Simulation

Statistical Forward Planning (SFP) algorithms use a forward model to consider future possible scenarios, making decisions based on statistics of previous simulations. Starting with random actions and working towards favourable outcomes, these methods often use bandit algorithms to balance exploitation with exploration. SFP is an anytime algorithm, providing immediate intelligent behaviour given a forward model.

This tutorial was presented at the Genetic and Evolutionary Computation Conference (GECCO) 2025. It covers core SFP methods including Monte Carlo Tree Search (MCTS), Rolling Horizon Evolutionary Algorithms (RHEA), and their variants, as well as recent advances in combining planning with learned models.

Simon M. Lucas, James Goodman, Marko Tot. Statistical Forward Planning Algorithms, in Proceedings of the Genetic and Evolutionary Computation Conference (GECCO) (2025).