Skip to content

Commit

Permalink
README: added videos, performance data and papers section
Browse files Browse the repository at this point in the history
  • Loading branch information
hazzah3 authored Nov 21, 2024
1 parent 92357ae commit 1680853
Showing 1 changed file with 28 additions and 3 deletions.
31 changes: 28 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,12 +14,15 @@ Contributions are very welcome!

- [DIAMOND](#diamond)
- [Minecraft](#minecraft)
- [Matrix](#matrix)
- [Other](#other)
- [Papers Please](#papers-please)

## DIAMOND
[![Homepage](https://img.shields.io/badge/homepage-gray)](https://diamond-wm.github.io/)
[![arXiv](https://img.shields.io/badge/arXiv-red)](https://arxiv.org/abs/2405.12399)
[![Discord](https://img.shields.io/badge/discord-purple)](https://discord.gg/74vha5RWPg)
[![Paper](https://img.shields.io/badge/paper-pink)](https://arxiv.org/pdf/2405.12399)

Open-source training and inference, initially launched with 26 atari games and CS:GO.

Expand All @@ -33,6 +36,7 @@ Open-source training and inference, initially launched with 26 atari games and C

### CS:GO
![??? FPS on RTX 4090](https://img.shields.io/badge/RTX%204090-%3F%3F%3F%20FPS-green)
![10 FPS on RTX 3090](https://img.shields.io/badge/RTX%203090-10%20FPS-cefad0)
[![Open Source](https://img.shields.io/badge/open%20source-blue)](https://github.com/eloialonso/diamond/tree/csgo)

<img src="https://github.com/user-attachments/assets/dcbdd523-ca22-46a9-bb7d-bcc52080fe00" width="500"/>
Expand All @@ -55,6 +59,8 @@ Open-source training and inference, initially launched with 26 atari games and C

One size of the model and inference code are open-source.

<img src="https://github.com/user-attachments/assets/e08a2fdd-232d-4b9c-8f1c-8f9b9f1babe7" width="500"/>

### Oasis
![<1 FPS on RTX 4090](https://img.shields.io/badge/RTX%204090-%3C1%20FPS-green)
[![Open Source](https://img.shields.io/badge/open%20source-blue)](https://github.com/etched-ai/open-oasis)
Expand All @@ -64,22 +70,41 @@ One size of the model and inference code are open-source.

A 500M model and inference code are open-source.


<img src="https://cdn.prod.website-files.com/671a36395e56dbd229da0e0e/6723b0a589d2f810325afdcb_4.webp" width="500"/>

## Matrix
![??? FPS on RTX 4090](https://img.shields.io/badge/RTX%204090-%3F%3F%3F%20FPS-green)
[![Open Source Coming Soon](https://img.shields.io/badge/(coming_soon)-open_source-blue)](#Matrix)
[![Homepage](https://img.shields.io/badge/homepage-gray)](https://thematrix1999.github.io/)
[![Paper](https://img.shields.io/badge/paper-pink)](https://thematrix1999.github.io/article/the_matrix.pdf)

2.7B model trained on data from Forza Horizon 5 and Cyberpunk 2077 as well as real-world footage. Open source and demo coming soon.

<img src="https://github.com/user-attachments/assets/c3fbae6d-1393-4fd9-97d8-db61bed139e0" width="500"/>

## Other

### Doom (GameNGen)
![??? FPS on RTX 4090](https://img.shields.io/badge/RTX%204090-%3F%3F%3F%20FPS-green)
[![Homepage](https://img.shields.io/badge/homepage-gray)](https://gamengen.github.io/)
[![arXiv](https://img.shields.io/badge/arXiv-red)](https://arxiv.org/abs/2408.14837)
[![Paper](https://img.shields.io/badge/paper-pink)](https://arxiv.org/pdf/2408.14837)

It runs Doom! The grandfather paper of the field.

<video src="https://gamengen.github.io/static/videos/e1m3.mp4" controls></video>

[![Watch the video](https://img.youtube.com/vi/O3616ZFGpqw/0.jpg)](https://www.youtube.com/watch?v=O3616ZFGpqw)

### Pokemon Overworld
![??? FPS on RTX 4090](https://img.shields.io/badge/RTX%204090-%3F%3F%3F%20FPS-green)
[![Demo](https://img.shields.io/badge/demo-yellow)](https://madebyoll.in/posts/game_emulation_via_dnn/demo)
[![Blog](https://img.shields.io/badge/blog-orange)](https://madebyoll.in/posts/game_emulation_via_dnn/)

An early model trained on a 4th Generation Pokemon game.

![Pokemon](https://madebyoll.in/posts/game_emulation_via_dnn/neural_notwork_3.gif)

## Papers Please

- [Diffusion for World Modeling: Visual Details Matter in Atari](https://arxiv.org/pdf/2405.12399) - Diamond Paper (2024)
- [Diffusion Models are Real-time Game Engines](https://arxiv.org/pdf/2408.14837) - GameNGen Paper (2024)
- [The Matrix: Infinite-Horizon World Generation with Real-Time Moving Control](https://thematrix1999.github.io/article/the_matrix.pdf) - Matrix Paper (2024)

0 comments on commit 1680853

Please sign in to comment.