Raw code is here:
https://drive.google.com/open?id=1tS6fdXBzrlsgbnye8lo9P4VoOuKqfaZJOkay, I bought one of these tiny 0.96 inch OLED matrices/displays. Ordered it from eBay for a few dollars. It uses SPI via an SSD1306 controller chip. I had to add a Micro SD card to the ESP32 to store the video data (about 6MB for 4 minutes).
Each frame requires sending 1024 bytes to the OLED controller. This is the 8192 bits that the OLED needs for a 128x64 display. So I used OpenCV Python to convert the video frames to BW and resize to 128x64, then I arranged the pixel states into bits and bytes that could be sent directly to the OLED controller, then I saved them into a single file on the SD card.
The ESP32 just has to read 1024 bytes at a time from the file, and send it to the SPI port. I could do that at 100 frames per second (too fast). This is slowed to about 24 FPS, which is the same as the original video.
I did use the IOMUX pins for the SPI connections, both for the SD card and the matrix, so I could get the full 80Mhz speed.