Pokemon Online Advanced Beta 1.0 — complete teardown
A complete static teardown of Pokemon Online Advanced Beta 1.0 (Konidias, September 2000): the Clickteam PAPP container, all 191 images, the object and instance layout, the entire event logic including a second event table that drives the menu, the four screens the game cannot reach, and the extraction toolchain.
From the desktop window
pol-beta-readme.txt — Pokemon Online Advanced Beta 1.0 teardown
# Pokemon Online Advanced, Beta 1.0 -- complete teardown
A full static analysis of **Pokemon Online Advanced Beta 1.0**, a Multimedia
Fusion 1.x fan game from the Pokemon Online (POL) scene that orbited
bulbagarden.com around 2000-2002. It ships as three files: `pol.exe`
(605756 bytes, the runtime), `pol.gam` (594404 bytes, the game) and
`cncs32.dll` (171520 bytes).
Everything below was recovered from those bytes. The game was never run, no
debugger was attached, and nothing here was derived from a screenshot. Screen
captures are used once, at the very end, only to check the result -- and the
reconstruction turns out to be pixel-identical to them.
The container is Clickteam's flat **`PAPP`** format, which is not the chunked
`PAME` format that CTFAK and mmfparser understand. All 102 event
groups are decoded, so what the game does is read out of the file rather than
inferred from watching it run.
**Section 10 explains how, and section 11 is the complete toolchain**, so you
can do this to any Games Factory or MMF 1.x application yourself.
---
## Contents
1. [What it is](#1-what-it-is)
2. [Provenance and identification](#2-provenance-and-identification)
3. [How it behaves](#3-how-it-behaves)
4. [Data inventory](#4-data-inventory)
5. [The program logic](#5-the-program-logic)
6. [Hidden and never-visible content](#6-hidden-and-never-visible-content)
7. [Bugs and loose ends](#7-bugs-and-loose-ends)
8. [The servers and the dead links](#8-the-servers-and-the-dead-links)
9. [Credits and rights](#9-credits-and-rights)
10. [How to reproduce this](#10-how-to-reproduce-this)
11. [The toolchain](#11-the-toolchain)
12. [About this teardown](#12-about-this-teardown)
---
## 1. What it is
A single-player walk-around demo of a Pokemon MMO that never shipped.
You get a title screen, one town, one house, and a person in the house who
asks you to turn his radio on. That is the whole game. What makes it worth
reading is everything around that: the file also contains a trainer ID card,
a "Pokemon Pocket Organizer" menu screen, a DirectPlay multiplayer lobby with
Host and Join buttons, and a networking object wired to broadcast your
position to other players -- none of which you can reach, because nothing in
the program jumps to those screens.
Beta 1.0 is a demo of the parts that worked, cut out of a larger project whose
other half is still sitting in the file.
The game's own summary of itself, from the readme screen, verbatim:
```
Pokemon Online Beta 1.0
This is a small upgrade to the alpha version. Things you can now do are:
-Walk in and out of a building
-Talk to a cpu player
-Use the tv
-Use the radio
-Walk faster
-View the credits in the menu and visit the sites
Controls:
Arrow keys move character
Shift- Read signs, talk to people, activate things
Space- Close any speech boxes that are open
```
"Read signs" is not true -- see section 7.
---
## 2. Provenance and identification
| | |
|---|---|
| Application name | `Pokemon Online Advanced` |
| Author | `Konidias` |
| Authoring tool | Clickteam Multimedia Fusion 1.x / The Games Factory (`PAPP` container, version 0x0207) |
| Runtime | `pol.exe`, the MMF stand-alone stub, PE timestamp **2000-02-08 12:42:20 UTC** |
| Support library | `cncs32.dll`, PE timestamp **1997-03-03 12:15:34 UTC** |
| Linker | 4.20, GUI subsystem |
| Overlay in `pol.exe` | 425532 bytes -- six extension DLLs, no game data |
### Hashes
```
pol.gam 594404 sha256 4884c2cdff9ffd1569ff046e4f2efe6a70567f5201d539e57241bdc949844957
md5 f32f71c4192f161e9f4ee82d3564eef1
pol.exe 605756 sha256 f592c25570890ffb33c9acff0b1d4a6c9974b9e0128fa36a15a06b0eab174b36
md5 d6789b9fc9433608a9096d33a8201dc6
cncs32.dll 171520 sha256 f048f89e8c5f3f8d926df63e75273db70d100071ee2f002e25e3cd760b5ba2b7
md5 653a4e64dfd35d8c11bd28cfbe0711a9
```
The distributed file date is 17 September 2000. (The copy this was run
against has had its timestamps reset by copying, so that date comes from the
distribution rather than from these bytes; the PE timestamp inside `pol.exe`,
2000-02-08 12:42:20 UTC, is the runtime's build date, not the game's.)
`cncs32.dll` was built in 1997 and `pol.exe` in February 2000, which places the
authoring tool: this is the Click&Create-era engine, one generation before the
MMF 1.5 that later POL-scene applications were built with.
The runtime knows what it is. Two of its own error strings:
```
This game has been created with an old version of The Games Factory.
Please load and save it using the current version.
This game has been modified since it was saved as a stand-alone game. Edit the original game and then save a new stand-alone game.
```
### Where it was distributed
The one thing in this document not derived from the bytes is where the file
came from, and for that there is a primary source: the Wayback Machine holds
pokemonol.com's own POL:A download page as it stood on **9 November 2000**,
seven weeks after the build date.
<https://web.archive.org/web/20001109143300/http://www.pokemonol.com/polrpg/downloads.html>
The page offered five files. None of them was saved by the crawler -- only
the page listing them -- but the listing itself is informative:
| link text on the page | file | size given |
|---|---|---|
| New POL Advanced Beta Version 1.0 | `polb1.zip` | 541 kb |
| (OLD) POL Advanced Alpha Version 1.0 | `polbeta.zip` | 319 kb |
| (OLD) Extreme Beta version 1.0 | `exbeta.zip` | 883 kb |
| CNCS32.dll (if needed when starting pol) | `cncs32.dll` | 167 kb |
| Character Creator Dev Kit | `charcreator.zip` | 8 kb |
Two of those can be checked against what is in hand. The preserved archive
is `polb1.zip` -- the same filename -- at **554,117 bytes, which is 541.1 KB**,
against the 541 kb the page claimed. `cncs32.dll` is **171,520 bytes, 167.5
KB**, against 167 kb. Both agree to the rounding, which is consistent with
these being the files that page was serving.
Three details worth keeping from it. `cncs32.dll` was a **separate download**,
offered "if needed when starting pol" -- this game shipped as a runtime plus a
data file plus a support library the player might have to fetch by hand, which
is why the DLL sits beside the game rather than inside it. The **alpha** was
still up as `polbeta.zip`, and it is the build whose leftovers are all over
this one -- the unused "Welcome to Pokemon Online Advanced Alpha City" text in
section 6, and the Pocket Organizer the alpha's own text says you open with
Enter or Ctrl. And the page carried an 8 kb **Character Creator Dev Kit**,
about which nothing further is known: 8 kb is too small to be a tool, none of
the file survives, and nothing in this game refers to it.
That page has been captured four times between 2000 and 2013, so the state of
the download area is recoverable across the project's whole life; this teardown
uses only the first. One other page from the same site is used, in section 8:
the POL:A information page of 16 December 2000, which published the planned
feature list.
### Where the game actually lives
`pol.exe` is just the runtime. Its PE overlay is 425532 bytes and holds
the extension pack -- six `.GOX` DLLs, stored as plain uncompressed PE images
with a list of filenames in front of them. There is no game data in the
executable at all.
The game is `pol.gam`, and its magic is `PAPP`. That is the older flat
container: one absolute offset table for the frames, and a repeating
self-describing bank structure for the media. There are no `id/flag/size`
chunk headers anywhere in the file, which is why tools written for the chunked
`PAME` format cannot open it -- they are not failing to parse it, they are
looking for a structure that is not there.
---
## 3. How it behaves
### The menu
The application menu is stored in `pol.gam` as a Windows-style nested item
list with UTF-16 labels, so it can be read without running anything:
```
&Main
&Restart id 1010 [F2]
---
&Pause id 1012
---
&Exit Game id 1009
&Options
Play &sounds id 1020 [Ctrl+S]
Play &music id 1021 [Ctrl+M]
---
&Hide menu id 1022 [F8]
---
&Full Screen id 1025
&Other
Help Guides
Game Guide id 2005
Site Guide id 2006
Character Editor
New Character id 2008
Character Archive id 2009
---
Credits and Sites
Game Credits id 2011
---
Visit BulbaGarden.com id 2012
Visit Pokemonol.com id 2013
Visit Gamedn.com id 2014
---
Trainer Pages
Create/Edit Your Page id 2017
View Trainer Archive id 2019
```
`Main` and `Options` are the stock runtime menu; `Other` is entirely the
author's. Two things stand out.
**`&Full Screen`, id 1025, is in the file but not on screen.** It sits in the
Options menu between `Hide menu` and the end. Asking Windows for the running
program's `HMENU` returns an Options menu of three items, without it, so the
runtime strips it at load time.
**The accelerator table**, stored immediately after the menu, is four entries:
```
Ctrl+S -> menu id 1020
Ctrl+M -> menu id 1021
F2 -> menu id 1010
F8 -> menu id 1022
```
`Full Screen` has no accelerator, which is consistent with an item the runtime
never intends to show.
### The menu the author started from
`pol.exe` carries the stock runtime menu as resource MENU 350, and it is the
template this one was edited out of:
```
&Game &New F2 | Pass&word | &Pause Ctrl+P | Pla&yers Ctrl+Y | &Quit Alt+F4
&Options Play &samples Ctrl+S | Play &musics Ctrl+M |
&Hide the menu F8 | &Full Screen Alt+Enter
&Help &How to use... F1 | &About...
```
Compare it with the game's: `&Game` became `&Main`, `&New` became `&Restart`,
`Pass&word` and `Pla&yers` were deleted, `&Quit` became `&Exit Game`, `&Help`
was replaced wholesale by `&Other`, and the Options menu was kept as-is. That
is also where the orphan `&Full Screen` comes from -- it was inherited from
this template, which gives it Alt+Enter, and the copy in the game lost the
accelerator on the way.
### Seven dialogs the game can never show
The runtime brings its own dialogs too, and the game exposes almost none of
them. With `Pass&word` and `Pla&yers` deleted from the menu there is nothing
left to open half of these:
| id | title | what is in it |
|---|---|---|
| 700 | Game paused | Continue |
| 701 | Keyboard setup | Down, Left, Right, Button 1, Button 2, "Press a key... or ESC to cancel." |
| 703 | Enter password | Cancel |
| 704 | Players controls | Joystick 1, Joystick 2, Keyboard, Default Controls |
| 705 | About | Copyright |
| 706 | Warning | "This game was created in a True / High color mode, running it in 256 color mode may result in slower loading times." |
| 707 | Warning | True / High color mode (Recommended) vs 256 color mode (Expect longer loading times) |
Those last two date the thing as neatly as anything in the file: in 2000 it
was still worth asking the player which color depth to run in.
### The frames
```
| # | name | size | objects | instances | events | reachable |
|---|-------------|----------|---------|-----------|--------|-----------|
| 0 | Opening | 500x400 | 4 | 4 | 7 | yes |
| 1 | Beta City 1 | 1024x800 | 32 | 985 | 38 | yes |
| 2 | House 1 | 384x256 | 33 | 72 | 39 | yes |
| 3 | ID Card | 500x400 | 23 | 21 | 1 | NO |
| 4 | Menu | 500x400 | 12 | 12 | 3 | NO |
| 5 | Untitled | 500x400 | 4 | 4 | 5 | NO |
| 6 | Untitled | 500x400 | 3 | 3 | 3 | NO |
| 7 | Untitled | 500x400 | 4 | 4 | 1 | yes |
```
Only four of the eight can be reached. That is not a guess -- it comes from
walking every "next frame", "previous frame" and "jump to frame" action in the
event tables and seeing which frames the graph reaches from frame 0:
```
Opening --(click START GAME / Enter / Space / Shift / Ctrl)--> Beta City 1
Opening --(click ABOUT)--------------------------------------> readme (frame 7)
Opening --(click QUIT)---------------------------------------> end application
Beta City 1 --(walk UP into door1)---------------------------> House 1
House 1 --(walk DOWN onto the mat)---------------------------> Beta City 1
readme --(Enter)---------------------------------------------> Opening
```
`ID Card`, `Menu` (the Pocket Organizer) and the two multiplayer frames are
never jumped to by anything.
A jump action's parameter is not a frame index; it is an index into the
application's frame-handle table, which for this game is
```
handle: 0 1 2 3 4 5 6 7
frame: 3, 4, 1, 5, 6, 7, 2, 0
```
so "jump to handle 6" means House 1. Reading that parameter as a frame number
gives a plausible-looking wrong answer, which is the sort of mistake that
survives review.
### The title screen
Three click zones, exact:
| zone | action |
|---|---|
| (259,151)-(447,172) | START GAME -- next frame |
| (260,187)-(448,206) | ABOUT -- jump to the readme frame |
| (279,222)-(435,240) | QUIT -- end application |
Enter, Space, Shift and Control all also start the game. Five separate event
groups, one per key, each doing "next frame".
---
## 4. Data inventory
### 4.1 The container, end to end
```
0x000 char[4] "PAPP"
0x004 u16 version
0x006 char[80] application name
0x056 char[80] author
0x0F8 u16 u16 window width, height 500 x 400
0x100 u32 application flags
0x110 u16[4][6] control keys, four players up down left right fire1 fire2
0x15C u32 frame count
0x166 u32[n] frame offsets, absolute
u16 u16 frame-handle count, then handle -> frame index
0x19E u32 + 832 the application icon
... application-level object and event tables
... extension table, 276-byte records
... menu (UTF-16), then the accelerator table
... the frames
... image bank, font bank, music bank
```
Every bank has the same shape, which is what makes them findable without
guessing at offsets:
```
u32 totalSize includes this field
u32 count
(u32 offset, u32 size) * count offsets relative to the count field,
contiguous except for deleted slots,
offset[0] == 4 + count*8
```
The file is fully accounted for. Header, eight frames and three banks cover
every byte except two 2-byte gaps immediately before the font and music banks,
which look like a bank-kind tag (0x0A images, 0x0B fonts, 0x0C music).
### 4.2 Images -- 191 of them
191 images decode, 1725249 pixels of art in total: 185
run-length compressed, 6 stored raw. Three bank slots
(170, 171, 176) are dead -- their table entries point back into the bank's own
offset table and their graphic mode is 0. They are images the editor deleted
and never compacted, and there is nothing in them to recover.
The image codec is the part of `PAPP` that takes real work, and it is
described byte for byte in section 10.2. The short version: the transparency
mask and the color data are stored as two separate planes, the color plane
holds only the opaque pixels, and when it is compressed it is **one run-length
stream that runs straight through row boundaries**. Decoding it per row --
which is what the row-offset table invites you to do -- gives images that are
almost right, which is worse than images that are obviously wrong.
Five invariants are checked before any pixel is believed, and all five hold
for all 191 images. They are listed in section 10.2.
Two other handles are worth naming, because the object names do not:
image **190** is Caveman -- a 32x32 brown-haired sprite in a gray shirt, the
only NPC in the game, placed as in House 1 -- and image
**192** is the 64x32 red bar that is , the invisible mat by
the door that sends you back outside.
Three handles are worth naming, because the object names do not. Image **3**
is the trainer card blank -- 380x230, a cream license with a rounded border,
**OFFICIAL POL LICENSE** across it and a printed barcode along the bottom.
Image **190** is Caveman, a 32x32 brown-haired sprite in a gray shirt, the
only NPC in the game. Image **192** is the 64x32 red bar that is House 1's
`Active object 2` -- the invisible mat by the door that sends you back
outside.
### 4.3 The application icon
The `.gam` carries its own 32x32 and 16x16 icon in a fixed 832-byte block with
no palette and no DIB header -- 4bpp icons of this era used the fixed Windows
16-color table, so only the indices need storing. It decodes to the stock
Games Factory **"gf"** badge. There is no custom Pokemon icon anywhere in
this application; the blue badge in the title bar of every screenshot is the
authoring tool's, not the author's.
### 4.4 Objects and instances
115 objects across the eight frames, placed 1105 times.
The object record is 68 bytes of header followed by an optional data block:
```
+0 char[42] name fixed field, NOT cleared on rename
+42 u8 objectType 0 quick backdrop, 1 backdrop, 2 active, 3 text,
7 counter, 32..37 extension, 0xFF deleted stub
+43 u8 obstacle backdrops: MMF's Obstacle property
+44 u16 instances how many times this object is placed
+46 u16 handle
+54 u32 image THE IMAGE BANK HANDLE
+64 u32 objectID application-wide, strictly increasing
+68 optional (u16 0x0000, u32 size, data) type-specific block
optional (u16 0x8002, u32 0) end marker
```
Two of those fields are worth calling out.
**`+54` is the image bank handle, directly.** The instance record also carries
an image reference, at its offset 8, but that one is numbered per frame rather
than per bank and does not decode to anything useful -- Beta City's reference
149 is the water tile and House 1's reference 149 is a plank wall. It can be
left alone. Every object's artwork is available without it, from this field
for backdrops and from the animation table for Actives.
**`+44` is the instance count**, and it is the strongest self-check in the
format: across all eight frames it equals the number of instances the instance
list actually places, for every object but one. Beta City's `Grass` says 530
and there are 530; `Small House` says 4 and there are 4.
The exception is worth recording rather than rounding away. In frame 7, the
readme screen, the `PowerApp Lite` object declares **0** instances and the
instance list places **1**, parked off-frame at (-29,434). Every other object
in the application agrees with its own count, so this is one stale field --
most likely the object was pasted in and the count never updated -- rather
than a misreading of the format. Treat the field as a check, not as a source
of truth: the instance list is the truth.
The name field is not cleared when an object is renamed, so shorter new names
leave a tail of the old one behind: House 1's `tv` is stored as
`tv\0kdrop object 4`, and frame 4, now called `Menu`, still has `led` sitting
after its terminator from when it was `Untitled`.
The instance list is self-validating in a way that removes all guesswork about
where it starts:
```
u16 version
u32 blockSize
u16 slotCount
u32 slot[slotCount] offset of a live record, or 0xFFFFFFFF for a slot
whose instance was deleted while editing
u8 record[live][32]
blockSize - 2 - slotCount*4 == live*32 exactly, no remainder
the live slot values are a permutation of 0, 32, 64, ...
```
The record array order is the draw order, back to front. The slot table is
the editor's edit-order index; rendering by it is visibly wrong.
### 4.5 Fonts, and a typeface you almost certainly do not have
The font bank holds 18 records of 104 bytes. Each one is a 14-byte header
followed by a LOGFONT whose numeric fields are 16-bit, the face name at +32
and a style name at +64. Reading the LOGFONT from the start of the record --
the obvious thing to do -- puts every field 14 bytes early and makes all 18
fonts look blank.
They are not blank:
| face | records |
|---|---|
| Tahoma | 9 |
| **Pokemon** | 6 |
| Verdana | 1 |
| (uninitialized) | 2 |
Six of them ask for a typeface named `Pokemon` -- stored with an accented e,
as the bytes `50 6f 6b e9 6d 6f 6e`. That is the fan-made Pokemon TTF that
circulated on fan sites at the time, and it is not a font any Windows machine
ships with. Unless the player had installed it, six of this game's text
objects have always fallen back to whatever Windows substituted, and the
typography the author designed has never been seen as intended by anyone who
did not already have the font.
The two uninitialized records are the more interesting ones -- see 4.8.
### 4.6 Audio
**There are no sound effects.** There is no sound bank in the file, and no
`RIFF`, `OggS` or bare PCM anywhere in `pol.gam`, `pol.exe` or `cncs32.dll`.
The Options menu's "Play sounds" and its Ctrl+S accelerator are the stock
runtime menu, present whether or not an application ships samples.
There is exactly one piece of audio, a MIDI in the music bank:
```
technosong.mid 29333 bytes, SMF format 1, 12 tracks, division 120, 133.0 BPM
```
Its embedded text:
```
Copyright (C) 1995 Kevin Newman
s9510929@babel.ee.up.ac.za
Verse 1
Chorus
Verse 2
Chorus
Bridge
Chorus
```
Track names: Saw Wave, Saw Wave, Syn. Calliope, Organ 1, Choir Aahs, Synth Bass 2, Overdrive Gt, Reverse Cym., Electronic Set, Electronic Set, Electronic Set.
**The music is not the game's.** It is a 1995 amateur MIDI that the game
shipped with, and its sequencer's name and university email address are still
in it. That is a different rights position from the rest of the file and it
is why section 9 lists it separately.
It is also not a soundtrack. It is the radio's -- see section 5.
### 4.7 The six extension DLLs
They are stored uncompressed in `pol.exe`'s overlay, one after another, and
come out with a byte copy:
```
| file | bytes | linked | cnd | act | exp |
|--------------|--------|------------|-----|-----|-----|
| DPlay.GOX | 93450 | 2000-01-11 | 0 | 0 | 0 |
| GFEDIT.GOX | 46858 | 2000-02-02 | 7 | 31 | 7 |
| GFBUTTON.GOX | 49930 | 2000-02-02 | 4 | 14 | 5 |
| powerapp.GOX | 102666 | 2000-03-11 | 1 | 77 | 23 |
| GFINI.GOX | 49930 | 2000-02-02 | 0 | 7 | 2 |
| moo2.GOX | 82432 | 2000-01-25 | 10 | 26 | 22 |
```
Each one carries a version resource, and between them they name everybody who
worked on the parts of this game the author did not write:
```
DPlay.GOX SMMP Informatique Copyright (C) 1999-2000 Michel Plante
GFEDIT.GOX Europress Software Copyright (C) 1996 Europress Software
GFBUTTON.GOX Europress Software Copyright (C) 1996 Europress Software
powerapp.GOX Core Media Aust. Copyright (C) 2000 Core Media Australia www.core.com.au
GFINI.GOX Europress Software Copyright (C) 1996 Europress Software
moo2.GOX Sean Poling, spoling@3ee.com Copyright (C) 1998-2000 Third Eye Entertainment
```
Three things fall out of that table. `moo2` -- the networking object the
`Communication` event groups are built on -- is **Sean Poling** of Third Eye
Entertainment, with his address `spoling@3ee.com` still in the resource.
`DPlay` is **Michel Plante** of SMMP Informatique, with a GeoCities-era home
page at `pages.infinit.net/smmp`. `powerapp`, the object that puts up the
Game Credits box, is **Core Media Australia**. The three Europress objects
are 1996 and shipped with the tool.
The original filenames are worth a second look: `DPLAY.COX`, `MOO2.CCX`,
`POWERAPP.CCX`, `GFEDIT.GFX`. The same DLLs were shipped under `.cox` for
Multimedia Fusion, `.ccx` for Click & Create and `.gfx`/`.gox` for The Games
Factory; only the extension changes. An extension written for one of those
tools worked in all of them.
These matter for more than completeness. An extension's conditions, actions
and expressions are stored in the `.gam` as bare numbers; the names live in
the DLL's own string resources, at 28000+n, 28200+n and 28400+n. Pulling
those out is the difference between
```
DO powerapp.GOX act#68 "http://www.pokemonol.com"
```
and
```
DO Set About Website URL To ? "http://www.pokemonol.com"
```
`DPlay.GOX` is the exception: it carries no ACE strings at all, so the
DirectPlay events in section 5 keep their numbers. Its menus lived in the MMF
editor rather than in the DLL.
`powerapp.GOX` is worth a look on its own. It is a general-purpose Windows
automation object with 77 actions, most of them "open this control panel
applet" -- Display-Background, Joystick Properties, Modems Properties, the
Install New Hardware wizard. It can also set the desktop wallpaper, change
the Internet Explorer toolbar skin and flash the application's title bar. This
game uses exactly two of its capabilities: opening a URL, and its built-in
About box.
### 4.8 What leaked out of the editor's memory
Several structures in this format are written straight out of buffers the
editor never fully initialized, so whatever happened to be in memory at save
time went into the file with them. The font bank's two blank records are
like this, and so is the tail of every global-object record in section 6.
That garbage is readable, and it dates the machine. **96 dwords in the file
fall in the range 0xBFF00000-0xBFFFFFFF, and there are only 11 distinct
values**, repeated over and over:
```
0xbff562ce seen 10 times first at 0x089805
0xbff713e2 seen 2 times first at 0x089e77
0xbff741ed seen 1 times first at 0x089e7f
0xbff7b457 seen 13 times first at 0x000952
0xbff7b4c6 seen 13 times first at 0x00095a
0xbff80010 seen 1 times first at 0x045e51
0xbff814b0 seen 13 times first at 0x000966
0xbff814f7 seen 13 times first at 0x00096a
0xbffc9490 seen 12 times first at 0x00094a
0xbfff0087 seen 9 times first at 0x00172a
0xbfff00e7 seen 9 times first at 0x001572
```
On Windows 95 and 98 the shared DLL arena runs from 0x80000000 to 0xBFFFFFFF
and KERNEL32.DLL loads at about 0xBFF70000, so `0xBFF7B457` is KERNEL32 plus
0xB457 and `0xBFF814B0` is KERNEL32 plus 0x114B0. These are return addresses
and function pointers from inside the operating system, captured mid-save.
They could not appear in a file written on Windows NT, 2000 or XP, where the
system DLLs sit near 0x77000000 and nothing is mapped at 0xBFFxxxxx at all.
**`pol.gam` was saved on a Windows 9x machine**, and the same five-pointer
stack frame -- 0xBFFC9490, 0xBFF7B457, 0xBFF7B4C6, 0xBFF814B0, 0xBFF814F7 --
was written out thirteen separate times, once per structure that had that
particular piece of stack still lying in its buffer.
---
## 5. The program logic
This is the part that had not been read before.
97 event groups across the eight frames -- 166 conditions and
177 actions -- plus 5 more in a second, application-level
table hidden in the header. Every group, condition, action and parameter
consumes exactly its declared size, which is the check that the decode is
right rather than merely plausible.
### 5.1 Movement, and how collision actually works
The player is two objects. `Solid` is an invisible block that does the moving
and the colliding; `Player` is the sprite, and every cycle it is simply set to
`Solid`'s position. The camera centers on `Solid`, not on `Player`.
Movement is four mutually exclusive groups, one per arrow key, each of the
form "while Down is held and Up and Left and Right are not":
```
IF KeyDown key Down
IF NOT KeyDown key Up
IF NOT KeyDown key Left
IF NOT KeyDown key Right
DO Set direction <Solid> direction down
DO Set Y position <Solid> obj18.YPosition + 3
DO Set direction <Player> direction down
DO Force animation <Player> animation 1 "Walking"
```
**Three pixels per cycle.** At the engine's default 50 frames per second that
is 150 pixels a second, which is what the character does on screen.
Collision is then four more groups that simply undo the move:
```
IF Is facing in direction <Solid> direction down
IF Is overlapping a backdrop <Solid>
DO Set Y position <Solid> obj18.YPosition - 3
```
That is worth stating plainly, because it changes how the map should be read:
**collision is MMF's per-pixel backdrop obstacle test, not a tile grid.** The
obstacle flag on each backdrop object decides which artwork participates, and
the test itself is against the sprite's actual mask. The obstacle maps this
teardown renders (`extract/render/*-obstacles.png`) are the object footprints,
which is exact for the tile art that makes up nearly all of the map and
slightly generous for the irregular sprites.
A fifth group covers the idle case: if no arrow key is down, force animation 0,
which the file names `"Stopped"`.
### 5.2 The radio
The radio is the one thing the game asks you to do, and it is two event groups
with the trigger box spelled out to the pixel:
```
[2.36]
IF Number of objects in zone <Player> zone (288,47)-(306,69), 1
IF KeyPressed key Shift
IF Is facing in direction <Player> direction up
DO PlayMusic
DO Set alterable value <Player> 1
[2.37]
IF Number of objects in zone <Player> zone (288,47)-(306,69), 1
IF KeyPressed key Space
IF Is facing in direction <Player> direction up
DO StopMusic
DO Set alterable value <Player> 0
```
Stand in that 18x22 pixel box, face up, Shift to start the music and Space to
stop it. The value it sets is what Caveman's dialogue reads back.
### 5.3 Caveman, the TV, and one text object doing three jobs
There is only one speech text object in House 1, and it holds three
paragraphs. Which one is displayed depends on where you are standing and on
the radio flag:
| zone | key | radio flag | paragraph shown |
|---|---|---|---|
| (254,47)-(275,65) | Shift | 0 | 0 -- Caveman asks you to turn the radio on |
| (254,47)-(275,65) | Shift | 1 | 2 -- "Cool music!! I'm moving and grooving!" |
| (219,39)-(240,67) | Shift | any | 1 -- the TV's "News Flash!" |
| anywhere | Space | -- | erased, and the panel is parked at (-129,-196) |
The speech panel itself is positioned with
```
DO Set X position <Active object 3> XLeftFrame
DO Set Y position <Active object 3> YTopFrame
DO Display paragraph <text> paragraph N at <Active object 3>+(14,12)
```
so the panel is pinned to the top-left of the visible frame, and the text is
inset 14 across and 12 down from it. House 1 is 384x256 inside a 500x400
window and MMF letterboxes it, so the frame origin sits at
((500-384)/2, (400-256)/2) = **(58,72)**, and that is where the speech box
lands on screen.
### 5.4 The doors
```
Beta City -> House 1: Solid facing UP, Player overlapping <door1>
House 1 -> Beta City: Solid facing DOWN, overlapping <Active object 2>
```
`Active object 2` is the invisible mat by the door. Both are direction-gated,
so brushing past sideways does not trigger them.
### 5.5 Where you come back to
Beta City has a "Positions" event group that decides the start position from
global value 16:
```
IF CompareGlobalValue 16, 1 DO Set position <Solid> at (290,150)
IF CompareGlobalValue 16, 0 DO Set position <Solid> at (224,192)
```
Global 16 is set to 1 on House 1's start-of-frame and never set back, so the
first entry to the town starts you at **(224,192)** and every later one puts
you outside the front door at (290,150). Both are literals in the file.
### 5.6 The multiplayer that is still wired up
Beta City and House 1 both open with an event group the author named
`Communication`, and it is a complete networked-player implementation:
```
IF DirectPlay is connected DO sign on
IF a new client connects DO create <Other Players> at (224,192)
DO store its client id
IF a client disconnects DO destroy the matching one
IF Every 10 ms and connected DO send this player's position
IF position received DO set the remote sprite's X, Y,
direction and animation
```
The remote-player prototype is destroyed on start of frame, so with no server
nothing appears. The lobby that would connect you -- frame 5, with `Host` and
`Join` buttons and the session name `polgame` -- is not reachable from
anywhere.
### 5.7 Six blank comment lines
Both playable frames open with the `Communication` group, and inside it are
three event-editor comment lines -- six in total across the two frames. Every
one of them is empty.
A remark is not stored as a string: it is a LOGFONT, then a color, then the
text. Read it as a string and you get the font's bytes and conclude the file
is corrupt. Read it properly and all six say the same thing: Arial, 13 point,
white, **no text at all**. They are blank lines the author left between the
sub-sections of the group to space them out in the editor.
### 5.8 The menu, and where it really lives
None of the eight frames contains a single "menu item selected" condition. On
the frame tables alone the whole custom `Other` menu looks dead.
It is not. There is a **second event table at application level**, sitting in
the header between the icon and the extension list, with no chunk tag and no
signature. It has five groups, and it is the entire menu implementation:
```
IF MenuSelected 2012 DO Open URL ? "http://www.bulbagarden.com"
IF MenuSelected 2013 DO Open URL ? "http://www.pokemonol.com"
IF MenuSelected 2011 DO Set About To Vert[2]Text Small
DO Set About Title To ? "Game Credits"
DO Set About Text Heading 1 To ? "POL Team"
DO Set About Text 1 To ? "Konidias - ..."
DO Set About Text Heading 2 To ? "Creator Message"
DO Set About Text 2 To ? "If you have ..."
DO Set About Email to ? "bquest@usa.net"
DO Set About Website URL To ? "http://www.pokemonol.com"
DO Open About Box
```
That is all of it. **The only menu ids handled anywhere in the application
are 2011, 2012, 2013.** So, from the file:
- `Game Credits` works, and the dialog is `powerapp.GOX`'s built-in About box
with its fields filled in at click time.
- `Visit BulbaGarden.com` and `Visit Pokemonol.com` work -- they open URLs.
- **`Visit Gamedn.com` (2014) has no handler.** It is the odd one out among
its two working siblings, and it does nothing.
- `Help Guides`, `Character Editor` and `Trainer Pages` -- all six items --
have no handlers. They are menu entries for features that do not exist.
Clicking through the menu will not tell you all of that. Six of the nine
custom items do nothing visible, and so does `Visit Gamedn.com` -- but the two
that work open a web browser rather than changing anything on screen, so from
the outside they look dead too. The table separates them.
---
## 6. Hidden and never-visible content
Of the 1,105 object instances the file places, **93 can never be seen**:
25 parked outside their frame, 22 completely covered by
something drawn later, and 46 belonging to objects with no artwork at
all (the extension objects, counters that are never displayed, and text that
is only ever drawn somewhere else).
### Four screens you cannot get to
`ID Card`, `Menu`, and the two multiplayer frames are unreachable, and they are
not stubs. Section 8 has the other half of this: the ID card, the organizer's
cellphone and the character editor were all published as planned features of
POL:A, so these screens are the announced feature list getting as far as being
drawn and no further.
**The trainer ID card** is a finished 500x400 screen. Image handle 3 is the
card blank itself, 380x230 -- a cream license card with a rounded border, the
words **OFFICIAL POL LICENSE** across it and a printed barcode along the
bottom -- and the frame lays 21 objects over it: the labels `Name:`, `Nicks:`,
`Age:`, `Gender:`, `AIM:`, `ICQ#:`, `Email:`, `Pmail:` and `User Message:`,
the filled-in values beside them, two counters using the gray digit set, and
a portrait. Its one event group is `On start of frame -> do nothing`.
**The "Pokemon Pocket Organizer"** is a finished menu screen with four
sections -- Main Menu, Options, Communication, Help Area -- and a working news
panel. Its three event groups implement exactly one hotspot: clicking
(56,68)-(144,82) shows an object, clicking anywhere outside two zones hides it
again. The rest of the screen is drawn but inert.
Its news panel is a development diary, and it has exactly one entry. That is
not an extraction limit: the text object's own header declares a paragraph
count of 1, and the only date-shaped string anywhere in the 594,404-byte file
is `09-01-00`, appearing twice -- once in the object's name field and once in
the text itself.
The entry explains why there is only one of it:
```
09-01-00 - Currently working on the organizer...
I hope to have most of the organizer completed today.. as much as I can
anyway.. So what does everyone think so far? By the way, this is actually in
the game, I didnt make this in paint. LOL Oh, and for right now, this update
is inside the game itself... but once I can get my server(sometime within the
next 3 days I estimate) then I can make the news uploader and I can auto
upload it whenever I need. The menu is gonna layout somewhat like this: Main
Menu>Save Game, Exit Game, Blah.. Options>Edit Profile, View Pokedex, View
Items, Current Stats, Game Stats, Edit Display... Communication>Use Cellphone,
Check Messages, Members, More Options... Help Area>How To Use,
TroubleShooting... Thats all for this moment that I can think of... See ya
later!
```
That is dated sixteen days before the build. The news panel was meant to be
filled by an uploader running against a server that did not exist yet, so
what shipped is the one hand-typed placeholder entry, still sitting where the
feed was going to go. The organizer never got connected to anything.
The screen is more finished than that makes it sound. It has a close button
(a text object whose entire content is `X`), four section headings, a
`News/Updates` label, and the panel heading `Pokemon Online News and Updates`.
Only one of those is wired to anything.
**Frames 5 and 6** are the multiplayer path. Frame 5 is a DirectPlay lobby
with `Host` and `Join` buttons that creates or joins a session called
`polgame`, and on connection it advances to frame 6.
Frame 6 is the connecting screen, and it is nicer than it needed to be. Its
three objects are a **spinning Pokeball** -- image handles 125, 130, 131, 132,
a four-frame animation of the ball rotating -- a 138x18 gray bar, and an 8x18
end cap the author named `leftedge`. Its three event groups roll the ball
along the bar and restart it when it reaches the end:
```
IF Animation finished <Active object 1> animation 7
IF Is overlapping object <Active object 1> <Active object 2>
IF Animation finished <Active object 2> animation 3
```
It is a loading bar with a Pokeball rolling across it. Nobody has seen it
run, because the only thing that reaches frame 6 is a successful DirectPlay
connection from frame 5, and frame 5 cannot be reached from the title screen.
Frame 6 has no exit either.
### The alpha's welcome text
Beta City contains a text object holding
```
Welcome to Pokemon Online Advanced Alpha City! You can walk through tall
grass, check out your options by pressing enter or ctrl, and many other
things! Explore around the small world! Have fun! ~Konidias
```
Nothing displays it. It is parked at (109,-147), off the top of the map, and
no event references it. It is left over from the alpha -- and note that the
"press enter or ctrl" it advertises is the Pocket Organizer, which by this
build is one of the screens you cannot reach. On the title screen, Enter and
Ctrl now just start the game.
### Ninety-six sprites that can never be drawn, and do not look like anything
Handles **26 to 121** are three consecutive runs of exactly 32 images. The
breaks are at 58 and 90 -- 26+32 and 58+32 -- and they are color breaks: the
first run is navy and green, the second olive and magenta, the third teal and
brown. Within a run the silhouette rotates smoothly through a full turn, and
handle 26, 58 and 90 have identical silhouettes, so the three runs are the same
shape rotated the same way in three colorings.
Everything about them says MMF generated them rather than a person drawing
them. Their bounding boxes change size frame to frame the way a rotated
rectangle's does (29x23, 32x24, 23x29, 24x33 ...). Their hotspots are
**centered** -- (14,9) on a 32x24, and so on -- where every hand-drawn sprite in
this bank has its hotspot at (0,0). And they are referenced only by the high
animation-direction slots of the `Player` and `Other Players` objects, which
carry 100 and 108 direction records each. MMF bakes a rotated copy of a sprite
for each of its 32 directions; this game only ever sets directions 0, 8, 16 and
24 (right, up, left, down), so **none of these 96 images can appear on screen.**
What they are rotations *of* is not settled, and the honest answer is that the
color plane for these images does not reconstruct into anything a person drew:
- every hand-drawn sprite in the bank uses **11 to 20 distinct colors**;
these use **99 to 189**, in about 300 opaque pixels;
- **95% of their pixels are colors that occur nowhere else in the image
bank**, over the whole 96-image band -- 540 distinct colors, of which 44
appear in any other image;
- their silhouettes do not match a rotation of any clean sprite in the bank:
the best mask overlap against the trainer sprites, over all angles, is about
0.49, where a genuine rotation would be above 0.8.
They are structurally sound -- all five image-codec invariants hold for every
one of them, the masks are clean and continuous, and the run structure is
coherent, which is why they render as solid rotating shapes rather than noise.
So either MMF's rotation bake blends the packed 16-bit pixel words
arithmetically instead of per channel, which would produce exactly this (and
would mean the file honestly contains these colors), or there is something
about the color encoding of generated frames that this decoder does not
understand. This teardown cannot tell those two apart from one file, and says
so rather than picking one.
The tempting explanation -- that these are just rotations of the trainer and
the engine's rotator is crude -- does not survive those numbers. A rotation
that copies source pixels cannot introduce 500 colors the source palette does
not contain, and a rotation that blends them per channel cannot leave the
source gamut, which pure blue and pure magenta are well outside.
**Either way, over half the image bank is artwork the engine generated for
itself and never uses.**
### One image nobody uses at all
Exactly one image in the bank is referenced by nothing: **handle 0, 395x300**.
It is a blank window panel -- a light gray field with a title strip across the
top and a border, in four colors. It is the right size and shape to be the
shell for the Pocket Organizer, and it is the only piece of art in the file
with no owner.
(Two digit fonts, one red and one gray, look unreferenced too until you decode
the `CNTR` block inside each counter object, which lists the image handles it
uses for its digits. They are both in use -- red for the position counters,
gray for the ID card. Worth mentioning because "unused font" is the sort of
finding that is easy to publish and wrong.)
### Every object still remembers what it used to be called
Object names live in a fixed 42-byte field that is not cleared when an object
is renamed, so the tail of the old name survives after the terminator. Across
the file that recovers the order in which the map was built, because MMF's
default names are numbered as you create them:
```
Beta City 1 Grass was Backdrop object 1 House 1 floor was ...object 1
Road was Backdrop object 2 tv was ...object 4
water was Backdrop object 3 table was ...object 5
Bush was Backdrop object 4 radio was ...object 6
Sign was Backdrop object 5 rug was ...object 7
door1 was ...object 6 shelf1 was ...object 9
shelf2 was ...object 10
picture was ...object 11
window was ...object 12
plant1 was ...object 14
```
So the town was laid down grass first, then the road, then the river, then the
trees, then the sign, then the door. The house went floor, television, table,
radio, rug, shelves, picture, window, plant -- the television before the
furniture, which is a choice.
The multiplayer lobby's `Host` button was `Button 2`, so `Join` was made
first. And frame 4, now called `Menu`, still has `led` sitting after its
terminator: it was `Untitled` until the organizer was built.
### Scenery dragged off the map
Beta City has a small graveyard of objects parked in negative coordinates --
grass, road, a bush, a sign, a small house, tall grass, flowers -- at x
between -288 and -192. These are the author's palette: one of each tile, kept
just off the left edge of the map where they can be copied from. The editor
saved them along with everything else.
### A table of shared objects, padded with the editor's memory
After the event-system limits, five of the eight frames -- Opening, ID Card,
Menu and the two multiplayer frames, but not Beta City, House 1 or the
readme -- carry a table of the objects they share with the application:
```
Opening 3 entries, 1344 bytes Xpos:fs64-@:Counter, Counter 1:fs64-@:Counter, ypos:fs64-@:Counter
ID Card 2 entries, 1240 bytes Xpos:fs64-@:Counter, (uninitialized)
Menu 3 entries, 1344 bytes Xpos:fs64-@:Counter, Counter 1:fs64-@:Counter, (uninitialized)
Untitled 1 entries, 1136 bytes (uninitialized)
Untitled 3 entries, 1344 bytes Xpos:fs64-@:Counter, Counter 1:fs64-@:Counter, (uninitialized)
```
Each entry is 104 bytes: a marker, two small numbers, then the object written
as `name:fs64-@:type`, and then a long tail of nothing in particular. The
table as a whole is 1,032 bytes of mostly-zero header plus those records, so
it runs to 1,136-1,344 bytes a frame for three counters' worth of information.
The tail is where section 4.8's leaked memory lives. It is also why this
block reads as "almost all zeros" if you only look at how much of it is zero:
about 250 bytes of each are not, and those bytes are the operating system.
---
## 7. Bugs and loose ends
**The sign is not interactive.** The ABOUT screen says "Shift- Read signs,
talk to people, activate things", and Beta City has a `Sign` object placed
twice. No event anywhere references it. It is a Backdrop, which in MMF
cannot carry events at all, so this was never going to work -- and the welcome
text that would presumably have been its message is the orphaned alpha
leftover above.
**Two global values are written every cycle and never read.** Beta City's
`XYPos` group does
```
IF Always
DO SetGlobalValue 14, obj19.CounterValue
DO SetGlobalValue 15, obj20.CounterValue
```
storing the player's X and Y into globals 14 and 15 continuously. Nothing ever
reads them. The two places that could have used them -- the return-from-house
positions -- use hardcoded coordinates instead. This is the remains of a
save-your-position feature that was wired up from one end only.
**`Visit Gamedn.com` does nothing**, while its two siblings work. See 5.7.
**Right-clicking your own character shows a label.** In both playable frames:
```
IF MouseOnObject <Player>
IF ObjectClicked button 2, <Player>
DO CreateObject <Active object 4> at <Player>+(-25,-70)
DO Display paragraph <UsernameID...> paragraph 0 at <Active object 4>+(5,7)
```
and moving the mouse off the character destroys it again. Button code 2 --
every other click condition in the file uses 0. This is the placeholder for
the name labels that would have floated over other players' heads. It is
undocumented on the controls screen, and it works.
**House 1's start position does not match its parked player.** The frame's
`Player` instance sits at (256,192), but start-of-frame sets `Solid` -- which
is what actually positions the player -- to (274,216). The stored instance
position is only where the object sat in the editor; the event table wins.
**`Level %d`.** The runtime's string table has it; nothing in this game has
levels. That is the stub's, not the author's.
---
## 8. The servers and the dead links
Everything this game was pointed at is gone.
| where | what it was |
|---|---|
| `http://www.pokemonol.com` | the project's own site. In the menu, in the About box, in the readme screen, and in Caveman's radio dialogue |
| `http://www.bulbagarden.com` | the community it grew out of. Still a going concern, at a different address |
| `Gamedn.com` | in the menu as a label only -- the item has no handler, so the URL is not even stored anywhere. It was not a random link: gamedn.com hosted POL's message board, at `/cgi-bin/forum/Ultimate.cgi` |
| `bquest@usa.net` | the author's contact address, in the About box. usa.net's free mail service is long gone |
| `polgame` | the DirectPlay session name. DirectPlay's lobby and matchmaking services were retired years ago |
| `Meowth@sienna.pol` | on the ID card. A fictional address in a fictional TLD -- the in-game mail system the organizer's "Use Cellphone" and "Check Messages" entries were for |
| `webmaster@pokemonol.com` | the ID card's "Pmail" field |
The `.pol` address is the interesting one. It is not a typo or a dead domain;
it is a designed in-fiction email system, and the Pocket Organizer's
Communication section was built to be its client. The news post explains that
the author was three days from having a server.
### The custom menu is the website's navigation
Section 5.8 establishes that six of the nine items under `Other` have no
handler behind them. The November 2000 capture of pokemonol.com explains why
they exist at all: **the menu is a copy of the site's own navigation.** Its
POL:A section listed Information, Downloads, Screenshots, Member Pages, Online
Guide and Development Team, and the game's menu maps onto it almost item for
item:
| menu item in the game | the section of the site it points at |
|---|---|
| `Help Guides` > `Game Guide`, `Site Guide` | the site's Online Guide |
| `Character Editor` > `New Character`, `Character Archive` | a planned game feature, not a page -- see below |
| `Trainer Pages` > `Create/Edit Your Page`, `View Trainer Archive` | the site's Member Pages |
| `Credits and Sites` > `Game Credits` | the site's Development Team |
| `Visit Gamedn.com` | the message board |
So they are not aspirational stubs for features nobody had designed. They name
sections of a website that existed; the game simply never got the code to open
them, and only the three items PowerApp Lite could handle with a URL or an
About box were ever wired up.
### What POL:A was going to be
A second capture, of the POL:A information page on **16 December 2000**, is
what makes the rest of the menu legible:
<https://web.archive.org/web/20001216135800/http://www.pokemonol.com/polrpg/>
It describes POL:A as the successor to an earlier POL RPG that never reached a
finished version, and then publishes a planned feature list. Several entries
on it are visible in this file three months earlier, as screens with no way in:
| planned feature, as the page describes it | what is in `pol.gam` |
|---|---|
| a **Character Editor** for making your own characters and submitting them to be added to the game | the dead `Character Editor` > `New Character`, `Character Archive` menu |
| every user assigned a customizable **ID card**, a pokedex and pokeballs | the finished, unreachable `ID Card` frame |
| a **cellphone** with an email feature, plus TVs and newspapers among fifty-odd new items | the Pocket Organizer's Communication section, the `Meowth@sienna.pol` address on the ID card, and the working TV in House 1 |
| a **profile page** created on the site for every player who signs up | `Trainer Pages` and the site's Member Pages |
| **over thirty towns**, gyms with CPU leaders, all 251 pokemon | one town, one house, one NPC |
That last row is the shape of the whole project. What shipped in September
2000 is the first town of a thirty-town world, with the ID card, the organizer
and the character editor built as screens and never connected -- which is
exactly what the unreachable frames in section 6 are. They are not abandoned
experiments; they are the parts of the announced feature list that got as far
as having a screen.
The same capture shows the site had added a Developer's Log by December and had
passed 70,000 hits, against 50,000 five weeks earlier. The status box still
read `POL RPG: (N/A)`.
---
## 9. Credits and rights
The game's own credits, from the About box that `Game Credits` opens, verbatim:
```
POL Team
Konidias - Developer, Graphics - Caveman - Graphic Artist, Ideas -
Optimus Primus - Concept Artist, Ideas - Dragoness - Story,
Characters - Special Thanks to everyone else that helped make
pol!
Creator Message
If you have any questions, comments, ideas, or you find any bugs, email me as
soon as possible.
```
The title screen carries its own disclaimer, which the game never displays
long enough to read comfortably:
```
Pokemon Online is (C)Konidas 2000 and is in no way affiliated with Nintendo,
GameFreak, Creatures, or any other respective companies. This is a free fan
game created for enjoyment only. No profits are being made from the game.
Any original content, ideas, or characters are copyright Nintendo, GameFreak,
Creatures.
```
(The author's handle is spelled `Konidias` everywhere except in his own
copyright line, where it is `Konidas`.)
Separately, and not the game's own work:
- **The music.** `technosong.mid` is `Copyright (C) 1995 Kevin Newman`, with
the sequencer's university address `s9510929@babel.ee.up.ac.za` still in the
file. It is credited here because it is a different rights position from
the rest of the game, and because attributing it to the POL team would be
wrong.
- **The engine and the extensions.** `pol.exe` and `cncs32.dll` -- the
latter identifying itself as the "Click & Create Graphic Library",
(C) Europress Software 1996 -- are the authoring tool's. The six `.GOX`
DLLs belong to the people named in section 4.7: Michel Plante
(SMMP Informatique), Sean Poling (Third Eye Entertainment), Core Media
Australia, and Europress Software. They are described here, not
redistributed by this document.
- **Pokemon** is Nintendo, Game Freak and Creatures', as the disclaimer says.
The `ID Card` frame -- which is unreachable, and was sample data filling in a
mock-up -- contains what appear to be the author's own details of the time.
They are quoted in the extraction output because they are in the file, and
they are not repeated here.
Corrections and takedown requests from anyone named above are welcome; see
section 12.
---
## 10. How to reproduce this
### 10.1 Find the game
Do not start with the executable. `pol.exe` is the Multimedia Fusion
stand-alone runtime with an overlay appended after its last PE section, and
that overlay is the extension pack, not the game. Walk the section table,
take the highest `rawPtr + rawSize`, and everything after it is the overlay.
The game is the `.gam` beside it, and its first four bytes say `PAPP`.
If you find `GAPP` instead, it is the same format from The Games Factory
rather than MMF, and everything below applies unchanged.
### 10.2 The image codec, which is the part that takes work
Each image record is 24 bytes of header and then a payload:
```
u32 uncompressedSize
(u32 maskRowOffset, u32 colorRowOffset) * height
mask plane
color plane
```
Three things about this will cost you an afternoon each if you have to find
them yourself.
**The row-offset table is stored uncompressed even when the payload is
compressed.** So the first few hundred bytes of a compressed image look
perfectly sensible, which makes it look as though the whole payload is raw.
**The color row offsets are destination offsets, not source offsets.** They
tell you where each row lands in the expanded buffer. Using them to index
into the compressed stream produces garbage that still decodes to the right
number of pixels.
**When compressed, the color plane is one run-length stream that spans row
boundaries.**
```
c = next byte
c < 0x80 : repeat the following u16 word c times
c >= 0x80 : (c - 0x80) literal u16 words follow
```
with exactly one padding byte at the end. Decoding it per row fails, and it
fails subtly.
The mask plane is never compressed. Per row it is byte pairs
(transparent run, opaque run) until the pairs sum to the width. The color
plane holds **only the opaque pixels**, as 15-bit RGB555 words.
Check all five of these before believing a pixel:
1. every mask row's runs sum to exactly `width`;
2. the color plane holds exactly `2 * sum(opaque runs)` bytes;
3. the RLE stream consumes its source to within the one padding byte;
4. `uncompressedSize` reconciles to
`4 + height*8 + maskBytes + 2*opaquePixels`;
5. grayscale ramps decode to exact `r == g == b`, which is what proves RGB555
and not RGB565.
All five hold for all 191 images in this file.
Clickteam's own LZ (`decompressOld`) is **not** used here. It returns 0 on
every image payload, with or without the leading length word.
### 10.3 The events
The event tables sit at the end of each frame, after the instance list, in a
chunk of their own: `u16 0x8004`, `u32 size`, `u32 groupBytes`, then the
groups, then 36 bytes of event-system limits.
The group, condition, action and parameter structures are the ordinary
Multimedia Fusion 1.5 ones. That is the useful discovery: the flat `PAPP`
container and the chunked `PAME` container disagree completely about how to
*store* the event tables and agree completely about what an event *is*. So
Mathias Kaerlev's mmfparser structures, written for `PAME`, decode `PAPP`
events byte for byte once you have found them.
Two places where they do not, and both bite:
**Expression tokens are laid out differently.** In `PAPP` a token is
```
i8 objectType
i8 num
u16 size total token length including these four bytes
u8[size-4] payload
```
terminated by a bare `(0, 0)` with no size word. Read it as
`(i16 type, u16 num, u16 size)` -- the `PAME` layout -- and every expression
in the file decodes to nonsense, while still consuming plausible-looking
numbers of bytes.
**The built-in object opcode numbering is not MMF2's.** mmfparser's *system*
tables (object type < 0) are correct and can be used unchanged. Its
*common object* tables are MMF2's, and MMF 1.5 numbers those differently:
"set position" is action 3, not 1; "set X" and "set Y" are 4 and 5, not 2 and
3; "force animation" is 18. Naming them from the MMF2 table produces an event
dump that reads fluently and is wrong -- which is the worst possible failure
mode for this kind of work.
The table this file pins down is in `events.py`, with the evidence for each
entry recorded next to it. Each was derived from the parameter types the
opcode carries plus what the game is known to do at that moment: the opcode
that takes a POSITION parameter is "set position"; the one that takes a
direction mask 16 times, once per arrow-key group, is "set direction"; the one
whose animation-selector payload carries the animation's own name, `"Walking"`
when a key is held and `"Stopped"` when none is, is "force animation".
Anything that could not be pinned that way keeps its number.
**Direction parameters are a 32-bit mask** over MMF's 32 directions, numbered
counter-clockwise from 0 = right. So right is bit 0, up is bit 8, left is
bit 16 and down is bit 24. Read the mask as 16 bits and left and down both
come out as zero -- a bug that produces a decode which looks fine until you
notice two directions have gone missing.
**Look for the application-level table.** It has no chunk tag and no
signature, and in this game it is the entire menu implementation. Find it by
parsing: try every offset between the icon and the extension table and keep
the one whose group chain runs cleanly to the extension table. Exactly one
does.
### 10.4 Check yourself against the pictures
The renders in `extract/render/` are built only from the container:
coordinates from the instance list, artwork from each object's own image
handle, order from the record array. Nothing in that chain looks at a picture
of the running program, so any screen capture is an independent test of the
whole decode at once -- container, image codec, palette, instance layout and
draw order. Two captures were used here, one of the town and one of the house
interior.
```
03_house_interior vs the House 1 reconstruction: mean |difference| 0.000 of 255, 100.00% of pixels identical
04_overworld vs the Beta City reconstruction: mean |difference| 0.000 of 255, 100.00% of pixels identical
```
Both are exact. A patch taken out of the middle of each screenshot's client
area matches the reconstruction with a mean absolute difference of **0.000 of
255** and **100% of pixels identical**.
And the byte accounting: header, eight frames and three banks account for
every byte of the 594,404-byte file except two 2-byte gaps, at offset 562995 (2 bytes) and offset 565021 (2 bytes), which
sit immediately before the font and music banks and look like a bank-kind tag.
### 10.5 A prompt for doing this yourself
If you want to point a model at another Games Factory or MMF 1.x file, this is
the shape of the instruction that works:
```
Here is a Clickteam .gam file (magic PAPP or GAPP) and the .exe beside it.
Do not run either. Recover, from the bytes alone:
1. The container: header fields, frame offset table, frame-handle table,
the application icon, the extension list, the menu (UTF-16) and the
accelerator table.
2. Every media bank. They all have the shape
(u32 totalSize, u32 count, (u32 offset, u32 size) * count) with offsets
relative to the count field. Find them by that structure, not by
scanning for signatures.
3. The image bank. The color plane is RLE that spans row boundaries; the
row-offset table is destination offsets and is stored uncompressed even
for compressed images. Verify with the five invariants before believing
any pixel: mask rows sum to width; color bytes == 2 * opaque pixels;
the RLE consumes its source to within one padding byte;
uncompressedSize reconciles; grayscales decode to r == g == b.
4. Per frame: the object list (68-byte records, name[42], type at +42,
obstacle at +43, instance count at +44, IMAGE BANK HANDLE at +54,
objectID at +64, then optional data chunks ending with tag 0x8002),
and the instance list (32-byte records; record-array order is the draw
order, not the slot table).
5. The event tables (chunk 0x8004 per frame, plus an untagged one at
application level). Use mmfparser's MMF1.5 group/condition/action
structures, but note that expression tokens are i8 type, i8 num,
u16 size, and that the built-in object opcode numbers are NOT MMF2's.
Derive the opcode names from the parameter types and from what the game
visibly does; do not assume the MMF2 table.
6. Carve the extension DLLs out of the .exe overlay and read their ACE
names from string resources 28000+n / 28200+n / 28400+n, so extension
events read in English.
Validate as you go, and state what did not decode rather than guessing:
- object lists must yield exactly objectCount records ending exactly on
listSize, for every frame;
- each object's instance count field must equal the instances placed;
- every group, condition, action and parameter must consume exactly its
declared size;
- every Text object must yield exactly the paragraph count its own TEXT
header declares -- scanning the block for printable runs instead will
look like it works and will quietly drop every short paragraph;
- render the frames and diff them against any screenshots you have.
Then account for every byte in the file and report what is left over.
```
The validation clauses are the important half. Every one of them caught a
real error while this was being written.
---
## 11. The toolchain
11 files, about 2810 lines of Python, no dependencies beyond
Pillow and numpy.
### Layout and what gets produced
Put the three original files in a folder with `tools/` beside them and run one
command. Nothing is written to the originals; `extract/` is created on the
first run and everything lands in it.
```
<any folder>/
+-- pol.exe the MMF runtime + the extension overlay
+-- pol.gam the game
+-- cncs32.dll the Click & Create graphics library
+-- tools/ this toolchain
+-- extract/ created on the first run
|
+-- images/ one RGBA PNG per image-bank entry
| +-- <handle>.png 191 of them, 0-193 with three gaps
+-- sounds/
| +-- technosong.mid the only audio in the game
+-- icon/ the application icon out of the .gam
| +-- app-icon-32.png 32x32, and _4x for a readable version
| +-- app-icon-16.png 16x16, and _4x
+-- extensions/ the six DLLs carved out of the exe overlay
| +-- DPlay.GOX GFEDIT.GOX GFBUTTON.GOX
| +-- powerapp.GOX GFINI.GOX moo2.GOX
+-- render/ every frame rebuilt from the container
| +-- <frame>.png the frame at its authored size
| +-- <frame>-view.png what the 500x400 window shows, letterboxed
| +-- <frame>-obstacles.png the collision map, white = blocked
| +-- <frame>-with-parked.png padded canvas, so objects parked outside
| the frame are visible
|
+-- app.json header, menu, accelerators, frames, banks
+-- images.json image manifest, empty slots, the font bank
+-- extensions.json per-DLL version info and ACE name tables
+-- frames.json object lists, animation tables, instances
+-- events.json every event group, structured
+-- EVENTS.txt the same thing, readable
+-- visibility.json per-instance: inside the frame, and visible
+-- verify.json render-vs-capture scores
+-- inventory.json provenance, strings, dialogue, reachability,
| leaked pointers, byte accounting
+-- contact-sheet.png all 191 images on one page
```
```sh
python tools/run_all.py # everything, about 20 seconds
python tools/run_all.py events # a single stage
```
Order matters in two places: `extensions` must run before `events`, because
the extension opcode names come out of the DLLs, and `images` must run before
`render`. Each stage prints its findings and its self-checks to stdout;
redirect them if you want them on disk.
The document itself is built separately, from the JSON above, so the prose
cannot drift from the extraction.
### 11.1 `polpaths.py`
```python
"""Where everything lives.
The teardown is run from the folder that holds the three original files
(pol.exe, pol.gam, cncs32.dll) as they were distributed. Everything the
toolchain produces goes under extract/, so the originals are never touched
and the whole output tree can be deleted and rebuilt.
"""
import os
HERE = os.path.dirname(os.path.abspath(__file__))
ROOT = os.path.dirname(HERE)
GAM = os.path.join(ROOT, 'pol.gam')
EXE = os.path.join(ROOT, 'pol.exe')
DLL = os.path.join(ROOT, 'cncs32.dll')
OUT = os.path.join(ROOT, 'extract')
IMAGES = os.path.join(OUT, 'images')
SOUNDS = os.path.join(OUT, 'sounds')
ICON = os.path.join(OUT, 'icon')
EXTENSIONS = os.path.join(OUT, 'extensions')
RENDER = os.path.join(OUT, 'render')
for _d in (OUT, IMAGES, SOUNDS, ICON, EXTENSIONS, RENDER):
os.makedirs(_d, exist_ok=True)
def out(*parts):
return os.path.join(OUT, *parts)
def read(path):
with open(path, 'rb') as fh:
return fh.read()
```
### 11.2 `papp.py`
```python
"""papp.py -- the PAPP/GAPP container.
`pol.gam` is a Clickteam "PAPP" file: the application container written by
Multimedia Fusion 1.x / The Games Factory (Click&Create engine). It is NOT
the chunked "PAME" format that CTFAK 2.0 and mmfparser understand -- there
are no `id/flag/size` chunk headers anywhere in it. It is a flat file with
one absolute offset table for the frames and a repeating bank structure for
the media.
Everything here was derived from the bytes of pol.gam and cross-checked
against Epic.gam (a `GAPP` sibling).
0x000 char[4] magic "PAPP" (MMF 1.x) or "GAPP" (Games Factory)
0x004 u16 version 0x0207
0x006 char[80] application name
0x056 char[80] author
0x0A6 char[78] copyright / about (empty here)
0x0F4 u16 ? 29
0x0F6 u16 ? 6
0x0F8 u16 window width 500
0x0FA u16 window height 400
0x0FC u32 border color 0x00000000
0x100 u32 app flags 0x00028105
0x104 u16 ? 1
0x106 u16 ? 4
0x108 u16[4] ? 3,3,3,3 (one per player)
0x110 u16[4][6] control keys VK codes, per player:
up, down, left, right, fire1, fire2
0x140 u32 ? 3
0x148 u32 ? 9
0x154 u32 ? 9999999 (score/counter ceiling)
0x15C u32 frame count 8
0x160 u32 ? 0x00200000
0x166 u32[n] frame offsets absolute
... u16 frame-handle count, then u16[n] handle -> frame index
... application icon, 4bpp
... global-object blobs (PowerApp Lite credits data)
... extension table, 276-byte records
... menu (UTF-16), then the accelerator table
... frames
... image bank / font bank / music bank [/ sound bank]
The banks all share one shape, which is what makes them findable without
guessing:
u32 totalSize includes this field
u32 count
(u32 offset, u32 size) * count offsets relative to the count field,
entries perfectly contiguous,
offset[0] == 4 + count*8
"""
import json
import os
import re
import struct
import sys
import polpaths as P
try:
sys.stdout.reconfigure(encoding='utf-8', errors='replace')
except Exception:
pass
VK = {
0x08: 'Backspace', 0x09: 'Tab', 0x0D: 'Enter', 0x10: 'Shift',
0x11: 'Control', 0x12: 'Alt', 0x1B: 'Esc', 0x20: 'Space',
0x21: 'PageUp', 0x22: 'PageDown', 0x23: 'End', 0x24: 'Home',
0x25: 'Left', 0x26: 'Up', 0x27: 'Right', 0x28: 'Down',
0x2D: 'Insert', 0x2E: 'Delete',
}
for _i in range(10):
VK[0x30 + _i] = str(_i)
for _i in range(26):
VK[0x41 + _i] = chr(65 + _i)
for _i in range(1, 25):
VK[0x6F + _i] = 'F%d' % _i
def vkname(v):
return VK.get(v, '0x%02X' % v)
def cstr(d, off, size):
return d[off:off + size].split(b'\0')[0].decode('latin1')
def wstr(d, off, limit=512):
"""NUL-terminated UTF-16LE string; returns (text, bytes consumed)."""
end = off
while end + 1 < off + limit:
if d[end] == 0 and d[end + 1] == 0:
break
end += 2
return d[off:end].decode('utf-16-le', 'replace'), end + 2 - off
# ---------------------------------------------------------------- header
def read_header(d):
h = {}
h['magic'] = d[:4].decode('latin1')
h['version'] = struct.unpack_from('<H', d, 4)[0]
h['name'] = cstr(d, 0x06, 80)
h['author'] = cstr(d, 0x56, 80)
h['copyright'] = cstr(d, 0xA6, 78)
h['unk0F4'], h['unk0F6'] = struct.unpack_from('<HH', d, 0xF4)
h['width'], h['height'] = struct.unpack_from('<HH', d, 0xF8)
h['borderColor'] = struct.unpack_from('<I', d, 0xFC)[0]
h['flags'] = struct.unpack_from('<I', d, 0x100)[0]
h['unk104'], h['unk106'] = struct.unpack_from('<HH', d, 0x104)
h['unk108'] = list(struct.unpack_from('<4H', d, 0x108))
keys = []
for p in range(4):
k = struct.unpack_from('<6H', d, 0x110 + p * 12)
keys.append(dict(zip(('up', 'down', 'left', 'right', 'fire1', 'fire2'),
[vkname(v) for v in k])))
h['controls'] = keys
h['unk140'] = struct.unpack_from('<I', d, 0x140)[0]
h['unk148'] = struct.unpack_from('<I', d, 0x148)[0]
h['unk154'] = struct.unpack_from('<I', d, 0x154)[0]
h['frameCount'] = struct.unpack_from('<I', d, 0x15C)[0]
h['unk160'] = struct.unpack_from('<I', d, 0x160)[0]
n = h['frameCount']
h['frameOffsets'] = list(struct.unpack_from('<%dI' % n, d, 0x166))
p = 0x166 + 4 * n
# frame-handle table: u16 count, u16 ?, u16 handle[count]
hcount = struct.unpack_from('<H', d, p)[0]
h['frameHandleCount'] = hcount
h['unkAfterHandleCount'] = struct.unpack_from('<H', d, p + 2)[0]
h['frameHandles'] = list(struct.unpack_from('<%dH' % n, d, p + 6))
h['headerEnd'] = p + 6 + 2 * n
return h
# ---------------------------------------------------------------- app icon
def read_icon(d, start):
"""The application icon, stored raw with no palette and no DIB header.
A u32 size of 0x340 = 832 bytes, which is exactly the two standard icon
images concatenated with nothing between them:
512 32x32 4bpp XOR plane (16 bytes per row)
128 32x32 1bpp AND mask
128 16x16 4bpp XOR plane ( 8 bytes per row)
64 16x16 1bpp AND mask
There is no color table because 4bpp icons of this era used the fixed
Windows 16-color palette, so only the indices need storing.
"""
for p in range(start, start + 16, 2):
if struct.unpack_from('<I', d, p)[0] == 0x340:
b = p + 4
return dict(offset=p, size=0x340, dataOffset=b,
big=dict(w=32, h=32, xor=b, and_=b + 512),
small=dict(w=16, h=16, xor=b + 640, and_=b + 768))
return None
# ---------------------------------------------------------------- extensions
EXT_MAGIC = 0x59082516
def read_extensions(d):
"""276-byte records, each ending in a MAX_PATH filename field.
The record is located by its magic; the filename follows 8 bytes later.
"""
out = []
for m in re.finditer(struct.pack('<I', EXT_MAGIC), d[:0x2000]):
p = m.start()
name = cstr(d, p + 8, 264)
if not name or not name.lower().endswith('.gox'):
continue
out.append(dict(offset=p, name=name,
prev=struct.unpack_from('<H', d, p - 2)[0]))
return out
# ---------------------------------------------------------------- menu
MF_POPUP = 0x0010
MF_END = 0x0080
MF_SEPARATOR = 0x0800
MF_GRAYED = 0x0001
MF_DISABLED = 0x0002
MF_CHECKED = 0x0008
def read_menu(d, start, end):
"""MMF stores the application menu as a Windows-style nested item list
with UTF-16 labels. Each item is:
u16 flags
[u16 id] only when the item is not a popup
wchar[] label NUL-terminated; a TAB separates the accelerator text
flags & 0x0010 opens a submenu; flags & 0x0080 closes the current level.
A label of "-" (or flags 0x0800) is a separator. The whole thing is
prefixed by a small header holding its byte size.
"""
p = start
root = []
# each level remembers whether the popup that opened it was itself the
# last item in ITS parent -- closing such a level closes the parent too
levels = [dict(items=root, closesParent=False)]
done = False
guard = 0
while not done and p < end:
guard += 1
if guard > 4000:
break
flags = struct.unpack_from('<H', d, p)[0]
p += 2
if flags & MF_POPUP:
label, used = wstr(d, p)
p += used
kids = []
levels[-1]['items'].append(dict(label=label, popup=True,
items=kids, flags=flags))
levels.append(dict(items=kids, closesParent=bool(flags & MF_END)))
continue
mid = struct.unpack_from('<H', d, p)[0]
p += 2
label, used = wstr(d, p)
p += used
item = dict(label=label.split('\t')[0], id=mid, flags=flags)
if '\t' in label:
item['accel'] = label.split('\t', 1)[1]
if not label or flags & MF_SEPARATOR:
item['separator'] = True
levels[-1]['items'].append(item)
if flags & MF_END:
while len(levels) > 1:
if not levels.pop()['closesParent']:
break
if len(levels) == 1:
done = True
break
return root, p
ACCEL_FVIRTKEY = 0x01
ACCEL_FSHIFT = 0x04
ACCEL_FCONTROL = 0x08
ACCEL_FALT = 0x10
ACCEL_FLASTKEY = 0x80
def read_accelerators(d, p, limit=64):
"""Win32 ACCEL table, widened to u16 fVirt: (fVirt, key, cmd, pad)."""
out = []
for _ in range(limit):
fv, key, cmd, pad = struct.unpack_from('<4H', d, p)
if fv == 0 or key == 0:
break
mods = []
if fv & ACCEL_FCONTROL:
mods.append('Ctrl')
if fv & ACCEL_FALT:
mods.append('Alt')
if fv & ACCEL_FSHIFT:
mods.append('Shift')
mods.append(vkname(key))
out.append(dict(offset=p, fVirt=fv, key=key, cmd=cmd,
combo='+'.join(mods), last=bool(fv & ACCEL_FLASTKEY)))
p += 8
if fv & ACCEL_FLASTKEY:
break
return out, p
# ---------------------------------------------------------------- banks
def find_banks(d, lo=0):
"""Every media bank has the same self-describing shape, so they are found
by structure rather than by signature or by a hardcoded offset."""
n = len(d)
out = []
p = lo
while p < n - 16:
total, count = struct.unpack_from('<II', d, p)
if 1 <= count <= 20000 and count * 8 + 8 <= total <= n - p:
off0, size0 = struct.unpack_from('<II', d, p + 8)
# entries are contiguous EXCEPT for slots the editor deleted, whose
# offset is left pointing back into the bank's own offset table.
# So the test is "every entry lies inside the bank and the last one
# reaches its declared end", not strict contiguity.
if off0 == 4 + count * 8 and size0 > 0:
lo, reach, ok, dead = off0, 0, True, 0
for i in range(count):
o, s = struct.unpack_from('<II', d, p + 8 + i * 8)
if s == 0: # slot deleted in the editor
dead += 1
continue
if o < lo or o + s > total:
ok = False
break
reach = max(reach, o + s)
if ok and reach == total:
out.append(dict(offset=p, base=p + 4, total=total,
count=count, end=p + 4 + total,
deletedSlots=dead))
p += total
continue
p += 1
return out
def bank_entries(d, bank):
base = bank['base']
return [struct.unpack_from('<II', d, base + 4 + i * 8)
for i in range(bank['count'])]
def bank_record(d, bank, i):
off, size = struct.unpack_from('<II', d, bank['base'] + 4 + i * 8)
a = bank['base'] + off
return a, size, d[a:a + size]
# ---------------------------------------------------------------- frames
def read_frames(d, h):
frames = []
offs = h['frameOffsets']
for i, o in enumerate(offs):
x, y, w, hh = struct.unpack_from('<hhHH', d, o)
raw = d[o + 8:o + 8 + 64]
name = raw.split(b'\0')[0].decode('latin1')
# the name field is fixed-size and is not cleared on rename, so the
# tail of the previous name survives after the terminator
tail = raw[len(name) + 1:]
m = re.match(rb'[\x20-\x7e]{2,}', tail)
frames.append(dict(index=i, offset=o, editorX=x, editorY=y,
w=w, h=hh, name=name,
staleName=m.group().decode('latin1') if m else None))
return frames
def load():
d = P.read(P.GAM)
h = read_header(d)
return d, h
def main():
d, h = load()
icon = read_icon(d, h['headerEnd'])
exts = read_extensions(d)
frames = read_frames(d, h)
# the menu sits between the last extension record and the first frame
menu_lo = exts[-1]['offset'] + 276 if exts else h['headerEnd']
tag = d.find(b'&\x00M\x00a\x00i\x00n\x00', menu_lo)
# back up over the item's flags word
menu_start = tag - 2
menu, menu_end = read_menu(d, menu_start, frames[0]['offset'])
accels, accel_end = read_accelerators(d, menu_end + 4)
banks = find_banks(d, frames[-1]['offset'])
info = dict(file=os.path.basename(P.GAM), size=len(d), header=h,
icon=icon, extensions=exts, frames=frames,
menuOffset=menu_start, menu=menu,
acceleratorOffset=menu_end + 4, accelerators=accels,
banks=banks)
with open(P.out('app.json'), 'w') as fh:
json.dump(info, fh, indent=1)
print('%s v%04X %r by %r %dx%d %d frames %d bytes'
% (h['magic'], h['version'], h['name'], h['author'],
h['width'], h['height'], h['frameCount'], len(d)))
print('header ends %#x icon %s %d extensions'
% (h['headerEnd'], icon and hex(icon['offset']), len(exts)))
print('frame handles', h['frameHandles'])
for e in exts:
print(' ext %#06x %s' % (e['offset'], e['name']))
print('menu at %#x:' % menu_start)
def show(items, depth=1):
for it in items:
pad = ' ' * depth
if it.get('popup'):
print('%s%s >' % (pad, it['label']))
show(it['items'], depth + 1)
elif it.get('separator'):
print('%s---' % pad)
else:
print('%s%-24s id=%-5d flags=%#06x %s'
% (pad, it['label'], it['id'], it['flags'],
it.get('accel', '')))
show(menu)
print("accelerators at %#x:" % (menu_end + 4))
for a in accels:
print(' %-10s -> %d' % (a['combo'], a['cmd']))
print('frames:')
for f in frames:
print(' %d %-14s %4dx%-4d @%#08x editor(%d,%d) stale=%r'
% (f['index'], f['name'], f['w'], f['h'], f['offset'],
f['editorX'], f['editorY'], f['staleName']))
print('banks:')
for b in banks:
print(' @%-8d count=%-5d total=%-8d end=%d'
% (b['offset'], b['count'], b['total'], b['end']))
if __name__ == '__main__':
main()
```
### 11.3 `images.py`
```python
"""images.py -- the image bank, the font bank, the music bank and the icon.
The image record and its payload are the part of PAPP that takes real work,
because the color plane is compressed with a run-length scheme that ignores
row boundaries. Decoding it per row -- the obvious thing to try -- produces
images that are almost right, which is worse than images that are obviously
wrong.
image record
i16 checksum
i32 references
i32 dataSize == entry size - 24
i16 width, i16 height
u8 graphicMode 6 = 15-bit RGB555; 0 = an empty/deleted slot
u8 flags 0x40 = has data, 0x02 = color plane is RLE
i16 hotspotX, hotspotY, actionX, actionY
u8 payload[dataSize]
payload
u32 uncompressedSize
(u32 maskRowOffset, u32 colorRowOffset) * height
offsets relative to payload+4; the COLOR ones
are destination offsets, not source offsets,
which is the trap
mask plane per row, byte pairs (transparentRun, opaqueRun)
until the pair sums reach `width`. Never
compressed, even when the color plane is.
color plane ONLY the opaque pixels, as u16 RGB555 words,
row after row.
flags & 0x02 clear -> stored raw
flags & 0x02 set -> ONE linear RLE stream
that spans rows:
c = next byte
c < 0x80 : repeat the following u16 c times
c >= 0x80 : (c-0x80) literal u16 words
with exactly one padding byte at the end.
Five invariants are checked before any pixel is believed, and all five hold
for all 191 images: mask rows sum to `width` exactly; the color plane holds
exactly 2 * sum(opaque runs) bytes; the RLE stream consumes its source to
within the one padding byte; `uncompressedSize` reconciles to
4 + height*8 + maskBytes + 2*opaquePixels; and grayscale ramps decode to
exact r == g == b, which is what proves RGB555 rather than RGB565.
"""
import json
import os
import struct
import sys
import numpy as np
from PIL import Image
import polpaths as P
import papp
try:
sys.stdout.reconfigure(encoding='utf-8', errors='replace')
except Exception:
pass
# the fixed 16-color palette 4bpp icons of this era assume
WIN16 = [(0, 0, 0), (128, 0, 0), (0, 128, 0), (128, 128, 0), (0, 0, 128),
(128, 0, 128), (0, 128, 128), (192, 192, 192), (128, 128, 128),
(255, 0, 0), (0, 255, 0), (255, 255, 0), (0, 0, 255), (255, 0, 255),
(0, 255, 255), (255, 255, 255)]
def read_record(d, bank, i):
off, size = struct.unpack_from('<II', d, bank['base'] + 4 + i * 8)
a = bank['base'] + off
if size < 24:
return None
chk, ref, dsz = struct.unpack_from('<hii', d, a)
w, h = struct.unpack_from('<hh', d, a + 10)
hx, hy, ax, ay = struct.unpack_from('<hhhh', d, a + 16)
return dict(handle=i, offset=a, size=size, payload=size - 24, w=w, h=h,
graphicMode=d[a + 14], flags=d[a + 15],
hotspotX=hx, hotspotY=hy, actionX=ax, actionY=ay,
checksum=chk, references=ref)
def decode(d, r):
"""-> (RGBA array, None) or (None, reason)"""
if r is None:
return None, 'null entry'
w, h, n = r['w'], r['h'], r['payload']
if r['graphicMode'] != 6 or w <= 0 or h <= 0:
return None, 'graphic mode %d, %dx%d' % (r['graphicMode'], w, h)
a = r['offset'] + 24
base = a + 4
A, B = [], []
for i in range(h):
x, y = struct.unpack_from('<II', d, base + i * 8)
A.append(x)
B.append(y)
runs, opaque = [], []
for i in range(h):
e = A[i + 1] if i + 1 < h else B[0]
seg = d[base + A[i]:base + e]
rr, tot = [], 0
for j in range(0, len(seg) - 1, 2):
rr.append((seg[j], seg[j + 1]))
tot += seg[j] + seg[j + 1]
if tot != w:
return None, 'mask row %d sums to %d, not %d' % (i, tot, w)
runs.append(rr)
opaque.append(sum(o for _, o in rr))
need = sum(opaque)
src = d[base + B[0]:r['offset'] + 24 + n]
if not r['flags'] & 0x02:
if len(src) < need * 2:
return None, 'raw color plane short'
words = np.frombuffer(src[:need * 2], dtype='<u2')
r['rleTail'] = len(src) - need * 2
else:
out = np.empty(need, dtype='<u2')
p = got = 0
while got < need:
if p >= len(src):
return None, 'RLE underrun at %d of %d' % (got, need)
c = src[p]
if c < 0x80:
if c == 0 or p + 3 > len(src) or got + c > need:
return None, 'bad RLE run %d at %d' % (c, p)
out[got:got + c] = struct.unpack_from('<H', src, p + 1)[0]
got += c
p += 3
else:
cnt = c - 0x80
if cnt == 0 or p + 1 + 2 * cnt > len(src) or got + cnt > need:
return None, 'bad RLE literal %d at %d' % (cnt, p)
out[got:got + cnt] = np.frombuffer(src, dtype='<u2', count=cnt,
offset=p + 1)
got += cnt
p += 1 + 2 * cnt
words = out
r['rleTail'] = len(src) - p
stated = struct.unpack_from('<I', d, a)[0]
r['sizeCheck'] = stated - (4 + h * 8 + sum(len(x) * 2 for x in runs)
+ 2 * need)
img = np.zeros((h, w, 4), dtype=np.uint8)
k = 0
for y in range(h):
x = 0
for t, o in runs[y]:
x += t
if o:
v = words[k:k + o].astype(np.uint32)
k += o
rc = ((v >> 10) & 0x1F).astype(np.uint8)
gc = ((v >> 5) & 0x1F).astype(np.uint8)
bc = (v & 0x1F).astype(np.uint8)
img[y, x:x + o, 0] = (rc << 3) | (rc >> 2)
img[y, x:x + o, 1] = (gc << 3) | (gc >> 2)
img[y, x:x + o, 2] = (bc << 3) | (bc >> 2)
img[y, x:x + o, 3] = 255
x += o
return img, None
# ---------------------------------------------------------------- app icon
def icon_png(d, spec, path, scale=4):
w, h = spec['w'], spec['h']
stride = (w // 2 + 3) & ~3
img = np.zeros((h, w, 4), dtype=np.uint8)
mstride = ((w + 31) // 32) * 4
for y in range(h):
row = h - 1 - y # DIB rows run bottom-up
for x in range(w):
b = d[spec['xor'] + row * stride + x // 2]
idx = (b >> 4) if x % 2 == 0 else (b & 0x0F)
m = d[spec['and_'] + row * mstride + x // 8]
if not (m >> (7 - x % 8)) & 1:
img[y, x, :3] = WIN16[idx]
img[y, x, 3] = 255
im = Image.fromarray(img, 'RGBA')
im.save(path)
im.resize((w * scale, h * scale), Image.NEAREST).save(
path.replace('.png', '_%dx.png' % scale))
return im
def read_fonts(d, bank):
"""The font bank. Each record is 104 bytes: a 14-byte header, then a
LOGFONT with 16-bit numeric fields, then the face name at +32 and a
style name at +64.
Reading the LOGFONT from the start of the record -- the obvious thing --
makes every font look blank, because the header is in front of it.
"""
out = []
for i in range(bank['count']):
off, size = struct.unpack_from('<II', d, bank['base'] + 4 + i * 8)
a = bank['base'] + off
hgt, wid, esc, ori, wgt = struct.unpack_from('<5h', d, a + 14)
face = d[a + 32:a + 64].split(b'\0')[0].decode('latin1', 'replace')
out.append(dict(index=i, offset=a, size=size, face=face,
height=hgt, weight=wgt, italic=d[a + 24],
underline=d[a + 25], strikeOut=d[a + 26],
charSet=d[a + 27]))
return out
def contact_sheet(items, path, cols=16, cell=64):
rows = (len(items) + cols - 1) // cols
sheet = Image.new('RGBA', (cols * cell, rows * (cell + 12)),
(24, 24, 28, 255))
from PIL import ImageDraw
dr = ImageDraw.Draw(sheet)
for i, (handle, im) in enumerate(items):
cx, cy = (i % cols) * cell, (i // cols) * (cell + 12)
t = im.copy()
t.thumbnail((cell - 4, cell - 4), Image.NEAREST)
sheet.alpha_composite(t, (cx + (cell - t.width) // 2,
cy + (cell - t.height) // 2))
dr.text((cx + 2, cy + cell), str(handle), fill=(180, 180, 190, 255))
sheet.save(path)
def main():
d = P.read(P.GAM)
h = papp.read_header(d)
banks = papp.find_banks(d, h['frameOffsets'][-1])
imgbank = banks[0]
manifest, thumbs, failed = [], [], []
tails, checks = [], []
for i in range(imgbank['count']):
r = read_record(d, imgbank, i)
img, err = decode(d, r)
if img is None:
failed.append(dict(handle=i, reason=err,
graphicMode=r['graphicMode'] if r else None))
continue
im = Image.fromarray(img, 'RGBA')
im.save(os.path.join(P.IMAGES, '%d.png' % i))
thumbs.append((i, im))
tails.append(r.get('rleTail', 0))
checks.append(r.get('sizeCheck', 0))
manifest.append({k: r[k] for k in
('handle', 'w', 'h', 'hotspotX', 'hotspotY',
'actionX', 'actionY', 'graphicMode', 'flags',
'payload', 'references', 'offset')})
print('images: %d decoded, %d empty slots' % (len(manifest), len(failed)))
print(' RLE tail bytes: min %d max %d (one padding byte expected)'
% (min(tails), max(tails)))
print(' uncompressedSize reconciles exactly on %d/%d'
% (sum(1 for c in checks if c == 0), len(checks)))
print(' empty slots: %s' % ', '.join(str(f['handle']) for f in failed))
print(' compressed %d, raw %d'
% (sum(1 for m in manifest if m['flags'] & 2),
sum(1 for m in manifest if not m['flags'] & 2)))
print(' total art: %d px' % sum(m['w'] * m['h'] for m in manifest))
contact_sheet(thumbs, P.out('contact-sheet.png'))
# ---- application icon out of the .gam header
ic = papp.read_icon(d, h['headerEnd'])
if ic:
icon_png(d, ic['big'], os.path.join(P.ICON, 'app-icon-32.png'))
icon_png(d, ic['small'], os.path.join(P.ICON, 'app-icon-16.png'))
print('app icon: 32x32 + 16x16 from .gam offset %#x' % ic['offset'])
# ---- fonts
fonts = []
if len(banks) > 1:
fonts = read_fonts(d, banks[1])
import collections
used = collections.Counter(f['face'] or '(blank)' for f in fonts)
print('fonts: %d records -- %s'
% (len(fonts), ', '.join('%s x%d' % kv for kv in
used.most_common())))
# ---- the other banks
sounds = []
for b in banks[1:]:
for i in range(b['count']):
off, size = struct.unpack_from('<II', d, b['base'] + 4 + i * 8)
rec = d[b['base'] + off:b['base'] + off + size]
import re
nm = re.search(rb'[\x20-\x7e]{3,}\x00', rec)
nm = nm.group()[:-1].decode('latin1') if nm else 'item%d' % i
kind = None
for sig, ext in ((b'MThd', '.mid'), (b'RIFF', '.wav'),
(b'OggS', '.ogg')):
j = rec.find(sig)
if j >= 0:
fn = re.sub(r'[^A-Za-z0-9_.-]', '_', nm) + ext
with open(os.path.join(P.SOUNDS, fn), 'wb') as fh:
fh.write(rec[j:])
kind = fn
break
sounds.append(dict(bank=b['offset'], index=i, name=nm, size=size,
file=kind))
print('bank @%d: %d entries, %s'
% (b['offset'], b['count'],
'fonts' if b['count'] > 2 else
[s['file'] for s in sounds if s['bank'] == b['offset']]))
with open(P.out('images.json'), 'w') as fh:
json.dump(dict(bank=imgbank, images=manifest, emptySlots=failed,
fonts=fonts, otherBanks=sounds), fh, indent=1)
if __name__ == '__main__':
main()
```
### 11.4 `pe.py`
```python
"""pe.py -- a small read-only PE reader.
Enough of the format to pull icons, string tables, dialogs and version
information out of pol.exe, cncs32.dll and the six extension DLLs carved out
of the exe's overlay. No dependencies, no execution, no unpacking: these
files are ordinary uncompressed PE images.
"""
import re
import struct
RT_CURSOR, RT_BITMAP, RT_ICON, RT_MENU, RT_DIALOG = 1, 2, 3, 4, 5
RT_STRING, RT_ACCELERATOR = 6, 9
RT_GROUP_ICON, RT_VERSION = 14, 16
RT_NAME = {1: 'CURSOR', 2: 'BITMAP', 3: 'ICON', 4: 'MENU', 5: 'DIALOG',
6: 'STRING', 7: 'FONTDIR', 8: 'FONT', 9: 'ACCELERATOR',
10: 'RCDATA', 11: 'MESSAGETABLE', 12: 'GROUP_CURSOR',
14: 'GROUP_ICON', 16: 'VERSION', 24: 'MANIFEST'}
class PE(object):
def __init__(self, data, base=0):
self.d = data
self.base = base
pe = struct.unpack_from('<I', data, base + 0x3C)[0] + base
if data[pe:pe + 4] != b'PE\0\0':
raise ValueError('not a PE image')
self.peOffset = pe
(self.machine, self.numSections, self.timeStamp, _sym, _nsym,
self.optSize, self.characteristics) = struct.unpack_from(
'<HHIIIHH', data, pe + 4)
opt = pe + 24
self.magic = struct.unpack_from('<H', data, opt)[0]
self.linker = (data[opt + 2], data[opt + 3])
self.imageBase = struct.unpack_from(
'<I' if self.magic == 0x10B else '<Q', data, opt + 28)[0]
self.sections = []
end = 0
for i in range(self.numSections):
o = opt + self.optSize + i * 40
name = data[o:o + 8].rstrip(b'\0').decode('latin1')
vsize, rva, rsize, raw = struct.unpack_from('<IIII', data, o + 8)
self.sections.append(dict(name=name, vsize=vsize, rva=rva,
rawSize=rsize, rawPtr=raw))
end = max(end, raw + rsize)
self.imageEnd = end
def rva2off(self, rva):
for s in self.sections:
if s['rva'] <= rva < s['rva'] + max(s['vsize'], s['rawSize']):
return self.base + s['rawPtr'] + (rva - s['rva'])
return None
# ------------------------------------------------------------ resources
def _dir(self, off, root):
n_named, n_id = struct.unpack_from('<HH', self.d, off + 12)
out = []
for i in range(n_named + n_id):
e = off + 16 + i * 8
name, data = struct.unpack_from('<II', self.d, e)
if name & 0x80000000:
p = root + (name & 0x7FFFFFFF)
ln = struct.unpack_from('<H', self.d, p)[0]
key = self.d[p + 2:p + 2 + ln * 2].decode('utf-16-le')
else:
key = name
out.append((key, data))
return out
def resources(self):
sec = next((s for s in self.sections if s['name'] == '.rsrc'), None)
if not sec:
return {}
root = self.base + sec['rawPtr']
tree = {}
for t, d1 in self._dir(root, root):
if not d1 & 0x80000000:
continue
tree[t] = {}
for name, d2 in self._dir(root + (d1 & 0x7FFFFFFF), root):
if not d2 & 0x80000000:
continue
for lang, d3 in self._dir(root + (d2 & 0x7FFFFFFF), root):
p = root + d3
rva, size = struct.unpack_from('<II', self.d, p)
off = self.base + sec['rawPtr'] + (rva - sec['rva'])
tree[t].setdefault(name, {})[lang] = (off, size)
return tree
def strings(self):
"""RT_STRING is stored in bundles of 16 length-prefixed UTF-16
strings; bundle `n` covers ids (n-1)*16 .. (n-1)*16+15."""
out = {}
for bundle, langs in self.resources().get(RT_STRING, {}).items():
if not isinstance(bundle, int):
continue
off, size = list(langs.values())[0]
p, end = off, off + size
for i in range(16):
if p + 2 > end:
break
ln = struct.unpack_from('<H', self.d, p)[0]
p += 2
if ln:
out[(bundle - 1) * 16 + i] = self.d[p:p + ln * 2].decode(
'utf-16-le', 'replace')
p += ln * 2
return out
def _vs_node(self, p):
"""One VS_VERSIONINFO node: (length, valueLength, type, key, value
offset). Keys and values are UTF-16 and every field is 4-aligned."""
length, vlen, vtype = struct.unpack_from('<3H', self.d, p)
q = e = p + 6
while struct.unpack_from('<H', self.d, e)[0] != 0:
e += 2
key = self.d[q:e].decode('utf-16-le', 'replace')
val = (e + 2 + 3) & ~3
return length, vlen, vtype, key, val
def version(self):
"""The StringFileInfo block: CompanyName, LegalCopyright and the
rest. This is where an extension DLL says who wrote it."""
res = self.resources().get(RT_VERSION, {})
if not res:
return {}
off, size = list(list(res.values())[0].values())[0]
out = {}
def walk(p, end, depth=0):
while p < end - 6 and depth < 6:
length, vlen, vtype, key, val = self._vs_node(p)
if length < 6 or p + length > end:
return
if vtype == 1 and vlen:
out[key] = self.d[val:val + vlen * 2].decode(
'utf-16-le', 'replace').rstrip('\0')
else:
child = (val + vlen + 3) & ~3 if vlen else val
walk(child, p + length, depth + 1)
p = (p + length + 3) & ~3
walk(off, off + size)
return out
def resource_strings(self, rtype, minrun=3):
"""Every UTF-16 run inside a resource of this type. Dialog and menu
templates interleave text with fixed-size control records; picking the
text out with a scan is crude but does not misreport anything."""
out = {}
for name, langs in self.resources().get(rtype, {}).items():
o, size = list(langs.values())[0]
blob = self.d[o:o + size]
runs = [m.group().decode('utf-16-le', 'replace') for m in
re.finditer(rb'(?:[\x20-\x7e]\x00){%d,}' % minrun, blob)]
out[name] = dict(size=size, strings=runs)
return out
def icons(self):
"""(id, DIB bytes) for every RT_ICON."""
out = []
for name, langs in self.resources().get(RT_ICON, {}).items():
off, size = list(langs.values())[0]
out.append((name, self.d[off:off + size]))
return sorted(out, key=lambda x: str(x[0]))
```
### 11.5 `extensions.py`
```python
"""extensions.py -- the six extension DLLs living inside pol.exe.
`pol.exe` is the Multimedia Fusion stand-alone runtime with an overlay
appended after the last PE section. The overlay is not the game -- the game
is `pol.gam` -- it is the extension pack:
+0 u16 ?
+2 u32 ?
+6 char[] names six NUL-terminated .GOX filenames, back to back
... zero padding
+266 the six DLLs, concatenated, each a complete uncompressed PE image
Nothing is compressed and nothing is encrypted, so the DLLs come straight
out with a byte copy. That matters for reading the event tables: an
extension's conditions, actions and expressions are numbered, not named, in
the .gam, and the names live in the DLL's own STRING resources. MMF's
convention is
28000 + n condition n
28200 + n action n
28400 + n expression n
with a leading "%s" standing for the object the event acts on. Pulling
those out is what turns `moo2.GOX act#12` into readable English.
A .GOX is just a renamed DLL; the four Clickteam objects here are pre-CC2 and
some carry no ACE strings at all, in which case their menus lived in the MMF
editor and those events stay numeric.
"""
import json
import os
import re
import struct
import sys
import polpaths as P
import pe as PEmod
try:
sys.stdout.reconfigure(encoding='utf-8', errors='replace')
except Exception:
pass
ACE_BASE = {'conditions': 28000, 'actions': 28200, 'expressions': 28400}
# object type -> .GOX, in the order the .gam's extension table lists them.
# The mapping is confirmed by the object names in the frames: object type 32
# objects are called "DirectPlay", 35 "PowerApp Lite", 36 "Ini", 37 "MOO2".
OBJECT_TYPE_GOX = {32: 'DPlay.GOX', 33: 'GFEDIT.GOX', 34: 'GFBUTTON.GOX',
35: 'powerapp.GOX', 36: 'GFINI.GOX', 37: 'moo2.GOX'}
def overlay_offset(data):
p = PEmod.PE(data)
return p.imageEnd
def carve(data):
"""Split the overlay into its PE images by locating each MZ/PE pair."""
start = overlay_offset(data)
names = []
p = start + 6
while len(names) < 64:
e = data.find(b'\0', p)
s = data[p:e].decode('latin1')
if not s.lower().endswith('.gox'):
break
names.append(s)
p = e + 1
offs = []
for m in re.finditer(b'MZ', data[start:]):
q = start + m.start()
try:
po = struct.unpack_from('<I', data, q + 0x3C)[0]
except struct.error:
continue
if 0 < po < 0x400 and data[q + po:q + po + 4] == b'PE\0\0':
offs.append(q)
out = []
for i, q in enumerate(offs):
end = offs[i + 1] if i + 1 < len(offs) else len(data)
out.append(dict(name=names[i] if i < len(names) else 'ext%d' % i,
offset=q, size=end - q))
return dict(overlayOffset=start, overlaySize=len(data) - start,
names=names, images=out)
def ace_strings(dll):
p = PEmod.PE(dll)
s = p.strings()
out = {}
for kind, base in ACE_BASE.items():
got = {}
for i in range(0, 200):
if base + i in s:
got[i] = s[base + i]
if got:
out[kind] = got
return out, p
def main():
e = P.read(P.EXE)
info = carve(e)
report = dict(overlayOffset=info['overlayOffset'],
overlaySize=info['overlaySize'],
names=info['names'], extensions=[])
print('overlay at %d, %d bytes, %d names, %d PE images'
% (info['overlayOffset'], info['overlaySize'],
len(info['names']), len(info['images'])))
for img in info['images']:
blob = e[img['offset']:img['offset'] + img['size']]
path = os.path.join(P.EXTENSIONS, img['name'])
with open(path, 'wb') as fh:
fh.write(blob)
try:
aces, p = ace_strings(blob)
except Exception as exc:
print(' %-14s %7d bytes (unreadable: %s)'
% (img['name'], img['size'], exc))
continue
ver = p.version()
rec = dict(name=img['name'], offset=img['offset'], size=img['size'],
timeStamp=p.timeStamp, linker='%d.%d' % p.linker,
sections=[s['name'] for s in p.sections],
version=ver,
aceCounts={k: len(v) for k, v in aces.items()},
ace=aces)
report['extensions'].append(rec)
import datetime
ts = datetime.datetime.fromtimestamp(p.timeStamp,
datetime.timezone.utc)
print(' %-14s %7d bytes linked %s %s'
% (img['name'], img['size'], ts.strftime('%Y-%m-%d %H:%M'),
', '.join('%s %d' % (k, len(v)) for k, v in aces.items())
or 'no ACE strings'))
if ver:
print(' ' + '; '.join('%s=%s' % kv for kv in ver.items()))
with open(P.out('extensions.json'), 'w') as fh:
json.dump(report, fh, indent=1)
print('wrote', P.out('extensions.json'), 'and', P.EXTENSIONS)
if __name__ == '__main__':
main()
```
### 11.6 `frames.py`
```python
"""frames.py -- frame header, object list and object-instance (placement) list.
frame + 0 i16 x, i16 y the frame's window position in the editor
frame + 4 u16 width, height
frame + 8 char[] frame name (fixed field, not cleared on rename)
frame + 130 RGBQUAD[256] palette
frame + 1154 OBJECT LIST
INSTANCE LIST
EVENT LIST (see events.py)
--- OBJECT LIST ----------------------------------------------------------
u16 version always 2
u32 listSize byte length of the whole list, header included
u16 objectCount
u16 ? always 60
then objectCount records:
+0 char[42] name fixed field, NOT cleared on rename, so a
shorter new name leaves a tail of the old
one ("tv\\0kdrop object 4")
+42 u16 objectType 0 Quick backdrop, 1 Backdrop, 2 Active,
3 Text, 7 Counter, 32..37 extension objects,
0x0FFF = a deleted stub the editor left behind
+43 u8 obstacle backdrops only: MMF's Obstacle property
+44 u16 instances how many times this object is placed
+46 u16 handle object handle within the frame
+48 u16 instances same value again
+54 u32 image IMAGE BANK HANDLE, or 0xFFFF for a Quick
backdrop and 0 for objects that own their
artwork through an animation table
+64 u32 objectID application-wide, strictly increasing
+68 optional (u16 0x0000, u32 size, u8[size]) type-specific data
optional (u16 0x8002, u32 0) end marker
A deleted stub (0x0FFF) is 68 bytes and carries neither. The end marker is
also absent on the final record, which ends flush with `listSize`.
The parse is self-validating: every one of the eight frames yields exactly
`objectCount` records ending exactly on `listSize`, and the `instances` field
of every object equals the number of instances the instance list places for
it (see verify.py).
--- ACTIVE OBJECT DATA (the 0x0000 block) --------------------------------
An Active's block holds a "SPRI" sprite section: one record per animation
direction, each
u8 minSpeed, u8 maxSpeed, u16 repeat, u16 backTo, u16 nFrames,
u16 imageHandle[nFrames]
`imageHandle` values are real image-bank indices.
--- INSTANCE LIST --------------------------------------------------------
u16 version always 3
u32 blockSize measured from the slotCount field
u16 slotCount capacity of the editor's slot table
u32 slot[slotCount] byte offset of a live record within the record
array, or 0xFFFFFFFF for a slot whose instance was
deleted while building the map
u8 record[live][32]
blockSize - 2 - slotCount*4 == live*32 exactly, no remainder
the live slot values are a permutation of 0, 32, 64, ... (live-1)*32
record:
+0 u16 objIndex index into the frame's object list
+2 i16 x top-left corner, frame coordinates
+4 i16 y negative = parked off-frame
+6 u8 objType same encoding as the object record's +42
+7 u8 flags bit 0 = Obstacle
+8 u16 imageRef per-frame image reference (see below)
+16 u8[4] color1 Quick backdrop gradient color A
+20 u8[4] color2 Quick backdrop gradient color B
+26 u16 width Quick backdrop / Text display box
+28 u16 height
+30 u16 0xFFFF end-of-record marker
The RECORD ARRAY order is the draw order, back to front. The slot table
is the editor's edit-order index and rendering by it is visibly wrong.
`imageRef` is a per-frame image reference whose numbering does not match
the image bank; it is left undecoded because it is not needed -- the
object record's own `image` field at +54 gives the bank handle directly.
"""
import json
import re
import struct
import sys
import polpaths as P
import papp
try:
sys.stdout.reconfigure(encoding='utf-8', errors='replace')
except Exception:
pass
PALETTE = 130
PALETTE_LEN = 1024
OBJECT_LIST = PALETTE + PALETTE_LEN # 1154
DELETED = 0x0FFF
BLOCK_DATA = 0x0000
BLOCK_END = 0x8002
TYPE = {
0: 'Quick backdrop', 1: 'Backdrop', 2: 'Active', 3: 'Text',
4: 'Question', 5: 'Score', 6: 'Lives', 7: 'Counter',
32: 'DirectPlay', 33: 'Edit', 34: 'Button', 35: 'PowerApp Lite',
36: 'INI', 37: 'MOO2', DELETED: '(deleted)',
}
def read_palette(d, frame):
p = frame['offset'] + PALETTE
return [tuple(d[p + i * 4:p + i * 4 + 3]) for i in range(256)]
# ---------------------------------------------------------------- objects
def read_object_list(d, frame):
base = frame['offset'] + OBJECT_LIST
ver, size, count, tail = struct.unpack_from('<HIHH', d, base)
end = base + size
p = base + 10
objs = []
for i in range(count):
start = p
raw = d[p:p + 42]
name = raw.split(b'\0')[0].decode('latin1', 'replace')
otype = struct.unpack_from('<H', d, p + 42)[0]
if otype != DELETED:
otype, obstacle = otype & 0xFF, d[p + 43]
else:
obstacle = 0
ninst, handle, ninst2 = struct.unpack_from('<3H', d, p + 44)
image = struct.unpack_from('<I', d, p + 54)[0]
oid = struct.unpack_from('<I', d, p + 64)[0]
q = p + 68
data = None
if otype != DELETED and q + 6 <= end:
tag, blen = struct.unpack_from('<HI', d, q)
if tag == BLOCK_DATA and 0 < blen <= end - q:
data = (q + 6, blen)
q += 6 + blen
if q + 6 <= end:
tag, blen = struct.unpack_from('<HI', d, q)
if tag == BLOCK_END and blen == 0:
q += 6
# the name field is fixed-size and is not cleared on rename, so
# the tail of the previous name survives after the terminator --
# which is how the original generic names, and with them the order
# the author created the objects, are still recoverable
stale = raw[len(name) + 1:]
m = re.match(rb'[\x20-\x7e]{2,}', stale)
objs.append(dict(index=i, offset=start, name=name,
staleName=m.group().decode('latin1') if m else None,
objectType=otype,
typeName=TYPE.get(otype, 'type %d' % otype),
obstacle=bool(obstacle) if otype in (0, 1) else None,
instanceCount=ninst, handle=handle,
image=None if image in (0, 0xFFFF) else image,
objectID=oid,
dataOffset=data[0] if data else None,
dataSize=data[1] if data else 0,
recordLength=q - start,
deleted=otype == DELETED))
p = q
return objs, dict(offset=base, version=ver, size=size, count=count,
tail=tail, end=end, consumed=p - base, exact=p == end)
def sprite_frames(d, obj):
"""Every animation-direction frame list inside an Active's data block."""
if not obj['dataOffset']:
return []
lo, hi = obj['dataOffset'], obj['dataOffset'] + obj['dataSize']
i = d.find(b'SPRI', lo, hi)
if i < 0:
return []
out, p = [], i + 4
while p + 10 <= hi:
mn, mx = d[p], d[p + 1]
rep, back, n = struct.unpack_from('<HHH', d, p + 2)
if (1 <= mn <= 100 and 1 <= mx <= 100 and rep <= 4 and back <= 64
and 1 <= n <= 16 and p + 8 + 2 * n <= hi):
fr = struct.unpack_from('<%dH' % n, d, p + 8)
if all(0 < hh < 4000 for hh in fr):
out.append(dict(minSpeed=mn, maxSpeed=mx, repeat=rep,
backTo=back, frames=list(fr)))
p += 8 + 2 * n
continue
p += 1
return out
def counter_digits(d, obj):
"""A Counter's data block carries a "CNTR" section ending in the list of
image-bank handles it uses for its digits:
'CNTR' u32 ? ... u16 ? u16 ? u16 ? u16 count u16 handle[count]
Without this the digit artwork looks unreferenced, which is how a bank
ends up wrongly described as holding two unused fonts."""
if not obj['dataOffset']:
return []
lo, hi = obj['dataOffset'], obj['dataOffset'] + obj['dataSize']
i = d.find(b'CNTR', lo, hi)
if i < 0:
return []
for p in range(i + 4, hi - 2):
n = struct.unpack_from('<H', d, p)[0]
# the handle list is the last thing in the block, so it ends flush
if 2 <= n <= 16 and p + 2 + 2 * n == hi:
hs = struct.unpack_from('<%dH' % n, d, p + 2)
if all(0 < x < 4000 for x in hs):
return list(hs)
return []
def paragraphs(d, obj):
"""A Text object's paragraphs, out of its "TEXT" section.
'TEXT'
u16 size of everything after this field
u16 5 constant
u16 count how many paragraphs
u16 headerLen the first paragraph's text starts at TEXT+14+this
... per-paragraph formatting
char[] text NUL-terminated
then, for each further paragraph:
u8[10] a length, a color and a flags word
char[] text NUL-terminated
Scanning the block for printable runs instead -- the obvious shortcut --
silently drops every short paragraph, which here means most of the ID
card ("Age:", "AIM:", "Male", "X") and half the organizer's menu. The
count in the header is what says how many there should be, and it is
what this is checked against.
"""
if not obj['dataOffset']:
return []
lo, hi = obj['dataOffset'], obj['dataOffset'] + obj['dataSize']
i = d.find(b'TEXT', lo, hi)
if i < 0:
return []
count, hdr = struct.unpack_from('<HH', d, i + 8)
out = []
p = i + 14 + hdr
for n in range(min(count, 64)):
if n:
p += 10
if p >= hi:
break
e = d.find(bytes(1), p, hi)
if e < 0:
break
out.append(d[p:e].decode('latin1'))
p = e + 1
return out
def paragraph_count(d, obj):
"""What the block says it holds, for checking against what came out."""
if not obj['dataOffset']:
return 0
lo, hi = obj['dataOffset'], obj['dataOffset'] + obj['dataSize']
i = d.find(b'TEXT', lo, hi)
return struct.unpack_from('<H', d, i + 8)[0] if i >= 0 else 0
# ---------------------------------------------------------------- instances
def find_instance_block(d, lo, hi):
for p in range(lo, hi - 8):
size = struct.unpack_from('<I', d, p + 2)[0]
slots = struct.unpack_from('<H', d, p + 6)[0]
if not (0 < slots <= 20000) or not (8 <= size <= hi - p):
continue
rem = size - 2 - slots * 4
if rem < 0 or rem % 32:
continue
live = rem // 32
idx = struct.unpack_from('<%dI' % slots, d, p + 8)
vals = [v for v in idx if v != 0xFFFFFFFF]
if (len(vals) == live and all(v % 32 == 0 for v in vals)
and set(v // 32 for v in vals) == set(range(live))):
return dict(offset=p, size=size, slots=slots, live=live,
order=[v // 32 for v in vals],
recordsAt=p + 8 + slots * 4,
end=p + 6 + size)
return None
def read_instances(d, blk):
base = blk['recordsAt']
recs = []
for i in range(blk['live']):
a = base + i * 32
oi, x, y, ty, fl, ref = struct.unpack_from('<HhhBBH', d, a)
w, h = struct.unpack_from('<HH', d, a + 26)
r = dict(z=i, objIndex=oi, x=x, y=y, objType=ty, flags=fl,
obstacle=bool(fl & 1), imageRef=ref,
endMarker=struct.unpack_from('<H', d, a + 30)[0])
if ty == 0:
r['color1'] = list(d[a + 16:a + 20])
r['color2'] = list(d[a + 20:a + 24])
if ty in (0, 3) and w and h:
r['w'], r['h'] = w, h
recs.append(r)
for slot, k in enumerate(blk['order']):
recs[k]['slotOrder'] = slot
return recs
# ---------------------------------------------------------------- driver
def decode(d, header):
frames = papp.read_frames(d, header)
out = []
for i, f in enumerate(frames):
objs, meta = read_object_list(d, f)
for o in objs:
o['animations'] = sprite_frames(d, o)
o['counterDigits'] = counter_digits(d, o)
o['paragraphs'] = paragraphs(d, o)
o['paragraphCount'] = paragraph_count(d, o)
hi = (frames[i + 1]['offset'] if i + 1 < len(frames)
else header.get('imageBankOffset', len(d)))
blk = find_instance_block(d, meta['end'], hi)
inst = read_instances(d, blk) if blk else []
for r in inst:
o = objs[r['objIndex']] if r['objIndex'] < len(objs) else None
r['name'] = o['name'] if o else '?'
r['typeName'] = TYPE.get(r['objType'], 'type %d' % r['objType'])
r['image'] = o['image'] if o else None
out.append(dict(frame=f, objectList=meta, objects=objs,
instanceBlock=blk, instances=inst))
return out
def main():
d = P.read(P.GAM)
h = papp.read_header(d)
banks = papp.find_banks(d, h['frameOffsets'][-1])
h['imageBankOffset'] = banks[0]['offset'] if banks else len(d)
out = decode(d, h)
bad = 0
for fr in out:
f, m = fr['frame'], fr['objectList']
counted = {}
for r in fr['instances']:
counted[r['objIndex']] = counted.get(r['objIndex'], 0) + 1
mism = [o['name'] for o in fr['objects']
if not o['deleted'] and o['instanceCount'] != counted.get(o['index'], 0)]
para = [o['name'] for o in fr['objects']
if o.get('paragraphCount') != len(o.get('paragraphs') or [])]
bad += len(mism) + (0 if m['exact'] else 1)
print('%-12s %4dx%-4d objects %2d/%-2d %s instances %4d '
'count-check %s'
% (f['name'], f['w'], f['h'], len(fr['objects']), m['count'],
'exact' if m['exact'] else 'OVERRUN',
len(fr['instances']),
'ok' if not mism else 'MISMATCH %s' % mism))
if para:
print('%-12s paragraph-count MISMATCH %s' % ('', para))
print('---', 'all frames validate' if not bad else '%d problems' % bad)
with open(P.out('frames.json'), 'w') as fh:
json.dump(out, fh, indent=1)
if __name__ == '__main__':
main()
```
### 11.7 `events.py`
```python
"""events.py -- the program logic.
The event tables are the part of a PAPP file that carries the program itself.
CTFAK cannot open the container at all, so without this the only way to say
what the game does is to watch it run.
They sit at the end of each frame, immediately after the object instance list,
in a chunk of their own:
u16 0x8004
u32 size everything below
u32 groupBytes byte length of the event-group list
... event groups terminated by an i16 0 where a group size
would be
u8[36] event-system limits, identical in every frame,
so application-level rather than per-frame
[global-object table] present on five of the eight frames -- see
global_objects() below
The group / condition / action / parameter structures themselves are the
ordinary Multimedia Fusion 1.5 ones, which is the useful discovery here: the
flat PAPP container and the chunked PAME container disagree about how to
*store* the event tables but agree completely about what an event IS. So
Mathias Kaerlev's mmfparser structures, written for PAME, decode PAPP events
byte for byte once you find them. The layouts below are his; the opcode name
tables in ace-names.json are transcribed from mmfparser too.
group:
i16 -size NEGATIVE byte length, including this field
u8 conditionCount
u8 actionCount
u16 flags
i16 restrict
i16 restrictCpt
i16 identifier
i16 undo
condition[conditionCount]
action[actionCount]
condition:
u16 size
i8 objectType <0 system, 0..9 built-in object, >=32 extension
i8 num opcode; for objectType > 2 it is stored +32
u16 objectInfo which object in the frame
i16 objectInfoList
u8 flags, u8 otherFlags, u8 paramCount, u8 defType
i16 identifier
parameter[paramCount]
action: same, minus the trailing identifier, and `num` is stored -32
for objectType > 2
parameter:
u16 size
u16 code see PARAM_NAMES
u8[] payload
A "jump to frame" action's parameter is an index into the application's
frame-handle table (papp.read_header -> frameHandles), NOT a frame index.
Verification, before trusting any of it: every one of the eight frames parses
with the group list ending exactly on `groupBytes`, every group's declared
size matching the bytes its conditions and actions actually consume, and
every parameter ending exactly on its declared size. Two independent
sanity checks on the *meaning*: the title frame's three "go to next frame"
groups fire on a mouse click inside (259,151)-(447,172), on Enter and on
Space -- which is exactly what the title screen offers -- and the readme
frame's single group is "if Enter pressed, jump to frame handle 7", which
resolves through the handle table to frame 0, the title. Both match the
running game.
"""
import json
import os
import struct
import sys
import polpaths as P
import papp
import frames as F
try:
sys.stdout.reconfigure(encoding='utf-8', errors='replace')
except Exception:
pass
HERE = os.path.dirname(os.path.abspath(__file__))
EVENT_CHUNK = 0x8004
HEADER_LEN = 36
# Extension ACEs are numbered, not named, in the .gam. The names come out of
# each .GOX's own STRING resources -- see extensions.py. Object types 32..37
# are the six extensions in the order the .gam's extension table lists them,
# which the object names in the frames confirm one for one (32 "DirectPlay",
# 35 "PowerApp Lite", 36 "Ini", 37 "MOO2").
EXT_BY_TYPE = {32: 'DPlay.GOX', 33: 'GFEDIT.GOX', 34: 'GFBUTTON.GOX',
35: 'powerapp.GOX', 36: 'GFINI.GOX', 37: 'moo2.GOX'}
EXT_ACE = {}
try:
_x = json.load(open(P.out('extensions.json')))
for _e in _x['extensions']:
EXT_ACE[_e['name']] = _e['ace']
except Exception:
pass
def ext_ace(gox, kind, index):
"""An extension ACE label. The stored string carries a two-digit prefix
(object placeholders, then parameter count) which is stripped here."""
tbl = EXT_ACE.get(gox, {}).get(kind, {})
s = tbl.get(str(index))
if s is None:
return None
if len(s) > 2 and s[0].isdigit() and s[1].isdigit():
s = s[2:]
return s.replace('%s', '?').strip()
_ace = json.load(open(os.path.join(HERE, 'ace-names.json')))
COND_SYS = {int(k): {int(a): b for a, b in v.items()}
for k, v in _ace['conditions']['system'].items()}
COND_EXT = {int(k): v for k, v in _ace['conditions']['extension'].items()}
ACT_SYS = {int(k): {int(a): b for a, b in v.items()}
for k, v in _ace['actions']['system'].items()}
ACT_EXT = {int(k): v for k, v in _ace['actions']['extension'].items()}
# Parameter type names, transcribed from mmfparser's
# chunkloaders/parameters/names.py, including its spellings.
PARAM_NAMES = {
1: 'OBJECT', 2: 'TIME', 3: 'SHORT', 4: 'SHORT', 5: 'INT', 6: 'SAMPLE',
7: 'SAMPLE', 9: 'CREATE', 10: 'SHORT', 11: 'SHORT', 12: 'SHORT',
13: 'Every', 14: 'KEY', 15: 'EXPRESSION', 16: 'POSITION',
17: 'JOYDIRECTION', 18: 'SHOOT', 19: 'ZONE', 21: 'SYSCREATE',
22: 'EXPRESSION', 23: 'COMPARISON', 24: 'COLOR', 25: 'BUFFER4',
26: 'FRAME', 27: 'SAMLOOP', 28: 'MUSLOOP', 29: 'NEWDIRECTION',
31: 'TEXTNUMBER', 32: 'Click', 33: 'PROGRAM', 34: 'OLDPARAM_VARGLO',
35: 'CNDSAMPLE', 36: 'CNDMUSIC', 37: 'REMARK', 38: 'GROUP',
39: 'GROUPOINTER', 40: 'FILENAME', 41: 'STRING', 42: 'CMPTIME',
43: 'PASTE', 44: 'VMKEY', 45: 'EXPSTRING', 46: 'CMPSTRING',
47: 'INKEFFECT', 48: 'MENU', 49: 'GlobalValue', 50: 'AlterableValue',
51: 'FLAG', 52: 'VARGLOBAL_EXP', 53: 'AlterableValueExpression',
54: 'FLAG_EXP', 55: 'EXTENSION', 56: '8DIRECTIONS', 57: 'MVT',
58: 'GlobalString', 59: 'STRINGGLOBAL_EXP', 60: 'PROGRAM2',
61: 'ALTSTRING', 62: 'ALTSTRING_EXP', 63: 'FILENAME',
64: 'FASTLOOPNAME',
}
EXPR_SYSTEM = {
0: {0: 'End', 2: '+', 4: '-', 6: '*', 8: '/', 10: '%', 12: '^',
14: '&', 16: '|', 18: '~'},
-1: {0: 'Long', 1: 'Random', 2: 'GlobalValueExpression', 3: 'String',
4: 'ToString', 5: 'ToNumber', 6: 'ApplicationDrive',
7: 'ApplicationDirectory', 8: 'ApplicationPath',
9: 'ApplicationFilename', 10: 'Sin', 11: 'Cos', 12: 'Tan',
13: 'SquareRoot', 14: 'Log', 15: 'Ln', 16: 'Hex', 17: 'Bin',
18: 'Exp', 19: 'LeftString', 20: 'RightString', 21: 'MidString',
22: 'StringLength', 23: 'Double', 24: 'GlobalValue', 28: 'ToInt',
29: 'Abs', 30: 'Ceil', 31: 'Floor', 35: 'Not', 40: 'Min',
41: 'Max', 46: 'LoopIndex', 47: 'NewLine', 48: 'Round',
49: 'GlobalStringExpression', 50: 'GlobalString',
51: 'LowerString', 52: 'UpperString', 53: 'Find',
54: 'ReverseFind', 55: 'GetClipboard',
-1: '(', -2: ')', -3: ','},
-3: {0: 'CurrentFrameOld', 1: 'PlayerCount', 2: 'XLeftFrame',
3: 'XRightFrame', 4: 'YTopFrame', 5: 'YBottomFrame',
6: 'FrameWidth', 7: 'FrameHeight', 8: 'CurrentFrame',
10: 'FrameRate'},
-4: {0: 'TimerValue', 1: 'TimerHundreds', 2: 'TimerSeconds',
3: 'TimerHours', 4: 'TimerMinutes'},
-6: {0: 'XMouse', 1: 'YMouse'},
-7: {0: 'PlayerScore', 1: 'PlayerLives', 2: 'PlayerInputDevice',
3: 'PlayerKeyName', 4: 'PlayerName'},
2: {80: 'GetColorAt', 81: 'GetXScale', 82: 'GetYScale', 83: 'GetAngle'},
3: {80: 'CurrentParagraphIndex', 81: 'CurrentText', 82: 'GetParagraph',
83: 'TextAsNumber', 84: 'ParagraphCount'},
7: {80: 'CounterValue', 81: 'CounterMinimumValue',
82: 'CounterMaximumValue'},
}
EXPR_EXTENSION = {
0: 'XPosition', 1: 'YPosition', 2: 'AnimationFrame', 3: 'Speed',
4: 'Acceleration', 5: 'Deceleration', 6: 'Direction', 7: 'AlterableValue',
8: 'XLeft', 9: 'XRight', 10: 'XTop', 11: 'XBottom', 12: 'YLeft',
13: 'YRight', 14: 'YTop', 15: 'YBottom', 16: 'NumberOfObjects',
17: 'IdentifierOfObject', 18: 'AlterableString', 19: 'Layer',
20: 'FlagValue', 21: 'AngleOfObject', 22: 'XScale', 23: 'YScale',
}
# Built-in objects index their expressions directly in MMF 1.5. 0 and 1 are
# pinned by the two CenterDisplay actions, which take the X then the Y of the
# same object; the rest follow MMF's documented order.
EXPRESSION_NAMES = {
2: {0: 'XPosition', 1: 'YPosition', 2: 'AnimationFrame', 3: 'Speed',
6: 'Direction', 7: 'ValueA', 8: 'ValueB', 9: 'ValueC'},
7: {0: 'CounterValue', 1: 'CounterMinimum', 2: 'CounterMaximum'},
3: {0: 'CurrentText'},
}
def expression_name(object_type, num):
t = EXPRESSION_NAMES.get(object_type)
if t and num in t:
return t[num]
return 'expr%d' % num
OBJ_TYPE_LABEL = {-7: 'Player', -6: 'Mouse/Keyboard', -5: 'Create',
-4: 'Timer', -3: 'Game', -2: 'Speaker', -1: 'System',
0: 'Quick backdrop', 1: 'Backdrop', 2: 'Active', 3: 'Text',
4: 'Question', 5: 'Score', 6: 'Lives', 7: 'Counter',
8: 'RTF', 9: 'Sub-application'}
class R(object):
def __init__(self, d, p=0):
self.d, self.p = d, p
def read(self, n):
v = self.d[self.p:self.p + n]
self.p += n
return v
def u8(self):
v = self.d[self.p]
self.p += 1
return v
def i8(self):
v = self.d[self.p]
self.p += 1
return v - 256 if v > 127 else v
def u16(self):
v = struct.unpack_from('<H', self.d, self.p)[0]
self.p += 2
return v
def i16(self):
v = struct.unpack_from('<h', self.d, self.p)[0]
self.p += 2
return v
def u32(self):
v = struct.unpack_from('<I', self.d, self.p)[0]
self.p += 4
return v
DIRECTION_NAMES = {0: 'right', 4: 'up-right', 8: 'up', 12: 'up-left',
16: 'left', 20: 'down-left', 24: 'down', 28: 'down-right'}
def directions(mask):
"""NEWDIRECTION is a 32-bit mask over MMF's 32 directions, numbered
counter-clockwise from 0 = right, so 8 = up, 16 = left, 24 = down."""
bits = [i for i in range(32) if mask >> i & 1]
if not bits:
return 'none'
return ' '.join(DIRECTION_NAMES.get(b, 'dir%d' % b) for b in bits)
def parse_expression_list(r):
"""PAPP expression tokens are
i8 objectType
i8 num
u16 size total token length including these four bytes
u8[size-4] payload
terminated by a bare (0, 0) with no size word. This is NOT the layout
mmfparser uses for PAME, which is where a naive port goes wrong: read it
as (i16 type, u16 num, u16 size) and every expression decodes to
nonsense. Built-in objects index their expressions directly here --
0 = X position, 1 = Y position -- rather than from 80 as in MMF2, and
the payload is (u16 objectInfo, u16 objectInfoList).
"""
out = []
guard = 0
while guard < 4096:
guard += 1
start = r.p
object_type = r.i8()
num = r.i8()
if object_type == 0 and num == 0:
out.append(dict(name='End'))
break
size = r.u16()
if size < 4:
out.append(dict(name='?'))
break
payload = r.d[start + 4:start + size]
tok = dict(objectType=object_type, num=num)
name = None
table = EXPR_SYSTEM.get(object_type)
if table and num in table:
name = table[num]
if object_type == -1 and num == 0:
tok['value'] = int.from_bytes(payload[:4], 'little', signed=True)
name = str(tok['value'])
elif object_type == -1 and num in (3, 50):
name = '"%s"' % payload.split(b'\x00')[0].decode('latin1')
elif object_type >= 0 and name is None:
tok['objectInfo'] = int.from_bytes(payload[:2], 'little')
name = 'obj%d.%s' % (tok['objectInfo'],
expression_name(object_type, num))
if name is None:
name = 'expr(%d,%d)' % (object_type, num)
tok['name'] = name
out.append(tok)
r.p = start + size
return out
def parse_parameter(r):
start = r.p
size = r.u16()
code = r.u16()
name = PARAM_NAMES.get(code, 'p%d' % code)
payload = r.d[start + 4:start + size]
if name in ('EXPRESSION', 'EXPSTRING', 'CMPSTRING', 'COMPARISON',
'VARGLOBAL_EXP', 'STRINGGLOBAL_EXP',
'AlterableValueExpression', 'FLAG_EXP', 'ALTSTRING_EXP'):
rr = R(r.d, start + 4)
comparison = rr.i16()
toks = parse_expression_list(rr)
val = {'comparison': comparison,
'expr': ' '.join(t['name'] for t in toks[:-1])}
elif name == 'REMARK':
# An event-editor comment line. It is NOT a bare string: it carries
# the font it is drawn in first, so reading it as text yields the
# LOGFONT's bytes and looks like mojibake.
val = {'height': int.from_bytes(payload[0:2], 'little', signed=True),
'weight': int.from_bytes(payload[8:10], 'little'),
'font': payload[18:50].split(b'\0')[0].decode('latin1'),
'color': list(payload[54:57]),
'text': payload[58:].split(b'\0')[0].decode('latin1')}
elif name in ('STRING', 'FILENAME'):
val = payload.split(b'\0')[0].decode('latin1')
elif name == 'NEWDIRECTION':
mask = int.from_bytes(payload[:4], 'little')
val = {'mask': mask, 'directions': directions(mask)}
elif name == 'SHORT':
# an animation selector: index plus the animation's own name
val = {'index': int.from_bytes(payload[:2], 'little', signed=True)}
nm = payload[2:].split(b'\x00')[0]
if nm and all(32 <= c < 127 for c in nm):
val['name'] = nm.decode('latin1')
elif name in ('TEXTNUMBER', 'FRAME', 'MENU', 'GROUPOINTER',
'GlobalValue', 'AlterableValue', 'FLAG', 'GlobalString',
'MVT', '8DIRECTIONS'):
val = int.from_bytes(payload[:2], 'little', signed=True)
elif code == 30:
# Text objects: where to draw the text, and which paragraph of it.
# Same head as POSITION -- parent object, flags, offset -- then a
# paragraph index. Two fields in the middle stay unidentified.
val = {'objectInfoParent': int.from_bytes(payload[0:2], 'little'),
'flags': int.from_bytes(payload[2:4], 'little'),
'x': int.from_bytes(payload[4:6], 'little', signed=True),
'y': int.from_bytes(payload[6:8], 'little', signed=True),
'unknown': int.from_bytes(payload[22:24], 'little'),
'paragraph': int.from_bytes(payload[24:26], 'little')}
elif name == 'INT':
val = int.from_bytes(payload[:4], 'little', signed=True)
elif name == 'OBJECT':
val = {'objectInfoList': int.from_bytes(payload[:2], 'little',
signed=True),
'objectInfo': int.from_bytes(payload[2:4], 'little'),
'objectType': int.from_bytes(payload[4:6], 'little',
signed=True)}
elif name == 'ZONE':
val = {'x1': int.from_bytes(payload[0:2], 'little', signed=True),
'y1': int.from_bytes(payload[2:4], 'little', signed=True),
'x2': int.from_bytes(payload[4:6], 'little', signed=True),
'y2': int.from_bytes(payload[6:8], 'little', signed=True)}
elif name in ('POSITION', 'CREATE', 'SYSCREATE', 'SHOOT'):
pos = {'objectInfoParent': int.from_bytes(payload[:2], 'little'),
'flags': int.from_bytes(payload[2:4], 'little'),
'x': int.from_bytes(payload[4:6], 'little', signed=True),
'y': int.from_bytes(payload[6:8], 'little', signed=True),
'direction': int.from_bytes(payload[12:16], 'little',
signed=True)}
val = (pos if name == 'POSITION' else
dict(pos, objectInstance=int.from_bytes(payload[22:24],
'little'),
objectInfo=int.from_bytes(payload[24:26], 'little')))
elif name == 'Every':
val = {'delayMs': int.from_bytes(payload[:4], 'little', signed=True)}
elif name == 'Click':
val = {'button': payload[0] if payload else 0,
'double': bool(payload[1]) if len(payload) > 1 else False}
elif name == 'TIME':
val = int.from_bytes(payload[:4], 'little', signed=True)
elif name in ('SAMPLE', 'CNDSAMPLE', 'CNDMUSIC'):
val = {'handle': int.from_bytes(payload[:4], 'little'),
'name': payload[4:].split(b'\0')[0].decode('latin1')}
elif name == 'GROUP':
val = {'flags': int.from_bytes(payload[:2], 'little'),
'id': int.from_bytes(payload[2:4], 'little'),
'name': payload[4:].split(b'\0')[0].decode('latin1')}
elif name in ('KEY', 'VMKEY'):
val = {'vk': int.from_bytes(payload[:2], 'little')}
else:
val = payload.hex()
r.p = start + size
return dict(code=code, name=name, value=val, size=size)
def parse_condition(r):
start = r.p
size = r.u16()
object_type = r.i8()
num = raw_num = r.i8()
if object_type > 2:
num -= 32
object_info = r.u16()
object_info_list = r.i16()
flags, other_flags, nparams, def_type = (r.u8(), r.u8(), r.u8(), r.u8())
identifier = r.i16()
params = [parse_parameter(r) for _ in range(nparams)]
over = r.p - (start + size)
r.p = start + size
return dict(kind='condition', objectType=object_type, num=num,
numRaw=raw_num,
objectInfo=object_info, objectInfoList=object_info_list,
flags=flags, otherFlags=other_flags, identifier=identifier,
params=params, sizeError=over)
def parse_action(r):
start = r.p
size = r.u16()
object_type = r.i8()
num = raw_num = r.u8()
if object_type > 2:
num += 32
object_info = r.u16()
object_info_list = r.i16()
flags, other_flags, nparams, def_type = (r.u8(), r.u8(), r.u8(), r.u8())
params = [parse_parameter(r) for _ in range(nparams)]
over = r.p - (start + size)
r.p = start + size
return dict(kind='action', objectType=object_type, num=num,
numRaw=raw_num, objectInfo=object_info, objectInfoList=object_info_list,
flags=flags, otherFlags=other_flags, params=params,
sizeError=over)
def parse_group(r):
start = r.p
size = -r.i16()
nconds, nacts = r.u8(), r.u8()
flags = r.u16()
restrict, restrict_cpt, identifier, undo = (r.i16(), r.i16(),
r.i16(), r.i16())
conds = [parse_condition(r) for _ in range(nconds)]
acts = [parse_action(r) for _ in range(nacts)]
over = r.p - (start + size)
r.p = start + size
return dict(offset=start, size=size, flags=flags, identifier=identifier,
restrict=restrict, conditions=conds, actions=acts,
sizeError=over)
GLOBAL_RECORD = 104
def global_objects(d, q, end):
"""After the event-system limits, five of the eight frames carry a table
of the objects they share with the application.
u32 size of the whole table, this field included
u16 ? a count
u16 count how many records follow
u16 0
u16 offset[count] each relative to the start of the table
... 1016 bytes of zero
record[count] 104 bytes each
A record is a marker, a couple of small numbers, then the object written
as "name:fs64-@:type", and then a long tail of whatever happened to be in
the editor's memory -- see inventory.py, which counts what leaked.
"""
if q + 12 > end:
return None
size = struct.unpack_from('<I', d, q)[0]
if size < 16 or q + size > end + 4:
return None
a, count, _z = struct.unpack_from('<3H', d, q + 4)
if not (0 < count <= 64) or q + 10 + count * 2 > end:
return None
recs = []
for i in range(count):
off = struct.unpack_from('<H', d, q + 10 + i * 2)[0]
r = q + off
if r + GLOBAL_RECORD > end + 4:
continue
raw = d[r + 12:r + 60].split(b'\0')[0]
txt = raw.decode('latin1', 'replace')
if ':fs64-@:' not in txt:
# nothing was written here; what is in the field is whatever the
# editor's buffer still held
txt = '(uninitialized)'
name, _, kind = txt.partition(':fs64-@:')
blob = d[r:r + GLOBAL_RECORD]
recs.append(dict(offset=r, text=txt, name=name or None,
kind=kind or None,
nonZero=sum(1 for c in blob if c)))
return dict(offset=q, size=size, count=count, records=recs)
def parse_events(d, p):
"""p is the offset of the 0x8004 chunk."""
tag, size = struct.unpack_from('<HI', d, p)
assert tag == EVENT_CHUNK, 'not an event chunk: %#06x' % tag
body = p + 6
group_bytes = struct.unpack_from('<I', d, body)[0]
r = R(d, body + 4)
end = body + 4 + group_bytes
groups = []
while r.p < end:
if struct.unpack_from('<h', d, r.p)[0] >= 0: # 0 terminator / not a group
r.p += 2
break
groups.append(parse_group(r))
limits = list(struct.unpack_from('<18H', d, end))
globals_ = global_objects(d, end + HEADER_LEN, p + 6 + size)
return dict(offset=p, size=size, groupBytes=group_bytes,
groupsEnd=r.p, exact=r.p == end or r.p == end - 0,
trailerBytes=size - 4 - group_bytes,
limits=limits, globalObjects=globals_, groups=groups)
def find_app_events(d, header):
"""There is a SECOND event table, at application level, in the header --
and it is the one that matters for the menu. None of the eight frames
contains a single "menu item selected" condition, so on the frame tables
alone the whole custom Other menu looks dead. It is not: the application
table handles it.
It is not in a 0x8004 chunk and has no signature, so it is located by
parsing: try every offset between the icon and the extension table, and
keep the one whose group chain runs cleanly all the way to the extension
table. Only one offset in that range does.
"""
exts = papp.read_extensions(d)
end = exts[0]['offset'] - 8 if exts else 0x2000
best = None
for start in range(0x400, min(0xA10, end)):
r = R(d, start)
groups = []
try:
while r.p < end:
if struct.unpack_from('<h', d, r.p)[0] >= 0:
break
g = parse_group(r)
if g['sizeError'] or g['size'] < 14:
raise ValueError
groups.append(g)
except Exception:
continue
if groups and abs(r.p - end) < 24 and (best is None
or len(groups) > len(best[1])):
best = (start, groups, r.p)
if not best:
return None
return dict(offset=best[0], groupsEnd=best[2], groups=best[1])
# ---------------------------------------------------------------- naming
# MMF 1.5's BUILT-IN object conditions and actions are NOT numbered the way
# MMF2's are, so mmfparser's tables mis-name them. (Its SYSTEM tables --
# objectType < 0 -- are fine, and are used unchanged above.) The entries
# below were derived from this file alone, each from the parameter types the
# opcode carries plus what the game is known to do at that moment:
#
# act 2 NEWDIRECTION, 16 uses, one per arrow-key group -> set direction
# act 3 POSITION, parent object + offset -> set position
# act 4/5 EXPRESSION; 4 fires on Left/Right, 5 on Up/Down -> set X / set Y
# act 18 animation selector whose payload carries the animation's own
# name, "Walking" when a key is held and "Stopped" when none is
# -> force animation
# act 1 applied at start of frame to the two invisible helper objects
# (Solid, the exit mat), and paired with act 0 in the Menu frame
# -> hide / show
# act 24 destroys the remote-player prototype and the hover label
# -> destroy
# act 27 written straight after a remote player is created, from the
# network object's client id; cond -19 compares it back
# act 37 written 1 by the radio-on group and 0 by the radio-off group;
# cond -25 compares it back
# cnd -8 NEWDIRECTION, mirrors act 2 -> is facing
# cnd -17 no parameters, and every use is immediately followed by undoing
# the move that was just made -> hit an obstacle
# cnd -18 OBJECT parameter -> overlapping obj
# cnd -27 ZONE + comparison -> count in zone
#
# Anything not listed keeps its raw number rather than being given a name it
# has not earned.
MMF15_COND = {
-8: 'Is facing in direction', -12: 'Animation finished',
-17: 'Is overlapping a backdrop', -18: 'Is overlapping object',
-19: 'Compare alterable value', -25: 'Compare alterable value',
-27: 'Number of objects in zone',
}
MMF15_ACT = {
0: 'Make object reappear', 1: 'Make object invisible',
2: 'Set direction', 3: 'Set position', 4: 'Set X position',
5: 'Set Y position', 18: 'Force animation', 24: 'Destroy',
27: 'Set alterable value', 37: 'Set alterable value',
}
MMF15_TEXT_ACT = {0: 'Erase text', 1: 'Display paragraph',
4: 'Set paragraph'}
MMF15_COUNTER_ACT = {0: 'Set counter value'}
def ace_name(entry, is_cond, objects):
t, n = entry['objectType'], entry['num']
sysd = COND_SYS if is_cond else ACT_SYS
extd = COND_EXT if is_cond else ACT_EXT
if t >= 32:
gox = EXT_BY_TYPE.get(t, '')
idx = (-n - 81) if is_cond else (n - 80)
real = ext_ace(gox, 'conditions' if is_cond else 'actions', idx)
if real:
return real
obj = objects.get(entry['objectInfo'], {})
return '%s %s#%d' % (obj.get('typeName', gox or 'extension'),
'cond' if is_cond else 'act', idx)
if t < 0:
if t in sysd and n in sysd[t]:
return sysd[t][n]
return '%s#%d' % (OBJ_TYPE_LABEL.get(t, 't%d' % t), n)
raw = entry.get('numRaw', n)
if is_cond:
if raw in MMF15_COND:
return MMF15_COND[raw]
else:
if t == 3 and raw in MMF15_TEXT_ACT:
return MMF15_TEXT_ACT[raw]
if t == 7 and raw in MMF15_COUNTER_ACT:
return MMF15_COUNTER_ACT[raw]
if t <= 2 and raw in MMF15_ACT:
return MMF15_ACT[raw]
return '%s %s#%d' % (OBJ_TYPE_LABEL.get(t, 't%d' % t),
'cnd' if is_cond else 'act', raw)
CMP = {0: '=', 1: '!=', 2: '<=', 3: '<', 4: '>=', 5: '>'}
def fmt_param(p, objects, frame_handles, frame_names):
v = p['value']
if p['name'] == 'FRAME' and isinstance(v, int):
if 0 <= v < len(frame_handles):
fi = frame_handles[v]
return 'frame %d "%s" (handle %d)' % (fi, frame_names[fi], v)
return 'handle %d' % v
if p['name'] in ('KEY', 'VMKEY'):
return 'key %s' % papp.vkname(v['vk'])
if p['name'] == 'NEWDIRECTION':
return 'direction %s' % v['directions']
if p['name'] == 'SHORT' and isinstance(v, dict):
return 'animation %d%s' % (v['index'],
' "%s"' % v['name'] if 'name' in v else '')
if p['code'] == 30:
par = objects.get(v['objectInfoParent'], {})
where = ('at <%s>+(%d,%d)' % (par.get('name', '?'), v['x'], v['y'])
if v['objectInfoParent'] != 0xFFFE
else 'at (%d,%d)' % (v['x'], v['y']))
return 'paragraph %d %s' % (v['paragraph'], where)
if isinstance(v, dict) and set(v) == {'x1', 'y1', 'x2', 'y2'}:
return 'zone (%d,%d)-(%d,%d)' % (v['x1'], v['y1'], v['x2'], v['y2'])
if isinstance(v, dict) and 'objectInfo' in v and 'objectType' in v:
o = objects.get(v['objectInfo'], {})
return '<%s>' % (o.get('name') or o.get('typeName') or
'h%d' % v['objectInfo'])
if isinstance(v, dict) and 'objectInfoParent' in v:
par = v['objectInfoParent']
who = ('<%s>' % objects.get(v.get('objectInfo', -1), {}).get('name')
if 'objectInfo' in v else '')
where = ('<%s>+(%d,%d)' % (objects.get(par, {}).get('name', '?'),
v['x'], v['y']) if par != 0xFFFE
else '(%d,%d)' % (v['x'], v['y']))
return ('%s at %s' % (who, where)).strip()
if isinstance(v, dict) and 'objectInfo' in v:
o = objects.get(v['objectInfo'], {})
return '<%s>' % (o.get('name') or '?')
if isinstance(v, dict) and set(v) == {'delayMs'}:
return '%d ms' % v['delayMs']
if isinstance(v, dict) and 'expr' in v:
c = v['comparison']
return v['expr'] if c == 0 else '%s %s' % (CMP.get(c, '?'), v['expr'])
if isinstance(v, dict) and 'name' in v and 'handle' in v:
return '"%s"' % v['name']
if isinstance(v, dict) and set(v) == {'flags', 'id', 'name'}:
return 'group "%s" (id %d)' % (v['name'], v['id'])
if isinstance(v, dict) and set(v) == {'button', 'double'}:
return '%s%s click' % ('right ' if v['button'] == 2 else '',
'double' if v['double'] else 'single')
if p['code'] == 37:
return ('%r in %s %dpt' % (v['text'], v['font'], -v['height'])
if v['text'] else '(blank line, %s %dpt)'
% (v['font'], -v['height']))
return repr(v)
def dump(evframes, decoded, header, out):
fh = header['frameHandles']
fnames = [f['frame']['name'] for f in decoded]
for fi, ev in enumerate(evframes):
objs = {o['index']: o for o in decoded[fi]['objects']}
out.write('\n' + '=' * 74 + '\n')
out.write('FRAME %d %s -- %d event groups\n'
% (fi, fnames[fi], len(ev['groups'])))
out.write('=' * 74 + '\n')
for gi, g in enumerate(ev['groups']):
out.write('\n[%d.%d]\n' % (fi, gi))
for c in g['conditions']:
o = objs.get(c['objectInfo'], {}) if c['objectType'] >= 0 else {}
label = ace_name(c, True, objs)
if c['otherFlags'] & 1:
label = 'NOT ' + label
oname = o.get('name') or ''
out.write(' IF %-32s %-20s %s\n'
% (label, ('<%s>' % oname) if oname else '',
', '.join(fmt_param(p, objs, fh, fnames)
for p in c['params'])))
for a in g['actions']:
o = objs.get(a['objectInfo'], {}) if a['objectType'] >= 0 else {}
label = ace_name(a, False, objs)
oname = o.get('name') or ''
out.write(' DO %-32s %-20s %s\n'
% (label, ('<%s>' % oname) if oname else '',
', '.join(fmt_param(p, objs, fh, fnames)
for p in a['params'])))
def main():
d = P.read(P.GAM)
h = papp.read_header(d)
banks = papp.find_banks(d, h['frameOffsets'][-1])
h['imageBankOffset'] = banks[0]['offset']
decoded = F.decode(d, h)
evs = []
problems = 0
for fr in decoded:
ev = parse_events(d, fr['instanceBlock']['end'])
bad = sum(1 for g in ev['groups']
if g['sizeError']
or any(c['sizeError'] for c in g['conditions'])
or any(a['sizeError'] for a in g['actions']))
problems += bad
nc = sum(len(g['conditions']) for g in ev['groups'])
na = sum(len(g['actions']) for g in ev['groups'])
go = ev.get('globalObjects')
if go:
print(' %-12s global-object table: %d entries -- %s'
% ('', go['count'],
', '.join(r['text'] or '(blank)'
for r in go['records'])))
print('%-12s %4d groups %4d conditions %4d actions %s'
% (fr['frame']['name'], len(ev['groups']), nc, na,
'clean' if not bad else '%d SIZE ERRORS' % bad))
evs.append(ev)
print('---', 'every group, condition and action consumes exactly its '
'declared size' if not problems else '%d problems' % problems)
app = find_app_events(d, h)
if app:
nc = sum(len(g['conditions']) for g in app['groups'])
na = sum(len(g['actions']) for g in app['groups'])
print('application-level table at %#x: %d groups, %d conditions, '
'%d actions' % (app['offset'], len(app['groups']), nc, na))
menu_ids = sorted({p['value'] for g in app['groups']
for c in g['conditions'] for p in c['params']
if p['name'] == 'MENU'})
print('menu ids handled anywhere in the application: %s' % menu_ids)
with open(P.out('events.json'), 'w') as f:
json.dump(dict(frames=evs, application=app), f, indent=1, default=str)
with open(P.out('EVENTS.txt'), 'w', encoding='utf-8') as f:
if app:
f.write('=' * 74 + '\n')
f.write('APPLICATION -- %d event groups (the menu lives here)\n'
% len(app['groups']))
f.write('=' * 74 + '\n')
for gi, g in enumerate(app['groups']):
f.write('\n[app.%d]\n' % gi)
for c in g['conditions']:
f.write(' IF %-32s %s\n'
% (ace_name(c, True, {}),
', '.join(fmt_param(pp, {}, h['frameHandles'],
[''] * 8)
for pp in c['params'])))
for a in g['actions']:
f.write(' DO %-32s %s\n'
% (ace_name(a, False, {}),
', '.join(fmt_param(pp, {}, h['frameHandles'],
[''] * 8)
for pp in a['params'])))
dump(evs, decoded, h, f)
print('wrote', P.out('events.json'), 'and', P.out('EVENTS.txt'))
if __name__ == '__main__':
main()
```
### 11.8 `render.py`
```python
"""render.py -- put the frames back together, and work out what is visible.
Every frame is rebuilt from the file alone: the instance list gives the
coordinates and the draw order, and the object list gives each object's
image-bank handle directly (see frames.py), so no screenshot matching is
needed anywhere.
Three things come out of this:
render/<frame>.png the frame as authored, at its full size --
including the parts of it that scroll off, and the
parts parked outside it that never show
render/<frame>-view.png what the 500x400 window actually shows at the
start position, letterboxed the way MMF does it
render/<frame>-obstacles.png
the collision map, from the Obstacle flag the
object record carries at +43
A "visibility ledger" is written alongside: for every instance, whether it
lies inside the frame at all, and if so whether anything drawn later covers
it completely. That is what turns "these objects exist" into "this object
can never be seen", which is the interesting part.
"""
import json
import os
import sys
import numpy as np
from PIL import Image
import polpaths as P
try:
sys.stdout.reconfigure(encoding='utf-8', errors='replace')
except Exception:
pass
VIEW_W, VIEW_H = 500, 400
def load_images():
out = {}
for name in os.listdir(P.IMAGES):
if name.endswith('.png'):
out[int(name[:-4])] = Image.open(os.path.join(P.IMAGES, name))
return out
def instance_image(o, imgs):
"""An object's artwork: its own image handle, or the first frame of its
first animation for an Active."""
if o.get('image') and o['image'] in imgs:
return imgs[o['image']], 'object record'
for a in o.get('animations') or []:
if a['frames'] and a['frames'][0] in imgs:
return imgs[a['frames'][0]], 'animation table'
if o.get('counterDigits'):
h = o['counterDigits'][0]
if h in imgs:
return imgs[h], 'counter digits'
return None, None
def render(fr, imgs, ledger):
f = fr['frame']
W, H = f['w'], f['h']
# authored canvas, plus a margin so parked objects are visible too
pad = 256
big = Image.new('RGBA', (W + 2 * pad, H + 2 * pad), (0, 0, 0, 0))
frame = Image.new('RGBA', (W, H), (0, 0, 0, 255))
obst = np.zeros((H, W), dtype=np.uint8)
objs = {o['index']: o for o in fr['objects']}
painted = []
for inst in fr['instances']:
o = objs.get(inst['objIndex'])
if o is None:
continue
im, src = instance_image(o, imgs)
x, y = inst['x'], inst['y']
if im is None:
if inst['objType'] == 0 and 'w' in inst: # Quick backdrop
im = Image.new('RGBA', (inst['w'], inst['h']),
tuple(inst.get('color1', [0, 0, 0, 0])[:3])
+ (255,))
src = 'quick backdrop color'
else:
ledger.append(dict(frame=f['name'], name=o['name'], x=x, y=y,
type=inst['typeName'], art=None,
inside=False, visible=False,
why='no artwork'))
continue
big.alpha_composite(im, (x + pad, y + pad))
inside = not (x + im.width <= 0 or y + im.height <= 0
or x >= W or y >= H)
if inside:
frame.alpha_composite(im, (max(x, 0), max(y, 0))
if x >= 0 and y >= 0 else (x, y))
if inst['obstacle']:
x0, y0 = max(x, 0), max(y, 0)
x1, y1 = min(x + im.width, W), min(y + im.height, H)
if x1 > x0 and y1 > y0:
obst[y0:y1, x0:x1] = 255
painted.append((inst, o, im, inside, src))
# ---- occlusion: is anything drawn later covering this instance whole?
cover = np.zeros((H, W), dtype=bool)
for inst, o, im, inside, src in reversed(painted):
x, y = inst['x'], inst['y']
if not inside:
ledger.append(dict(frame=f['name'], name=o['name'], x=x, y=y,
type=inst['typeName'], art=src, inside=False,
visible=False, why='parked outside the frame'))
continue
x0, y0 = max(x, 0), max(y, 0)
x1, y1 = min(x + im.width, W), min(y + im.height, H)
alpha = np.array(im)[..., 3] > 0
alpha = alpha[y0 - y:y1 - y, x0 - x:x1 - x]
region = cover[y0:y1, x0:x1]
shown = int((alpha & ~region).sum())
total = int(alpha.sum())
ledger.append(dict(frame=f['name'], name=o['name'], x=x, y=y,
type=inst['typeName'], art=src, inside=True,
visible=shown > 0,
pixels=total, visiblePixels=shown,
why=None if shown else 'completely covered'))
cover[y0:y1, x0:x1] |= alpha
return frame, big, obst
def main():
frames = json.load(open(P.out('frames.json')))
imgs = load_images()
ledger = []
for fr in frames:
f = fr['frame']
name = f['name'].replace(' ', '-').lower() + '-%d' % f['index']
frame, big, obst = render(fr, imgs, ledger)
frame.convert('RGB').save(os.path.join(P.RENDER, name + '.png'))
big.save(os.path.join(P.RENDER, name + '-with-parked.png'))
Image.fromarray(obst).save(os.path.join(P.RENDER,
name + '-obstacles.png'))
view = Image.new('RGB', (VIEW_W, VIEW_H), (0, 0, 0))
view.paste(frame.convert('RGB'),
(max(0, (VIEW_W - f['w']) // 2),
max(0, (VIEW_H - f['h']) // 2)))
view.save(os.path.join(P.RENDER, name + '-view.png'))
blocked = 100.0 * (obst > 0).mean()
print('%-12s %4dx%-4d %4d instances %5.1f%% of the frame is '
'obstacle' % (f['name'], f['w'], f['h'],
len(fr['instances']), blocked))
with open(P.out('visibility.json'), 'w') as fh:
json.dump(ledger, fh, indent=1)
never = [l for l in ledger if not l['visible']]
print('---')
print('%d instances placed, %d of them can never be seen'
% (len(ledger), len(never)))
for l in never:
print(' %-12s %-24s (%5d,%5d) %s'
% (l['frame'], l['name'][:24], l['x'], l['y'], l['why']))
if __name__ == '__main__':
main()
```
### 11.9 `verify.py`
```python
"""verify.py -- check the reconstruction against pictures of the real thing.
The renders in render/ are built purely from the container: coordinates from
the instance list, artwork from each object's own image handle, order from
the record array. Nothing in that chain looks at a picture of the running
program, so any screen capture is an independent test of the whole decode --
container, image codec, palette, instance layout and draw order at once.
The method is deliberately blunt: take a patch out of the middle of the
screenshot's client area, slide it over the render, and report the best sum
of absolute differences. If any part of the chain were wrong the best match
would not be exact.
"""
import json
import os
import sys
import numpy as np
from PIL import Image
import polpaths as P
try:
sys.stdout.reconfigure(encoding='utf-8', errors='replace')
except Exception:
pass
SHOTS = [
('screenshot_04_overworld.png', 'beta-city-1-1.png'),
('screenshot_03_house_interior.png', 'house-1-2.png'),
]
def client_area(im):
"""Strip the Windows title bar and menu bar. Both screenshots were taken
on Windows 11, whose frame is 1px of border plus a 32px caption plus a
~19px menu; rather than hardcode that, find the first row from the top
whose colors stop looking like chrome."""
a = np.array(im.convert('RGB'))
h, w = a.shape[:2]
for y in range(h):
row = a[y]
# chrome rows are near-uniform light gray; game rows are not
if row.std() > 40:
return a[y:, :]
return a
def best_match(patch, target):
"""Two passes: find candidate offsets with one row of the patch, using a
strided view so the search is a single vectorised operation, then score
the full patch only at those offsets."""
from numpy.lib.stride_tricks import sliding_window_view
ph, pw = patch.shape[:2]
th, tw = target.shape[:2]
if th < ph or tw < pw:
return None
row = patch[ph // 2].astype(np.int16)
win = sliding_window_view(target.astype(np.int16), (pw, 3), axis=(1, 2))
win = win[:, :, 0] # (th, tw-pw+1, pw, 3)
err = np.abs(win - row).mean(axis=(2, 3))
ys, xs = np.unravel_index(np.argsort(err, axis=None)[:4000], err.shape)
best = None
for y, x in zip(ys, xs):
y0 = y - ph // 2
if y0 < 0 or y0 + ph > th:
continue
d = np.abs(target[y0:y0 + ph, x:x + pw].astype(np.int16)
- patch.astype(np.int16))
s = float(d.mean())
if best is None or s < best[0]:
best = (s, int(x), int(y0), float((d.max(axis=2) == 0).mean()))
if s == 0:
break
return best
def main():
results = []
for shot, render in SHOTS:
sp = os.path.join(P.ROOT, shot)
rp = os.path.join(P.RENDER, render)
if not (os.path.exists(sp) and os.path.exists(rp)):
print('missing %s / %s' % (shot, render))
continue
cli = client_area(Image.open(sp))
tgt = np.array(Image.open(rp).convert('RGB'))
# a patch from the middle, small enough to search quickly and large
# enough that a false match is not credible
# try several patches: the player sprite and the animated NPC move,
# so a patch that happens to contain them can never match exactly
ph, pw = min(96, cli.shape[0] - 8), min(128, cli.shape[1] - 8)
got = None
for oy, ox in ((4, 4), (cli.shape[0] - ph - 4, 4),
(4, cli.shape[1] - pw - 4),
(cli.shape[0] - ph - 4, cli.shape[1] - pw - 4),
((cli.shape[0] - ph) // 2, (cli.shape[1] - pw) // 2)):
patch = cli[oy:oy + ph, ox:ox + pw]
r = best_match(patch, tgt)
if r and (got is None or r[0] < got[0]):
got = r
# the render offset that corresponds to the frame origin
got = (r[0], r[1] - ox, r[2] - oy, r[3])
if got and got[0] == 0:
break
if got is None:
print('%-34s render too small to search' % shot)
continue
err, x, y, exact = got
print('%-34s best match at (%d,%d): mean |difference| %.3f of 255, '
'%.2f%% of pixels identical' % (shot, x, y, err, 100 * exact))
results.append(dict(screenshot=shot, render=render, x=x, y=y,
meanAbsError=err, exactFraction=exact,
patch=[pw, ph]))
with open(P.out('verify.json'), 'w') as fh:
json.dump(results, fh, indent=1)
if __name__ == '__main__':
main()
```
### 11.10 `inventory.py`
```python
"""inventory.py -- provenance, the text, the reachability graph, the slack.
Pulls together the things that are interesting about the file but do not
belong to any one decoder: what built it and when, every string it contains,
which frames can actually be reached, and which bytes nothing accounts for.
"""
import datetime
import hashlib
import json
import os
import re
import struct
import sys
import polpaths as P
import papp
import pe as PEmod
try:
sys.stdout.reconfigure(encoding='utf-8', errors='replace')
except Exception:
pass
def stamp(ts):
return datetime.datetime.fromtimestamp(
ts, datetime.timezone.utc).strftime('%Y-%m-%d %H:%M:%S UTC')
def midi_info(path):
d = open(path, 'rb').read()
fmt, ntrk, div = struct.unpack_from('>HHH', d, 8)
out = dict(bytes=len(d), format=fmt, tracks=ntrk, division=div,
text=[], trackNames=[])
p = 14
tempo = None
for _ in range(ntrk):
if d[p:p + 4] != b'MTrk':
break
ln = struct.unpack_from('>I', d, p + 4)[0]
body = d[p + 8:p + 8 + ln]
q = 0
while q < len(body) - 3:
if body[q] == 0xFF and body[q + 1] in (0x01, 0x02, 0x03, 0x04,
0x05, 0x06, 0x07, 0x51):
meta = body[q + 1]
n = body[q + 2]
payload = body[q + 3:q + 3 + n]
if meta == 0x51 and n == 3 and tempo is None:
tempo = (payload[0] << 16) | (payload[1] << 8) | payload[2]
elif meta != 0x03:
s = payload.decode('latin1').strip()
if s:
out['text'].append(s)
elif meta == 0x03:
out['trackNames'].append(payload.decode('latin1').strip())
q += 3 + n
continue
q += 1
p += 8 + ln
if tempo:
out['bpm'] = round(60000000.0 / tempo, 1)
return out
def frame_graph(events, header, frames):
"""Which frame can reach which, straight out of the actions."""
fh = header['frameHandles']
edges, ends = [], []
for i, ev in enumerate(events):
for g in ev['groups']:
for a in g['actions']:
if a['objectType'] != -3:
continue
if a['num'] == 0:
edges.append((i, i + 1, 'next frame'))
elif a['num'] == 1:
edges.append((i, i - 1, 'previous frame'))
elif a['num'] == 2:
v = a['params'][0]['value']
if isinstance(v, int) and 0 <= v < len(fh):
edges.append((i, fh[v], 'jump'))
elif a['num'] == 4:
ends.append(i)
reach = {0}
changed = True
while changed:
changed = False
for a, b, _ in edges:
if a in reach and b not in reach and 0 <= b < len(frames):
reach.add(b)
changed = True
return edges, sorted(reach), ends
def main():
d = P.read(P.GAM)
h = papp.read_header(d)
banks = papp.find_banks(d, h['frameOffsets'][-1])
frames = papp.read_frames(d, h)
dec = json.load(open(P.out('frames.json')))
ev = json.load(open(P.out('events.json')))
events, appevents = ev['frames'], ev.get('application')
rep = {}
# ---- provenance
files = []
for path in (P.GAM, P.EXE, P.DLL):
raw = P.read(path)
st = os.stat(path)
rec = dict(name=os.path.basename(path), size=len(raw),
sha256=hashlib.sha256(raw).hexdigest(),
md5=hashlib.md5(raw).hexdigest(),
modified=datetime.datetime.fromtimestamp(
st.st_mtime).strftime('%Y-%m-%d %H:%M:%S'))
if raw[:2] == b'MZ':
p = PEmod.PE(raw)
rec.update(peTimeStamp=stamp(p.timeStamp),
linker='%d.%d' % p.linker,
sections=[s['name'] for s in p.sections],
imageBase=hex(p.imageBase),
overlay=len(raw) - p.imageEnd)
files.append(rec)
rep['files'] = files
print('== files ==')
for f in files:
print(' %-12s %8d %s' % (f['name'], f['size'], f['sha256'][:32]))
if 'peTimeStamp' in f:
print(' linked %s, linker %s, overlay %d bytes'
% (f['peTimeStamp'], f['linker'], f['overlay']))
# ---- runtime strings worth quoting
exe = PEmod.PE(P.read(P.EXE))
rep['runtimeStrings'] = exe.strings()
res = exe.resources()
rep['runtimeResources'] = {PEmod.RT_NAME.get(k, str(k)):
sorted(str(x) for x in v)
for k, v in res.items()}
# ---- every string in the .gam, and where
strings = []
for m in re.finditer(rb'[\x20-\x7e]{6,}', d):
s = m.group().decode('latin1')
where = 'header'
for i, f in enumerate(frames):
if f['offset'] <= m.start():
where = 'frame %d %s' % (i, f['name'])
if m.start() >= banks[0]['offset']:
where = 'banks'
strings.append(dict(offset=m.start(), where=where, text=s))
rep['strings'] = strings
print('\n== strings ==\n %d printable runs of 6+ characters' % len(strings))
# ---- the paragraphs, which is the dialogue
paras = []
for fr in dec:
for o in fr['objects']:
for i, t in enumerate(o.get('paragraphs') or []):
paras.append(dict(frame=fr['frame']['name'], object=o['name'],
index=i, text=t))
rep['paragraphs'] = paras
print(' %d text paragraphs across %d text objects'
% (len(paras), len({(p['frame'], p['object']) for p in paras})))
# ---- reachability
edges, reach, ends = frame_graph(events, h, frames)
rep['menuIdsHandled'] = sorted({p['value'] for g in (appevents or {}).get('groups', [])
for c in g['conditions']
for p in c['params']
if p['name'] == 'MENU'})
rep['frameEdges'] = edges
rep['reachableFrames'] = reach
print('\n== frames ==')
for i, f in enumerate(frames):
outs = sorted({b for a, b, _ in edges if a == i})
print(' %d %-12s %4dx%-4d %-10s -> %s'
% (i, f['name'], f['w'], f['h'],
'reachable' if i in reach else 'UNREACHABLE',
', '.join(str(x) for x in outs) or '(nothing)'))
# ---- music
mid = os.path.join(P.SOUNDS, 'technosong.mid')
if os.path.exists(mid):
rep['midi'] = midi_info(mid)
m = rep['midi']
print('\n== music ==\n technosong.mid %d bytes, SMF format %d, '
'%d tracks, division %d, %s BPM'
% (m['bytes'], m['format'], m['tracks'], m['division'],
m.get('bpm', '?')))
for t in m['text']:
print(' text: %s' % t)
print(' tracks: %s' % ', '.join(t for t in m['trackNames'] if t))
# ---- what leaked out of the editor's memory
#
# Several structures in this format are written from buffers the editor
# never fully initialized, so whatever was in memory at save time went
# into the file. On Windows 95/98 the shared DLL arena sits at
# 0x80000000-0xBFFFFFFF with KERNEL32 loaded around 0xBFF70000, and
# pointers into it are unmistakable. They cannot occur in a file written
# on NT, 2000 or XP, where the system DLLs live near 0x77000000.
ptrs = {}
for off in range(0, len(d) - 4):
v = struct.unpack_from('<I', d, off)[0]
if 0xBFF00000 <= v <= 0xBFFFFFFF:
ptrs.setdefault(v, []).append(off)
rep['win9xPointers'] = {'%#010x' % k: v for k, v in sorted(ptrs.items())}
total = sum(len(v) for v in ptrs.values())
print('\n== leaked memory ==')
print(' %d dwords in the Win9x shared-DLL range, %d distinct values'
% (total, len(ptrs)))
for v, offs in sorted(ptrs.items()):
print(' %#010x x%-3d first at %#08x' % (v, len(offs), offs[0]))
# ---- the runtime's own menu and dialogs, which the game never shows
rep['runtimeMenus'] = exe.resource_strings(PEmod.RT_MENU, minrun=2)
rep['runtimeDialogs'] = exe.resource_strings(PEmod.RT_DIALOG, minrun=4)
print('\n== runtime resources ==')
for name, v in sorted(rep['runtimeMenus'].items()):
print(' MENU %s: %s' % (name, ' | '.join(v['strings'])))
for name, v in sorted(rep['runtimeDialogs'].items()):
keep = [s for s in v['strings'] if sum(c.isalpha() for c in s) > 2]
print(' DIALOG %s: %s' % (name, ' | '.join(keep)[:170]))
# ---- byte accounting: is anything unexplained?
spans = [(0, frames[0]['offset'], 'header, icon, extensions, menu')]
for i, f in enumerate(frames):
end = (frames[i + 1]['offset'] if i + 1 < len(frames)
else banks[0]['offset'])
spans.append((f['offset'], end, 'frame %d %s' % (i, f['name'])))
for b in banks:
spans.append((b['offset'], b['end'],
'bank of %d' % b['count']))
spans.sort()
gaps, cur = [], 0
for a, b, why in spans:
if a > cur:
gaps.append((cur, a - cur))
cur = max(cur, b)
if cur < len(d):
gaps.append((cur, len(d) - cur))
rep['unaccounted'] = gaps
print('\n== byte accounting ==')
print(' %d bytes total; unaccounted: %s'
% (len(d), gaps if gaps else 'none, the file is fully mapped'))
with open(P.out('inventory.json'), 'w') as fh:
json.dump(rep, fh, indent=1)
if __name__ == '__main__':
main()
```
### 11.11 `run_all.py`
```python
"""run_all.py -- the whole extraction, in order.
papp the container: header, menu, accelerators, frame table, banks
images the image bank, the icon, the music bank -> extract/images
extensions carve the six .GOX DLLs out of the exe overlay
frames object lists, animation tables, instance lists
events the program logic, frame tables and the application table
render rebuild every frame, plus the visibility ledger
verify score the rebuilds against screen captures of the real thing
inventory provenance, strings, dialogue, reachability, byte accounting
`extensions` must run before `events`, because the extension ACE names come
out of the DLLs. `images` must run before `render`. Everything else is
ordered for readability rather than necessity.
"""
import os
import subprocess
import sys
import time
HERE = os.path.dirname(os.path.abspath(__file__))
STEPS = ['papp', 'images', 'extensions', 'frames', 'events', 'render',
'verify', 'inventory']
def main():
only = sys.argv[1:]
for name in STEPS:
if only and name not in only:
continue
print('\n' + '=' * 70)
print('== %s' % name)
print('=' * 70)
t = time.time()
r = subprocess.run([sys.executable, os.path.join(HERE, name + '.py')],
cwd=HERE)
print('-- %s: %s in %.1fs'
% (name, 'ok' if r.returncode == 0 else
'FAILED (%d)' % r.returncode, time.time() - t))
if r.returncode != 0:
return r.returncode
return 0
if __name__ == '__main__':
sys.exit(main())
```
## 12. About this teardown
Produced by **FieRcE YeD**, 12 August 2026, for
**[doors98.com](https://doors98.com)**.
### Why bother
Pokemon Online Advanced Beta 1.0 still runs, which makes it a less obvious
candidate for this than a program that cannot. But running it only shows you
the four screens it lets you reach, and the interesting half of this file is
the half it does not: a finished trainer ID card, a finished pocket organizer
with a development diary inside it, a DirectPlay lobby, and a complete
networked-player implementation sitting in both playable frames waiting for a
server that was three days away in September 2000 and never arrived.
Playing it also cannot tell you that the sign was never wired up, that two
global values are written every cycle and read by nothing, that one of the
three "Visit" menu items has no handler while the other two work, or that
right-clicking your own character does something the controls screen does not
mention.
### Why a teardown and not a description
Getting the artwork out of one of these files is the easy half, and it is
where most attempts stop: CTFAK cannot open the container at all, the event
tables carry no signature to search for, and the obvious way to name the
opcodes -- borrow mmfparser's tables -- produces output that reads fluently
and is wrong.
That is why section 11 carries the code rather than a summary, and why section
10.3 spends as much space on the two places mmfparser's structures do *not*
apply as on the places they do. A description of a format helps one person; a
working decoder helps whoever finds the next file. Standing on Mathias
Kaerlev's **Anaconda / mmfparser**, which documented the MMF 1.5 structures in
the first place, this fills in the flat `PAPP` container those structures live
in for Games Factory-era files, and the opcode numbering that changed
underneath them.
### doors98.com
doors98.com is a Windows 98 desktop rebuilt in a browser -- a parody, an
archive, and a place to put things that no longer have anywhere to live. It
already hosts recreations and teardowns of several recovered applications from
this era, including the POL Map Editor, POL Lounge V 1.0, Epic's Memorial and
Square's Weakest Link -- and a playable browser recreation of this game.
Where things live:
| | |
|---|---|
| The original files | <https://archive.org/details/doors98-pokemon-online-beta-1-0> |
| This document, raw | <https://doors98.com/apps/pol/beta/pol-beta-readme.txt> |
| This document, as Markdown | <https://doors98.com/apps/pol/beta/pol-beta-readme.md> |
| This document, on the desktop | <https://doors98.com/downloads/pol-beta-readme> |
| The download entry | <https://doors98.com/downloads/pol-beta-1> |
| The browser recreation | <https://doors98.com/pol-beta> |
| Other recovered software of the era | <https://archive.org/details/@fierceyed> |
(The two document files are byte-identical. Web servers have no mime mapping
for `.md`, so that copy is sent as `application/octet-stream` and browsers
download it instead of showing it; the `.txt` is `text/plain` and renders
inline, which is why the desktop viewer and the sitemap both point at it. The
document is written in Markdown and kept pure ASCII so it reads correctly
either way.)
The findings, the format notes and the toolchain are the part worth keeping.
### Corrections
Almost everything here was derived from one set of files, and some of it is
inference. Anything uncertain is marked as such in the text -- particularly
the derived opcode table in section 10.3, where four built-in opcodes are
still printed as numbers because nothing in this file pins them down, and the
click-button encoding in section 7, which follows MMF's ordering rather than
observation.
The exceptions are section 2's "Where it was distributed" and section 8's "What
POL:A was going to be", which are the only parts sourced from outside the
files. Both rest on Wayback captures of pokemonol.com -- the download page of
9 November 2000 and the POL:A information page of 16 December 2000. The two
size agreements quoted in section 2 are consistent with the preserved archive
being the file that page served; they are not proof of it. And a published
feature list is a statement of intent: it says what was planned, not what was
built, which is the whole point of comparing it against what the file actually
contains.
If you were part of the POL scene, or are Konidias, Caveman, Optimus Primus or
Dragoness: corrections are genuinely welcome, and so are takedown requests
from any rights holder named in section 9. A copy of `polbeta.zip`, `exbeta.zip`
or `charcreator.zip` would be especially welcome -- the crawler saved the page
that listed them but none of the files, so the alpha this build is haunted by
is, as far as anyone knows, gone.
A complete static teardown of Pokemon Online Advanced Beta 1.0 (Konidias, 17 September 2000), a Clickteam Multimedia Fusion 1.x fan game. Produced without running the program: the flat PAPP container, all 191 images, the object and instance layout, the font bank, the six extension DLLs carved out of the executable, and the entire event logic were decoded from the bytes.
Pokemon Online Advanced Beta 1.0 ships as pol.exe, pol.gam and cncs32.dll. The executable is only the Multimedia Fusion runtime; its overlay holds six uncompressed extension DLLs and no game data at all. The game is pol.gam, whose magic is PAPP - the older flat Clickteam container with one absolute frame offset table and self-describing media banks, not the chunked PAME format that CTFAK and mmfparser understand, which is why standard tools cannot open it. The image codec is the part that takes work: the transparency mask and the color data are separate planes, the color plane holds only the opaque pixels as 15-bit RGB555 words, and when compressed it is a single run-length stream that runs straight through row boundaries while the row-offset table in front of it is stored uncompressed and holds destination rather than source offsets. All 191 images decode with five structural invariants holding, and rebuilding the frames from the container alone - coordinates from the instance list, artwork from each object's own image handle, order from the record array - reproduces screen captures of the running game with a mean absolute difference of 0.000 and 100 percent of pixels identical. All 102 event groups decode, including a second, untagged event table at application level that turns out to be the entire custom menu: only three menu ids are handled anywhere in the application, so Game Credits and two of the three Visit links work and Visit Gamedn.com, Help Guides, Character Editor and Trainer Pages have no handlers at all. Four of the eight frames cannot be reached by anything: a finished trainer ID card, a Pokemon Pocket Organizer holding a single dated development diary entry from 1 September 2000, a DirectPlay lobby with Host and Join buttons that creates a session called polgame, and a connecting screen with a Pokeball rolling along a loading bar. Ninety-three of the 1,105 placed object instances can never be seen, ninety-six images are engine-generated rotation frames the game never uses, six of the eighteen font records ask for a fan-made typeface named Pokemon that no Windows machine ships, and 96 dwords scattered through the file are uninitialized memory whose pointers land in the Windows 9x shared DLL arena, fixing the machine it was authored on.