Next: The D64 disk image format, Previous: The T64 tape image format, Up: The emulator file formats [Contents][Index]
(This section was contributed by Peter Schepers and slightly edited by Ettore Perazzoli.)
This format was defined in 1998 as a cooperative effort between several
emulator people, mainly Per Håkan Sundell, author of the CCS64 C64
emulator, Andreas Boose of the VICE CBM emulator team and Joe
Forster/STA, the author of Star Commander. It was the first real public
attempt to create a format for the emulator community which removed
almost all of the drawbacks of the other existing image formats, namely
D64
.
The intention behind G64
is not to replace the widely used
D64
format, as D64
works fine with the vast majority of
disks in existence. It is intended for those small percentage of
programs which demand to work with the 1541 drive in a non-standard way,
such as reading or writing data in a custom format. The best example is
with speeder software such as Action Cartridge in Warp Save mode or
Vorpal which write track/sector data in another format other than
standard GCR. The other obvious example is copy-protected software
which looks for some specific data on a track, like the disk ID, which
is not stored in a standard D64
image.
G64
has a deceptively simply layout for what it is capable of
doing. We have a signature, version byte, some predefined size values,
and a series of offsets to the track data and speed zones. It is what’s
contained in the track data areas and speed zones which is really at the
heart of this format.
Each track entry in simply the raw stream of GCR data, just what a read head would see when a diskette is rotating past it. How the data gets interpreted is up to the program trying to access the disk. Because the data is stored in such a low-level manner, just about anything can be done. Most of the time I would suspect the data in the track would be standard sectors, with SYNC, GAP, header, data and checksums. The arrangement of the data when it is in a standard GCR sector layout is beyond the scope of this document.
Since it is a flexible format in both track count and track byte size, there is no “standard” file size. However, given a few constants like 42 tracks and halftracks, a track size of 7928 bytes and no speed offset entries, the typical file size will a minimum of 333744 bytes.
Below is a dump of the header, broken down into its various parts. After that will be an explanation of the track offset and speed zone offset areas, as they demand much more explanation.
Addr 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F ---- ----------------------------------------------- 0000: 47 43 52 2D 31 35 34 31 00 54 F8 1E .. .. .. ..
Offset | Description |
$0000-0007 | File signature (GCR-1541 ) |
$0008 | G64 version (presently only $00 defined) |
$0009 | Number of tracks in image (usually $54, decimal 84) |
$000A-000B | Size of each stored track in bytes (usually 7928, or $1EF8) in LO/HI format. |
An obvious question here is “why are there 84 tracks defined when a
normal D64
disk only has 35 tracks?” Well, by definition, this
image includes all half-tracks, so there are actually 42 tracks and 42
half tracks. The 1541 stepper motor can access up to 42 tracks and the
in-between half-tracks. Even though using more than 35 tracks is not
typical, it was important to define this format from the start with what
the 1541 is capable of doing, and not just what it typically does.
At first, the defined track size value of 7928 bytes may seem to be arbitrary, but it is not. It is determined by the fastest write speed possible (speed zone 0), coupled with the average rotation speed of the disk (300 rpm). After some math, the answer that actually comes up is 7692 bytes. Why the discrepency between the actual size of 7692 and the defined size of 7928? Simply put, not all drives rotate at 300 rpm. Some can be faster or slower, so a upper safety margin of +3% was built added, in case some disks rotate slower and can write more data. After applying this safety factor, and some rounding-up, 7928 bytes per track was arrived at.
Also note that this upper limit of 7928 bytes per track really only applies to 1541 and compatible disks. If this format were applied to another disk type like the SFD1001, this value would be higher.
Below is a dump of the first section of a G64
file, showing the offsets
to the data portion for each track and half-track entry. Following that
is a dump of the speed zone offsets.
Addr 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F ---- ----------------------------------------------- 0000: .. .. .. .. .. .. .. .. .. .. .. .. AC 02 00 00 0010: 00 00 00 00 A6 21 00 00 00 00 00 00 A0 40 00 00 0020: 00 00 00 00 9A 5F 00 00 00 00 00 00 94 7E 00 00 0030: 00 00 00 00 8E 9D 00 00 00 00 00 00 88 BC 00 00 0040: 00 00 00 00 82 DB 00 00 00 00 00 00 7C FA 00 00 0050: 00 00 00 00 76 19 01 00 00 00 00 00 70 38 01 00 0060: 00 00 00 00 6A 57 01 00 00 00 00 00 64 76 01 00 0070: 00 00 00 00 5E 95 01 00 00 00 00 00 58 B4 01 00 0080: 00 00 00 00 52 D3 01 00 00 00 00 00 4C F2 01 00 0090: 00 00 00 00 46 11 02 00 00 00 00 00 40 30 02 00 00A0: 00 00 00 00 3A 4F 02 00 00 00 00 00 34 6E 02 00 00B0: 00 00 00 00 2E 8D 02 00 00 00 00 00 28 AC 02 00 00C0: 00 00 00 00 22 CB 02 00 00 00 00 00 1C EA 02 00 00D0: 00 00 00 00 16 09 03 00 00 00 00 00 10 28 03 00 00E0: 00 00 00 00 0A 47 03 00 00 00 00 00 04 66 03 00 00F0: 00 00 00 00 FE 84 03 00 00 00 00 00 F8 A3 03 00 0100: 00 00 00 00 F2 C2 03 00 00 00 00 00 EC E1 03 00 0110: 00 00 00 00 E6 00 04 00 00 00 00 00 E0 1F 04 00 0120: 00 00 00 00 DA 3E 04 00 00 00 00 00 D4 5D 04 00 0130: 00 00 00 00 CE 7C 04 00 00 00 00 00 C8 9B 04 00 0140: 00 00 00 00 C2 BA 04 00 00 00 00 00 BC D9 04 00 0150: 00 00 00 00 B6 F8 04 00 00 00 00 00 .. .. .. ..
Offset | Description |
$000C-000F | Offset to stored track 1.0 ($000002AC, in LO/HI format, see below for more) |
$0010-0013 | Offset to stored track 1.5 ($00000000) |
$0014-0017 | Offset to stored track 2.0 ($000021A6) |
… | |
$0154-0157 | Offset to stored track 42.0 ($0004F8B6) |
$0158-015B | Offset to stored track 42.5 ($00000000) |
00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F ----------------------------------------------- 0150: .. .. .. .. .. .. .. .. .. .. .. .. 03 00 00 00 0160: 00 00 00 00 03 00 00 00 00 00 00 00 03 00 00 00 0170: 00 00 00 00 03 00 00 00 00 00 00 00 03 00 00 00 0180: 00 00 00 00 03 00 00 00 00 00 00 00 03 00 00 00 0190: 00 00 00 00 03 00 00 00 00 00 00 00 03 00 00 00 01A0: 00 00 00 00 03 00 00 00 00 00 00 00 03 00 00 00 01B0: 00 00 00 00 03 00 00 00 00 00 00 00 03 00 00 00 01C0: 00 00 00 00 03 00 00 00 00 00 00 00 03 00 00 00 01D0: 00 00 00 00 03 00 00 00 00 00 00 00 03 00 00 00 01E0: 00 00 00 00 02 00 00 00 00 00 00 00 02 00 00 00 01F0: 00 00 00 00 02 00 00 00 00 00 00 00 02 00 00 00 0200: 00 00 00 00 02 00 00 00 00 00 00 00 02 00 00 00 0210: 00 00 00 00 02 00 00 00 00 00 00 00 01 00 00 00 0220: 00 00 00 00 01 00 00 00 00 00 00 00 01 00 00 00 0230: 00 00 00 00 01 00 00 00 00 00 00 00 01 00 00 00 0240: 00 00 00 00 01 00 00 00 00 00 00 00 00 00 00 00 0250: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0260: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0270: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0280: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0290: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 02A0: 00 00 00 00 00 00 00 00 00 00 00 00 .. .. .. ..
Offset | Description |
$015C-015F | Speed zone entry for track 1 ($03, in LO/HI format, see below for more) |
$0160-0163 | Speed zone entry for track 1.5 ($03) |
… | |
$02A4-02A7 | Speed zone entry for track 42 ($00) |
$02A8-02AB | Speed zone entry for track 42.5 ($00) |
Starting here at $02AC is the first track entry (from above, it is the first entry for track 1.0)
The track offsets (from above) require some explanation. When one is set to all 0’s, no track data exists for this entry. If there is a value, it is an absolute reference into the file (starting from the beginning of the file). From the track 1.0 entry we see it is set for $000002AC. Going to that file offset, here is what we see…
00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F ----------------------------------------------- 02A0: .. .. .. .. .. .. .. .. .. .. .. .. 0C 1E FF FF 02B0: FF FF FF 52 54 B5 29 4B 7A 5E 95 55 55 55 55 55 02C0: 55 55 55 55 55 55 FF FF FF FF FF 55 D4 A5 29 4A 02D0: 52 94 A5 29 4A 52 94 A5 29 4A 52 94 A5 29 4A 52
Offset | Description |
$02AC-02AD | Actual size of stored track (7692 or $1E0C, in LO/HI format) |
$02AE-02AE+$1E0C | Track data |
Following the track data is filler bytes. In this case, there are 368 bytes of unused space. This space can contain anything, but for the sake of those wishing to compress these images for storage, they should all be set to the same value. In the sample I used, these are all set to $FF.
Below is a dump of the end of the track 1.0 data area. Note the actual track data ends at address $20B9, with the rest of the block being unused, and set to $FF.
00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F ----------------------------------------------- 1FE0: 52 94 A5 29 4A 52 94 A5 29 4A 52 94 A5 29 4A 52 1FF0: 94 A5 29 4A 52 94 A5 29 4A 52 94 A5 29 4A 52 94 2000: A5 29 4A 52 94 A5 29 4A 52 94 A5 29 4A 52 94 A5 2010: 29 4A 52 94 A5 29 4A 52 94 A5 29 4A 52 94 A5 29 2020: 4A 52 94 A5 29 4A 52 94 A5 29 4A 52 94 A5 29 4A 2030: 55 55 55 55 55 55 FF FF FF FF FF FF FF FF FF FF 2040: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF 2050: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF 2060: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF 2070: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF 2080: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF 2090: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF 20A0: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF 20B0: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF 20C0: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF 20D0: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF 20E0: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF 20F0: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF 2100: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF 2110: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF 2120: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF 2130: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF 2140: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF 2150: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF 2160: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF 2170: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF 2180: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF 2190: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF 21A0: FF FF FF FF FF FF .. .. .. .. .. .. .. .. .. ..
The speed offset entries can be a little more complex. The 1541 has four speed zones defined, which means the drive can write data at four distinct speeds. On a normal 1541 disk, these zones are as follows:
Track Range | Speed Zone |
1-17 | 3 (highest writing speed) |
18-24 | 2 |
25-30 | 1 |
31 and up | 0 (lowest writing speed) |
Note that you can, through custom programming of the 1541, change the speed zone of any track to something different (change the 3 to a 0) and write data differently. From the dump of the speed offset entries above, we see that all the entries are in the range of 0-3. If any entry is less than 4, this is not considered a speed offset but defines the whole track to be recorded at that one speed.
In the example I had, there were no offsets defined, so no speed zone dump can be shown. However, I can define what should be there. You will have a block of data, 1982 bytes long. Each byte is encoded to represent the speed of 4 bytes in the track offset area, and is broken down as follows:
Speed entry $FF: in binary %11111111 |'|'|'|' | | | | | | | +- 4'th byte speed (binary 11, 3 dec) | | +--- 3'rd byte speed (binary 11, 3 dec) | +----- 2'nd byte speed (binary 11, 3 dec) +------- 1'st byte speed (binary 11, 3 dec)
It was very smart thinking to allow for two speed zone settings, one in the offset block and another defining the speed on a per-byte basis. If you are working with a normal disk, where each track is one constant speed, then you don’t need the extra blocks of information hanging around the image, wasting space.
What may not be obvious is the flexibility of this format to add tracks and speed offset zones at will. If a program decides to write a track out with varying speeds, and no speed offset exist, a new block will be created by appending it to the end of the image, and the offset pointer for that track set to point to the new block. If a track has no offset yet, meaning it doesn’t exist (like a half-track), and one needs to be added, the same procedure applies. The location of the actual track or speed zone data is not important, meaning they do not have to be in any particular order since they are all referenced by the offsets at the beginning of the image.
Given that a Commodore VIC 1541/1570/1571 can read/write at most 42 tracks per side, it is quite natural to use the half-tracks from 1 to 84 for the first side and the following 84 half-tracks (i. e. halg.track 85 to 168) for the second size.
In order for this to work, byte 9 of the image (i. e. the maximum number of tracks) usually contains 168 for double sided disks.
To make identifying such images easy, the file signature in the first 8 bytes of
such files will be GCR-1571
).
G64s created by the DTC tool from Kryoflux dumps contain extra info for each track, usually at offset $02ac right after the speedzone entries.
First comes the extra info tag:
Offset | Description |
$00-$02 | extra info tag (EXT) $45,$58,$54 |
$03 | extra info version ($01) |
Followed by 16 bytes of extra info for each track in the image:
Offset | Description |
$00 | write splice position |
$04 | write area size |
$08 | bitcell size in nanoseconds (eg 3500 means 3.5us) |
$0c | track fill value |
$0d | n/a |
$0e | format code |
$0f | format extension |
A zero value in any of these fields means that the respective default/standard value applies.
Format codes:
ID | Description |
0 | Unknown format |
1 | GCR Data |
2 | CBM DOS |
3 | CBM DOS Extended |
4 | MicroProse |
5 | RapidLok |
6 | Datasoft |
7 | Vorpal |
8 | V-MAX! |
9 | Teque |
10 | TDP |
11 | Big Five |
12 | OziSoft |
Format Extensions:
ID | Description |
0 | Unknown protection |
1 | Datasoft with Weak bits |
2 | CBM DOS with Cyan loader, Weak bits |
3 | CBM DOS with Datasoft, Weak bits |
4 | RapidLok Key |
5 | Data Duplication |
6 | Melbourne House |
7 | Melbourne House, Weak bits |
8 | PirateBusters v1.0 |
9 | PirateBusters v2.0, Track A |
10 | PirateBusters v2.0, Track B |
11 | PirateSlayer |
12 | CBM DOS, XEMAG |
This section is taken from "P64 file format specification" by Benjamin ’BeRo’ Rosseaux.
All values are in little endian order !
0 1 2 3 4 5 6 7 8 9 A B C D E F +---------------------------------------------------------------+ 0000: |'P'|'6'|'4'|'-'|'1'|'5'|'4'|'1'| Version | Flags | +---------------+---------------+-------------------------------+ 0010: | Size | CRC32Checksum | +-------------------------------+
Version: File format version, current is 0x00000000
Size Size of the following whole chunk content stream
Flags: Bit 0 = Write protect Bit 1 = Number of sides (0 for single sided, 1 for double sided) Bit 2-31 = Reserved, all set to 0 when creating a file, preserve existing value when updating
CRC32CheckSum: CRC32 checksum of the following whole chunk content stream
0 1 2 3 4 5 6 7 8 9 A B C D E F +-----------------------------------------------+ 0000: |Chunk Signature| Size | CRC32Checksum | +-----------------------------------------------+
Chunk signature: Signature of chunk
Size: Size of the chunk data
CRC32CheckSum: CRC32 checksum of the chunk data
| x = half track index byte | +—————————+ Bit 7 of the half track index byte contains the side.
Track 18 = Half track 36 = Half track index byte decimal value 36
Half track NRZI transition flux pulse data chunk block
0 1 2 3 4 5 6 7 8 9 A B C D E F +---------------------------------------------------------------+ 0000: | Count pulses | Size | ..... Range-encoded data .... | +---------------------------------------------------------------+
Count pulses: Count of the NRZI transition flux pulses in half track
Size: Size of the range-encoded data
Hint: For a working C implememtation see p64.c and p64.h
The range coder is a FPAQ0-style range coder combined with 12-bit 0-order models, one model per byte with one bit per byte processing.
+--------------------------------------------------------------------------+ | Sub stream | Count of models | Size per model | Total value bits | +------------------+-----------------+------------------+------------------+ | Position | 4 | 65536 | 32 | +------------------+-----------------+------------------+------------------+ | Strength | 4 | 65536 | 32 | +------------------+-----------------+------------------+------------------+ | Position flag | 1 | 2 | 1 | +------------------+-----------------+------------------+------------------+ | Strenth flag | 1 | 2 | 1 | +------------------+-----------------+------------------+------------------+ +===Total models===| 10 |==================|==================| +--------------------------------------------------------------------------+
All initial model state values are initialized with zero.
All initial model probability values are initialized with 2048.
These model probability values will be updating in a adaptive way on the fly and not precalculated before the encoding and even not loaded before the decoding, see pseudo code below.
16000000 Hz / 5 rotations per second at 300 RPM = maximal 3200000 flux pulses
So NRZI transition flux pulse positions are in the 0 .. 3199999 value range, which is also a exact single rotation, where each time unit is a cycle at 16 MHz with 300 RPM as a mapping for the ideal case.
The NRZI transition flux pulse stength are in the 0x00000000 .. 0xffffffff value range, where 0xffffffff indices a strong flux pulse, that always triggers, and 0x00000001 indices a weak flux pulse, that almost never triggers, and 0x00000000 indices a flux pulse, that absolutly never triggers.
For 32-bit values, the model sub streams are subdivided byte wide in a little-endian manner, and each byte is processed bitwise with model probability shifting of 4 bits, just as:
Pascal-Style pseudo code:
procedure WriteDWord(Model, Value : longword); var ByteValue, ByteIndex, Context, Bit : longword; begin for ByteIndex := 0 to 3 do begin ByteValue := (Value shr (ByteIndex shl 3)) and $ff; Context := 1; for Bit := 7 downto 0 do begin Context := (Context shl 1) or RangeCoderEncodeBit( RangeCoderProbabilities[ RangeCoderProbabilityOffsets[Model + ByteIndex] + (((RangeCoderProbabilityStates[Model + ByteIndex] shl 8) or Context) and $ffff)], 4, (ByteValue shr Bit) and 1); end; RangeCoderProbabilityStates[Model+ByteIndex] := ByteValue; end; end;
And for 1-bit flag values it is much simpler, but also with model probability shifting of 4 bits, just as:
Pascal-Style pseudo code:
procedure WriteBit(Model, Value : longword); begin RangeCoderProbabilityStates[Model] := RangeCoderEncodeBit(RangeCoderProbabilities[ RangeCoderProbabilityOffsets[Model] + RangeCoderProbabilityStates[Model]], 4, Value and 1); end;
The position and strength values are delta-encoded. If a value is equal to the last previous value, then the value will not encoded, instead, a flag for this will encoded. First the position value will encoded, then the stength value. If the last position delta is 0, then it is a track stream end marker.
Pascal-Style pseudo code:
LastPosition := 0; PreviousDeltaPosition := 0 LastStrength := 0; for PulseIndex := 0 to PulseCount - 1 do begin DeltaPosition := Pulses[PulseIndex].Position - LastPosition; if PreviousDeltaPosition <> DeltaPosition then begin PreviousDeltaPosition := DeltaPosition; WriteBit(ModelPositionFlag, 1) WriteDWord(ModelPosition, DeltaPosition); end else begin WriteBit(ModelPositionFlag, 0); end; LastPosition := Pulses[PulseIndex].Position; if LastStrength <> Pulses[PulseIndex].Strength then begin WriteBit(ModelStrengthFlag, 1) WriteDWord(ModelStrength, Pulses[PulseIndex].Strength - LastStrength); end else begin WriteBit(ModelStrengthFlag, 0); end; LastStrength := Pulses^[PulseIndex].Strength; end; // End code WriteBit(ModelPositionFlag, 1); WriteDWord(ModelPosition, 0);
The decoding is simply just in the another direction way.
Pseudo code for a FPAQ0-style carryless range coder:
Pascal-Style pseudo code:
procedure RangeCoderInit; // At encoding and decoding start begin RangeCode := 0; RangeLow := 0; RangeHigh := $ffffffff; end; procedure RangeCoderStart; // At decoding start var Counter : longword; begin for Counter := 1 to 4 do begin RangeCode := (RangeCode shl 8) or ReadByteFromInput; end; end; procedure RangeCoderFlush; // At encoding end var Counter : longword; begin for Counter := 1 to 4 do begin WriteByteToOutput(RangeHigh shr 24); RangeHigh := RangeHigh shl 8; end; end; procedure RangeCoderEncodeNormalize; begin while ((RangeLow xor RangeHigh) and $ff000000) = 0 do begin WriteByteToOutput(RangeHigh shr 24); RangeLow := RangeLow shl 8; RangeHigh := (RangeHigh shl 8) or $ff; end; end; function RangeCoderEncodeBit(var Probability : longword; Shift, BitValue : longword) : longword; begin RangeMiddle := RangeLow + (((RangeHigh - RangeLow) shr 12) * Probability); if BitValue <> 0 then begin inc(Probability, ($fff - Probability) shr Shift); RangeHigh := RangeMiddle; end else begin dec(Probability, Probability shr Shift); RangeLow := RangeMiddle + 1; end; RangeCoderEncodeNormalize; result := BitValue; end; procedure RangeCoderDecodeNormalize; begin while ((RangeLow xor RangeHigh) and $ff000000) = 0 do begin RangeLow := RangeLow shl 8; RangeHigh := (RangeHigh shl 8) or $ff; RangeCode := (RangeCode shl 8) or ReadByteFromInput; end; end; function RangeCoderDecodeBit(var Probability : longword; Shift : longword) : longword; begin RangeMiddle := RangeLow + (((RangeHigh - RangeLow) shr 12) * Probability); if RangeCode <= RangeMiddle then begin inc(Probability, ($fff - Probability) shr Shift); RangeHigh := RangeMiddle; result := 1; end else begin dec(Probability, Probability shr Shift); RangeLow := RangeMiddle + 1; result := 0; end; RangeCoderDecodeNormalize; end;
The probability may be never zero! But that can’t happen here with this adaptive model in this P64 file format, since the adaptive model uses a shift factor of 4 bits and initial probabilities value of 2048, so the probability has a value range from 15 up to 4080 here. If you do want to use the above range coder routines for other stuff with other probability models, then you must to ensure that the probability output value is never zero, for example with "probability |= (probability < 1); " in C.
This is the last empty chunk for to signalize that the correct file end is reached.
Next: The D64 disk image format, Previous: The T64 tape image format, Up: The emulator file formats [Contents][Index]