Preserve Macintosh MFS floppies

I have boxes of old floppies in storage and thought it best I take some steps to archive them. Sounds simple, but like everything to do with retro computing, it's never as simple as it sounds.

Greaseweazle

I have quite a variety of disks to back up... both 5.25" and 3.5" disks from Apple ][ through Atari, Amiga, Macintosh and early PC. A little research suggested I needed a device capable of reading the raw magnetic flux from the disks to have the best chance at preserving these relics of history. The choice between a $x00 Kryoflux and a $x0 GreaseWeazle was an easy one.

While waiting for the GreaseWeazle to arrive from overseas, I trawled through my stash of old computers as well as some piles of e-waste and all the local thrift shops and I built a computer to house it. This is based on an X58 motherboard (a nice balance between old and new for a workhorse machine) and contains a CD-ROM, two spare 5.25" slots, a Panasonic 3.5" floppy and an NEC 3.5" floppy drive. The spare 5.25" bays are earmarked for a Plextor DVD drive and a 5.25" floppy drive. This will be the ultimate retro archival engine!

The system can triple-boot from a single SSD to Windows XP, Windows 10 and Ubuntu 22.04. Building a drive which can triple-boot is a whole other week-long story. This setup gives access to a really wide range of software and utilities and it can also be booted to DOS from USB or floppy if required for flashing optical drive firmware or other fun shenanigans.

The GreaseWeazle arrived just in time to coincide with my week-long Christmas leave, and with high hopes and no sign of lessons learned from past experience I expected to archive a few hundred floppies. Alas this was not to be, but rest assured the story has a happy ending.

Floppies

First off, I dug out a few floppies with nothing useful on them. I selected some old PC format 1.44MB floppies containing X-Wing and Xtree Gold, the content being already available through archive.org. So if I trashed these disks it didn't matter.

The learning curve here was pretty gentle, and I soon had a grasp of the different formats (KF-Raw, SCP, img) and the various software tools (GW.exe, Kryoflux, HxC) and I developed what I thought was a pretty nice process for recording on the NEC FD231M the raw magnetic flux transitions for posterity as well as generating a disk image which exactly matched the archive.org versions.

mkdir "C:\Floppies\xtree-3.0-d1"
gw.exe read --revs=5 --drive=A --tracks=c=0-84:h=0-1 "C:\Floppies\xtree-3.0-d1\xtree-3.0-d1.00.0.raw"
gw convert --format ibm.1440 "C:\Floppies\xtree-3.0-d1\xtree-3.0-d1.00.0.raw" C:\Floppies\xtree-3.0-d1.img"

This:
- creates an empty directory to hold raw flux image, one file per track
- reads the disk with lots of retries, saving 5 complete revolutions of the disk for each track and reads 84 tracks despite there generally only being 79 tracks on a disk
- converts the raw files to a standard 1.44 MB floppy image.

At this stage, I'm feeling pretty pleased with myself, despite being 2/3 of the way through my holiday and with only 8 floppies completely imaged. I'm thinking it will all be good now.

Macintosh disks

And then I inserted a Macintosh floppy from 1988.

Well, long story told short, two weeks later I have developed a process to generate an image from these Mac disks. The process is simple, with just three simple command lines. But the process of crafting them was long and the tools are not well documented. I have well over 100 tabs open, each containing a hint, but none containing a solution.

Neither the NEC nor the Panasonic drives were able to deal with the Mac MFS disk. They both reported read errors, generally on the outer tracks of the disk. After a number of trials, I settled on a Sony MPF-920 drive salvaged from e-waste. This drive was able to read the Mac floppies where other drives had failed. Rumour has it that TEAC drives are also good, but alas my only TEAC drive is faulty.

The process for MFS floppies is:

For single-sided 400k disks:

Capture the raw flux files, one disk per directory:
mkdir "c:\Floppies\disk_name_d1"
gw.exe read --revs=5 --drive=A --tracks=c=0-79:h=0 "c:\Floppies\disk_name_d1\disk_name_d1.00.0.raw"

Convert to Disk Copy 4.2 compatible image:
fluxengine.exe read mac --400 -s kryoflux:"c:\Floppies\disk_name_d1" -o c:\Floppies\disk_name_d1.diskcopy

Convert to MacBinary format which adds the file creator and type so it's usable on a Mac emulator:
convert_to_macbinary.py "C:\Floppies\disk_name_d1.diskcopy" "C:\Floppies\disk_name_d1.img.bin"

Copy this file into a Mac emulator (Basilisk II running System 7.5 works well), drop it onto StuffItExpander, and the disk image should mount on the desktop.

For double-sided 800k floppies:

mkdir "c:\Floppies\disk_name_d2"
gw.exe read --revs=5 --drive=A --tracks=c=0-79:h=0-1 "c:\Floppies\disk_name_d2\disk_name_d2.00.0.raw"
fluxengine.exe read mac --800 -s kryoflux:"c:\Floppies\disk_name_d2" -o c:\Floppies\disk_name_d2.diskcopy
convert_to_macbinary.py "C:\Floppies\disk_name_d2.diskcopy" "C:\Floppies\disk_name_d2.img.bin"

In both instances, watch the initial rip to make sure it completes without error. If it fails to read any sectors, that needs to be fixed (clean the drive or the disk). Also watch the conversion to diskcopy, if that doesn't convert 100% of the disk, there will be trouble.

I've not yet had to deal with unrecoverable errors on disks, so this process is only known to work on the golden path of success. I'll likely post a followup discussing salvage from failing disks.

Dealing with simple errors

The gw.exe command above just reads raw flux from disk and writes it to file. It doesn't validate the data in any way. So for example including the option --retries=20 on this command will have no effect because the command isn't looking for errors. But at the next step, occasionally the fluxengine command will return some errors indicating the raw flux was corrupted and the original data was unable to be reconstructed from the flux stream.

The first step in this situation is to clean the drive heads and closely inspect the floppy disk and ascertain whether it's damaged or dirty. My disks have been stored in sealed boxes in a dry environment so I've not yet seen any disks which are rotting away from humidity or mold. Some have been a bit dusty or crusty, and I gently wipe the disk surface with a microfibre cloth dampened with 65% isopropyl alcohol. I've heard this can be a bit harsh and can actually contribute to degradation of damaged disks, and I'm watching carefully to learn more about this. Another option is soap and water.

Following this simple remediation, I make sure the original dump is safe and secure and run a new command:

mkdir "c:\Floppies\disk_name_d3"
gw.exe read --retries=20 --revs=9 --format=mac.800 --raw --drive=A --tracks=c=0-79:h=0-1 "c:\Floppies\disk_name_d3\disk_name_d3.00.0.raw"
fluxengine.exe read mac --800 -s kryoflux:"c:\Floppies\disk_name_d3" -o c:\Floppies\disk_name_d3.diskcopy
convert_to_macbinary.py "C:\Floppies\disk_name_d3.diskcopy" "C:\Floppies\disk_name_d3.img.bin"

This command allows gw.exe to interpret the flux, convert to Mac sector data and validate that the data is correct. If a sector is detected as bad, the retries argument then takes effect it will be re-read for another set of revolutions and re-validated until the entire track has been successfully read. The --raw argument causes gw.exe to output the raw flux dumps, which can generate some pretty large files. A shortcoming of this technique is that it can read from the disk for quite a long time, and if the disk has any remaining dust in the casing, or if the disk surface is fragile, this could result in irreparable damage to the disk.

Update: Following a quick read of the .raw file spec, it is a block-based format with each block identifiable by a few header bytes. Simply concatenating a new .raw file to the original works fine with fluxengine (although "gw.exe convert" will not find the additional revolutions). Using this technique, gw.exe can be used to dump additional revolutions of problematic tracks, this data can be combined with the original dump and re-analysed. This is useful for iteratively fine tuning the dump with a series of short reads rather than attempting one massive scan which could potentially take an hour.

Example of this technique, watch track 64:

mkdir "c:\Floppies\tmp2"
gw.exe read --revs=5 --raw --drive=A --tracks=c=0-79:h=0-1 "c:\Floppies\tmp2\tmp2.00.0.raw"
fluxengine.exe read mac -800 -s kryoflux:"c:\Floppies\tmp2" -o c:\Floppies\tmp2.diskcopy

Extract of the result:
64.1: 595 ms in 150284 bytes
      no more data; giving up
      48 raw records, 20 raw sectors; 3.97us clock (252kHz)
      sectors: 64.1.0! 64.1.1! 64.1.2! 64.1.3 64.1.4? 64.1.5 64.1.6! 64.1.7
      3668 bytes decoded

     Tracks -> 1         2         3         4         5         6         7
H.SS 01234567890123456789012345678901234567890123456789012345678901234567890123456789
1. 0 ................................................................BBB.B..B........
1. 1 ................................................................BBBB?B..........
1. 2 ................................................................BB?BB...........
1. 3 ..................................................................BBBB..........
1. 4 ................................................................?B?BBBB.B.......
1. 5 .................................................................BBB............
1. 6 ................................................................BBBB.B..........
1. 7 .................................................................BBBBB..........
1. 8 ................................................................XXXXXXXXXXXXXXXX
1. 9 ................................................XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
1.10 ................................XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
1.11 ................XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Good sectors: 1496/1920 (77%)
Missing sectors: 320/1920 (16%)
Bad sectors: 104/1920 (5%)

mkdir "c:\Floppies\tmp3"
gw.exe read --retries=3 --revs=7 --format=mac.800 --raw --drive=A --tracks=c=64-79:h=1 "c:\Floppies\tmp3\tmp3.00.0.raw"
# In git bash:
cd /c/Floppies/tmp3 ; for FN in * ; do FO=$(echo $FN | sed -e 's/tmp3/tmp2/') ; cat $FN >> ../tmp2/$FO ; done
fluxengine.exe read mac -800 -s kryoflux:"c:\Floppies\tmp2" -o c:\Floppies\tmp2.diskcopy

64.1: 3160 ms in 796879 bytes
      no more data; giving up
      254 raw records, 111 raw sectors; 3.97us clock (252kHz)
      sectors: 64.1.0 64.1.1? 64.1.2? 64.1.3 64.1.4 64.1.5 64.1.6 64.1.7
      3144 bytes decoded

     Tracks -> 1         2         3         4         5         6         7
H.SS 01234567890123456789012345678901234567890123456789012345678901234567890123456789
1. 0 .................................................................B..............
1. 1 ................................................................?BB.............
1. 2 ................................................................?.BBB...........
1. 3 ................................................................................
1. 4 .................................................................BB.............
1. 5 ................................................................................
1. 6 ..................................................................BB....C.......
1. 7 .................................................................BB.B...........
1. 8 ................................................................XXXXXXXXXXXXXXXX
1. 9 ................................................XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
1.10 ................................XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
1.11 ................XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Good sectors: 1526/1920 (79%)
Missing sectors: 320/1920 (16%)
Bad sectors: 74/1920 (3%)

mkdir "c:\Floppies\tmp4"
gw.exe read --retries=5 --revs=7 --format=mac.800 --raw --drive=A --tracks=c=64-72:h=1 "c:\Floppies\tmp4\tmp4.00.0.raw"
# In git bash:
cd /c/Floppies/tmp4 ; for FN in * ; do FO=$(echo $FN | sed -e 's/tmp4/tmp2/') ; cat $FN >> ../tmp2/$FO ; done
fluxengine.exe read mac -800 -s kryoflux:"c:\Floppies\tmp2" -o c:\Floppies\tmp2.diskcopy

64.1: 13068 ms in 3295385 bytes
      1052 raw records, 475 raw sectors; 3.97us clock (252kHz)
      sectors: 64.1.0 64.1.1 64.1.2 64.1.3 64.1.4 64.1.5 64.1.6 64.1.7
      4192 bytes decoded

     Tracks -> 1         2         3         4         5         6         7
H.SS 01234567890123456789012345678901234567890123456789012345678901234567890123456789
1. 0 ................................................................................
1. 1 ..................................................................B.............
1. 2 ...................................................................B............
1. 3 ................................................................................
1. 4 ..................................................................B.............
1. 5 ................................................................................
1. 6 ..................................................................BB....C.......
1. 7 .................................................................BB.............
1. 8 ................................................................XXXXXXXXXXXXXXXX
1. 9 ................................................XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
1.10 ................................XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
1.11 ................XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Good sectors: 1532/1920 (79%)
Missing sectors: 320/1920 (16%)
Bad sectors: 68/1920 (3%)

This process has recovered an additional 36 sectors from the disk and could potentially be continued, providing the disk surface is not degrading.

References

I can be contacted about this process at: archivist@keepondigging.com

That's all for now. Have fun, and
KeepOnDigging.