Saturday, 9 August 2003
Sorry, trackback should actually work now. Forgot to actually click the “enable trackback” checkbox. Oops.
Back to work…
11:43 PMHere's something I posted on the Boston BBS, which I figure might be useful to someone else. It traces the terminology relating to disks from Mac OS 9 to Mac OS X in a hopefully intelligible fashion.
Q. I'm trying to understand the differences between a disk image, a folder, and a partition. Does anyone have the energy?
A. There can be a lot of confusion over terminology. Mac OS 9 and Mac OS X can use different terms to refer to the same entities. Unix adds some more terms of its own, which have filtered down into Mac OS X in some places. I hadn't thought about all the complexity until I attempted to write this message. I'll try to define each of these, sometimes more than once: disk, drive, folder, partition, volume, slice, image.
In Mac OS 9, the icons you see at the top right corner of your desktop represent disks. Most Macs come from the factory with a single disk, named “Macintosh HD.” In most cases, each of the disks you see on the desktop corresponds to a physical object. Insert a CD or DVD: you'll see its icon appear on the desktop. Attach your iPod, and the same thing happens.
Folders are organizational aids which exist within a disk. When you move an item into a different folder on the same disk, you're just updating the disk directory to give the item a new parent. The file itself doesn't move on disk.
Mac OS 9 also employs the concept of a drive, or container for one or more disks. For floppy disks, the drive is the enclosure, and a disk is a physical floppy disk. For fixed drives such as hard drives, using Apple's Drive Setup or a similar utility, you can partition the drive into one or more partitions, the contents of which appear as separate disks.
In Mac OS 9's Apple System Profiler, one of the tabs is labeled “Devices and Volumes”. The Mac OS 9 concept of devices include drives, but also memory cards, keyboards, mice, cameras, scanners and so on: anything that can be connected to your Mac via ADB, USB, SCSI, FireWire, and so on.
You can think of a partition like a space on a bookshelf, and a volume as the book filling that space. All the partition specifies is where the book begins and ends on the shelf; the volume contains its own internal structure such as titles, tables of contents, indices, etc, as well as text and graphics (your files). This internal structure is known as a filesystem. Common filesystems are Mac OS Extended (HFS+) and its journaled variant on Mac OS X, and FAT32 and NTFS on Windows 2000/XP.
Volumes can also exist without enclosing disks (partitions): one example is AppleShare. The AppleShare client asks you which volumes you want to mount, after you've connected to a server. Since the corresponding storage area on the file server may be a folder, or an entire disk, it can't be realistically referred to as a disk. Network filesystems do not have corresponding physical formats on disk, but implement network protocols—the Apple Filing Protocol, in this case.
The disk/drive separation began to break down with the introduction of high-capacity removable, desktop-mountable media (CD, Bernoulli/SyQuest, etc.) Each removable disc or cartridge could be partitioned, just like a hard drive, and it would appear on the desktop as multiple “disks”. Referring to a CD or Bernoulli cartridge as a “drive”, or relying on the difference between “disc” and “disk” would be to nit-pick, but it took until Mac OS X for Apple to redefine “disk” for consistency.
What Mac OS 9 calls a drive, Mac OS X calls a disk. Disks in Mac OS X include hard drives, floppy drives, flash memory cards, and so on. Note that “Drive Setup” in Mac OS 9 has been replaced by “Disk Utility”, indicating the change in wording. Where Mac OS 9 may refer to the contents of a floppy disk or hard drive partition as a disk, Mac OS X only uses the term volume (“disk” was already taken). From now on, I will only use the Mac OS X meaning of “disk”.
I mentioned above that filesystems and volumes can exist without partitions, in the case of network filesystems, or in Unices, special-purpose filesystems such as device filesystems. (More on the expanded meaning of “device” under Mac OS X in a minute.) Partitions can also contain other things than filesystems. If you're using Mac OS X, open Terminal and type “sudo pdisk /dev/rdisk0 -dump” to receive a list of partitions on your startup drive. On my PowerBook, the output looks like this:
/dev/rdisk0 map block size=512 #: type name length base ( size ) 1: Apple_partition_map Apple 63 @ 1 2: Apple_Driver43*Macintosh 56 @ 64 3: Apple_Driver43*Macintosh 56 @ 120 4: Apple_Driver_ATA*Macintosh 56 @ 176 5: Apple_Driver_ATA*Macintosh 56 @ 232 6: Apple_FWDriver Macintosh 512 @ 288 7: Apple_Driver_IOKit Macintosh 512 @ 800 8: Apple_Patches Patch Partition 512 @ 1312 9: Apple_HFS Ti 69947904 @ 1824 ( 33.4G) 10: Apple_HFS Puma 8190424 @ 69949728 ( 3.9G) 11: Apple_Free 0+\@ 78140152d
There are 11 partitions on my hard drive. The first is the partition map itself, which contains the information above. Without it, a Mac would have no idea where to find the partitions on the hard disk—it'd just be one long stream of data. Each partition in the map has a type, a name (which does not necessarily correspond to the displayed name on the Desktop), a length and base address. The two Apple_HFS partitions do indeed contain HFS(+) filesystems; the other partitions with name “Macintosh” contain driver code (for pre-OS X SCSI Manager 4.3, ATA, FireWire, and Mac OS X's IOKit, respectively). Disk Utility will copy the first few driver partitions only if the “Include Mac OS 9 Drivers” box is checked.
Drivers and partition maps aren't the only thing non-filesystem partitions are used for. Most Unix variants can use designated swap partitions to handle virtual memory paging, and the iPod, for example, uses a partition to hold its operating system separate from your music and data.
“Devices” are a more generic term, yet can refer to more specific entities under OS X, thanks to its Unix heritage. A device is any item which is individually addressible through a Unix device node (a special type of file, which is generally hidden). For disks, you typically have a “raw” (unbuffered) disk device (the “/dev/rdisk0” of above), which applications such as Disk Utility use to format and partition disks, a corresponding entire-disk device, and indvidual devices for each partition. The buffered disk device for my hard disk is /dev/disk0, and the individual partitions are /dev/disk0s0, disk0s1, and so on. The 's' indicates a slice number, which is a BSD-derived word for something resembling a partition. (FreeBSD uses partitions inside slices because of limitations in the PC partition table format, which are labeled by letters such as “b” for the swap space and “c” for the entire slice… but this is going too far off topic.)
An image file (on the Mac, one typically ending in .img or .dmg) consists of the data in a disk or partition. Apple's imaging utility, Disk Copy, in Mac OS 9 only makes volume images, which contain a single partition and no partition map. Disk Copy can construct a volume image from an existing volume, or create a volume to image on the fly from the contents of a folder you drag onto it.
With the redefinition of “disk” in Mac OS X, the new Disk Copy gains the ability to make disk images. These images come complete with a partition map and, potentially, multiple partitions, though I've never seen the latter in practice. OS X's Disk Copy can still make volume or folder images if you wish, and introduces a new format called a sparse image file, which starts off small and grows as you add data to the image through normal operations. (Sparse images are inefficient for everyday usage; they're mainly useful if you're copying a bunch of files to an image and don't know in advance how large the image will be.)
You can do almost anything with an image file you can do with any other disk or volume. Starting with an old control panel called MountImage, you could mount images on the desktop (on other operating systems, this is known as a loopback mount), or you can write them to real disks or partitions. Internet-enabled disk images in Mac OS X 10.2.3 and later are automatically mounted, their contents copied to a folder in place of the image, and the images deleted—which results in their behaving much like a StuffIt archive.
As much as I've written, I'm only covering the simplest scenarios here. There are as many ways to map your files to physical storage as you can think, from logical volume managers to RAID, HSM, SAN and NAS, various forms of distributed caching and disconnected operation. Most of these are not easily available on the Mac, though managing your files is hard enough already that I hope Mac OS X migrates beyond fixed partitions and flat files in the near future.
I wrote most of this from memory, so corrections are appreciated.
Thursday, 7 August 2003
This week, I managed to fry both of the external ports on my FireWire PCI card, and the supposedly protected FireWire port on my PowerBook. Many months ago I dumped my old cheap FireWire enclosure, and now only use a WiebeTech AC-powered bridge, an iSight and iPod, and a high-quality Adaptec cable to connect my two machines for FireWire over IP. Since I haven't plugged anything except the Adaptec cable into my PowerBook for several weeks, I think my use of FireWire over IP is responsible, possibly when combined with the static electricity that is rampant at the lab. So I'm going to buy an Ethernet card to use instead, and send back my PowerBook for warranty repairs when I get a chance.
Just ordered a replacement FireWire card; hopefully it will last me until it's time to replace my desktop machine (most likely next summer).
None of this is particularly helping with my research, which is proceeding at an astonishingly slow pace since I got back from Norway. I haven't had any time to spend on my Cocoa projects in weeks. Nor have I finished writing my journal or organizing photos. But if I lose a hard drive and have no current backup, it will be even worse, so I'm pouring money and time into making my computers work again.
On the bright side, my apartment is in great shape, Steve is about 80% moved in, I went to Kuk Sool on Monday, our cable modem install is scheduled for next Monday, and I finally have made some progress on getting my car registration and insurance moved out to Illinois.
Monday, 4 August 2003
My parents have been visiting Champaign since Thursday and Friday, respectively. We've been cooking, cleaning, and fixing up my apartment all weekend. Steve moves in tomorrow.
This blog has trackback now, thanks to Jake.
Somehow my spam load went from 3-5 messages per day to over 100 messages per day over the weekend. Time to install a spam filter.
Monday, 21 July 2003
I'm back; a power outage while I was in Norway accounts for my lack of updates, as I did end up with decent Internet access for much of the time. I got arnold back up and running when I was back in Oslo, and byron is now up though flaky (need to get that USB hub replaced).
I had a really fabulous time in Norway. Lots of stories to tell. I took about 1100 photos, and was by no means the most prolific. But my first priority this week is research. Before I crawl back into my hole, here's an ad that caused me to crack up, from the Oslo subway:
