I know Calvin and Hobbes has been offline for a long time, and I know why, but it always surprises me how lively and current it feels even after all this time. The connection is still there. It’s still funny. Mr. Watterson, thank you.

Jodi Picoult : Vanishing Acts.

Yes, I know it’s a book by a girl. And I’m starting to notice that entirely too many girl-written books have the same attributes:

1. Girl hero character is always object of hidden, secret love that comes out in book, even when it does nothing for the story.

2. Love triangle thing always happens.

3. Girl hero is always right. Even when it’s illogical, and stupid. Because she just is.

4. Girl hero character sleeps with forbidden member of love triangle in 85% of these books.

Which begs the question – are girl authors not getting enough action? Or is this just playing to the audience of girls that don’t get enough action? Either way, I’m getting tired of this pattern.

Flipping the same flashlight on guy-written books you get:

Action books:

1. Manly man hero character is always allowed to kill someone. At any time. With anything. And rarely gets hurt.

2. Manly man rarely if ever has secret burning love of character in book. Either they’re in bed or he’s forgotten.

3. He’ll just jump in bed with whoever.

Fantasy books:

1. Wimp at beginning of story is wronged.

2. Wronged wimp suddenly discovers magic/fantastic power or lifelong friend.

3. Wronged wimp and comedy tagalongs must escape/take on noble journey.

4. Immediate family or equivalent bond is sworn to kill them, or everyone, or both, or destroy the world, or whatever.

5. Powers have no cost, and are entirely overused.

6. Readers lose interest and only finish out of commitment to finishing what they started in the dire hopes that it gets better, which it never does.

Scifi books:

1. One neat or perhaps clever idea is presented at the beginning as the crux of a theoretical society.

2. The idea is beat to death.

3. Every instance of, “Dammit!” is replaced with a slightly altered version that reflects on point 1, such as “By Space!”, or “By Selden!”, as if it made sense.

4. Every other aspect of human life is ignored as a convenience to the reader. The only elements left to these societies is science, politics, and sex. Love, art, music, children, funding, etc. cease to exist. Conveniently.

Back to Vanishing Acts.

There are a few things that I liked. The pace was good. The characters were okay. The imagery was good. The shock value was up there. But why in the world does it have to follow the girl pattern? He has always loved her, but left it alone for her happiness. She’s had CharA’s baby, but decides suddenly it’s time to jump beds and take up with his and her Other best friend, because once you’ve had one guy’s baby, why not have the other one’s too? He gives up Everything to be near her.

Has this woman not been to Louisiana? Has she not met women before? Has she not met a GUY before? Which leads me to the obvious alternative: they’re symbolic. So what do they represent:

Hero Girl: I stopped maturing at 17 and my hormones are uncontrolled. I’m the mental projection of every locked-in housewife south of the snowline. I’m slim, attractive, emotionally wrecked, always right, and nothing is my fault!

Baby’s Father: I’m the drunk you fell in love with in high school and wonder what would have happened if you stayed with. I clean up, become a lawyer, and then go back on the bottle, so aren’t you glad you broke up with him/me? And when we break up and you take away my baby, I give in because you’re always right, and nothing is your fault.

Best Friend: I’m the intellectual and moral backbone of your friendships in high school that you wish you had slept with (or slept with more). I should get the girl, so in this book I will! I will leave my job and home to follow you around. Aren’t I cooler than the person that provides for you? Wow, you’re always right.

Dad: If I were real, you’d love to have had me instead of whatever defective real people brought you up and cared for you. I represent the overidealized parent that would do anything. Wouldn’t you rather have me than your own parents? I thought so. Because you’re always right.

Really girls – get over this. It’s ruining your books.

I just had a pleasant evening after a rough day. Work stuff – very boring.

Come home and get my hands in the dishwater as the kids finish reading books with mom. “Dada! Someone very small wants you!” comes from upstairs. The three year old is there, one-piece jammies on, hearing aids away, bottom lip out, standing on the side of the crib, pointing to the guest bed and murmuring, “Dada, bed”. He’s asking to be held on the bed while he falls asleep. This warms me up especially because this has been a Mama month, the Dada being out of favor for constantly putting on The Wrong Shoes, The Wrong Shirt, Restraining Me While Walking in Parking Lots, and all sorts of other 3-year-old infractions.

Up on the shoulder he goes, head down, arms splayed across dada, and we lay down on the bed. In 10 minutes, he’s all floppy.

I’m going to miss that all too soon.

“What, Dave, more?” you ask. Yes, I’m afraid so. I’m afraid so. Trying to keep it short this time.

All builds have a server, 4+ disk RAID5, and offer flexible storage expansion to keep them around for about 2 to 4 years. Each has a “Priced from scratch” number. Remember that this includes 4x $100 drives (currently the 1.5 to 2TB range) that you might already own, or be willing to substitute your old 200-300 gig drives for instead, or whatever. I think you get the idea.

Optimize for money, with Mac guest:

Priced from scratch: $1900ish

• Mac mini (any flavor), add RAM to 4. ($600-1000) ($300-700 on craigslist)
• VMware Fusion ($70)
• Mac OS Server (presumably the license you got from buying the Mini Server, right?) ($500 or included with mac)
• Drobo via local connection for the host, guests live on the Drobo ($400)
• disks ($400)
• Apple USB dongle (technically support is only for the MBAir, but it works. Trust me.) if you want to use a VM router. ($30)

Pro:

• Low electricity.
• Low heat.
• Few surprises.
• Drobo adds the ability to just keep expanding your storage for either more VMs or more space on them without having to worry about iSCSI components losing power between you and the array.
• Easy UI and remote control with Back to my Mac and/or VNC.
• Low TCO – macs depreciate slowly. If it’s time to upgrade to something more powerful later, you can sell it.

Con:

• Few server-quality expansion options.
• No internal high-speed RAID.
• The Drobo will only hit 40ish meg a second, but honestly for home, you’ll probably not need that much all the time.
• VMs limited to 100mbit connections.

Optimize for money, without Mac guest:

New

Priced from scratch: $1550ish

• DIY Intel Core2 quad based on LGA775, 8 gig of ram (priceout on newegg: $550 + hard drives)
• ESXi $0
• Adaptec 2405 RAID controller ($200ish) and SAS case/enclosure ($400)(or other supported card/case/enclosure setup)
• disks ($400)
• Optional substitution: iSCSI remote storage (like FreeNAS or Drobo Pro)(-1000, +cost of chosen iSCSI target)

Pro:

• ESXi is really the right tool for the job.
• Can sometimes get away with a very cheap hardware build.
• Some VMs can go up to gigabit connections if you absolutely must have it.
• Optional iSCSI/fiberchannel storage growth paths open to you.
• Faster and more powerful than the Mac Mini option, but you’ll have some surprises building the box. There is always a hidden surprise.

Con:

• ESXi doesn’t support lots of cheap hardware, so you’ll have to be careful about the hardware you pick.
• You still need a separate windows box/VM around to manipulate things, or get really comfortable with the VMware CLI.
• Higher overall TCO? It’s usually harder selling a used medium-low power server in two to four years.
• It will use more electricity for those extra cores and extra RAM, but if you do video work with this box, it may pay off to have them.
• Electricity/heat in the middle range. Initial expense may convince you that the maintenance is worth it.
• Doesn’t support the mac guest, but there are alternatives to OS X Server’s two most attractive features for the home: iCal server and TimeMachine on flexible storage. Dang, that’s a pro. Moving on.

Used

I will point out that the previous generation of HP/Dell ESX supported servers are regularly on craigslist for cheap – usually $400 and under. Rackmounted and everything, but usually running “real” SCSI disks, very loud, very hot, very small. They still will need $1000 in storage, the same as the rest, so buyer beware, but if you’re just going for the VM server and can skip the storage, you can get in on the cheap.

Optimize for speed, with Mac guest:

Priced from scratch: $5000ish + 2 kidneys

• Mac Pro 8-cores Nehalem, 16GB RAM ($kidney)
• Mac OS X Supported RAID controller plus enclosure and disks. ($other kidney + $1000)
• Mac OS X Server (bundle with Mac?) ($500)
• VMware Fusion ($70)
• disks ($400)
• Alternate to RAID controller and enclosure: Drobo plus disks ($300-3000)

Pro:

• Fast.
• Very fast.
• Lots of memory for lots of VMs.
• Lots of very fast storage.
• Potential no-downtime storage growth.

Cons:

• Dude, you’re out of kidneys.
• Limited hardware choices for SAS RAID.
• Limited iSCSI options.
• Speed gain over Core-2 chips might not enough to warrant price differential unless you do a lot more math than most of us do.
• TCO for depreciation can be large when initial investment is also large.
• Hot. Cooling is expensive.

Optimize for speed, without Mac guest:

Stop. At this point we’re priced out of reasonable home options. A similar spec to the Mac Pro can be built on Newegg for about $1200 with a RAID card. But at that point, you might as well get one of the Asus barebones servers that are fully VMware certified and attach to external RAID storage. When I get as far as this, I know we’ve exited “home” and reached “business” expense levels, at which point the economics of consolidation have already broken down. How many systems do you have to sell and turn off – in your house - to cover the depreciation of a Mac Pro, over the same for a Mini?

Hope that was short enough. Comment away!

One of the abstraction problems I had with plotting out home virtualization is that I’ve already solved a number of these problems, and I don’t have a dedicated budget to go about replacing my whole – working – infrastructure. But knowing how much electricity is lost and heat generated with each little power brick and redundant device I owned, I needed a way to plan the flow from real to virtual. I finally found a way I like, and I’ll offer it as a time-saver for you. Comments welcome, maybe there’s a better way.

Eventually I settled on a layer cake analogy, more or less a table that showed what I was doing and where it was possible, and the game was to shift as much as possible to the left, into the VM column. But this also let me pick how the progression went. I can keep several of my layer cake tables showing a phased motion toward the virtual.

My first step was to document exactly what services require this mess at home and where it’s currently being handled:

Some of those things were not yet implemented but were wish-list items, like the proxy, video encoder as VM, and MythBackend. Anyway, the point is that now I want to move as many things as possible, independent of each other, to the leftmost column available. So the next table shows the eligible places that each service could move:

For the record? This is a lot easier on a pad of paper.

There are now two missing pieces. One is to investigate the technical options for moving each service into a separated VM to provide that service. Second is to plot the order and requirements in such a way that everything eventually ends up where you actually want it. That’s a lot harder than it sounds.

Options are Optional
Now I want to start with the observation that while VMware does have an appliance marketplace, there are entirely too many all-in-one appliances in there. But with the advent of this newfangled thing called the internet, I found that it’s almost as easy to do an easy-install of Ubuntu and install the one package I want, and make that the new dedicated virtual appliance I need. After all, to me the point of this is to have a lot of little VMs doing dedicated jobs, not to have yet another monolithic appliance that does it all.

So I started on my list of service requirements and did some reading. For instance I found instructions on how to build up an iCal server using Apple’s server, but on Ubuntu instead of the integrated OS X Server one. So that’s a VM I built and have on hand. The only issue there will be handling upgrades manually if they change the schema. I’m not sure I’m willing to risk it, but I have a backup server anyway.
Next I built a MythBuntu backend-only install and configured it for my network, then another for just a frontend. I don’t have an encoder card installed anywhere anymore, but it’s good to know it’s ready when I want it. And it’s SO MUCH EASIER THAN IT WAS 5 YEARS AGO. Wow.
The proxy was probably the easiest one – I found one prebuilt in the VM Library that was nothing more or less than a proxy server with virus scanning built in. Perfect!
RadioLover was easy, I just copied my license and settings into the OS X Server VM and started it up. The only Mac I can virtualize is server, so that’s pretty much it unless I switch to other software.
Indigo did not go so well. I was moving this service off an old G4 mac mini where it had been running time out of mind. The issues started when I decided to try to run it inside the VM. Something about the driver doesn’t get along with the USB control unit, and no amount of fiddling fixed it. I’ll have to bring it in to the USB devices team and have them poke about. However, as my mac mini host is still going to be on, I just reinstalled it there, where it was happy enough.
At some point I’m going to have to resolve (in money) my storage solution. It has to live in the noisy, hot garage where I don’t have to hear it. It has to be reasonably fast. There is an option to have it as a virtualized software RAID, so it’s available. I know, I’ve discussed this before, but it needles me so.
So after a day or two of contemplation and migration, this is the new reality:

Items in parenthesis are alternative options, or things I’ve tried but plan not to use. But now it’s time to see how much more can make the leftward shift.

I hope this makes sense to other people, it sure helped me see how I could start doing this at home. I honestly think we’ll start to see a commercial product doing something like this in the near future.

Apparently it’s time to start in on the project, as my Linksys router is melting. Getting any decent load on the router causes it to reset. This is especially problematic because it is the central switch in the network. That backup you were doing? Not so much. I borrowed a switch from the basement storage servers and put the wireless router on as a bump-0n-the-log, so it can reset all it wants so long as I’m doing my backup on the wired connection.

First recovery step was to set up a m0n0wall VM as a wired router. This is working surprisingly well so long as I don’t try to update anything (keep snapshots!). Adding the Apple USB to Ethernet adapter was seamless. In the host I disabled it entirely, and set it up bridged to the m0n0wall VM. Connect it bridged to the m0n0wall VM and connect that directly to the DSL modem and everything comes up perfectly.

After some contemplation of physical demands and so on, I came up with two models – unified and distributed. Unfortunately, I need 4 switches to physically connect all the bits of my house I plan to use; I just can’t get around that. Because of this, I don’t have much motivation to pick up one of those $200 plus big switches and connect every port in the house (dang). But here’s what it would look like now if I were to pick up one big multiport switch to finish the current implementation:

And here’s what it will probably look like after the meltdown is over:

(Don’t take the diagrams too seriously, some things are just empty slots that are reserved for moving things around. I didn’t want to spend any more time dinking around with it. )

The essentials being that instead of having two nodes with most of the hardware, I spread it around a bit. The net savings in hardware being $100-200 difference in the primary switch in the wiring closet. If they had finished the wiring in my house properly, I could likely reduce this to the single switch in the wiring cabinet. But without doing physical rewiring in the house I can just do this by replacing a single gigabit switch ($40-80) and picking up a wifi-N access point (which I did today, $120 Netgear).

I did toy with setting up VLAN support on the network, but having to replace ALL the switches to move the router and servers to the basement was just too expensive for home implementation. If  I was buying all new, I’d gladly pay the $20 premium per-switch to do it.

I will also point out that this marks the beginning of my infrastructure virtualization project, as the m0n0wall VM not only works, but appears speed up “normal web access” compared to my Linksys 330N. Three cheers for the m0n0wall folks.

I should write up another post detailing why I didn’t use the gateway built into OS X Server, which is a bummer.

In my previous post on the topic, I presented a couple alternative ways to get a lot of home infrastructure with little redundancy and some virtualization. The fact is, it’s pricey to do it ‘properly’. So I set out to look at the pros and cons of some alternative methods that make it a more practical for home (a.k.a. cheapskate) implementation.

Know thyself

Part of the problem I realized is that I probably don’t need an 8-core Xeon server on 24/7/365 to power my firewall and backup streaming (sorry Apple!). As much as I’d love to have one, I just can’t justify the amount of calories it would produce and feed into my [garage|basement] and the number of kilowatt-hours it would consume. Even with clever power management schemes as turning it off during the 1 A.M. to 1 P.M. periods, spinning down hard drives, etc., I’m just not using that much processor that much time. So I started dividing up jobs into two major categories: always-on services vs. on-demand services. Then I came up with this list, which I’m sure will be different than other people’s:

Always On:

• PPPoE dialer
• Firewall/router/gateway
• wifi-n access point
• DHCP
• dyndns.com updater daemon
• Remote access; VPN, Back to my Mac, whatever
• iCal server; reachable from the outside
• vCenter, maybe; reachable from the outside
• Backup server (RAID, growable)
• PS3 media server (or by schedule, details later) (RAID, growable)
• Web proxy
• “Warm storage” file server – stuff that can go to cold storage if unused for a while
• Home control software (lights, security, HVAC, whatever)
• [unplanned] MythTV backend

On Demand:

• Quicken
• Audiobook downloads
• Video encoding
• Work-VPN VM (it’s just easier this way)
• iTunes to manage files in PS3 media server directories
• daily use items (raw hardware)
• Games (raw hardware)

Note that I have re-ordered this list to make grouping them easier than I had originally done on paper.

Group into hardware

At this point I started drawing circles around things that were obviously easy to tie together, such as the first 5 items. These are currently done by my $80 linksys router with a power draw of something like 10w and almost no maintenance or downtime. Hard to argue with that. So I left those together. Then I looked at how I want to access my home resources from work (or the road) and decided that I still like Back to my Mac as an easy way to get in and see if I really do have that video on my home iTunes server. So that’s not hard. It’s functional so long as I feel like coughing up for the $100/yr service. Which I may be able to stop doing after this project is over. But I digress. The next set of things on the list is the set of “services provided from home”. A free account with dyndns.com or similar services allow me to reach home, so long as the bandwidth and processor power required to provide these to basically myself any my immediate family is small, I don’t need to shell out to a remote company to provide it. So I can host my own iCal server. It’s even open source if I don’t care to come up with an OS X Server license.

Now we get into the range of things that I really had a hard time deciding I could live with leaving in their current state. The backup server and the media server. Currently I have a pair of beat up ReadyNAS sparc-based units. One I bought new after a particularly bad hard drive crash, and the other I bought in a ‘company is folding, see if it works’ sale. I love these units for three main reasons: 1. they don’t need a lot of playing with. 2. they grow the shared volumes without me having to do much work to make it happen. 3. they don’t use a whole lot of power. But: these much older units that I have are not really good at sharing time. As single-purpose appliances they shine, especially because you can let them resize the volume by just inserting new drives. And they don’t ask for lots of maintenance upgrades and security upgrades, etc. They really are great little units if you can find them under $600. So with the luxury of having two already on hand, I decided that with jumbo frames turned on at both ends and a suitable Netgear switch in the middle, they really do perform these two jobs quite suitably. So my cheapest and most sane option is to repurpose them as dedicated appliances instead of general-purpose storage systems. And I also realized this was my migration path – essentially turn them into hardware incarnations of the VMs that they will be replaced by in the eventual ideal solution. This made me happy. So now I can dedicate one to be the growable backup unit and the other to be the growable media server unit. Which they already do.

Web proxy is honestly an optional extra I’d love to implement if I can arrange it. It has its own pitfalls, as the fabled “one more thing to go wrong” as well as being something else to maintain – something I’m usually quite set against. It’s there, we’ll see if it can be done.

“Warm storage” is the place I put things like setup files for downloaded products, project files for Audiobook Builder, iMovie project files I’m done working on but not ready to shelve, that kind of thing. Anything that is in warm storage for about 6 months will eventually get moved to offline cold storage and kept powered off, in a box, either at work or at home. As this is something that I want access to and won’t use often, and really is just about a terabyte of things that I’m not quite done with yet, I could group this with the hardware that does the backups. Speed isn’t an issue, just the access and the space, as these are quite regularly bigger than a DVD. Again, as this could easily be defined as something like a FreeNAS VM or a slice of a file server VM, I don’t mind this as a standalone role that can be virtualized in the final ideal solution.

Now we’re down to the on-demand services. Re-evaluating what these roles are helped me see that most of this can be handled in two places before the “final ideal solution” comes around. Currently the Quicken VM lives on my laptop. The Audiobook downloader lives on my low power mini server at home. Video encoding is done on the native OS of the mini server. The work VPN VM is on my laptop, as that’s where I use it most often. The iTunes library is on the native OS of the mini. I don’t mind this arrangement, but it helped me realize again the eventual arrangement in virtual machines I’d like to use, breaking them into specific roles let me see the migration path as hardware permits.

Now that I have the new role-based model to help me see what is where now and where I want it to be later, I came up with this diagram:

Big line flow charty thing

There are still a few stray points to clear up. I could entirely abandon the Linksys if I can find a way to control a wifi card like an access point. I don’t have to give up my growable storage arrays yet, but I could if I can find a 3ware card and enough drive bays (I already have, next post, I swear).

Note that this is just the role flowchart. I have additional charts that I’m working on to deal with network paths, storage, and so on. It’s not a trivial study as I originally thought. But there you have the original thoughts on how to organize it into hardware groups in a more realistic home environment.

So I work for this place called VMware that takes the idea of consolidating servers into a single monolithic megaserver that should (and nearly always does) save you money and increase data travel speeds and so on. I’m looking at the house and I was able to count 7 computers, 4 of which are user-centric and 3 of which (maybe 4) are data-centric, meaning to me that I could really turn 4 of them off in exchange for running one far more powerful one. And throw into the mix that I already use 3 VMs that I would rather not run on my laptop. So in a small, cheap way, I am a potential target for such consolidation already. But I don’t run megasuper servers with fiberchannel cards to mega-racked storage arrays. I run a pair of slowish RAID units, an old busted down windows box, a newer mac mini, an older mac mini, a pair of apple laptops, a PS3, and a Linksys firewall/wifi device. Looking into the pile I figure there’s some electricity and storage to be consolidated and a lot of upgrading to do. So what would it take.

• First I had to identify what actually needs to get done on the network.
• Second I had to identify how much space that is, and how and where to store it.
• Third I had to identify how much to get that kind of performance.
• Fourth I had to identify how to sell a kidney. Because that’s how much it would take. But, seeing as I’m rather attached to mine, I think I’ll just look for a less expensive way.

So let’s look.

1: What gets done and where will it go.

Central Server: I’m going to skip a long discussion of why and tell you now that I want  to run OS X Server on my home network to do a number of jobs. We can put why in another article. It will handle the backups properly, the file sharing properly, the calendar serving properly, etc. I like what it does and how it does it. I’m not going back into serving email with it this round, but it’s nice to know it can. This can be virtualized, but only on Apple host hardware. I can live with that. It will hold offline photos, backups, media server for sharing to the PS3, and well anything else I can think of that can just be piled into one server.

Satellite VMs: I also run Quicken in a VM. And I have to download Audiobooks from the public library in Windows. And so help me I want a web proxy. And so long as we’re on it, I’d like to have a VM that does most of the work the Linksys box does. Possible, but not probable.

Backup server: currently on a ReadyNAS box, but getting hard to administer.

Cold storage server: For keeping uncompressed ISO images of movies, lossless compressed audio of CDs, audiobook builder project files, iMovie project files, etc. that should be kept, but maybe not online. This is currently done by cold-swapping in older hard drives via USB and leaving them in the desk, off.

Then the user-only boxes I don’t plan to make virtual currently:

Desktop mac mini: My scanner/iTunes/movie sync station.

Gaming PC: I don’t have one any more, because mine is so out of date, but I want one. I’ll put it in the list.

Wife laptop: Photo editing, daily tasks, etc.

My laptop: Photo editing, daily tasks, etc.

Kid browser box: web games, run Indigo (to turn on/off lights in the house).

2: How much space do I really need in the coming year?

Doing all the math on a sheet of real dead-tree paper (with blue lines no less!) I worked out that I need about 16 TB total for all that, online all at once:

7 online data – VMs, media, photos.

4 backups

4 cold storage space

2 growing space to prevent having to rebuild next year.

3: How?

In order to get all that storage in one server, I’ll want something quite serious. But I can’t afford quite serious. Which leads to a problem. How fast is fast enough? My current network is gigabit, but the transfer rate to the low-power out-of-date RAID units I have caps out at 5 meg a second. Hardly fast enough for the kind of things I plan to make them do. In fact a serious factor in doing this all in the first place is that the two units I have keep bogging down doing things like playing a movie while deleting files or well, anything else, at the same time. So they’re essentially serial RAID units, you can only use them from one location at a time. And that won’t fly.

So the problems to solve are now boiled down a bit to this:

I want a powerful server to run VMs.

It will need about 8 GB of RAM and a super-fast 12-16TB RAID shared over the network.

I want to run MacOS X Server, preferably as a VM.

I want to be able to share lots and lots of data from a hopefully centralized point.

I want to use less power and rack mount this mess in the garage, out of reach of little fingers.

I want to be able to grow the storage with a minimum of fuss, and incrementally.

4: Can it be done in a house?

The simple solution would be to buy an Intel based XServe with a fiberchannel card and XRaid, install them in the rack (sideways, so I can still fit the car in), run the VMs in Fusion and call it a day. But the price puts that idea out of reach.

Price ballpark: Over 5 kilobucks.

The next notch down is to get a Mac Pro with a 3ware 9690 card plus gobs of RAM, run regular OS X on it, Fusion in the middle, OS X Server in a VM, other VMs on Fusion.

Price ballpark: Over 3 kilobucks.

The next notch down after that is to decouple the mac jobs from the non-mac jobs, so keep the mini and the McServer VM together, but move all the other VMs to a beefy-ish ESXi server and slowly learn to do each of those other tasks on the ESXi server, and eventually write off OS X Server. But as the idea was to consolidate instead of fragmenting more, this is counterproductive at the moment.

Price ballpark: 1.5 kilobucks.

The least attractive option (from a consolidation view) is to just rebuild the ReadyNAS units I have with bigger drives and continue to live with the slower transfers, put bigger drives in all the user boxes and only transfer backups and cold storage over the network. This would also require putting something like a Drobo on the mini to give the OS X Server VM enough storage for the media library and other tasks that the Mini would have to absorb. This method has its advantages, as the mini and storage would be ‘local’ on USB, it would support incremental storage growth, etc. But it only has RAM space enough for 1 VM (the server), the mini is NOT a powerhouse, the bus speed is really too low for heavy use, and while it’s low power, it probably won’t stand up well to long term usage at that level. I’d probably need a cheap used Dell or something similar to run the other VMs, and I have a hardware raid card I can throw into that. Storage probably 1 TB (x2 for redundant), 8 GB of inexpensive RAM, would need another $220 in hard drives for it.

Price ballpark: .5 kilobucks.

But the options aren’t quite over. There are a set of machines called “Hackintoshes” that purport to be quasi-legally able to run Mac OS. The fact is they’re not entirely on the legal end of the stick, but may be technically capable. Do I dare risk building a machine that might run Mac OS X for half the price of a Mac Pro? Is it worth the risk? Personally, I’m inclined to wait out the lawyers. This would reduce the price of building an XServe quality unit from $3000 to $1100. That’s not insignificant.

And since I don’t need cutting-edge abilities, I could go with a used Mac Pro/XServe, which go around $1400 instead of $2500.

Then consider the selloff of the existing infrastructure components. Maybe $500 for the RAID units would be something. So we’re not entirely lost here.

Initial speed tests done by installing a 3 year old RAID card in my Windows box shows I can get a 4x network share speed increase by using a new server, so it can be done, and that box is so old it only has a 133 bus and runs the RAID card at half speed. There is much to be gained.

So what to do? Comment away, I’ll read and consider, then we’ll see what real dollars get spent.

Oh, and there was research along the way into using iSCSI to store the RAID in the garage and keep the brainy server local, but the lack of iSCSI initiators for the mac under $200 a pop discouraged me. The free one doesn’t like Snow Leopard x64 at the moment. And the option of using afp protocol on top of zfs surfaced, but the combination in FreeNAS proved unstable and the zfs availability issue keeps getting in the way. It’s possible, but the hassle doesn’t pay for the payoff in my situation.

The economy is bad, and I realized early this year that my audiobook subscription was next on the chopping block, and, screaming in my heart, I clicked “Close” on my Audible subscription. And then I resumed reading the last 2 books I picked up.  Unfortunately, the second was a serial with 3 other parts, and I rather liked it. So I was in trouble. What to DO!?

So I did the next most logical thing. I went to the public library’s web site. Now, of course, this is where many of you will get angry with me, because I’m part of the San Jose Public Library system (www.sjlibrary.org) and you’re probably not. I can only say that this is documenting how I got around this problem, and you’ll have to adjust your experiences accordingly. Moving on.

So I still did the next most logical thing. I went to the public library’s web site. And sure enough: there on the left is a search catalog link. Which is exactly what I was after. I typed in the title and clicked search.  The list came back with over 300 entries. Most were paper, most by other authors, but the great thing was: they had audiobooks in there too. And I don’t just mean the CDs you can get on the shelves, which was all I was expecting – but electronic audiobooks. So I clicked, and fell down the rabbit hole.

For Father’s Day this year I got my audiobook subscription to Audible renewed for one year, so I have book credits on hand. However, I’m surprised by two changes; first – I love the new Audible site design with the NYT Bestseller lists and favorite-ever lists, and second – now that I know how to get so many books from other sources for ‘free’, I’m surprised how little is left on my Audible list that can’t be found elsewhere.  There are some exceptions, like there are a hand full of Orson Scott Card books that I just don’t find elsewhere, and some Michael Crichton (I limit myself to one Crichton book a year or I stop enjoying them). But even many of the newest books appear in the public library in short order.

For the impatient, I’ll just shortcut to the:

Workflow Cheat Sheet

• Keep a master wish list at Audible or Amazon. They have the big catalogs of books, with a variety of reviews available and star ratings that may keep you out of a book that sounds like one thing but turns out to be another. And sometimes they will have books that aren’t available from public locations. I hate forgetting: what-was-that-book-that-sounded-so-good-but-then-I-forgot-to-write-it-down-and-then -I-got-busy-and-now-I’ll-never-know-if-it-was-any-good-or-not. Get in the habit of using the voice recorder on your phone if you don’t feel like typing. I use the iPhone camera in book stores and the public library to snap the title and author of anything that looks good. They don’t have to like me.
• Periodically go to your wish list at either site above and start searching your public library catalog. I’m finding that about 2/3 of the books I search for can be had at one of the free sources, albeit with a wait. Even if there is a wait, I have 12 books queued up, I can be patient.
• Go to the public library site and search, sort by media kind and scroll all the way down – ebooks and audiobooks are at the end mostly. This catalogs both what is in the ebooks site and what is in the physical holdings. First pick the electronic version if you can get it. No CDs to import, quality is sufficient for translation into medium or low quality Audiobook Builder format. And no driving.

The San Jose ebook web site has its own flow. The flow here is [ optional - wishlist]->waiting list->bookshelf for 1,2,3 weeks. They manage it in the same way you would manage the paper copies – only so many copies are available for checkout, and you have to take turns with the others. Their wish list has each books current “Get it now” or “You’ll have to wait” status.

Spend some time one day just throwing piles of stuff into your wishlist from your master Audible/Amazon list, or from browsing. If you keep your checked-out list (a.k.a. bookshelf) about half full, you’ll still have room if a couple of your waiting list books come to you.  Otherwise you’ll miss your reservation window. Go ahead and keep your waiting list books more or less full and don’t be afraid to put yourself back in a queue if you miss it because of a full bookshelf. Remember about once a week to go back and check the wish list for available books and check out enough to keep it around half full and you’ll be happy.

Second choice is to get the physical CD set. Sometimes there is also an mp3 CD set. I was quite impressed with the quality of this. If the library is between work and home, I’ll drive it myself. If not, I’ll request a delivery to the one closest to home.

• Then turn to LibriVox. A surprising amount is available there, so don’t discount it.  Just don’t cringe at the narrators – quality varies widely. But also because they aren’t hindered by things like production cost, you can find a lot more content there than in most commercial collections, like their Sherlock Holmes collections are far more complete. The problem is that the LibriVox site is hard to navigate; but again, the price is right. If I liked my voice, I might consider doing some narrating myself. Dracula, Sherlock Holmes, seriously good stuff available.
• If all else fails, money talks. Audible or Amazon can help you find nearly anything in audiobook form.

Okay, now the details.

Welcome to DRM – we hates it forever

First: At home I’m a Mac user by default. I have a Windows box and 3 linux boxes, but my preference for the UI is the mac. iTunes is run on my mac mini – which is its main job really, followed by the endless video encoding I do. So when I showed up at the San Jose Library’s ebooks section (run by Overdrive DRM software, piggybacking Windows DRM), there were a few things of note:
1. They do have mp3 audiobooks with “do it yourself” DRM – please delete when you are done with a reminder from the download client.
2. They have far more DRMd WMA books.
3. But hey, the price is right.
4. Books can be checked out for 1, 2, or 3 weeks at a time, with a max limit of 10 at a time.
5. You can also put yourself on the waiting list for individual books. If you have a book slot open on the day your number comes up, you can check it out.  If not, just put yourself back in the queue tomorrow. Be sure to check your email.
6. They recently added “iPod friendly WMA audiobooks”, which apparently means that if you run iTunes from Windows, you can sync them to Apple devices. I haven’t tried them.

So, being an Mac iTunes-iPod-iPhone person, this limited me to just a subset of those books, but you can search just within that range and still get quite a bit.

Bundling and Organizing

So, just the same, I got busy with the checking out some MP3 audiobooks and some CD audiobooks, and remembered one of my pet peeves about the whole business. The “Audiobook” handling in iTunes is only scalable to the point at which you copy one or maybe two to the device you use. The interface on the iPhone and iPod for their Audiobooks navigation is annoying – titles only. So I’ve taken to storing them as Music, so I can navigate like so:

1. Music
2. Genres
3. Audibooks or Radio for radio shows
4. Author as Artist
5. Series name or Title as Album
6. Title Part N

I also keep a set of playlists based on the intended listener:

• Book Queue New

- for my new books. I keep them here to get around to them in some sort of order, whatever I feel like. I find that if a book is too dark or too much work, I only listen to it on the way to work and prefer something lighter for the trip home.

• Book Queue Reread

- for the lighter books or just anything I feel like I want to come back to soon.

• Book Queue Kids

- The public library has lots of audibooks geared toward the short set. I’ve picked up some for them and put them in here so we can get to them on the school run. They loved the Star Wars Radio Shows I got a couple years ago and a Peter Pan radio play.

So if Apple ever offers Audiobooks the same navigation, I’ll switch them back, but it’s a simple change.

Each publisher has different policies when dealing with the track length, and the books you get from the public library could be chapter divisions or they could be disc divisions, or something else entirely. Enter Audiobook Builder by Splasm software. It does exactly what I want it to do – combine lots of different little chapter or whatever files into a single set of m4b audiobook format files for iTunes. Bingo! Well worth the $10 I paid for it three years ago.

So now that I have my books and can listen to them and they aren’t 6 bizillion little files, I pop them into iTunes and the silence is gone. I have recovered – and the only thing I had to buy (but already owned, so no expense for me) was Audiobook Builder. $10 for a serious pile of audiobooks.

So I guess that’s the conclusion as well. I have a lot of credits for the things I can’t get from the Library, and the Library does a spectacularly efficient job of giving me access to books I can hear.Happy listening!

Thought I’d add this one to the list simply because I discussed it with a friend recently and didn’t have a chance to put it up here.

Orson Scott Card’s Intergalactic Medicine Show

Now generally I like short story collections, especially in audio form, because the change from story to story is a lot of fun, especially with a 30m – 1h commute most mornings. Sometimes you complete a whole story on the way to or from work and you feel like you’ve accomplished something, and you get the whole picture in a short time. It’s nice. These were especially fun because they were extremely well thought out, sometimes silly, sometimes sad, and they touched me beyond the normal, “heh, that was cool.” The one about Elvis had me rolling in my seat in the parking lot, 10 minutes after I’d arrived at work. I’m glad nobody saw me there. As it was, I had to re-tell the story to my wife, doubled over in laughter the whole time. Highly suggested collection.