I won't go into how to use an Arduino as an ISP, there are plenty of instructables and other how-to's on the net for that.
What I needed was a simple way to program (and burn the bootloader) onto my ATtiny chips. I have made a breadboard breakout board (soon to be a different instructable) for my ATTiny84/44s as well as one for my ATTiny85/45s.
These sub-boards are for breadboard prototyping so I don't have to setup the crystal, the power, and the reset every time I want to move to a breadboard.
This is the ATTiny*4 Breadboard breakout. It's the board's traces are a mess I know. I made it with a sharpie marker as I didn't have access to a proper laser printer late at night when I made it. But it works like a charm.
The ICSP header breaks out the chip's MOSI, MISO, SCK, VCC, GND, and RESET pins. That is wired to a normal 6 wire cable. The other end is where the trick comes in.
Typically the ICSP on the arduino boards is used to program the chip on that board. So more like an "IN" as opposed to an "OUT". The reason for that is pin 5 is wired to the reset pin on that chip, and not the reset needed inside the "Arduino as ISP" sketch. I've see "how-to's" that tell you to cut those traces and rewire it. I had no interest in modifying my board permanently so I found it way easier to modify the cable.
using an 8 pin cable I scavenged I remove the previous cable, then cut 6 wires from an old IDE ribbon cable. After that I pulled the 5th wire from the ribbon cable and made it about 3 inches longer, and did NOT put it into the punchdown for the header.
I punched the other wires in and cut them flush. I put a dab of super glue on both sides of the ribbon cable to try and prevent it from splitting down the cable any more. at the end of the reset pin (5th wire) I put a male connector on it so it would fit into the female pin 10 for programming.
Now all I need to do to program (or burn the bootloader) onto really any AVR that I have a bootloader for is attach this to the ICSP on the slave board and then the other end to my main arduino board and wire up pin 10 for reset. I am able to use the stock "Arduino as ISP" sketch with no mods to the boards or the sketch.