So my friend brought over his modded XBOX that was showing "Open Tray" every time he tried to play a game. I had fixed this once before on another box, and decided to write a tutorial to help anybody out who may be having this problem.
So, to begin, let me first repeat that this fix will only work for an XBOX with a Liteon DVD drive - you can tell your DVD drive manufacturer by looking at the disc tray. Check this site out:
Scroll down to find a picture comparing all the drives. If you happen to have a BenQ drive, the tray will look the same as a Liteon. You can tell the difference by reading the big sticker on top of the drive (which will be visible when you open up the XBOX).
Be aware that an "Open Tray" being displayed in the dash could also be the fault of a dying laser. Luckily, this fix is free, and if you continue to have issues after you have done it, the next logical step would be to replace the laser.
Keeping this issue from recurring will require modification to a component of the DVD drive. If you are uncomfortable with doing this, you CAN still get your drive to work by simply moving the laser. However, the constant "Open Tray" message will most likely come back after awhile. More on this later.
I also want to say that this will ABSOLUTELY VOID YOUR WARRANTY. Check on xbox.com to make sure that your standard warranty is up before you crack open your box to perform this fix. Alright, with all that out of the way, lets get it done.
Step 1: The Problem
So the root cause of the "Open Tray" error is the DVD laser getting stuck at the back of the drive. You can see this when you take the casing off and set a disk in the tray. I drew what this looks like in Paint, because I forgot to take a picture lol. The white circle is supposed to be the lens BTW, which was almost completely outside of the radius of the disc on friends box. Anyways, the laser rides a worm gear, which slides it from the center of the disk to the far edge so that it can read data imprinted into the disk.
My theory is that the worm gear is the root of the problem. The very last turn on it, the one closest to the motor, is slightly thicker than the one before it. It took me awhile, but I finally managed to get a good picture of this. I think this causes the white plastic piece tying the laser to the worm gear (which is basically a clamp with two teeth to grip the gear), to get stuck when it slides over this thick turn. This is because only one of its teeth can catch the track and the other one pops out, causing it to skip and stay put as the worm gear spins.
This seems to happen rarely. I think that the laser was actually never meant to go so far out on the worm gear, but it obviously does under just the right circumstances. So, to keep it from recurring, I decided to work around the worm gear issue by modifying the white grip piece. I used an exacto knife to thin out the plastic "hinge" above the spring. This lets the spring push the grip's teeth further and harder into the worm gear, which should help it stay in the gear's track even when close to the thick turn. If, for whatever reason, one of the teeth does pop out on the thick turn again, in theory the added flexibility of the piece will help hold it tight enough to the worm gear so that it will not get stuck.
I based this whole idea on my comparison between a grip piece from a BenQ XBOX drive (which, hardware wise, is almost identical to a Liteon) and the one from my friends drive. In my pics, you can see how the BenQ piece bends more because of its thinner "hinge", and thus has more grip on the worm gear as compared to the Liteon. And I've never had to fix an "Open Tray" BenQ, so that also assured me.