This is really turning into a recurring theme here, but we had another drive failure.
Last time it was the actual RAID array, this time it was the SSD drive that got put into the server a while ago. The replay files are stored on the SSD because it’s much faster to unpack them from there than it is from the main SATA drives, and it seems the SSD drive did the same thing as another one did a while back; it just up and died.
I did get it back to life for a short bit, but filesystem recovery netted me 200 replays out of the 13500 that were on the site; which is not good. A few repeated attempts at copying off the data resulted in the thing packing up permanently, it’s not responding, and the OS doesn’t even see it anymore as being there.
On the whole:*well shit!
This means that while the replay*pages are still there, the files themselves are not. Not a total disaster, mind you, but still not exactly what I had in mind. I’ll be doing some updates on the site to enable re-uploading files that have already been uploaded to restore them back to where they’re supposed to be going, but any replay files that don’t get reuploaded are irrevocably*gone.
I’ll be pre-emptive here:
Any whining about how there were no backups will be ignored; I have in the past explained why there are no backups of the actual replay files, and only the database, but to recap: it’s all about finances. The server itself already runs more than $60 a month; and while the past two months that has been paid by donations, only a few people have been generous enough to donate. I can’t afford (read: justify) spending an additional $20 to $40 on backup space; neither can I use Amazon S3 for it because I’m one of those strange people that doesn’t have a credit card. I have been talking to people to work on a backup solution, but inbetween my real life and my work, I haven’t had a chance to finalize that.
tl;dr: replay files went poof. pages are fine. re-upload should match the file back to it’s proper page. complaints go to /dev/null.