Ads displayed for guests and not donating members only. Get ad-free by donating. If you have already donated, please read here.
  • Server on it's limit, applying new storage policy for Replays

    The last weeks were very hard for us. Busy messages with ADU, slow responses in high peak times. Especially the last two Sundays were extremely demotivating.

    Sorry for the hiccups, vBAddict is a slave of it's own success.

    Beside the processing unit, there is now another limit the server reached yesterday: Disk Storage. "Incompatible Dossier" was a result of this.
    I already asked my Hoster about upgrading the server, but they said it's not possible. I cannot order more than four drives, as the space inside the server is limited. And ordering a new one does not solve the problem as the space is the same. It only reduces the headaches a bit by expanding the disk space - with much more costs.

    One of the solutions to this I see only by reducing the limits for the storage of Replays.
    Right now vBAddict is hosting over two million replays. Replays are deleted after 60 days. I have to reduce this timeframe to 45 days. Replays, which have no views/downloads after 7 days and additionally have low experience earned, will be deleted immediatly.

    ATS and Battle Results are not stored on the server, they affect only the size of the database.

    First actions reduced the occupied storage from 100% to 72%, which gives us a little more air to breathe.
    Comments 23 Comments
    1. KamonegiX.eu's Avatar
      KamonegiX.eu -
      Maybe think about building your own server and colocate it with hetzner?
      I did the same thing with my projects... I was outgrowing their servers and none of them fit my needs anyway so I configured my own server and sent it to them for colocating. The veeeeeery cheap traffic costs from them are still the main reason why I did stick with hetzner. Also the fact that their colocation costs are really affordable as well. I have a 2 hights space in their rack and all in all I pay ~100 Euros a month for power, space in the rack and traffic (currently almost 30TB a month).

      Just start a donation drive for the server components and build your own server. It's cheaper in the long run than paying for all the stuff from the hoster in the first place.

      Edit: Oh well, seems hetzner does not offer single space colocation anymore. Only 1/3 rack and 14 hight units are a bit overkill for this I think...
      Too bad, but I'm pretty sure there are enough comparable hosters out there.
    1. Phalynx.eu's Avatar
      Phalynx.eu -
      The costs for own hardware are too high. I'm lucky people are donating enough to pay the bill of 132 Euro.
      It would take few months to collect that alone from the donations, so no way.

      Will switch probably on a later date to a newer server with 3x 2 TB (4 TB @ RAID5) and 1x 480 SSD. That would double the current disk space (2 TB @ RAID0)
    1. ltd_webber.eu's Avatar
      ltd_webber.eu -
      Hi,

      i dont realize that the replays deleted after 60 days.
      As I delete them immediately after the upload, it would be nice if I could download my entire replays to archive them. Some of them I would like to keep.

      Maybe you can etablish an donationbar like wikipedia, so the people can see how many money you need to pay the bill or to relalize an special project like an new server.
    1. Phalynx.eu's Avatar
      Phalynx.eu -
      There is no function to download this as an archive.

      The 60 days was already defined 10 months ago, also because of disk usage:
      http://www.vbaddict.net/content/31-Cleanup-on-Replays
    1. Scrambled.asia's Avatar
      Scrambled.asia -
      Yeah, your host (and mine) isn't too good with upgrades. You can get more disks installed but I think they just replace your current ones, so you'd still have to backup a stupid amount of data.

      Or do what I did, get a new one at the same place, then copy the old stuff to the new one and switch DNS. Either that, or go for the cheapest server they have purely for the storage space and use it as a NFS mount or something...

      Success can be a beetch
    1. Phalynx.eu's Avatar
      Phalynx.eu -
      Yes, inserting 2x 4 TB Disks would solve this issue, but a reinstall of OS and all other stuff would be the result. This is a downtime of at least 2 days, which is not acceptable.

      I can reduce the downtime to a few hours with the ordering of a new server. That way I can create the environment, copy all stuff, switch vBAddict offline and copy the deltas. Probably this step will follow by reaching the next limits…
    1. Scrambled.asia's Avatar
      Scrambled.asia -
      Downtime bad(tm).

      Dunno, seems swapping to a new server is the less painful option If you do it near the end of the month (e.g. order the new one) you only get invoiced a few euros for it, and if you then cancel your old one before the first week of the next month, you won't be invoiced for it, except a few euros, so all in all your bill will be 10 euros higher that one month. At least that's how it turned out for me when I did the server move.

      As always, if you need a hand, temporary storage space, hit me up and I'll help out whatever way I can
    1. Phalynx.eu's Avatar
      Phalynx.eu -
      Thanks for your offer, will come back than
    1. Huzar79.eu's Avatar
      Huzar79.eu -
      While ADU can't parse files before uploading them and replays are needed anyway for certain statistics calculation - how about additional option allowing every user to filter/decide which replays to keep and which should be deleted from the server immediately after relevant data is processed? Of course all that on top of limits you already introduced.

      Say: XP, frags, efficiency or WR8 thresholds, certain map, tank or mode, won/lost, achievements (probably Top Gun, Ace Tanker and Scout would suffice) etc... Similar to sharing options in profile settings.

      Either that or any way to mark for deletion multiple replays at once (maybe checkbox along with "delete all checked" button on Quick Summary/Battle History page).

      I bet it would reduce amount of replays stored even more...

      Also - as someone mentioned above - donation bar showing how much is still needed to pay the bill would be a good idea imho

      *As a side note - possibility to create additional RSS feed based on certain criteria would be awesome.


      Thank you for creating this site and providing such great service to WoT players.

      *thumbs_up*
    1. Phalynx.eu's Avatar
      Phalynx.eu -
      Thanks for adding this to the tracker. Will implement most of the stuff as my time allows it.
    1. Scrambled.asia's Avatar
      Scrambled.asia -
      Quote Originally Posted by Phalynx.eu View Post
      Thanks for your offer, will come back than
      I'll be where I usually am, head deep in replay packets

      Depending on my work situation (and of course, when you'd really really need it), I'd be willing to split the cost for an SX60 at Hetzner, should end up at 8Tb in RAID-1 (4x4Gb SATA), but would add about $35 to our respective server bills. Again, kind of depends on work, being a freelancer I need to justify to everyone (including my wife, otherwise known as she-who-must-be-obeyed) why I'm spending money so... yeah. Consider it a "maybe" for now

      Outside of that I've got about a Tb of storage available that's not in use right now, so if you need that space temporarily, hit me up and I'll set you up with rsync, scp, or ftp - whichever you prefer
    1. MiSTiE.ru's Avatar
      MiSTiE.ru -
      Hey, Phalynx. Thanks for doing a great job running this fantastic resource! I find it extremely useful for my research on how individual merits (net of other potential factors) impact the probability of winning a battle in WoT. But I have a question: are there any particular reasons, why it is the Russian and Korean server for which battle statistics are currently unavailable?
    1. Radiopatrol.na's Avatar
      Radiopatrol.na -
      it is too bad you are in Europe ... I am about to surplus a stack of DL 360 G'5s

      although they do not have huge storage arrays ....

      the max for the Chassis is: Hot Plug SATA 3.0TB SATA 6 x 500GB
    1. Phalynx.eu's Avatar
      Phalynx.eu -
      Quote Originally Posted by Scrambled.asia View Post
      I'll be where I usually am, head deep in replay packets

      Depending on my work situation (and of course, when you'd really really need it), I'd be willing to split the cost for an SX60 at Hetzner, should end up at 8Tb in RAID-1 (4x4Gb SATA), but would add about $35 to our respective server bills. Again, kind of depends on work, being a freelancer I need to justify to everyone (including my wife, otherwise known as she-who-must-be-obeyed) why I'm spending money so... yeah. Consider it a "maybe" for now

      Outside of that I've got about a Tb of storage available that's not in use right now, so if you need that space temporarily, hit me up and I'll set you up with rsync, scp, or ftp - whichever you prefer
      Sounds promising. That would be the final destination for our replay sharing stuff.
    1. Phalynx.eu's Avatar
      Phalynx.eu -
      Quote Originally Posted by MiSTiE.ru View Post
      Hey, Phalynx. Thanks for doing a great job running this fantastic resource! I find it extremely useful for my research on how individual merits (net of other potential factors) impact the probability of winning a battle in WoT. But I have a question: are there any particular reasons, why it is the Russian and Korean server for which battle statistics are currently unavailable?
      That is really simple to explain. There are not really much users from RU and KR, therefore only a limited set of data is available. (RU: 2671 Users, KR: 1255 Users, Compared to EU: 15584 Users)
    1. Phalynx.eu's Avatar
      Phalynx.eu -
      Quote Originally Posted by Radiopatrol.na View Post
      it is too bad you are in Europe ... I am about to surplus a stack of DL 360 G'5s

      although they do not have huge storage arrays ....

      the max for the Chassis is: Hot Plug SATA 3.0TB SATA 6 x 500GB
      Need to find a solution where I have the control of the data.

      Currently there is no need to upgrade the diskspace, as the 45 days rule has removed the need for more storage.
    1. Scrambled.asia's Avatar
      Scrambled.asia -
      Quote Originally Posted by Phalynx.eu View Post
      Sounds promising. That would be the final destination for our replay sharing stuff.
      Yeah would at least solve having to cross-post it all; but ... still, 69 euro/mo extra split 2 ways, plus a one-off 69 euro setup fee. But 4x4Tb SATA... that's a lot of replays; even if it's RAID-1 so "only" 8Tb...
    1. DarkRemedy.eu's Avatar
      DarkRemedy.eu -
      "There are currently 419 users online." couldn't they all just subscribe for €1 a month and solve this?
      I'm not rich, so I just subscribed for €3 a month. Thanks, and keep up the good work!

      -Edit: make you Donate button more prominent ^^ I Don't think most people come here, hell it took me a month.
    1. Phalynx.eu's Avatar
      Phalynx.eu -
      There are 25k users which are actively using vBAddict.... imagine just 1% of them would spent 1 EUR.
    1. DarkRemedy.eu's Avatar
      DarkRemedy.eu -
      That would be awesome!
      I know for a fact people like Quickybaby/Quickfingers use VBAddict (that's how I found out about it), I imagine Jingles does as well. They have a great community of followers that are also willing to pay for all things good in WoT. Perhaps you could have a chat with any of them in order to see if they're interested in helping you out by mentioning your mod and that servers cost money. Of course it would probably also increase your traffic.