1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.
  2. Greetings Guest!!

    In order to combat SPAM on the forums, all users are required to have a minimum of 2 posts before they can submit links in any post or thread.

    Dismiss Notice

Regarding Great Lakes' Revert

Discussion in 'UHall' started by Wallenstein, Jul 28, 2009.

  1. Wallenstein

    Wallenstein Guest

    So most of us probably know Great Lakes was reverted today to fix an issue that caused almost all locked down and secure items in Luna city homes to vanish. But, how can we be sure this won't happen tomorrow at server-up? As far as I know the issue took place somewhere between early this morning and 10:00am ET, or right around the time of the maintenance period.

    Regardless, backups of the servers should be made every 6 hours or so instead of 24. In an MMORPG thousands of players accomplish things in a 24-hour period, and to revert an entire day is a big loss to some. A 6-hour revert is nowhere near as much of a hardship. Why isn't this possible? Why must all reverts go back 24 hours?

    Just food for thought..
     
  2. Mapper

    Mapper Crazed Zealot
    Stratics Veteran Alumni Stratics Legend

    Joined:
    Oct 24, 2005
    Messages:
    3,528
    Likes Received:
    110
    When each server goes down each day isnt that it backing up? Surely it couldnt backup every 6 hours unless it went down every 6 hours.
     
  3. Viper09

    Viper09 Grand Poobah
    Stratics Veteran

    Joined:
    May 16, 2008
    Messages:
    6,680
    Likes Received:
    824
    Well from what Chrissay had already said, the issue has been resolved. Surely they would want to fix whatever bug caused it so they don't have to revert the shard again.


     
  4. Devil_Woman

    Devil_Woman Guest

    From EA Support in answer to when shards go down for maintenance:
    So, obviously back ups are run of each server several times each day. The problem may be that whatever issue caused the problem was imbedded in the later back ups therefore the revert has to go back to a known "clean" back up.

    And all reverts are not 24 hours. Every time the shards are taken down in the middle of the day for a publish, and the last time for that was a week or two ago, they "revert" to the most recent back up. When was the last time you recall a publish in the middle of the day reverting you to the day before?
     
  5. KrazyFrog

    KrazyFrog Guest

    This conversation brings back really old memories of days past. It use to be that you would actually get a message every few hours when the server was performing its backup. Of course, back then there was not nearly as much to back up and they didn't take hours to complete.

    Which things brings back the memories of old when there would be "server down wars". It use to be you would get a message informing you that the server would be going down "soon" for its maintenance and all chaos would break loose all over because you knew none of it was going to be saved (except that one time...).

    Fun days those were... back in '97/'98
     
  6. Beastmaster

    Beastmaster Guest

    The part that I find most upsetting is that we are told only "the issue was resolved". Well DUH! In EA's eyes that could simply mean a)something went wrong and stuff was lost, b)we reverted the shard, c)issue resolved. In otherwords the items were returned and thus they resolved the issue of missing items. It does not speak to what caused the items to go missing in the first place.

    I'm sure some of you will say, well it was a bug, glitch, etc., and EA fixed it because they are a huge gaming company and surely they want to make things right.

    Others will say, EA figures it was a glitch but has no clue what it really was so they just crossed their fingers and reverted.

    Personally, I think it was somewhere in the middle. Like they found a likely cause, but still aren't sure it actually was the cause and cannot reproduce it. So we continue on and hope it doesn't rear its ugly head again.
     
  7. JC the Builder

    JC the Builder Crazed Zealot
    Stratics Veteran

    Joined:
    May 12, 2008
    Messages:
    3,154
    Likes Received:
    708
    You have to go to a backup where the issue doesn't exist. If you go to the 1 hour backup there are items missing. The 6 hour backup there are items missing. The issue began at server up, so they had to revert to that.

    If you ever login after a server crash, you usually don't lose more than an hour.
     
  8. Lady Michelle

    Lady Michelle Sprite Full SP
    Stratics Veteran The DarkOutlaws, TDO

    Joined:
    May 12, 2008
    Messages:
    2,451
    Likes Received:
    1,113
    I wasnt happy about it, but I would rather lose 24 hours than see anyone lose years of things they have worked for, or collected.I am just glad they fixed the issue, and hope it doesnt happen again.
     
  9. Basara

    Basara UO Forum Moderator
    Moderator Professional Wiki Moderator Stratics Veteran Stratics Legend Campaign Supporter

    Joined:
    Jul 16, 2003
    Messages:
    8,457
    Likes Received:
    582
    IF people would just check the knowledge base, Stratics, or any of the other sources (trying to remember if JC's wiki includes the information) -

    It has been stated repeatedly that UO backs up every 45 MINUTES, in a staggered pattern for the specific server farm. That's why, for example, shards go down for morning maintenance, in usually the same order, as the game completes the 45 minutes for each shard before dropping it. So, for example, a couple weeks ago, Lake Austin would go down first, Pacific & Origin last, of the Western US servers. Shortly after the latter two dropped, LA would come back up, and the time between it coming back up and Pacific coming up was typically the same time difference between them dropping.

    If it's a simple system failure, reverts are typically back up to 45-90 minutes, if it's known that the failure didn't corrupt the save data, or wasn't caused by something in the software that a restore would reinstitute.

    Things like what happened to GL, get complicated, especially if they occur when the server guys and programmers aren't at work. consider how many 45 minute backups occurred between "The Vanishing" and someone finding out about it - especially if the GMs are going on about "can't replace lost items" off their script, and don't realize that something major has FUBARed with the lockdown/decay system for an entire subserver (and, that would probably be partly the pagers' fault, if they didn't phrase the correspondence properly to the effect of "all the houses in Luna are losing their contents!" as opposed to "My stuff is gone!").
     
  10. Tinsil

    Tinsil Guest

    Shouldn't they be able to back up certain things easily though?

    Like character skills, etc then revert?
     
  11. Experimental

    Experimental Lore Keeper
    Stratics Veteran Stratics Legend

    Joined:
    Jun 23, 2004
    Messages:
    780
    Likes Received:
    19
    Is it possible that this is what happened to Hemi??? Just a thought.
     
  12. Tinsil

    Tinsil Guest

    Naw. His supposedly wasn't even owned by him anymore.
     
  13. blueturtle

    blueturtle Lore Keeper
    Stratics Veteran

    Joined:
    Dec 8, 2007
    Messages:
    824
    Likes Received:
    0
    just a thought what if some1 xsftered a char to gl yesterday, guess it just got deleted for good?
     
  14. JC the Builder

    JC the Builder Crazed Zealot
    Stratics Veteran

    Joined:
    May 12, 2008
    Messages:
    3,154
    Likes Received:
    708
    You need to contact support for EA to restore the character. They have the logs of everything transferred.
     
  15. Tinsil

    Tinsil Guest

    Ouch. Transfers weren't backed up?

    That's goooing to suuuuuck for my few buddies that transferred to GL yesterday. They get to deal with EA support. lol
     
  16. Godiva_DF

    Godiva_DF Guest

    On the other hand if you transfered stuff FROM GL to another Shard did you legally dupe the items?
     
  17. Tinsil

    Tinsil Guest

    Was wondering that myself also. Of course I don't know that EA would admit if this was the case.
     
  18. Fink

    Fink Guest

    Perhaps if you're playing 6-12 hours a day, and need 6-hourly backups, you should simply appreciate your good fortune at having so much free time to play. Unless you're scripting 24/7 I can't see that a day's setback is too big a deal for most people.

    It shouldn't take you too long to recoup your losses. I understand it, a few hours' gain lost to a revert would be minimal compared to the losses some would have sustained with their homes torn open and their possessions wiped out.
     
  19. Wallenstein

    Wallenstein Guest

    I understand that, but the server had no issues for the entire day prior to the shard revert. So, to have the only backup available be the one that was made 24 hours earlier during the previous day's shard backup means that there were either; (A) no backups made for that entire 24-hour period prior to the shard revert, or (B) corrupt shard backups made for that entire 24-hour period prior to the shard revert. Either way, an issue somewhere needs to be addressed.

    This, of course, based on the posting that shard backups are made "multiple times per day". Apparently this isn't so.
     
  20. Shamus Turlough

    Shamus Turlough Lore Master
    Stratics Veteran Alumni

    Joined:
    May 19, 2008
    Messages:
    1,066
    Likes Received:
    207
    Ever stop to think that maybe there was a problem in the code that was only triggered a few hours after the Monday morning server up? If they needed to revert that code (or whatever) then they would have had to go back to the latest instance of the "good version". Sometimes problems are not immediately visible!
     
  21. Cetric

    Cetric Grand Inquisitor
    Stratics Veteran

    Joined:
    Aug 1, 2008
    Messages:
    4,110
    Likes Received:
    906
    UO is constantly being backed up, the problem probably showed its head early monday, and the backups between then and the revert had the problem on them. That is why in the IT world you keep backups for different intervals of time, and the past shows that this is something they have done effectively
     
  22. Chrissay_

    Chrissay_ Guest

    We reverted to the last verifiable point prior to the issue starting. We didn't want to take any chances with some players getting restored while others weren't.
     
  23. Xalan Dementia

    Xalan Dementia Slightly Crazed
    Stratics Veteran

    Joined:
    Jul 1, 2008
    Messages:
    1,355
    Likes Received:
    81
    Yes reverts are an annoyance, but not devastating. Everything lost can be regained, its not like someone received a one of a kind item then lost it due to the revert. (cloaks, crafted items, loot, it all can be regained.)