Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Desync Workaround using Dropbox and mklink

PsychoAngelicPsychoAngelic Junior MemberPosts: 9Registered Users
edited September 2013 in Total War Eras Support Forum
I also posted this in the Steam forums, hopefully this works for some of you.

Greetings!

While emailing saved games is OK, I found this troublesome and hit-or-miss as the desync can occur before the save and then you're screwed and have to start over. So, I decided to test using dropbox as a workaround with a friend. So far this has worked with no issues and is completely automated so there is no need to email or upload the save all the time.

The system I came up with is as follows, which works with a brand new game after turn 1 when a fresh save is made and not after:

Host Instructions:

1) Both parties will need a Dropbox account

2) Install Dropbox and create a save game folder to be shared.

3) Navigate to your saved game folder (normally found in "C:\users\<yourname>\appdata\roaming\the creative assembly\shogun2\save_games_multiplayer"

4) In the folder from step 3, move the file named "autosave.save_multiplayer" to your newly created Dropbox shared folder. In my case it was "C:\Dropbox\Shogun2"

5) Delete the file "autosave.save_multiplayer" from the folder identified in step 3.

6) Now using the command prompt you will need to set up a symbolic link in windows to allow the game to save correctly. The syntax for a symlink in this case is as follows, though you will need to change directory/username info specific to your system:

mklink "c:\users\<yourname>\appdata\roaming\the creative assembly\shogun2\save_games_multiplayer\autosave.save_multiplayer" "c:\dropbox\shogun2\autosave.save_multiplayer"

Effectively, this creates a shortcut to the file allowing you to change your autosave location.

7) The host will need to share the dropbox folder with whoever they need to.
8) Everything should be good to go.

Connecting Player Instructions:

The connecting player will need to do all the above steps except in step 4 instead of moving their autosave, they will need to take the autosave from the host's dropbox folder and place it in their dropbox folder which creates the initial saved game synchronization.


Other Notes:

-It is strongly recommended you do not have any other save game data in your multi-player save folders


Let me know if this works for anyone. This has been working for my friend and me for about 50 turns so far with multiple loads (not continuous play).

Update: Completed a game with no problems using this and am now playing a RotS game with no problems so far.
Edit1: Changed formatting to remove ambiguity on connecting player's instructions
Post edited by PsychoAngelic on
«1

Comments

  • sgtgumbosgtgumbo Member Posts: 40Registered Users
    edited March 2012
    Wow....

    Well done sir.. I will give this a try =)
  • HeroicSpurHeroicSpur Member Posts: 57Registered Users
    edited March 2012
    How do the multiple dropbox accounts work? I thought you could only share a dropbox folder when you used the same account. Can 2 different accounts share 1 folder?
  • HeroicSpurHeroicSpur Member Posts: 57Registered Users
    edited March 2012
    Ah sorry, I just checked dropbox site, found the answer. This looks the kind of thing that hits desync at their source (the moronic system which allows two computers to produce divergent outcomes, rather than forcing the host to determine all outcomes). Lets hope this works, and thank you PsychoAngelic.
  • PsychoAngelicPsychoAngelic Junior Member Posts: 9Registered Users
    edited March 2012
    This looks the kind of thing that hits desync at their source (the moronic system which allows two computers to produce divergent outcomes, rather than forcing the host to determine all outcomes).

    Correct. The autosave feature saves as an unique file on each person's computer. When a new game is started, the saves are likely to not be identical and the CPU calculates things differently during the AI moves (I'm assuming). This method bypasses the autsaving on the connecting person's end, forcing use of the host's autosave upon resuming the game after a break.

    I'll update again when I've tested it more.
  • BOTA:99BOTA:99 Junior Member Posts: 8Registered Users
    edited March 2012
    How is this different than the host doing a file transfer over skype? It does the same thing forcing both computers to use duplicate saves?
  • HeroicSpurHeroicSpur Member Posts: 57Registered Users
    edited March 2012
    I'm not sure how Skype works but dropbox works automatically. Any files in the dropbox folder are automatically synched. Therefore, if I have this right, the autosave is moved to the dropbox folder, which is shared. Both machines refer their Shogun 2 autosave to that dropbox folder. Therefore everytime an autosave is made, it is automatically synched and this means the connecting player always uses the hosts save file.

    My only concern is this, even if the game is continually using the same save file, does that stop the game becoming divergent during the actual game? Surely the game doesn't load the autosave every turn. So could the game still desync? I suppose it depends on how exactly the autosave file is used and what data it contains.

    On a further note, what impact do you think this has for non-autosave files? Do you think it's advisable to only use the autosave?
  • PsychoAngelicPsychoAngelic Junior Member Posts: 9Registered Users
    edited March 2012
    My only concern is this, even if the game is continually using the same save file, does that stop the game becoming divergent during the actual game? Surely the game doesn't load the autosave every turn. So could the game still desync?

    Possible, but unlikely. In my experience, desyncs would generally happen after resuming the game, not continuous play. I have noticed that the game auto saves two separate files for each save, one on the host the the other on the connecting player after each individual player's turn and both of those saves are the ones used to resume the game (which is a poorly designed system CA) and this is why emailing saved games sometimes works.

    So, why doesn't emailing always work?

    I do however think that previous save information is carried over into newly created saves (i.e. autosave data being used to create quick/manual save data). If my assumption is correct and based on my before mentioned observation, this suggests that due to the different autosave files for each player, the desync happens when either the game is resumed from a non synced autosave or a quick/manual save is used that utilized desynced autosaved data.
    On a further note, what impact do you think this has for non-autosave files? Do you think it's advisable to only use the autosave?

    For the reasons I describe, I would suggest only using the auto saves under this method until CA actually fixes the issue. Not only that, quick and manual saved have a unique file name in the Appdata folder, so using dropbox and symbolic links would be very problematic. The autosave file is the only file in there that is constant. If you insist on using the other saves, you should just stick to email/skype/etc.
  • RemairRemair Member Posts: 32Registered Users
    edited March 2012
    This sounds really promising Psycho. I'm a bit sceptical (too many years wanting real TW multiplayer and being disappointed to get my hopes up) but will keep an eye on this thread, let me know if you manage to finish a full campaign. If this does indeed fix it then frankly CA should pay you for doing what they've been unable to do for a year :)
  • insolentfoolinsolentfool Junior Member Posts: 18Registered Users
    edited March 2012
    A friend and I played like 8 hours straight in campaign without loading. on legendary so we only have the one save. about 40 turns in the host CTD. we loaded the game and 2 turns later had a battle on only my screen. Sure the desync was after the load, but i believe the CTD was caused by desyncing.
  • PsychoAngelicPsychoAngelic Junior Member Posts: 9Registered Users
    edited March 2012
    Remair wrote: »
    This sounds really promising Psycho. I'm a bit sceptical (too many years wanting real TW multiplayer and being disappointed to get my hopes up) but will keep an eye on this thread, let me know if you manage to finish a full campaign. If this does indeed fix it then frankly CA should pay you for doing what they've been unable to do for a year :)

    Works like a charm as far as I can tell. Just finished a 109 turn game with zero desyncs.

    I'm really interested in finding out if anyone else has had some success with this method.
  • RemairRemair Member Posts: 32Registered Users
    edited March 2012
    A friend and I played like 8 hours straight in campaign without loading. on legendary so we only have the one save. about 40 turns in the host CTD. we loaded the game and 2 turns later had a battle on only my screen. Sure the desync was after the load, but i believe the CTD was caused by desyncing.

    To me that strongly suggests that Psycho is right... you got that far with no desync then as soon as you load boom.. it goes. Thinking back I think every instance of desyncing I have experienced was always after the first play session.
  • PsychoAngelicPsychoAngelic Junior Member Posts: 9Registered Users
    edited April 2012
    Giblet wrote: »
    Having to do this is an absolute joke and I refuse to. This problem should have been fixed by now. HEY CA, FIX IT. CHEERS

    There is a whole thread (several actually) dedicated to non-constructive venting. So unless you have something constructive to say about this fix or a suggestion on how to make this post better, GET OUT OF MY THREAD.
  • IkoshiIkoshi Junior Member Posts: 20Registered Users
    edited April 2012
    Did you manage to end a coop campaign with this system, Psycho?
  • HeroicSpurHeroicSpur Member Posts: 57Registered Users
    edited April 2012
    'Works like a charm as far as I can tell. Just finished a 109 turn game with zero desyncs.'

    He says he did.
  • HonourMikeHonourMike Junior Member Posts: 1Registered Users
    edited April 2012
    nope :/ working for a while, but once again around 20ish turns in... We had to exit. On reconnecting, it gave us an error with the auto save : incompatible versions. We went back to earlier manual save, ran a turn. Exited, to load from the new auto-save. We were able to connect, but a few turns after, we started having problems like: it would appear it was both our turns at the same time!!?? and battles lead to desync's. >:'(

    3 years creative.... We bought this game for some co-op fun. My mate will now be eating smart price beans for the next month to afford a game he can't play. This feature is broken. If our progress was somehow recoverable, I wouldn't mind wasting 5mins to exit and reload every 2 hours.

    What you guys need to do, is create an override force sync button. Available at the host a game overview. When pressed it overwrites the info of the co-op partner, with the expected values your POV has calculated for them. If in my view its my tern, then it forces him to not be in his, if from my POV some assassin has killed one of his generals or something, then it force updates that.

    Maybe something that tests sync in game, so as to possibly identify what's out of sync. But more importantly we can immediately find out when we did something to break it and go back to an earlier save and try again.

    You're part of the steam cloud save services, cant you use that to somehow?

    Best of luck guys :'( ill be ever waiting for a fix
  • BOTA:99BOTA:99 Junior Member Posts: 8Registered Users
    edited April 2012
    Me and my friend pretty much only play coop and prolly have 15 games played from start to finish and only had 1 we could just never get to work. The key is the hosts computer. Do a file transfer from host to other player using whatever means you find works best. We use skype because the file transfers take only 5 or 6 seconds. Also make sure to clean all old files out! Do not keep them as the op suggested. If you find you restart and it still will not stay synced, avoid and agent actions, naval or ground combat for one turn, resave, and retransfer. It also is a huge help if you use a free program called teamviewer that allows you to see your partners desktop. You actually will be able to pinpoint the exact moment of desync. We have found that around 80% of the time it is agent actions, the majority of the rest from diplomacy actions, and for some reason occasionally naval battles are the problem.
    As i said, doing things in this manner has helped up finish every single game we have started except 1.
  • PsychoAngelicPsychoAngelic Junior Member Posts: 9Registered Users
    edited April 2012
    BOTA:99 wrote: »
    Also make sure to clean all old files out! Do not keep them as the op suggested..

    While cleaning old files out is a good idea, I never suggested you keep the old files.
    BOTA:99 wrote: »
    You actually will be able to pinpoint the exact moment of desync. We have found that around 80% of the time it is agent actions

    The game creates an autosave at the end of each turn on each person's computer, and that file is unique to each person and that is where the actual desync occurs. What you describe is the manifestation of the desync.
    BOTA:99 wrote: »
    Do a file transfer from host to other player using whatever means you find works best. We use skype because the file transfers take only 5 or 6 seconds.

    My way eliminates the need to do this with the same result being playable games. If you read my post I clearly say that doing file transfers all the time is fine, it is troublesome. I came up with this way to avoid not only desyncs but to automate the file transfer.
  • PsychoAngelicPsychoAngelic Junior Member Posts: 9Registered Users
    edited April 2012
    HonourMike wrote: »
    We had to exit. On reconnecting, it gave us an error with the auto save : incompatible versions. We went back to earlier manual save, ran a turn.

    That sucks. Make sure that there are no other files in your save game folder, only the symbolic link. I also suspect the game uses old save data to create new saves so the downside to this method is you have to play the same multiplayer game with the same person, you can't start a new one and then pick up the old one later.

    That said, I've been through two full games now without a desync using the method in the OP. Possibly try it again and make sure you follow the instructions to the letter?
  • GebugGebug Junior Member Posts: 6Registered Users
    edited April 2012
    So was using this method, and got to turn 86, with multiple reloads. When reconnecting at one point, my coop partner couldn't connect, and he kept getting the message 'Failed to connect'. It worked when we used a quicksave, but we very rapidly encountered a desync soon after. We're planning to see if we can swap quicksaves to make it work, but (after some questioning) we found that some things are dramatically different in each others games - his warrior class Ironclad was halfway around the map on his screen, but I had seen it sitting in his bay for the last 5 or so turns.

    Hopefully we can get our game working again - but after I read the last post;
    Make sure that there are no other files in your save game folder, only the symbolic link. I also suspect the game uses old save data to create new saves

    It doesn't look good...

    (By the way, when you say 'Follow the instructions to the letter', I have actually got to question something- you say the other person should do ALL steps except 4. Surely it should be till step 4? So in that way they aren't making a symbolic link too? Or am I missing something here?)

    Regardless, thank you for this solution. Its been a wonderful feeling to be able to actually play something for a good amount of time!
  • PsychoAngelicPsychoAngelic Junior Member Posts: 9Registered Users
    edited April 2012
    Gebug wrote: »
    When reconnecting at one point, my coop partner couldn't connect, and he kept getting the message 'Failed to connect'.

    We got that last night after reconnecting. Our autosaves were different so my friend just pulled the autosave off dropbox and replaced the one in his folder and we were rolling again with no problems. I strongly recommend you do not use quicksaves in any form as I strongly think the game uses old saves to create new ones from saves in the folder. By only having one save (the autosave) you force the game to create a new save from scratch each time.

    (By the way, when you say 'Follow the instructions to the letter', I have actually got to question something- you say the other person should do ALL steps except 4. Surely it should be till step 4? So in that way they aren't making a symbolic link too? Or am I missing something here?)

    Incorrect. Both players need to create symbolic links. The connecting player just takes the autosave from the host. Both players need to use the same save and if you had it set up so only the host did the symbolic link, you are missing the overall aspect of the method as the connecting player is still using their autosave and not the hosts.

    Using the symbolic links, you are moving the save game location to a shared Dropbox folder. The connecting person is then taking the autosave from the host (think of it as emailing the autosave after each turn only automated). Because Dropbox automatically will update the autosave everytime it is changed (when the host ends their turn), when the connecting player ends their turn and creates an autosave, Dropbox recognizes it is different and updates the save on either end.

    Update: I've finished a Shogun 2 game with no problems and am now 85 turns into a RotS game with no problems using this method. The only minor problem we've had is a "failed to connect" message though a simple check to see if the autosaves were the same solved it.
  • Zerobot1Zerobot1 Junior Member Posts: 5Registered Users
    edited April 2012
    I don't know if this works but I'm trying it out with a friend and thought I'd add to make it a little easier to set up.

    Link Shell Extension lets you set up symbolic links without using the command prompt.
    Download and install it first of all. Right click on your save file in your shared dropbox folder, click 'Pick Link Source', navigate to your "save_games_multiplayer" folder, right click anywhere inside the space inside the window, Drop As... Symbolic Link. And you're done.

    For XP users, if you scroll down the page there's a driver to download so you can make symbolic links using this method. Search the page for 'Symbolic links for Windows XP'; your antivirus probably won't like it but I'm pretty sure it's safe.
  • FlemeFleme Junior Member Posts: 13Registered Users
    edited April 2012
    We're going to give this a go on our next campaign. I'll come back with our experience sooner or later. Also, thanks for coming up with this. We've been using Dropbox manually and while FotS made this more convenient with the 1/10 save file size, this'll hopefully make the problem go away.
  • RemairRemair Member Posts: 32Registered Users
    edited April 2012
    All the feedback looks hopeful, will give it a shot at some point. We only do multiplayer stuff on weekends so might be some time before we get results.

    Cheers for this Psycho. And CA.. on the incredibly vague chance you're reading this: Implement an automatic system to do this in the next patch and pay Psycho for doing your job! The man deserves a medal.
  • BananaRepublicBananaRepublic Junior Member Posts: 2Registered Users
    edited April 2012
    Hey guys,

    a friend of mine and I were glad to find out that TW runs with only one version playing it via steam offline mod/LAN. We've been playing Coop-Campaigns since Napoleon came out, so that we got an amount of over 1k hours gaming time. At some point we stopped counting the failed campaigns, which were caused by desync.

    We keep looking for a fix or at least a workaround and as prevention before starting to play FotS I searched the forums.
    After I found this thread, I tried to get this method working for Fall of the Samurai, but the game doesn't create an autosave after turn 1. So what I did was manualy overwriting/saving the autosave (same letters), which still existed from Vanilla Shogun 2. After that I followed the steps posted here. When i tried to load the campaign the loading screen reset itself and the background image turned into one, from which I guess, it was used in Shogun 2 Vanilla (tree with birds).

    Well, guess I did it wrong, manually overwriting the old autosave. But why doesn't the game create one on its own after a player ends his turn? Also it didn't after a few rounds and it didn't later, when I played with a friend of mine.
    So up to now this didn't work for me, cause I think i'm to noobish to get this running :D

    Later that eve the friend of mine came to me and I was like: Lets just play the usual style we playing, trying to avoid desync by saving the game every time seasons change (only the host) and reload it, if desync occures, which actually seems to be counterproductive, after i read this thread. Playing LAN the advantage is, that we can easily compare our movements of armys, ships and agents. Further more we count casualties respectively survivors after a battle. Desyncs could be detected very easy back in Napoleon, cause both players had a screen with the results of a battle. That's why we got to count manualy right now :P

    While playing the campaign yesterday, I recognized some improvements in order to avoid the desync. Since Shogun 2 Vanilla they changed the opportunity to interact (especially repairing) while the turn changes. In FotS you are not allowed to manage your cities during the AI-turns, too. Furthermore when it's the other human players' turn, you neither can't watch "family and council" (guess it's called like this, got to excuse for my bad english) nor interact with this feature (addopting generals, give ranks etc.).

    Well, the point is: Yesterday we made it until turn 114 using my manual save-"strategy" without desynch. We played the whole session without a break. Gamemod was normal. Also a lot of agent moves, take-overs of troops with the Ishin-Shishi and even 4 or 5 naval battles. Hopefully this is one of some 100 campaigns, which we can bring to an end =)

    However I'd like to know about how to get an autosave, created by the game respectively this method running.
  • GebugGebug Junior Member Posts: 6Registered Users
    edited April 2012
    Thank you for your reply. We managed to finish that campaign after all in the end - seems as if you are correct, the past saves do affect the newer ones. After copying over all the saves, we managed to finish the campaign in RotS (turn 114 I think it was).

    Apologies for misunderstanding before - I thought only the host files could be used to load the game from, but if both the host and the partner can use the same save, then there is no issue it seems. However, if it is saving to the same place by two different people, (hence overwriting), how did you 'replace' the autosave? Presumably if you were both saving to the same place, there is only one autosave in use?

    But regardless of my queries, it works nicely using the symbolic link. Even if one player is not very technically competent (which is what happened in my case), just having one person link the autosave to dropbox seems to work nicely. I'd say spread this idea of yours about as much as possible, this might end up helping solve this coop issues officially!

    Thank you very much for this solution!
  • FlemeFleme Junior Member Posts: 13Registered Users
    edited April 2012
    Updating as promised. We've now played 101 turns of a Legendary campaign using this fix and so far so good. Obviously this is great for Legendary, which uses only autosaves to begin with.

    Thanks for coming up with a solution to the problem - but at the same time it boggles the mind that CA could not implement something this simple as a built in functionality to their games.

    I'll update again once we're finished with our campaign.
  • AlJabberwockAlJabberwock Moderator Posts: 7,293Moderators, Tech Moderators
    edited April 2012
    Nice thread.

    I sympathize strongly with the MPC folks, and I have pruned nonsense, nonconstructive posts that help the cause not at all.


    Note that something _is_ afoot in this area officially...
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    “Every so often things happen that can’t be rationalized in a conventional way. People wanna know their government has a response. I am that response.”
    ― Kent Mansley (in "Iron Giant")

    For most general problems, for which you have no idea of the culprit, your first port of call should be:
    https://help.sega.com/home

    If you are aware of a bug or a specific issue for which you know the cause, post in the support section for the specific title on our forums. Feel free to PM me or Matthias CA if you haven't received a response within 24 hours, 48 hours on the weekend. ~Al

    https://forums.totalwar.com/discussion/172193/forum-terms-and-conditions#latest
  • GebugGebug Junior Member Posts: 6Registered Users
    edited April 2012
    Update; have started another campaign, 119 turns in have run into a slight issue that meant we could not continue.

    Just as I was about to run out of time on my turn, my partner's game CTD'd, and they were disconnected from the game - however, the game continued and ended my turn going through the AI moves, just before I quit to allow them to come back in. When we did start the campaign back up, (while before they were playing as Nagoka) it registered they my partner was now playing as Jozai (whom I had killed a while back as Aizu), and their difficulty had been reset from Very Hard to very easy.
    When we started up the game, they CTD'd and I got a message saying my partner had been defeated, and so the game would end.

    We've checked the files are linked, verified the game, and tried a few more times to connect, all with the same effect. Seems that the autosave has been changed in such a way that now we are unable to carry on, and as we have stopped the use of Quicksaves, as per previous conclusions on this thread, I think this campaign is pretty much dead, alas.
  • GebugGebug Junior Member Posts: 6Registered Users
    edited April 2012

    Note that something _is_ afoot in this area officially...

    While this would be promising, can you point to where you got this 'official' information from?
    Considering how long its taken & the amount of failed fixes, you'll have to forgive my slight skepticism.

    (Hope springs eternal nevertheless....)
  • Billy RuffianBilly Ruffian Moderator UKPosts: 27,827Registered Users, Moderators
    edited April 2012
    Post from CraigTW on TWC http://www.twcenter.net/forums/showthread.php?p=11226100#post11226100. Last two paragraphs.

    "He uses statistics as a drunken man uses lamp-posts - for support rather than illumination." (Andrew Lang)

    |Takeda| Yokota Takatoshi

    Forum Terms and Conditions: - https://forums.totalwar.com/discussion/172193/forum-terms-and-conditions#latest

    "We wunt be druv". iot6pc7dn8qs.png
«1
Sign In or Register to comment.