Jump to content


How to do an forced "merged - Get Latest" with files not checked-out?

question resolve working offline

  • Please log in to reply
11 replies to this topic

#1 BJS

BJS

    Member

  • Members
  • PipPip
  • 16 posts

Posted 11 November 2019 - 08:59 AM

For different reasons, our software department has set perforce up such that we generally do not check-out files automatically.
Instead, people work with files having the <x> flag set and use "reconcile offline work" whenever they are ready to check-in their local work.

This generally works, but we run into issue when the following situation arises:

- Person A works locally on the files (i.e. the local files are modified, but not checked-out), so it has f.e. revision #3 + local changes
- The file on the depot gets a new revision #4
- Person A does a "Get latest" on the file.
Perforce now refuses to update the file with the "can't update modified file" error/warning.

How can I force a "Get Latest" update with automated merge-action (and potential prompt for manual merge if there are conflicts), without checking the file out?

i.e. I want to perform the action on my locally changed files, but I do not want to have the final file checked out, nor do I want to check the merged file into the depot.
I just want to have my local modified file but based on the "latest revision".

I guess a different way to ask the question is: How can I do a "Merge latest revision" operation rather than a "Get latest revision" ?

Side info: The workspace is set to "not clobber" and the "Force operation" option is not helpful, because I do not want to replace my local file with the revision, but merge my local file with the latest revision.

#2 Sambwise

Sambwise

    Advanced Member

  • Members
  • PipPipPip
  • 976 posts

Posted 11 November 2019 - 03:53 PM

The way that you tell Perforce "I'm working on this file, make sure other changes get merged into it nicely instead of overwriting my work" is to check it out.  Why make life difficult by forbidding yourself from checking the file out?  :)

If you really want the file to not be checked out while retaining your local content, you can revert -k it at the end:

p4 reconcile ... # open the file(s)
p4 sync ... # get latest (scheduling merges)
p4 resolve -am ... # auto-merge
p4 revert -k ... # revert the files, keeping local content

but if you just check out the files that you're working on (i.e. the standard workflow), you don't need to do any of this and can just work normally.

It's technically possible to reverse-engineer everything that Perforce does automatically during the sync/resolve process without actually letting Perforce open/resolve the file: you could construct local copies of the merge inputs (which you can determine by looking at what you synced and what the other user submitted), run a merge algorithm over them, replace your local file with the result, and then delete the temp files, but at that point you're implementing so much logic yourself that you almost might as well just have the files on a network file system instead of using a version control tool whose job is to handle all of that for you.

#3 BJS

BJS

    Member

  • Members
  • PipPip
  • 16 posts

Posted 13 November 2019 - 05:50 PM

View PostSambwise, on 11 November 2019 - 03:53 PM, said:

Why make life difficult by forbidding yourself from checking the file out?  :)

The reason for this is, that we've run into a different issue company-wide with having files checked-out. I don't know if this is a bug or a mis-configuration, but what we are seeing is this:

Person A is not checking-out or locally modifying any files, but just interested in "Get Latest" whenever building. (The stuff being build is a large code-base with a lot of inter-dependencies.)
Person B is working on a file, having it checked out and modified, but has not checked in the file with the changes yet (i.e an open pending change list).
Now Person C is checking out the same file, making changes, and checking the changes back in, increasing the revision number in the depot.

When Person A is now doing a "Get Latest", Perforce is not updating the file, despite it not being locally modified or checked-out. This completely breaks the workflow for Person A.

Additionally, we have a regular build-server which acts like Person A, so whenever we run into the situation described above, our automatic builds fails as well.

View PostSambwise, on 11 November 2019 - 03:53 PM, said:

p4 reconcile ... # open the file(s)
p4 sync ... # get latest (scheduling merges)
p4 resolve -am ... # auto-merge
p4 revert -k ... # revert the files, keeping local content

The
p4 revert -k
option sounds interesting. Can it be invoked via the P4V UI-interface somehow?

Side comment: Is there any way to call a console command from P4V directly? (It's a PITA to open a separate command-prompt and session for that.)

#4 Sambwise

Sambwise

    Advanced Member

  • Members
  • PipPipPip
  • 976 posts

Posted 13 November 2019 - 09:52 PM

Quote

When Person A is now doing a "Get Latest", Perforce is not updating the file, despite it not being locally modified or checked-out. This completely breaks the workflow for Person A.

I assume this is a typo and you're talking about Person B, since you described this as a problem particular to people that *did* check the file out.

Person A should just get the latest version of the file whenever they "get latest", regardless of what Person B is doing.  If they don't, there's something else they're doing that you're not describing -- my guess would be that they're rolling back their local copy after syncing the latest version, or something like that.  In any event, whatever it is that might be messing them up has nothing to do with Person B having the file open; that's just not how anything works.  :)

When Person B (who DID open the file, which is the right thing to do) does a "get latest," they should get a message saying "must resolve (Person C's change)".  (In P4V, I think this is represented by a little red question mark next to the file or something like that.)  When they "resolve", that's what merges C's change into B's file (which is the exact thing you're asking how to do).

Give that a try and let me know how it goes.  If you're having a problem with not syncing down other people's changes, implementing a policy whereby you never check out your files is the exact opposite of the way to fix it.  :)

#5 Sambwise

Sambwise

    Advanced Member

  • Members
  • PipPipPip
  • 976 posts

Posted 13 November 2019 - 11:37 PM

View PostSambwise, on 13 November 2019 - 09:52 PM, said:

In any event, whatever it is that might be messing them up has nothing to do with Person B having the file open; that's just not how anything works.  :)

Waiiiiiiit.... you aren't doing something completely bananas like having everyone share one client spec, are you?

#6 BJS

BJS

    Member

  • Members
  • PipPip
  • 16 posts

Posted 14 November 2019 - 09:03 PM

View PostSambwise, on 13 November 2019 - 09:52 PM, said:

I assume this is a typo and you're talking about Person B, since you described this as a problem particular to people that *did* check the file out.

Person A should just get the latest version of the file whenever they "get latest", regardless of what Person B is doing.

I wish. No it was no typo, that's why I was wondering if it is a bug or mis-configuration.


View PostSambwise, on 13 November 2019 - 11:37 PM, said:

Waiiiiiiit.... you aren't doing something completely bananas like having everyone share one client spec, are you?

Can you elaborate on what you mean with "everyone share on client spec" ? (i.e. what do I need to check to find out.)
Note that I haven't set up the system in our company, I'm just using it. So I need to relate everything you tell me here to someone else.
This, plus the fact that nobody really has experience with Perforce can very well be the source of our problems :rolleyes:

#7 Sambwise

Sambwise

    Advanced Member

  • Members
  • PipPipPip
  • 976 posts

Posted 14 November 2019 - 09:37 PM

View PostBJS, on 14 November 2019 - 09:03 PM, said:

Can you elaborate on what you mean with "everyone share on client spec" ? (i.e. what do I need to check to find out.)
Note that I haven't set up the system in our company, I'm just using it. So I need to relate everything you tell me here to someone else.
This, plus the fact that nobody really has experience with Perforce can very well be the source of our problems :rolleyes:

Okay, I'm pretty sure this is the source of your problems, because everyone sharing one client spec will basically break every single Perforce workflow, in roughly the way that you've described.

In P4V a client spec is called a "workspace" (technically a client spec is the thing in the server that describes a workspace; P4V just calls them both the same thing).  Everyone using Perforce should have their own user name and their own workspace.  When you look at the list of users in P4V, you should see each person's name there, and when you look at the list of workspaces, you should see all of their machine names.  If someone sets up Perforce on a new machine and they pick an existing workspace instead of creating a new workspace, then when they run a "get latest", they're not going to get any files because as far as Perforce is concerned they already have the files.  Perforce tries to protect you from doing this by having a "Host" that each workspace is bound to, but it's possible to zero that field out so that a workspace can be used independently of the files that are in that workspace.  This usually leads to confusion and heartbreak.

Anyone who's in the situation of someone else using their workspace and messing them up (this includes your build machine) can fix this by doing the following:

1) Set a password on their user.  This makes sure nobody else can run commands as them.
2) Make sure the Owner of the workspace matches their user name.  If it doesn't, you are using someone else's workspace!  Make a new workspace for yourself*
3) Make sure the Host of the workspace matches their computer name.  If it doesn't, you are using another computer's workspace!  Make a new workspace for this computer*
4) Set the workspace to locked.  Now only the Owner can use that workspace.

* a new workspace will always start with the current user as the Owner and the current client computer's hostname as the Host

Each person who goes through that process should now find that everything works correctly (for them at least) because nobody else is able to mess up their workspace state.  You will be able to "get latest" without having to force a re-download of everything, you will be able to check out files and merge changes submitted by other users, et cetera.

If you don't have a Perforce license and you're consequently not able to create a workspace for each user, then I regret to tell you that you've exceeded the limits of the free version and should either pay for a license or uninstall it and use another tool.  You'd be better off just putting your files on a network filesystem (or GDrive or Dropbox or what have you) than trying to use Perforce in a configuration where everyone's sharing a single workspace.

#8 BJS

BJS

    Member

  • Members
  • PipPip
  • 16 posts

Posted 15 November 2019 - 08:44 AM

Thanks for the details, but no, that does not seem to be the problem.
As far as I can see, we all have separate workspaces for each computer in use, but we'll possibly do a careful check of that again.
Certainly the Person A, B, C of my example above had all separate workspaces at the time of trying.
What we do have is multiple (different) workspaces per machine for different projects, but I think that is as it is supposed to be. But each workspace has a single (correct) Owner and Host.

View PostSambwise, on 14 November 2019 - 09:37 PM, said:

If you don't have a Perforce license and you're consequently not able to create a workspace for each user, then I regret to tell you that you've exceeded the limits of the free version and should either pay for a license or uninstall it and use another tool.

No worries there. Quite the opposite. We've rather recently changed over to Perforce from a different system.

#9 Sambwise

Sambwise

    Advanced Member

  • Members
  • PipPipPip
  • 976 posts

Posted 15 November 2019 - 09:06 AM

I'm still pretty sure that you have some kind of workspace-sharing issue going on, since that's the only way I can conceive of one user's open file affecting another user's sync, but if you do have the ability to create workspaces then at least it's solvable.  :)  Did you take the step of locking the workspaces and password-protecting user accounts to make absolutely sure that they can't be shared accidentally?

Quote

Certainly the Person A, B, C of my example above had all separate workspaces at the time of trying.

If this were a situation that could ever be recreated, then I'd be very interested in seeing the output of person A's "sync" command (to see what error it returns, if any) as well as an "opened" command (to verify that there are no files open on that client).

#10 BJS

BJS

    Member

  • Members
  • PipPip
  • 16 posts

Posted 18 November 2019 - 05:41 PM

View PostSambwise, on 15 November 2019 - 09:06 AM, said:

I'm still pretty sure that you have some kind of workspace-sharing issue going on, since that's the only way I can conceive of one user's open file affecting another user's sync.

Let me think... there is one thing which might be untypical going on in our setup...
All three persons A, B and C have their own workspace mapping onto their own local directories ( A_ws, B_ws, C_ws ).
However, all three (may) use batch-files of a different (common) workspace mapping onto the same local directories during some automated building process. (autoBuild_ws)

i.e. the workflow of person A may be:
  • Use workspace A_ws to sync from depot.
  • Build stuff locally. (no code was changed)
  • Call an automated build-script, which uses the workspace autoBuild_ws to sync from depot into the same local directory-tree (same mapping structure) and performs a series of automated builds.
  • Use workspace A_ws to sync from depot. --> now can see the problem described earlier
Person B and person C might also have run the same batch files (with autoBuild_ws) but when checking-out/-in files, they are always working on there respective unique workspaces.




View PostSambwise, on 15 November 2019 - 09:06 AM, said:

Did you take the step of locking the workspaces and password-protecting user accounts to make absolutely sure that they can't be shared accidentally?

Yes.


View PostSambwise, on 15 November 2019 - 09:06 AM, said:

If this were a situation that could ever be recreated, then I'd be very interested in seeing the output of person A's "sync" command (to see what error it returns, if any) as well as an "opened" command (to verify that there are no files open on that client).


It was fully reproducible, but I tested it with some colleagues when we were physically together. Unfortunately, I'm not now as I work remotely and several hrs time-difference in between. But it can be arranged...

#11 Sambwise

Sambwise

    Advanced Member

  • Members
  • PipPipPip
  • 976 posts

Posted 18 November 2019 - 08:10 PM

Quote

However, all three (may) use batch-files of a different (common) workspace mapping onto the same local directories during some automated building process. (autoBuild_ws)

This sounds sketchy.  The fact that things go wrong after you run this script is a good indication that something's not working right with it.  Is there any possibility that the script is actually modifying persistent configuration state (e.g. it's running "p4 set" or otherwise modifying config files) so that the shared workspace badness is "leaking" into the user's environment after they run the script?

By "same local directories" do you mean that autoBuild_ws and A_ws share a local client root (or that one is a subset of the other)?  If so, that's inherently bad even if everything else is being managed well.  The script should not be using autoBuild_ws in that case, it should just inherit the environment it's executed in so that it uses the user's own workspace (that way Perforce can actually track the workspace state correctly and it'll be a lot harder to mess things up in confusing ways).

The only way I can see this shared autoBuild_ws thing working well is if it's always using its own dedicated root directory *and* the only command you ever run with it is p4 sync -p (which is basically a "stateless" sync operation) so that there's not any workspace state that's being shared between multiple client hosts.  In addition, the script should always be passing the workspace option statelessly (i.e. it should run "p4 -c autoBuild_ws sync -p", not "p4 set P4CLIENT=autoBuild_ws;p4 sync -p") so that there's no possibility of leakage.

#12 BJS

BJS

    Member

  • Members
  • PipPip
  • 16 posts

Posted 19 November 2019 - 08:45 AM

Thanks for the additional info. I'll pass it on to the person managing our system.





Also tagged with one or more of these keywords: question, resolve, working offline

1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users