Jump to content


Differentiating between unresolved vs "resolve -ay" integrations

integrate resolve stream

  • Please log in to reply
4 replies to this topic

#1 Daniel B

Daniel B

    Newbie

  • Members
  • Pip
  • 3 posts

Posted 05 February 2019 - 08:20 AM

I'm trying to manage automated upstream integrations between 2 streams, with occasional manual downstream pulls. There's context below on how and why our streams are set up to get to this state, but the core of the issue is:

If a branched file is modified in both //stream/Parent and //stream/Child (or just related branches, this isn't exclusive to streams), then those changes are merged and resolved downstream into //stream/Parent, it seems to be impossible to safely automate getting the merged result back to //stream/Child.

p4 integ -S //stream/Child -r
p4 resolve -at
// AKA
p4 copy -S //stream/Child -r -F
is unsafe - it will blindly overwrite conflicts that have not been resolved.

p4 integ -S //stream/Child -r
p4 resolve -as
will usually work - leaves unresolved conflicts unresolved (exactly what we want), doesn't checkout conflicts that have been "resolve -at" in //stream/Parent, and copies back up changes from either "resolve -am" or p4merge by a user in //stream/Parent.
Where it fails is anything that was resolved in //stream/Parent with "resolve -ay" or a p4merge that resulted in content identical to //stream/Parent/file.

This is the end result:
resolveIssue.jpg

This seems pretty bugged to me, is it intended? If so, how can we mark a revision as "this version is correct and should be propagated to related streams"?

Context on stream setup:
I know the default "merge down, copy up" paradigm wouldn't allow this to happen because ALL changes would be resolved into one stream before blindly copying into the other, but that is not quite a perfect fit for our needs, whether //stream/Child is treated as stable or unstable.
We have an "Authority" mainline with occasional well-tested commits, and an "Outsource" child which frequent, potentially unstable changes are submitted to.
All commits to Authority should be integrated ASAP to Outsource - this is the automated process I'm maintaining. Since there will be as-yet unapproved changes in Outsource, it cannot be a "Stable" stream unless I go "against the flow" when integrating across. This seems like the best option, but hits the issue described above.

However, since all integrations from Outsource to Authority are performed by taking one or more (but not all) changes, resolving any merge conflicts, then testing in the stable environment before committing that piece of work, we need to merge into Authority, so Outsource can't be a "Development" stream either (again, unless we go "against the flow" and integrate to Authority, setting us up for the same issue).

#2 Sambwise

Sambwise

    Advanced Member

  • Members
  • PipPipPip
  • 889 posts

Posted 05 February 2019 - 03:19 PM

I think the crux of what you're hitting is that "resolve -ay" creates divergence between streams, and (more importantly) the default behavior of a subsequent "p4 merge" is to preserve that divergence in both directions.

In other words, in the first image, the merge into Outsource is attempting to propagate changes 77 and 78 into Outsource while also preserving change 75 (which exists in Outsource but not Authority).

In the second image, the merge is treating the difference between revisions Outsource#3 and Authority#4 as a change to be propagated (change 80, which effectively is a rollback of change 75), so the merge into Outsource rolls back change 75 and is a simple copy.

The difference is that in the first case, the previous merge was an "ignore" (creates a point of divergence) and in the second it was an "edit" (adds a new change that is a candidate for the next merge back, which will cause the two streams to converge again).

If you want *all* of your "resolve -ay" records to be treated like the second image, there's actually a configurable on the server to do exactly that:

dm.integ.tweaks 0 Modify integrate behavior (engine=3 only):
				1: Treat all 'copy' records as 'merge'
				2: Treat all 'ignore' records as 'edit'

See the "esoteric configurables" section of the "Base Picking" white paper: https://swarm.worksh...ase Picking.pdf

#3 Daniel B

Daniel B

    Newbie

  • Members
  • Pip
  • 3 posts

Posted 06 February 2019 - 12:29 AM

View PostSambwise, on 05 February 2019 - 03:19 PM, said:

I think the crux of what you're hitting is that "resolve -ay" creates divergence between streams, and (more importantly) the default behavior of a subsequent "p4 merge" is to preserve that divergence in both directions.

In other words, in the first image, the merge into Outsource is attempting to propagate changes 77 and 78 into Outsource while also preserving change 75 (which exists in Outsource but not Authority).

Yes, that's exactly it - if this is working as intended, it's bonkers! There's a world of difference between "This is the correct change to be implemented in this file" and "That change should be isolated to that stream/branch".
For starters, wanting to create a divergence like that is far more likely if there were no corresponding change in Authority - in which case one would have to manually force a resolve despite the lack of conflict, then do a "resolve -ay" to ignore the change from Outsource.

Secondly, in this example I actually never explicitly performed a "resolve -ay" - all conflicts were resolved through the p4merge GUI.
This was the merge back to Authority that led to change 80 in file1 (which fails to propagate back to Outsource):
resolve3merge.jpg
Whereas this is the same merge for file3 (which propagates as expected):
resolve3merge-both.jpg

The fact that 2 such logically similar operations create such a difference behind the scenes by default is concerning, especially when the difference could be as little as one character of whitespace left in or out.


View PostSambwise, on 05 February 2019 - 03:19 PM, said:

If you want *all* of your "resolve -ay" records to be treated like the second image, there's actually a configurable on the server to do exactly that:

dm.integ.tweaks 0 Modify integrate behavior (engine=3 only):
1: Treat all 'copy' records as 'merge'
2: Treat all 'ignore' records as 'edit'

See the "esoteric configurables" section of the "Base Picking" white paper: https://swarm.worksh...ase Picking.pdf

Thanks, I'll look into that. I can imagine there is definitely a use for marking a given change as something to be preserved but isolated to a given branch, but in the middle of conflict resolution is absolutely not it. Surely there should be a separate command to mark a change as such!

#4 Sambwise

Sambwise

    Advanced Member

  • Members
  • PipPipPip
  • 889 posts

Posted 06 February 2019 - 12:51 AM

My memory of this is hazy now since it was years ago, but there was a fix in the client API and the CLI to make the default suggestion when a merge tool produces the "yours" file "ae" (for edit) rather than "ay" (for ignore), for exactly the reasoning you give (that if you arrive at the "yours" file via an interactive merge tool, your intent is more likely to be "this is how I resolved this conflict" rather than "I'm opting to ignore these changes wholesale" as with "resolve -ay).  However, I'm pretty sure the GUI doesn't actually do anything with that "hint" the way the command line does.

(edit) oh, I found the actual code that does this.  :)  https://swarm.worksh...ntmerge3.cc#586

  // If the edited result equals yours, suggest edit instead.
  // In most cases if the user has picked out the yours diff
  // in an editor, they will want to reverse-integrate it.

  if( autoStat == CMS_YOURS )
	  autoStat = CMS_EDIT;
	 }

So if you did your resolves via the command line they'd do the thing you want (for all the good that might do).  The command line also has an undoc "-e" flag that forces conversion of ANY ignore record to an "edit"; this was a request from a site that hit the problem you describe on a pretty regular basis during a particular workflow, and wanted to have a flag they could add to the resolve commands in that workflow (while keeping the default behavior in all other cases -- oy).

#5 Daniel B

Daniel B

    Newbie

  • Members
  • Pip
  • 3 posts

Posted 06 February 2019 - 02:30 AM

Aha, that doesn't quite help in my situation, but pointed me in the right direction!

In practice, many of the files we're seeing this issue on are binaries (image files) and the people doing conflict resolution are inspecting each file and using P4V to select which one to keep - so unfortunately can't rely on them running a specific undocumented resolve option on the CLI :)

I'm also understandably wary of messing with server-wide configs that could affect other projects, regardless of how much I still think dm.integ.tweaks=2 should be true for most cases.

But both of these led me to what looks like the silver bullet I was hoping for: the integration script pushing from Authority to Outsource simply has to use the undoc "integrate -Znnn" option:
p4 integ -S //stream/Child -r -Z2
p4 resolve -as
and truly unresolved conflicts are still flagged, while anything resolved in favour of either parent is synchronised without issue.

Thanks for your help!





Also tagged with one or more of these keywords: integrate, resolve, stream

0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users