7 replies to this topic

[tmcsweeney]

When merging a add/delete pair into a stream with a view that excludes half of the pair P4V silently ignores the entire pair, (but merges everything else).  I'd expected it to merge the part of the pair relevant to the stream view. If you're merging a bunch of dependent stuff at the same time it is really easy to break the code in the stream.

Let me explain the full situation below, I'd love to know if I'm trying to do something beyond the pale, or if there is a better way to achieve what we want from the streams.


We've got a large Unity project stored in Perforce. The project is designed to be built into a number of different products that would each consist of a subset of the master project's features. There are a bunch of core shared features, and then a set of optional features for the various products, something like:

Assets
	+  Core
	+  Features
		+  Flowers
		+  Trees
		+  Vegetables
		+  Caterpillars

The mainline stream contains the super set of all of the features, but when working on a particular product child stream we'd like to exclude anything not used in that product.  This is partly to keep the size down (models, textures, animation and audio are large and slow iteration) and partly to enforce separation of dependencies between features. So the stream views are set up something like:

stream name: Product_Wilderness
Paths:
share ...
exclude /Assets/Features/...
share /Assets/Features/Flowers/...
share /Assets/Features/Trees/...

i.e. For a given product we: include everything, exclude all of the optional features, then re-include just the optional features we want.

The problem came when something was checked in to Core, that really should have been part of Features/Caterpillars.  The developer in the Caterpillar stream checked in a move/delete from Assets/Core/Caterpillar.cs -> Assets/Features/Caterpillars/Caterpillar.cs  This change copied up to mainline fine, and  it merges fine into any child that includes both Core and Features/Caterpillars but when trying to merge down in to the Product_Wilderness stream (with no Caterpillars) the add/delete always gets ignored leaving Caterpillar.cs in Core.  

I'd expect that from the Wilderness stream's point of view the delete part of the add/delete pair would still merge, because from the point of view of that stream that is exactly what has happened. Caterpillar.cs should disappear.

From fiddling around if we switch to merging use filespecs and specify either -Di or -1 in the advanced options then the delete half of the merge will come through. But it is annoying in that there is no indication that the merge has failed, and to fix it you have to drop out of the nice stream workflow.

Am I just asking too much of Perforce?

The other way we considered going was to make the product streams full copies of mainline, but then use virtual streams with limited views for anyone working in them.  We rejected this because it makes it a bit harder to create dev streams as children of the product streams, and requires anyone merging from mainline to remember to switch to a workspace that has the full product stream synced.

[tmcsweeney]

I've done some more testing and I'm now feeling more and more like this is a bug, or at least is very very unintuitive.  When I try and do the same thing the other way around it works.  

That is:
In the Caterpillars stream, move a file (UsefulSharedUtil.cs)  that is inside Assets/Features/Caterpillars/ and move it to Assets/Core/
Copy the change up to mainline.
Merge down in to Wilderness stream.
tada! The Add half of the Add/Delete pair DOES show up inside Assets/Core/ in the stream with a view that excludes the delete half of the pair.

[Sambwise]

The explanation in a nutshell of why this happens is that to avoid "evil twinning" (which is what happens in your second example, and is often considered undesirable), the two halves of a move are considered to be a single file with two paths, and the newer path (i.e. the move/add path) is the canonical identifier for that file.  To implement this, a file whose head revision is "move/delete" is ignored by a merge operation, and when a "move/add" revision is included as part of a merge range the resolve operation will include a filename resolve in order to account for the move/delete.  If the move/add revision isn't part of the merge, it's assumed that the corresponding move/delete has either been accounted for already or that it will be (e.g. if you were to cherry-pick around the move/add rev, you would not get the move/delete rev independently of it).

[tmcsweeney]

I think I understand what you re saying: "If the move/add revision isn't part of the merge, it's assumed that the corresponding move/delete has either been accounted for"  means that in my case where the move/add is in the part of the stream that is excluded the delete gets ignored as well.

The bit I don't understand is how are excludes in child streams supposed to work given this? Or to put it another way, how can I set up my streams so that the children don't need to have everything in their parent but moves still "work"?

[Sambwise]

Quote

The bit I don't understand is how are excludes in child streams supposed to work given this? Or to put it another way, how can I set up my streams so that the children don't need to have everything in their parent but moves still "work"?

It's not at all possible for moves to work (i.e. remain atomic, which is the point of having a special "move" action) if you exclude half of the move.  Generally the idea is that if you "move" a file you're moving it within a self-contained module, such that its atomicity isn't going to get broken by future branching operations or view partitioning.  This gets loosely (and indirectly) enforced by the requirement that two parts of a move are always submitted in the same changelist, and the related requirement that a changelist can only ever include files from one stream (which in the simple case represents a single module).

If the move doesn't need to be atomic, you can do a "delete/"add" combination, such that rather than having a single file whose location changes, you have one file that gets deleted and another that gets added, with no dependencies between them.  This is much more permissive of the possibility that a child stream might end up with no file at all, or two of the "same" file, and it also makes merging between the two variants more difficult, but it does permit you to operate on either file in isolation and have it just behave like any other file, without regard to the other variant.

[tmcsweeney]

I had an epiphany while having morning coffee, I think my whole mental model was wrong, When I edit the stream and change the "Paths" field, I'm really not changing the structure of the stream at all, rather I'm changing the view of it that any workspaces pointing at that stream get. Thinking about it this way makes a lot of my mistakes stand out, and makes your explanation click for me.

Replacing moves with adds and delete probably isn't practical, Unity (and other tools) already generate moves, they'd sneak in and we wouldn't find out that they're a problem until they are already committed and it is too late.  Plus losing history is gross.

Would it make more sense to switch to virtual streams? Have the child streams be full copies of mainline, and then set up virtual streams that do the exclusions? People working day to day would work in the virtual streams with the limited set of content, but anyone merging to pr from mainline would use a workspace pointing at the full stream.  There is a bit more management overhead, but if it is the only way to achieve the workflow we need....

Or... crazy idea... use the "Remapped" field on the stream to move the undesired content out of the Unity project path when syncing locally. Everyone would get the full massive set of content, but most of it would be in a location within their workspace where it would just be ignored by our tools.  Wasteful of bandwidth/time/disk space but it might work?

Or is there some other way to have limited views of streams that I'm not aware of?

[Sambwise]

Using virtual streams is the standard way of having a "limited" view, yeah.  Usually a virtual stream is *less* overhead than a child stream with "share" paths because you don't need to merge every time the parent has a new change; it just syncs down automatically.

[tmcsweeney]

Thanks so much for your help, I'll give it a go.