Jump to content


Exclude tree for p4 fetch

personal server dvcs fetch

  • Please log in to reply
7 replies to this topic

#1 danielku15

danielku15

    Newbie

  • Members
  • Pip
  • 4 posts

Posted 04 December 2018 - 03:00 PM

Hey everybody.

TL;DR: Is there a way to tell p4 fetch to ignore certain trees in the history completely? I do not have access to certain trees and/or they are not relevant for my fetch. But as the changes are part of integrations spanning multiple trees fetch is not possible anymore.


Long Version:
I am currently trying to fetch the source code from our product from the central server to my personal one on my machine. Unfortunately there seems a restriction on fetching I am currently fighting with I hope somebody can help me getting this running.

Before going into details first some introductional information:

We use a classical branching strategy for our product that is based on "folders". Historically one of our products lives below the main tree of our product suite.
  • //depot/OurSuite/OurSuite
    Trunk
  • //depot/OurSuite/OurSuite3.0
    Release 3.0
  • //depot/OurSuite/OurSuite2.0
    Release 2.0
  • //depot/OurSuite/OurSuite/SubProduct
    Trunk of "SubProduct"
  • //depot/OurSuite/OurSuite/OldProduct
    Trunk of "OldProduct
  • //depot/OurSuite/OurSuite3.0/SubProduct
    Release 3.0 of "SubProduct"
My goal is to fetch only "SubProduct" with all branches to my personal server.

Now to the restriction I am having issues with: We have some changelists that contains changes spanning both products. For instance when we update some version numbers there are some commits that bump the version file from "SubProduct" and "OldProduct" in one go. With this cross-folder changelist in place, a pull of "SubProduct" only seems impossible. Perforce requires you to widen your remote spec to include the whole "OurSuite" tree.

Typical error messages I encounter there are:
  • Change 748318 performs a move/add on //depot/OurSuite/OurSuite/OurProduct/SomeFile.cpp#1, but the parameters of this fetch, push, or zip command include only part of the full action. Specify a wider view to include both the source and target of the change, or specify a narrower view to exclude both the source and target of the change.
  • Change 750162 performs a move/delete on //depot/OurSuite/OurSuite3.0/OurProduct/SomeOtherFile.cpp#3, but the parameters of this fetch, push, or zip command include only part of the full action. Specify a wider view to include both the source and target of the change, or specify a narrower view to exclude both the source and target of the change.
I understand that typically it is a requirement that you need to include all files in your view that are related to your tree to fetch it. But requiring that really every dependency that might be once be part of a changelist is in the view, breaks the whole benefit of maybe consuming only a sub-tree you are working with. In my case the problem goes really far and I would end up in cloning our whole depot just to get "OurProduct". Also I have some trees where I do not even have read access. Just because another developer merged once in the past 2 trees in one shot, fetching for me is not possible anymore.

Is there some way how I can tell perforce to ignore these errors and maybe only fetch the parts of the CLs that are contained in my remote spec? I do not care that maybe with 750162 a whole other subtree below "OldProduct" was also merged back from the 3.0 branch to the trunk. I only want to consume the SubProduct tree with related changes.

My next step would be to push SubProduct to another perforce server with GitFusion enabled, hence I want to retain the whole history for SubProduct. The cross-tree merge happened quite lately which means if I fetch only partly the history, on git we will only see the changes of the last few weeks.

Home my intention is clear.

Kind Regards
Daniel

#2 Sambwise

Sambwise

    Advanced Member

  • Members
  • PipPipPip
  • 665 posts

Posted 04 December 2018 - 05:01 PM

I don't think the problem is that a changelist touched both projects, but specifically that you have files which are logically part of both "projects" due to having been "renamed" in between projects.  The idea is that a paired "move/add+move/delete" are logically a single atomic revision/file (with two different names) and most commands are loathe to split them up because that's how you end up with "evil twin" duplicate files.  (E.g. in a streams depot you aren't able to "move" files between streams since streams are supposed to be independent of each other; you can't submit a changelist with only one half of a move operation because changelists and moves are both supposed to be atomic; a fetch command won't fetch only one half of a move revision; integrate will detect that you're only integrating one half of a move and will either automatically pull in the other half or throw an error; etc.)

Most of those commands feature some sort of override flag or command where you can effectively "downgrade" the move and allow the evil-twinning to happen, but I don't see a flag on "fetch" that lets you do this.

What if you did the fetch with a shallower depth so that it's not trying to fetch the full history of each file (and will therefore disregard the "move" history that causes files to straddle namespaces that you do and don't have access to)?

#3 danielku15

danielku15

    Newbie

  • Members
  • Pip
  • 4 posts

Posted 05 December 2018 - 10:05 AM

Thanks for the quick response. Maybe I was not precise enough in my explanation but yes, I understand that if you want to fetch some files, you need to ensure that you do not fetch half of the atomic operations. But still I have the feeling there are some prerequisites as I described them. Maybe it helps to solve my issue if I provide the actual error I am currently facing. I hope just posting all this details is fine here. In some forums people are not happy with that kind of "posting all details and screaming for help"-kind of posts :unsure:

I cleaned up again all my earlier tests and started from scratch to start with the initial error I get. I have the following remote spec for fetching:

//stream/OurProduct/Trunk/... //depot/OurSuite/OurSuite/OurProduct/...

//stream/OurProduct/SomeBranch/... //depot/OurSuite/OurSuite_SomeBranch/OurProduct/...

//stream/OurProduct/Release7.2/... //depot/OurSuite/OurSuite7.2/OurProduct/...

//stream/OurProduct/Release7.1/... //depot/OurSuite/OurSuite7.1/OurProduct/...

//stream/OurProduct/Release7.0/... //depot/OurSuite/OurSuite7.0/OurProduct/...

[Some more here...]


And I get this error when executing the fetch:

Change 750162 performs a move/delete on //depot/OurSuite/OurSuite_SomeBranch/OurProduct/src/SomeFile.cs#2, but the parameters of this fetch, push, or zip command include only part of the full action. Specify a wider view to include both the source and target of the change, or specify a narrower view to exclude both the source and target of the change.


When I look at the history of this file I see this:


D:\Dev>p4 filelog -i //depot/OurSuite/OurSuite_SomeBranch/OurProduct/src/Testing/SomeFile.cs#2

//depot/OurSuite/OurSuite_SomeBranch/OurProduct/src/Testing/SomeFile.cs

... #2 change 750162 move/delete on 2017/10/04 by user1@user1_OurSuiteTmp (utf8) 'Merging using OurSuite_SomeBranch '

... #1 change 744444 branch on 2017/09/12 by user2@user2_OurSuiteTmp (utf8) 'Merging OurSuite Trunk (All rest)'

... ... branch from //depot/OurSuite/OurSuite/OurProduct/src/Testing/SomeFile.cs#1,#3

//depot/OurSuite/OurSuite/OurProduct/src/Testing/SomeFile.cs

... #3 change 741082 edit on 2017/08/29 by user3@user3_OurProduct (utf8) 'Some other change'

... ... branch into //depot/OurSuite/OurSuite_SomeBranch/OurProduct/src/Testing/SomeFile.cs#1

... #2 change 738357 edit on 2017/08/17 by user3@user3_OurProduct (utf8) 'Some change'

... ... branch into //depot/OurSuite/OurSuite7.1/OurProduct/src/Testing/SomeFile.cs#1

... #1 change 738094 add on 2017/08/16 by user4@user4_OurProduct (utf8) 'Some initial change'


From what I see is that all trees are included in my spec. It is unfortunate that perforce detects that the fetch only includes partly the action, but then does not tell me which path is exactly missing so I could add it or dig further.


Regarding the reducing of the depth I already tried to cover that point in my initial post: As I plan to push to a server with Git Fusion enabled, I would like to have the full history available on this server as it helps the devs to understand why changes might were made in the past.

Edit 1:

When I look at the changes of 750162 I see on one site that there are integrations across many trees. For the SomeFile.cs where it complains the entries look like this:

File Name   | Revision | Action      | FileType | In Folder

SomeFile.cs | 3        | Integrate   | utf8     | //depot/OurSuite/OurSuite_SomeBranch/OurProduct/src/Infrastructure

SomeFile.cs | 2        | Move/Delete | utf8     | //depot/OurSuite/OurSuite_SomeBranch/OurProduct/src/Testing



After some further tests and additional filters, I also found the same error and a similar change that was happening on the trunk.


Change 748318 performs a move/delete on //depot/OurSuite/OurSuite/OurProduct/src/Testing/SomeFile.cs#6, but the parameters of this fetch, push, or zip command include only part of the full action. Specify a wider view to include both the source and target of the change, or specify a narrower view to exclude both the source and target of the change.File Name   | Revision | Action      | FileType | In Folder

SomeFile.cs | 10       | Integrate   | utf8     | //depot/OurSuite/OurSuite/OurProduct/src/Infrastructure

SomeFile.cs | 6        | Move/Delete | utf8     | //depot/OurSuite/OurSuite/OurProduct/src/Testing



There are 2 files with the same name in different folders, and they are unrelated. But maybe it has an impact.

There was a deletion of SomeFile.cs that cannot be fetched.

#4 Sambwise

Sambwise

    Advanced Member

  • Members
  • PipPipPip
  • 665 posts

Posted 05 December 2018 - 04:45 PM

The fact that this changelist is a merge isn't really relevant.  This right here is the problem:

//depot/OurSuite/OurSuite_SomeBranch/OurProduct/src/Testing/SomeFile.cs
... #2 change 750162 move/delete on 2017/10/04 by user1@user1_OurSuiteTmp (utf8) 'Merging using OurSuite_SomeBranch '
... #1 change 744444 branch on 2017/09/12 by user2@user2_OurSuiteTmp (utf8) 'Merging OurSuite Trunk (All rest)'
... ... branch from //depot/OurSuite/OurSuite/OurProduct/src/Testing/SomeFile.cs#1,#3
There should be a line under #1 that says something like:
... ... moved into //depot/OurSuite/OurSuite_SomeBranch/OurProduct/src/Testing/SomeOtherFile.cs


because just like every move/delete must have a corresponding move/add (i.e. a file may not move into nonexistence), there must always be an integration record connecting the move/add to the revision prior to the move/delete (i.e. there must always be an unambiguous record of where the file moved to).

But in this case SomeOtherFile.cs is a path that you don't have access to, which hides it from the filelog output and from the fetch operation.  Check with the admin of this server and see if they can fix that; as it is now they've set up your protections so that you can only see part of the history of certain files, which makes it impossible to fetch the full history of each file as you're trying to do.

Another possibility is that the admin obliterated the path that this file moved to; if that's the case they probably want to obliterate the pre-move path as well (or at least obliterate the move/delete revision so that it's as if the entire move never happened).

Alternatively, you may want to modify your remote spec to exclude these files specifically (if you can't fetch the latest versions, you probably don't want to fetch the earlier versions on their own).

#5 Sambwise

Sambwise

    Advanced Member

  • Members
  • PipPipPip
  • 665 posts

Posted 05 December 2018 - 04:46 PM

(edit) wow, Invision's WYSIWYG editor is painful

#6 danielku15

danielku15

    Newbie

  • Members
  • Pip
  • 4 posts

Posted 06 December 2018 - 07:37 AM

Quote

Another possibility is that the admin obliterated the path that this file moved to
This sounds familiar. I think we asked the admin to obliterate some paths in the past because some things totally got messed up. An online research at that time mentioned to obliterate certain paths/files.

Maybe I really try to exclude this particular file and hope that nothing else pops up. The file only contains some unit tests where history might not be that important. I will update on this post once I have some new info.

Thanks so far.

#7 danielku15

danielku15

    Newbie

  • Members
  • Pip
  • 4 posts

Posted 06 December 2018 - 10:10 AM

It seems that really only this one particular file was causing issues. The fact that we obliterated something in the past was leaving us in an inconsistent state regarding this Changelist. I decided to completely exclude the file from the migration and add it later again manually to the repository.

Thanks a lot for your assistance and from my side this topic is closed.

#8 Sambwise

Sambwise

    Advanced Member

  • Members
  • PipPipPip
  • 665 posts

Posted 06 December 2018 - 08:04 PM

You may also want to pass on to the admin that they should obliterate //depot/OurSuite/OurSuite_SomeBranch/OurProduct/src/Testing/SomeFile.cs to keep other people from having the same problem.  :)





Also tagged with one or more of these keywords: personal server, dvcs, fetch

0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users