Jump to content


how to make labelsync label deleted files

label integrate delete

  • Please log in to reply
12 replies to this topic

#1 amolode

amolode

    Advanced Member

  • Members
  • PipPipPip
  • 41 posts

Posted 16 July 2013 - 01:44 PM

Hi,
We use two-branch policy for regular development in our project. All the developers submit their work into main-dev branch. An automatic integration process runs periodically on the client mapped to this branch. If the build and primary tests pass successfully the revisions in this build client are labeled with labelsync (meanwhile main-dev branch may contain more advanced revisions) and the integrate command transfers the labelled revisions into an official main-int branch.
The problem is that p4 labelsync command does not reflect deleted files and following integrate does not delete the files in main-dev branch. Is there any way to label the necessary snapshot without knowing the latest changelist when the build started? We tried regular p4 labelsync -l SUCCESS_BUILD and p4 labelsync -l SUCCESS_BUILD //depot/main-dev/...#have and p4 tag -l
SUCCESS_BUILD //depot/main-dev/...#have  but wthout avail.
Apaprently that if to write down the latest changelist in main-dev branch when the build starts and later use it for revision specification it does work. But maybe we miss something with labelsync and our method could work out?

#2 P4Matt

P4Matt

    Advanced Member

  • Members
  • PipPipPip
  • 1383 posts

Posted 16 July 2013 - 04:19 PM

Doesn't your build system have the changelist number it built at? I'm probably overly simplifying, but it seems like your label represents a known point in time. It should be able to use that changelist to do the integrate, or at least to fill the label. I know with our Jenkins builds we get the Perforce changelist as a environment variable that we can use in our build and post-build steps.

#3 amolode

amolode

    Advanced Member

  • Members
  • PipPipPip
  • 41 posts

Posted 17 July 2013 - 09:54 AM

So to store the changelist number is imperative <_< ... Currently we make Get Latest Revision from main-dev branch, build and test and then labelsync the snapshot in the builder client workspace. This labeled project version we integrate to main-int branch. There has not been need to store changelist numbers :) . But we will if there is no any other alternative to integrate files deletion (other chages are happily integrated).

#4 P4Matt

P4Matt

    Advanced Member

  • Members
  • PipPipPip
  • 1383 posts

Posted 17 July 2013 - 05:53 PM

No, we can work with this, I was just trying to understand your setup; there's always a chance there is an easier solution. =)

If you specify a file path to labelsync it will include deleted files. In my test server, with a client of "matt-test" and label of "test_label" I ran:

p4 tag -l test_labe //matt-test/...

A quick "p4 files @test_label | grep delete" after showed numerous deleted files.

#5 amolode

amolode

    Advanced Member

  • Members
  • PipPipPip
  • 41 posts

Posted 21 July 2013 - 06:45 AM

Unfortunately this kind of command does not suit because it labels the #head revision at the depot, whereas we need to label the #have versions which have been compiled and tested. The same command p4 tag -l tets_label //depot/...#have  somehow does not labels deleted files (at least at my try). To issue the suggested command with a temporary label before we are starting to build and after the successful testing apply a permanent one is the same bother as to remember the latest changelist.

#6 Patrick B

Patrick B

    Member

  • Members
  • PipPip
  • 29 posts

Posted 21 July 2013 - 02:59 PM

What about tag ...#delete ...#have ?

#7 amolode

amolode

    Advanced Member

  • Members
  • PipPipPip
  • 41 posts

Posted 22 July 2013 - 02:31 PM

View PostPatrick B, on 21 July 2013 - 02:59 PM, said:

What about tag ...#delete ...#have ?

With #have it does not work properly. How can I use #delete ?

#8 Michael Van Hoff

Michael Van Hoff

    Member

  • Members
  • PipPip
  • 14 posts

Posted 22 July 2013 - 02:43 PM

I did some testing, and I'm not seeing your issue...

I created two files, test1.txt and test2.txt.  I added and submitted them in CL#1234.
add //depot/path/test1.txt#1
add //depot/path/test2.txt#1

I then deleted test2.txt in CL#1235.
delete //depot/path/test2.txt#2

I then created my label, 'p4 label DELETE_TEST' and set the view to look at where these files were in my depot, '//depot/path/...'.

I then sync'd the label, 'p4 labelsync -l DELETE_TEST', but it said it only added test1.txt.
//depot/path/test1.txt - added

But as you'll see below, it was still aware that test2.txt was to be deleted...

I then sync'd the directory to CL#1234, 'p4 sync ...@1234' to get both files back and to simulate the state of the client/workspace before the label with the delete would be applied.
//depot/path/test1.txt#1 - added as C:\workspace\depot\path\test1.txt
//depot/path/test2.txt#1 - added as C:\workspace\depot\path\test2.txt

I then sync'd to the label, 'p4 sync ...@DELETE_TEST' and as expected, test2.txt was deleted:
//depot/path/test2.txt#1 - deleted as C:\workspace\depot\path\test2.txt

I am running server 2012.2 and client 2013.1.

Are you doing something that makes the havelist unaware of the files that aren't being deleted?  If the havelist doesn't think they are depot files, syncing to a label won't delete them as it doesn't think they are files it has control over.  Files that are NOT deleted and are not in the havelist, will be overwritten by the sync as it needs a place to write the files, that is if you set your client/workspace to do so. This means that if you are syncing to a filesystem directory that has files, but the client/workspace's havelist is unaware of thier state, it will overwrite existing depot files (files marked add and edit) and will leave non-existent depot files (files marked delete) alone.

If you're doing something like swapping client specifications over the same filesystem directory, you might see this behavior as the status of the files in the filesystem is stored in the client specification, not in the files themselves.  Also some of the offline tools might create a difference between the filesystem and the workspace's havelist.  Streams can put a different spin on this, but it doesn't sound like you're using Streams.

If what I just described is the case, and depending on how your client/workspace is getting out of sync with the filesystem, there are ways to recover without too much expense.  Post your case and I'm sure we can help you figure it out.

Also, does your label specification (created with p4 label) cover all the files, including the deleted files?  My specification's label view was //depot/path/... to cover the entire directory.  If your label specification's view was only something like //depot/path/test1.txt, then it would miss test2.txt, and the delete would not be labeled when you run labelsync later or applied when you go to sync to the label.

I'm wondering if you are modifying the label specification in your automation by looking at the files in the filesystem and using the results to create the label specification's view.  In my scenario above, when test2.txt is deleted, and if I had used automation to update my label specification's view to specific files, I would have had //depot/path/test1.txt and //depot/path/test2.txt in my view, but modified it to //depot/path/test1.txt only as test2.txt had been deleted.  Unfortunately, when you go to sync, since //depot/path/test2.txt is no longer in the label specification's view, the sync process doesn't know to even deal with it, so it will leave the file we want to be deleted at its last sync'd state.

If this is the case, I'd suggest leaving your label specification's view as wide as possible using paths with '/...' or '/*', and then use labelsync and tag to add AND exempt files you DON'T want the label to know about.  Let the sync process handle actually removing deleted files, not modifications to the label specification.  A Label specification is a control structure.  By removing paths and files from it, you don't remove the paths and files, you remove the control over them so they'll no longer update when that label is specified.

I'm just taking some wild guesses here, so post with some more details about the Perforce side of your process if this doesn't help.  Details about your client views and the commands you use in your automation would be helpful.

Hope this helps!

#9 amolode

amolode

    Advanced Member

  • Members
  • PipPipPip
  • 41 posts

Posted 23 July 2013 - 09:27 AM

Our problem is not syncing but integrating. When I labelsync the client workspace in which deletions occurred and try to integrate from this label to another branch, the deletions are not executed in the target branch. No changes to any specs are made during the process. Can you just perform the following test and let me know if you succeeded. Create a branch test-dev with a few files and a workspace with mapping to the branch files. Integrate it (head) to the new branch test-int. Delete some file(s) in the original workspace (p4 delete and p4 commit) for the test-dev branch, create TEST_LABEL label spec and make p4 labelsync -l TEST_LABEL in the source workspace. Again integrate test-dev to test-int, this time from TEST_LABEL. Please let me know if the deletion was performed in test-int branch as a result of the integration. In my case, no integration was performed at all, as Perforce considered that everything has been already integrated and no files were deleted in the test-int branch.  And this is a problem with our process, since it takes time from the build's start till the successful tests and we do not want to retain the latest changelist number included in the build.
Our problem is not syncing but integrating. When I labelsync the client workspace in which deletions occurred and try to integrate from this label to another branch, the deletions are not executed in the target branch. No changes to any specs are made during the process. Can you just perform the following test and let me know if you succeeded. Create a branch test-dev with a few files and a workspace with mapping to the branch files. Integrate it (head) to the new branch test-int. Delete some file(s) in the original workspace (p4 delete and p4 commit) for the test-dev branch, create TEST_LABEL label spec and make p4 labelsync -l TEST_LABEL in the source workspace. Again integrate test-dev to test-int, this time from TEST_LABEL. Please let me know if the deletion was performed in test-int branch as a result of the integration. In my case, no integration was performed at all, as Perforce considered that everything has been already integrated and no files were deleted in the test-int branch.  And this is a problem with our process, since it takes time from the build's start till the successful tests and we do not want to retain the latest changelist number included in the build.

#10 Michael Van Hoff

Michael Van Hoff

    Member

  • Members
  • PipPip
  • 14 posts

Posted 23 July 2013 - 02:47 PM

OK, now I DO see your issue.  I apologize for not reading deeper the first time.  I played with a number of the switches with integrate, most notably the –D switches, but to no avail.
What did work was using 'p4 copy', which is essentially "integrate with a hammer"! It does whatever it takes to make the destination look identical to the source.
p4 copy //depot/path/test-dev/…@TEST_LABEL //depot/path/test-int/…
That command DID do the deletion of the file, but read up on it.  It really does take a hammer to the target branch and makes it look just like the source by branching as it can, replacing as it needs to, and deleting if it doesn't think the file should be there.  If there are changes in test-int, you'll want to integrate them into test-dev before running 'p4 copy' or it'll overwrite them without asking permission…
If 'p4 copy' is too dangerous for you implementation, the only work around I can think of is to create second label, LAST_SUCCESS_BUILD.  Use that label to capture the last time you integrated SUCCESS_BUILD to main-int.  To use this, first, in main-dev, apply SUCCESS_BUILD to the new files to capture what just gave you the successful build.  Next, sync main-dev to LAST_SUCCESS_BUILD.  Then sync back to SUCCESS_BUILD , but this time, capture the output and parse for deleted files, capturing both their depot path and revision.  Then one-by-one integrate that path and revision to main-int.  Finally, labelsync LAST_SUCCESS_BUILD to main-dev (since we just integrated to SUCCESS_BUILD) to be prepared for the next build.
But easier yet, what I use with great success, and what has previously been suggested, is to forget labels and capture the changelist.  I use CruiseControl.Net and I wrote my own labeler to use the changelist as the build label, which gets handed to my script as a variable.  From one of your previous replies it sounds like you are already grabbing the CL in your Get Latest Revision, just not tracking it.  Changelist is the most resilient method of tracking changes, so I suggest using them to associate to your builds.
I took a look at the Jenkins Perforce plugin and it makes a variable, $P4_CHANGELIST, available to the scripting.  See "Advanced Configuration" heading's "Sync multiple builds to the same changeset" subheading on https://wiki.jenkins...Perforce Plugin for more information.  There are also some tutorials out there on how to write your own Jenkins plugins if you can't find one that fits the bill.
Sorry I couldn't be more help, it sounds like an "unintended feature" when 'p4 sync' respects the deleted files, but 'p4 integrate' and 'p4 changes' don't see them.  I even tried the older integration engines (-1 and -2, see 'p4 help undoc' for more info), but to no success.
Good luck!  And remember to have fun with it, whatever you choose to do!

#11 P4Matt

P4Matt

    Advanced Member

  • Members
  • PipPipPip
  • 1383 posts

Posted 23 July 2013 - 06:09 PM

Try this:

p4 labelsync -l test_label //<client name/...#have
p4 labelsync -a -l test_label //<client name>/...#delete


The first command will put the files in your have list in the label. The second will add all deleted files in your client view. The '-a' on label sync tells Perforce to only add files to the list, which prevents your have list files from being removed.

#12 amolode

amolode

    Advanced Member

  • Members
  • PipPipPip
  • 41 posts

Posted 24 July 2013 - 08:56 AM

Thank to both of you for your answers and suggestions! Apparently there is no escape from retaining the changelist number. The command
p4 labelsync -a -l test_label //<client name>/...#delete
does not suit. It labels ALL the recently deleted files, even those deleted in other client workspaces later than #have revision. That is if the last sync to our workspace was at 11:00 AM and the build and tests finished at 19:00 , p4 labelsync -l test_label //<client name/...#have   would label the right versions, but
p4 labelsync -a -l test_label //<client name>/...#delete  would label also all the files that were deleted between 11:00 AM and 19:00 in other workspaces and this is UNDESIRABLE behaviour.

#13 P4Matt

P4Matt

    Advanced Member

  • Members
  • PipPipPip
  • 1383 posts

Posted 24 July 2013 - 04:28 PM

Curses. My bad. One last try...

p4 changes -m 1 //<client name>/...#have

This will get you the changelist you are synced to.

p4 labelsync //<client name>/...#<changelist number I just generated>

With this approach you won't have to retain the changelist number, you can just generate it from your workspace. It's not flawless though; if the latest changelist was only deleted files, Perforce will report the previous changelist because you technically don't "have" the deleted files. The way around that is you can instead run

p4 cstat //<client name>/...

For every change you have potentially mapped into your workspace it will let you know if you have it or not. Just take the last change number with status "have" and you can be guaranteed that you can label your deleted files as you expect.

So no matter what you'll need a changelist number, but potentially you can generate it when you need it instead of tracking it.





Also tagged with one or more of these keywords: label, integrate, delete

0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users