Jump to content


p4v sync -f failing with unable to create a file that has been removed from repository

p4v

  • Please log in to reply
10 replies to this topic

#1 pjmat1

pjmat1

    Newbie

  • Members
  • Pip
  • 3 posts

Posted 21 April 2015 - 03:43 AM

We are seeing an issue where when a p4 sync -f is triggered from our Thoughtworks GO server ( http://www.go.cd/ ) we get a STDERR output.

STDERR: Filename 'D:\GoAgent\pipelines\XXXXXXXXXX-CI-WebRole\XXXXXXXXX.CreateApiEntity\Library.Cmdlets\templates\Adapters\Version1\XXXXXXXXX.IntegrationTests\XXXXXXX.IntegrationTests.csproj\{{api_entity}}\{{api_entity}}RepositoryTests.ReadWrite.cst' is length 264 which exceeds the internal length limit of 260.
Failed to run : --- Command ---
p4 sync -f //cruise-XXXXXX-WebRole-oyKkcDnuKKOVZXs5TkMlxqZatnQ/...@172002
--- Environment ---
{P4PORT=XXXXXXX.net:1666, P4CLIENT=cruise-XXXXX-WebRole-oyKkcDnuKKOVZXs5TkMlxqZatnQ, P4USER=XXXX}
--- INPUT ----

The file it is trying to write out no longer exists in the repository and has not done so for many months.

Any advice would be appreciated as it is occurring on new build agents but does happen on all agents.

Thanks

#2 P4Sam

P4Sam

    Advanced Member

  • Members
  • PipPipPip
  • 484 posts
  • LocationSan Francisco, CA

Posted 21 April 2015 - 04:55 AM

The error indicates that you're hitting a file path limit on the client side.  If the file doesn't exist in the repository then a sync wouldn't be trying to write it out to the workspace -- is the changelist in question (@172002) perhaps an older changelist from before that file was deleted?

The fact that you're only 4 characters over the limit means you might be able to squeeze under it with a shorter client root (e.g. if the client's Root is "D:\GoAgent\pipelines" maybe you could shorten it to "D:\GAP" or something similarly brief).

#3 pjmat1

pjmat1

    Newbie

  • Members
  • Pip
  • 3 posts

Posted 21 April 2015 - 04:59 AM

Hi There,

We have found that turning off the -f in the sync means everything is OK again (FYI: In this GO Server this is "Clean Directory"). The changelist is the most recent changes but there must be something in the -f that is looking through old data maybe. Is there a way we can work out why p4v thinks it needs to get that file given it is not in the repository (anymore)?

Changing the directory would be a more drastic approach (we have many build agents that would all need to be changed).

Appreciate your help on this.

#4 Robert Cowham

Robert Cowham

    Advanced Member

  • PCP
  • 269 posts
  • LocationLondon, UK

Posted 21 April 2015 - 08:20 AM

It's a little strange that the sync is giving different results with -f and without, but there are several potential explanations.

I would run something like:

p4 files //cruise-XXXXXX-WebRole-oyKkcDnuKKOVZXs5TkMlxqZatnQ/...@172002

And check the output for that particular file - e.g. you could use findstr (or grep) for RepositoryTests.ReadWrite.cst

Then run a "p4 filelog" on that file.

There are other options depending on the output of the above (e.g. p4 have/where but they require extra flags to impersonate the client workspace).

As Sam says, simple answer is to shorten the root dir.
Co-Author of "Learning Perforce SCM", PACKT Publishing, 25 September 2013, ISBN 9781849687645

"It's wonderful to see a new book about Perforce, especially one written by Robert Cowham and Neal Firth. No one can teach Perforce better than these seasoned subject matter experts"
  • Laura Wingerd, author of Practical Perforce, former VP of Product Technology at Perforce

#5 P4Sam

P4Sam

    Advanced Member

  • Members
  • PipPipPip
  • 484 posts
  • LocationSan Francisco, CA

Posted 21 April 2015 - 02:22 PM

Ah -- it's a deleted file, and the "-f" means the server is telling the client to try to delete the file even though it's not there (just like it'll try to add a file that already is there, update one that's already up to date, etc, "-f" means it'll always try to delete a deleted file).  The error should be benign in that case, since failing to delete a nonexistent file doesn't have any actual effect.

If you want to avoid the error, an alternative to "p4 sync -f" might be doing a "p4 clean" (just to make sure nothing is amiss) followed by a normal sync to update any files that need it.

#6 Patrick B

Patrick B

    Member

  • Members
  • PipPip
  • 29 posts

Posted 21 April 2015 - 02:24 PM

If -f is specified then Perforce will forcefully try to remove deleted files as well (to ensure they're really not there - which makes sense).
The problem is the max path error is broken in this case.  For *that* given path, it wouldn't have been able to create the file in the workspace in the first place so to error out saying it can't remove a file that it couldn't have put there is bogus.
This has caused us a lot of heartache lately with build servers that use -f for syncing.

#7 Robert Cowham

Robert Cowham

    Advanced Member

  • PCP
  • 269 posts
  • LocationLondon, UK

Posted 21 April 2015 - 02:50 PM

Have a look at the long filename option:

http://answers.perfo...ticles/KB/8794/

Note limitations.
Co-Author of "Learning Perforce SCM", PACKT Publishing, 25 September 2013, ISBN 9781849687645

"It's wonderful to see a new book about Perforce, especially one written by Robert Cowham and Neal Firth. No one can teach Perforce better than these seasoned subject matter experts"
  • Laura Wingerd, author of Practical Perforce, former VP of Product Technology at Perforce

#8 Patrick B

Patrick B

    Member

  • Members
  • PipPip
  • 29 posts

Posted 21 April 2015 - 03:00 PM

Limitations is right... it requires a server side setting (no big deal) AND an option specified by every single client/api everywhere for every command.  It's basically a non-starter.  Clearly if sync can't 'create' a file that is too long given the constraints, fine, but to treat as an error the 'attempt' to delete a file that isn't even there is pretty broken.  Windows long filename support has been there for a very long time (at the api/file system level). It really shouldn't have to be specified explicitly in my mind.  It should just work.  Perforce should handle it.  If other programs don't like the long filenames, that's not a problem Perforce should worry about.

#9 Robert Cowham

Robert Cowham

    Advanced Member

  • PCP
  • 269 posts
  • LocationLondon, UK

Posted 21 April 2015 - 04:06 PM

The server setting is if your p4d server is running on Windows - so unnecessary if your server is any shade of *nix. The P4CONFIG file is an option - for setting your client programs - and in this instance allowing your builds to proceed. Presumably you have a fairly standard build server config so that might help?

Given that it is sync -f causing the problems - an alternative is to avoid that call in your build server. E.g. "sync -k #0" followed by removing local files, and then just a "sync".

One enhancement might be to globally set the option for all Windows clients. I will certainly explore raising this.
Co-Author of "Learning Perforce SCM", PACKT Publishing, 25 September 2013, ISBN 9781849687645

"It's wonderful to see a new book about Perforce, especially one written by Robert Cowham and Neal Firth. No one can teach Perforce better than these seasoned subject matter experts"
  • Laura Wingerd, author of Practical Perforce, former VP of Product Technology at Perforce

#10 Patrick B

Patrick B

    Member

  • Members
  • PipPip
  • 29 posts

Posted 21 April 2015 - 06:14 PM

Exactly, it requires a change on every single client call everywhere or in every config file anywhere on all machines and all paths.  Windows supports long filenames.  Why this would be a config option you would have to set as opposed go just a 'I'm on at least version X of windows - we support it..'

The syncing behavior isn't something I control.  The CI server forces a sync -f in many cases.  Perforce failing because it can't delete a file it refuses to even allow itself to delete, and thus would have refused itself to even sync is... 'frustrating'.  That the file doesn't even exist locally in the first place is just a second slap in the face. :)

#11 pjmat1

pjmat1

    Newbie

  • Members
  • Pip
  • 3 posts

Posted 23 April 2015 - 01:28 AM

Thanks everyone for your feedback. Good to know it is not just us!

For the moment we are going to try deliberately excluding the folder containing the now non-existent long path from the view.. a bit clunky but should work.

Thanks
Paul





0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users