Jump to content


Determine if files on server should have been deleted by obliterate


  • Please log in to reply
2 replies to this topic

#1 Br.Bill

Br.Bill

    Member

  • Members
  • PipPip
  • 21 posts

Posted 11 January 2019 - 01:29 AM

Is there any way to determine a list of all the files in the server depots that should have been deleted by obliterate, but weren't?

We have good reason to believe that we have files on the system that are "orphaned" -- having no actual references in the p4db at all.

Looking for a way to report a list of these orphans. That way we could know what files could be safely deleted from the depot.

#2 Sambwise

Sambwise

    Advanced Member

  • Members
  • PipPipPip
  • 800 posts

Posted 11 January 2019 - 03:54 AM

The only way I know of to do this is to run "p4 fstat -Oc" across everything and then cross-reference that with the depot filesystem.  If a depot archive revision isn't referenced by any depot file (as displayed by fstat -Oc) then it should be safe to delete.

Most "orphaned" revisions get that way (I think) via failed submits -- the submit starts to write the new revision, something fails, the user either abandons the change or renumbers it, and that partial submit is never committed or cleaned up.  Obliterate never cleans these up because they aren't associated with any of the submitted revisions that it's purging.

#3 Matt Janulewicz

Matt Janulewicz

    Advanced Member

  • Members
  • PipPipPip
  • 164 posts
  • LocationSan Francisco, CA

Posted 07 March 2019 - 11:56 PM

Another tactic that I have used in the past because our servers had gotten so messy, and we have a fairly medium-sized network of edge servers and spares ...

Create a read-only replica of your server and populate the entire thing via 'p4 verify -t'. The '-t' does an archive file transfer if the file doesn't yet exist, so you can build out a fresh, clean, 100% concise server this way.

You'd have to take some precautions as there are certain other types of files you have to look out for. For us, that's just unload (p4 verify -qUt) and shelves (p4 verify -qSt @=<change#>) There may be others but we don't use them, yet (archive depots, graph depots, etc.)

After you think you've transferred everything, do an entire 'p4 verify -qz' on it to be sure. From there, you have two options:

1) Generate a list of all the files on both hosts, compare them and remove anything extra on the first host.
2) Swap in your new, clean replica for the old master (and reverify the whole thing again.)

About a year ago I went through the long process of building out a clean replica like this then we swapped it in to the commit server role. I'm pedantic like that sometimes. :)

Protip: Don't do 'p4 verify -qt //...' unless you have a fairly small server. It's easy to clog up the pull threads. For us, the ideal max in the queue is about 100,000 files.
-Matt Janulewicz
Staff SCM Engineer, Perforce Administrator
Dolby Laboratories, Inc.
1275 Market St.
San Francisco, CA 94103, USA
majanu@dolby.com




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users