Jump to content


Obliterate files with more than X revisions or delete all revisions but head revision


  • Please log in to reply
4 replies to this topic

#1 PandaCeleste

PandaCeleste

    Newbie

  • Members
  • Pip
  • 1 posts

Posted 11 March 2020 - 04:57 PM

Hello,

How would you obliterate all versions of all files in a directory and it's sub-directories with more than 10 revisions for example so that it keeps only the 10 latest revisions.
Or alternatively how can you obliterate all versions that are not the head revision in a directory and all of it's sub directories?

We're trying to clean up HDD space and we have quite a few old projects we only want to keep the most recent data from.
We applied a new typemap but this isn't relevant to all files currently on the depot.

Kind regards

#2 Miles O'Neal

Miles O'Neal

    Advanced Member

  • Members
  • PipPipPip
  • 182 posts

Posted 17 March 2020 - 08:27 PM

I generate a list of files & revs using p4 commands (such as "p4 filelog -a <path>". See https://community.pe...s/article/15075 for suggestions on generating the revision range you want, if you need help there. Then I have a script build a set of clients explicitly listing each <path>#<rev> . I only put a certain number in each client.[1] Then I have a script that runs obliterate using each client in turn. I typically run these during off hours to ensure minimal impact to users. IFF you can *guarantee* no one has these files, run obliterate with "-h" to speed things up.
Never, ever interrupt an obliterate. It can leave the metadata in a corrupt state, and you'll have to rebuild from the last checkpoint. Trust me on this.
Test the snot out of your scripts on a test server. A mistake here can be costly.
I've obliterated many hundreds of thousands of revs of files this way. Seriously. It's probably close to a million. I developed this method through sweat and tears, but thankfully no blood. 8^)

[1] I determine the number empirically, by testing, Pick the length of time you are OK with the server locking up, and use the max number of path/revs that comes in under that.

#3 Sambwise

Sambwise

    Advanced Member

  • Members
  • PipPipPip
  • 1037 posts

Posted 17 March 2020 - 10:48 PM

The quick and easy way to gather the list of revisions without having to write a script is via labelsync:  https://stackoverflo...9033327/3799759

#4 Matt Janulewicz

Matt Janulewicz

    Advanced Member

  • Members
  • PipPipPip
  • 217 posts
  • LocationSan Francisco, CA

Posted 17 March 2020 - 10:51 PM

I'd also reconsider doing this if the sole motivation is disk space. If this is code for legitimate projects, you're essentially destroying someone's past work. There are many reasons your future self wouldn't want to be responsible for doing that, but one that comes to mind for me (that I witnessed but did not initiate) is when your company/project is acquired and the buyer wants to know what happened to all the work that corresponds with all those closed bugs. The value/sale price of said product was greatly reduced due to lack of that history. Another good way to create havoc is if you happen to work in a regulated industry, and the auditors come along (FDA, PCI) and you can't answer their questions about historical code/data. One last bullet point off the top of my head is that you never know when an old project/product will become new again. Or when software out in the field will suddenly need a revision (for us older people, Y2K might still be showing up in our nightmares.) You might say I have an aversion to destroying code, and you should too.

That's not to say there are certainly some legit reasons to obliterate data this way. I'm sure we all have some stories from the trenches:

* Automation that checks in identical iso's hundreds of times over the course of a couple years.
* Someone checking in massive amounts of personally identifiable medical data to a 'public' area (severe HIPPA violation.)
* Someone(s) checking git repos (.git directory) directly into Perforce (WTH? Seriously, people.)

If you think about this for about a week and still want to obliterate, please heed what Miles says up there. A misguided or mis-applied obliterate, and the following recovery efforts, will be one of the worst things you have to deal with in your career (speaking from experience.)
-Matt Janulewicz
Currently unemployed, looking for work in Boise, ID!

#5 Miles O'Neal

Miles O'Neal

    Advanced Member

  • Members
  • PipPipPip
  • 182 posts

Posted 17 March 2020 - 11:34 PM

I concur with Matt's caveats about why.

I will also note that I developed the current scripts out of self-defense. We had met the enemy and he was us.




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users