Jump to content


Saving exact workspace state

client label tag state workspace

  • Please log in to reply
7 replies to this topic

#1 Miles O'Neal

Miles O'Neal

    Advanced Member

  • Members
  • PipPipPip
  • 146 posts

Posted 30 May 2018 - 08:14 PM

We're going to use labels to save state just before we clean up old on-disk workspaces and remove the p4 clients. I plan to use "p4 labelsync -l LABEL @CLIENT". We can't make any assumptions about what's in the on-disk workspace, so the changelist approach won't work.

Questions:
- Do I just leave the view at //... in the label to make sure the label hits streams, etc? Some of our clients are very complex.
- I assume the label will not include the fact that files are added/unsubmitted or opened for edit and I need to save pertinent info separately (we plan to copy those files out and then revert them). I was planning to run "p4 opened" and save the info from that so that we can restore that state later if necessary.

Most of these will eventually be deleted, but we'll give a grace period during which they can restore if they need to.

#2 Sambwise

Sambwise

    Advanced Member

  • Members
  • PipPipPip
  • 957 posts

Posted 30 May 2018 - 08:27 PM

Have you looked at "p4 unload"?

https://www.perforce.../p4_unload.html

That does exactly what it sounds like you're trying to implement, but it's all built in, including the restore process ("p4 reload").  It also has the benefit of writing the data to a flat file rather than a static label, so you're actually de-bloating the db rather than moving bloat from db.have to db.label.

#3 Matt Janulewicz

Matt Janulewicz

    Advanced Member

  • Members
  • PipPipPip
  • 191 posts
  • LocationSan Francisco, CA

Posted 30 May 2018 - 08:28 PM

You may want to consider just unloading the clients. This saves #have and #opened information about the client.

You can't _query_ that info while it's unloaded, as far as I know, but if you later reload it, the 'p4 have' and 'p4 opened' lists will be the same as they were before unloading them.

To be thorough and ensure you don't lose local changes to checked out files, consider shelving the changelist first, then unloading it.

Out of curiosity, what's the end goal here if you plan on deleting the workspaces anyway?
-Matt Janulewicz
Staff SCM Engineer, Perforce Administrator
Dolby Laboratories, Inc.
1275 Market St.
San Francisco, CA 94103, USA
majanu@dolby.com

#4 Matt Janulewicz

Matt Janulewicz

    Advanced Member

  • Members
  • PipPipPip
  • 191 posts
  • LocationSan Francisco, CA

Posted 30 May 2018 - 08:28 PM

View PostSambwise, on 30 May 2018 - 08:27 PM, said:

Have you looked at "p4 unload"?

https://www.perforce.../p4_unload.html

That does exactly what it sounds like you're trying to implement, but it's all built in, including the restore process ("p4 reload").  It also has the benefit of writing the data to a flat file rather than a static label, so you're actually de-bloating the db rather than moving bloat from db.have to db.label.

Jinx!
-Matt Janulewicz
Staff SCM Engineer, Perforce Administrator
Dolby Laboratories, Inc.
1275 Market St.
San Francisco, CA 94103, USA
majanu@dolby.com

#5 Miles O'Neal

Miles O'Neal

    Advanced Member

  • Members
  • PipPipPip
  • 146 posts

Posted 30 May 2018 - 08:47 PM

The goal? Less wasted disk space, less overall metadata pollution, shorter client lists to wade through, quicker backups, etc. This only applies to clients untouched for some time, or otherwise determined to no longer matter. ("I made thirty versions of this to try various tweaks. I only need to keep these three." "These people left." Etc.)

We have lots of old clients and associated workspaces (on-disk representation). Rather than bugging the engineers (who already have plenty to do) to clean up, we are going to "archive" the state, clean up the disk and client list, and send a list of cleaned up data to each engineer. In special cases, we may hold the data long term. In most cases, after some number of months, we'll throw the N-month old "archives" away.

Will the unload save the contents of opened files, or do I still need to do that? It will handle all the stream types in the client view as well, I assume.

Thanks.

#6 Matt Janulewicz

Matt Janulewicz

    Advanced Member

  • Members
  • PipPipPip
  • 191 posts
  • LocationSan Francisco, CA

Posted 30 May 2018 - 08:57 PM

View PostMiles O, on 30 May 2018 - 08:47 PM, said:

The goal? Less wasted disk space, less overall metadata pollution, shorter client lists to wade through, quicker backups, etc. This only applies to clients untouched for some time, or otherwise determined to no longer matter. ("I made thirty versions of this to try various tweaks. I only need to keep these three." "These people left." Etc.)

We have lots of old clients and associated workspaces (on-disk representation). Rather than bugging the engineers (who already have plenty to do) to clean up, we are going to "archive" the state, clean up the disk and client list, and send a list of cleaned up data to each engineer. In special cases, we may hold the data long term. In most cases, after some number of months, we'll throw the N-month old "archives" away.

Will the unload save the contents of opened files, or do I still need to do that? It will handle all the stream types in the client view as well, I assume.

Thanks.

Cool. Understandable and what I suspected but sometimes I like to ask in case someone is doing something weird. Well, I do weird things. Maybe weird things that are easier to do another way. :)

Anyway, I think unload is where it's at for this. It won't save the contents of opened files, thus the need to shelve them. You'd need to physically be on the host that has the files, though, as it needs to upload the content.

If you have a lot of these to do it might be worth getting into the habit of doing 'p4 unload -z' to store data zipped. By default it will store them (as Sam said) in flat files in an unload depot. The dumps/unloads of db.have can be significant in size if you have enough workspaces unloaded. On one of our more popular servers we have 22 GB of compressed unload data.

Which reminds me of one last thing. If you happen to be in a commit-->edge architecture, the unload depot on each edge (and commit) will be unique to that server. You'll want to treat it like a spec depot and back those depots up individually on each server.
-Matt Janulewicz
Staff SCM Engineer, Perforce Administrator
Dolby Laboratories, Inc.
1275 Market St.
San Francisco, CA 94103, USA
majanu@dolby.com

#7 Miles O'Neal

Miles O'Neal

    Advanced Member

  • Members
  • PipPipPip
  • 146 posts

Posted 30 May 2018 - 09:12 PM

Thanks! Shelves and streams are my weakest points with Helix.

We don't yet use edge/commit, although we are planning on it for automated build environments.

Zipping was already on the list, but thanks. And thanks for checking on the rest.

Not sure we'll shelve opened files since we have to save unmanaged files, anyway (results, copies, notes, references, etc.) Yes, those would ideally be somewhere else, but they aren't always.

If you aren't doing anything weird, you probably feel weird in a high tech environment. 8^)

#8 Matt Janulewicz

Matt Janulewicz

    Advanced Member

  • Members
  • PipPipPip
  • 191 posts
  • LocationSan Francisco, CA

Posted 30 May 2018 - 09:23 PM

View PostMiles O, on 30 May 2018 - 09:12 PM, said:

Thanks! Shelves and streams are my weakest points with Helix.

We don't yet use edge/commit, although we are planning on it for automated build environments.

Zipping was already on the list, but thanks. And thanks for checking on the rest.

Not sure we'll shelve opened files since we have to save unmanaged files, anyway (results, copies, notes, references, etc.) Yes, those would ideally be somewhere else, but they aren't always.

If you aren't doing anything weird, you probably feel weird in a high tech environment. 8^)

One other thought as we continue to spitball, is your mentioning of unmanaged files. Since you have to find those anyway, it might be worth doing a 'p4 reconcile' on the workspace. This will mark anything 'off' as add, edit or delete, but will not submit them. Rather than track them separately, you could theoretically do everything in the 'Perforce way' by doing:

> p4 reconcile
> p4 shelve (if reconcile found anything, or files are otherwise checked out)
> p4 unload
-Matt Janulewicz
Staff SCM Engineer, Perforce Administrator
Dolby Laboratories, Inc.
1275 Market St.
San Francisco, CA 94103, USA
majanu@dolby.com





Also tagged with one or more of these keywords: client, label, tag, state, workspace

0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users