Jump to content


Sambwise

Member Since 12 Sep 2016
Offline Last Active Today, 04:57 PM
*****

Posts I've Made

In Topic: GetDepotDirs() - client map too twisted for directory list

09 December 2019 - 10:09 PM

Gotcha -- so yeah, "sync -n" sounds like the way to go.  I'm imagining you'd do "p4 switch --no-sync", then "p4 sync -n" to create a list of files that you'll create placeholders for and hydrate with a real "p4 sync" on demand.

If you have local disk space to spare, another fix I might suggest is to have multiple workspaces so you don't need to throw away and re-download a bunch of files every time you switch streams (i.e. just have a workspace per stream and switch workspaces rather than switching streams within a workspace).

In Topic: GetDepotDirs() - client map too twisted for directory list

09 December 2019 - 03:06 PM

View Postcodersblock, on 09 December 2019 - 09:21 AM, said:

a lot of the files synced I don't even use.

I assume you already know about "p4 switch --no-sync".  Have you looked at creating virtual streams that only sync the files you use?  (Create a child stream with type "virtual" and edit the Paths to only include the files you need.)

In Topic: Using git as an emergency backup?

09 December 2019 - 05:20 AM

View Postbenchang, on 08 December 2019 - 06:57 PM, said:

I have P4 admin privileges but not filesystem-level access so I don't think there's much I can do on my own.

As an admin you can reclaim disk space by obliterating files, but this will only be effective if you have large files that are unnecessary (e.g. if somebody's been submitting giant binary files to the Perforce server).  If:

1) this disk is dedicated to the Perforce server
2) it filled up very suddenly
3) you don't have any limits in place to prevent users from submitting giant files

it's very possible that one student has (unwittingly?) denial-of-serviced the Perforce server by submitting enough data to it to fill up the disk.  You could do a quick check of the student directories (I assume you've got permissions set up so that each student is at least confined to a particular part of the depot) to see if any of them are abnormally large.  Here's an example of a query like that run against the //guest depot on public.perforce.com:1666:

% p4 -F %dirName%/... dirs "//guest/*" | p4 -x - sizes -s
...
//guest/yael_stern/... 53 files 574669 bytes
//guest/yariv_sheizaf/... 118 files 400574 bytes
//guest/ydatoor/... 15 files 523797 bytes
//guest/yonas_jongkind/... 155 files 3769823 bytes
//guest/zach_helke/... 1 files 3133 bytes
//guest/zachwhaley/... 42 files 54005 bytes
//guest/zardlove/... 1 files 16 bytes
//guest/zynthar/... 405 files 159849886 bytes


If one user seems to be taking up all the space, you can go into their directory and obliterate whatever seems superfluous.

If the disk isn't dedicated to the Perforce server, or if it's been creeping up for a while and nobody noticed until it hit the failure point, those are both good things to raise to the IT department as fixable problems.  :)

Quote

If they do this, would it work to use their existing P4 workspace directory also as their local git repository, so that when the helix server is back online they can sync the changes?  Or would it be safer to have them just use a clean directory for their git repository?

Using the workspace directly is fine, but they should make sure to exclude the .git directory from their workspace (or add it to their P4IGNORE file) since they probably don't want to add git's repo metadata to the Perforce depot.

Note also that students should be able to simply clone personal Perforce servers on their own machines.  That way when the central server comes online they'll be able to push their local history to it directly (whereas if it's in git it's probably not going to be easy to convert back into Perforce).

https://www.perforce...f/p4_clone.html

If the clone command still fails with the disk space error, a temporary workaround would be to bump down the filesys.P4LOG.min configurable (obviously you only have 250M worth of wiggle room, so this is not a long term solution; you might want to disable write access until the root cause is resolved).

In Topic: GetDepotDirs() - client map too twisted for directory list

04 December 2019 - 11:36 PM

Running fstat once on 40k files is going to be much faster than running where 40k times on one file at a time, yes, because network I/O is the main bottleneck for simple db queries like these.  :)

If you ran "p4 where" once to fetch the entire mapping and then did the translation of the "p4 files" output on the client side (which you can do via the MapApi class in the C++ API; I'm not sure if there's a .NET equivalent to that one) then that might end up being a little faster.  But definitely running "p4 where" thousands of times is going to be a lot slower than running one or two commands that do the whole thing in one shot.

Either "sync -n" or "have" will be a little faster than fstat because those both return less data (fstat is kind of a one stop shop for all kinds of random file metadata).  Note that in your original post:

Quote

the directories/files which are mapped to a given workspace, i.e. the directories and files which would be synced if I get latest

you're asking for two different things -- the files which are mapped to a given workspace is best represented by "files"/"fstat", whereas the files which will be synced if you get latest is best represented by "sync -n" (running a command like sync with the "-n" flag is precisely "show me what would happen if I did this").  More information about what your app is going to do with the information or what it's supposed to represent would make it easier to steer you in the right direction.  :)

In Topic: GetDepotDirs() - client map too twisted for directory list

04 December 2019 - 11:14 PM

View Postcodersblock, on 04 December 2019 - 05:01 PM, said:

they come back as depot paths which I then need to convert to workspace paths

In that case use "fstat", e.g.:

p4 -F %clientFile% fstat //my_workspace/...

Not sure offhand what the .NET API version of this is, but I'm sure there's a direct equivalent that you can pull all the fstat fields from.

You might also be interested in "p4 have" (which gives you the depot and client paths of everything you've previously synced) and "p4 sync -n" (which gives you the depot and client paths of everything that you need to sync).