Jump to content

Sambwise's Content

There have been 290 items by Sambwise (Search limited from 19-February 19)

By content type

See this member's

Sort by                Order  

#25959 The mem of P4D.exe grow so fast

Posted by Sambwise on Yesterday, 03:29 PM in Administration

If it's using that much memory when nothing's running it sounds like a leak.  :wacko:   You should contact support@perforce.com so they can work with engineering to debug it.

#25957 The mem of P4D.exe grow so fast

Posted by Sambwise on 17 February 2020 - 06:33 PM in Administration

That does seem suspicious.  What does p4 monitor show say is running?  Did you upgrade/change anything else at the same time you installed the new P4D version (i.e. add some new client app that's running lots of expensive queries)?

#25944 Merge reporting nearly identical files as a single massive conflict, refuses...

Posted by Sambwise on 13 February 2020 - 05:44 AM in P4V

Glad to hear you got it working!

View PostJozy Zim, on 13 February 2020 - 12:19 AM, said:

As I poked around, my theory is that p4 resolve was directly comparing the files, while P4Merge was creating a "difference map" using assumptions based on Perforce file history (and the bad conflicting result could make sense, given the unusual paths these files have taken). I was able to get the Resolve window, from additional actions, to give me a diff that looked like a '5 Conflict' solution. But I wasn't able to translate that into any kind of resolution via the GUI. It seemed informational only.

FWIW, this should not be true.  Each resolve is a simple three-way merge; all P4Merge should get as its inputs as far as the file contents go are those three inputs (the base and two legs).  The base and the "theirs" leg are determined on the server at the time that the resolve is scheduled (e.g. during "p4 integrate" or "p4 sync"), and the "yours" leg is always the local workspace file.  Regardless of whether you merge from the command line or from a GUI tool (and regardless of which GUI tool), those three input files should be the same because "p4 resolve" is driving the merge in all cases.

Based on your description it sounds like there's a decent chance that the latest version of P4Merge has some kind of bug pertaining to whitespace handling; that might be worth following up on with Perforce support...

#25942 Merge reporting nearly identical files as a single massive conflict, refuses...

Posted by Sambwise on 12 February 2020 - 11:24 PM in P4V

The difference between the resolve dialog and the merge tool is very interesting and worth investigating.  Here are the two things to investigate:

1. Is there a version discrepancy between the server (P4D), the client (P4V), and the merge tool (P4Merge)?  Each of these has the merge logic compiled into it, and if they're of wildly different vintages, you might get different behavior from them.  I believe the "merge preview" you get from the resolve dialog is calculated inside the client library (i.e. inside P4V) using flags and file data sent by the server, but I'm not positive about that...

2. Are the merge tool and the "p4 resolve" command using the same whitespace settings?  (e.g. if you see "p4 resolve -dw" in P4V's log pane that would correspond with "ignore whitespace differences" in P4Merge)

It sounds like whatever merge calculation "p4 resolve" is doing is the one you want, so it's just a matter of figuring out how to bring P4Merge into alignment with "p4 resolve".  :)

Alternatively, you could drop out to the command line and run "p4 resolve" there; one of the options it'll give you is to do a merge in your text editor.

#25932 Writeable flag for typemap folder not working

Posted by Sambwise on 11 February 2020 - 03:44 PM in General

The typemap is applied to new files at the time you open them with "p4 add"; it won't retroactively change existing ones, and it can be overridden with the "-t" flag.  To change existing files you have two options:

p4 edit -t +w //depot/mystream/UnrealEditor/...
p4 submit -d "Changing type to writable."

This will add a new revision with the +w type; once people sync this down the files will be made writable.  I recommend this approach because it's the most straightforward.

p4 retype -t +w //depot/mystream/UnrealEditor/...

This will retroactively change all the existing revisions to +w.  This requires admin permission, and the change won't be reflected automatically on sync (since as far as the clients are concerned they already have these revisions) until either a new revision is submitted or clients do something like sync to #none and then re-sync.

#25926 Why is it bad to backup db files?

Posted by Sambwise on 10 February 2020 - 05:11 PM in Administration

Checkpointing has some nice benefits over snapshotting the raw db.* files:

1. The fact that the checkpoint has to walk the entire the database gives you some assurance that the files are structurally sound.
2. The creation of the new checkpoint is synchronized with a rotation of the journal file, making it easy to recover transactions that happen in between checkpoints.
3. The checkpoint itself is both smaller and more portable than a db file backup.

None of the above are must-haves; you could verify your db snapshot offline, you could manage the journal rotation yourself (or parse out the timestamps during a recovery), portability doesn't matter unless/until you need to move your server, and disks are relatively cheap.  Checkpointing is all in all the most user-friendly way to make db recovery easy, though, as long as you can live with however much downtime it takes to create a checkpoint.

#25902 Delete files from child stream, but reintegrate them later

Posted by Sambwise on 02 February 2020 - 06:30 AM in Streams

If you use the “p4 undo” command to undo the integration and combine it with the “dm.integ.undo” configurable (this is an undoc setting that negates integration credit for undone integrations, designed with this exact use case in mind) you should be able to do this.

Another option, if you want to remove the files altogether and then restore them later like nothing happened, would be to exclude them from the stream's Paths (add "exclude" lines).  The files will still be in the depot but they'll be removed from workspaces of that stream.  When you remove the exclusions, the files will be mapped back in.

#25901 Having difficulty with virtual streams and Jenkins continuous integration server

Posted by Sambwise on 02 February 2020 - 06:28 AM in Streams

Creating a temp workspace is the only way, unfortunately.  

It’s bugged me for a long time that there’s no such thing as a “stream syntax” to handle this use case, but the fact that stream names correspond to their associated depot paths (instead of being a “domain” like clients/labels/etc) makes that difficult to do even in theory.

#25900 Revert is much slower than sync

Posted by Sambwise on 01 February 2020 - 05:17 PM in P4V

My guess is it doesn’t batch/parallelize the transfer the same way (especially if you’ve got any sort of distributed setup or if you’ve configured any special transfer settings that are particular to the sync command).  If you run the two commands with something like “-vnet=3” you can spy on the network messages and get an idea what the differences are.

#25899 merge problem - it wants to merge two files with completely different paths/n...

Posted by Sambwise on 01 February 2020 - 05:08 PM in General

Go to the command line, run “p4 resolve -n”, and paste the output here?  That’ll help me get an idea of what it is you’re trying to merge.  If the source/target file names are different, it usually means that the file got renamed and the two names are being reconciled automatically as part of the merge (that’ll show up in the output as “resolving filename” and it’ll say what the two names are).

If you want to “accept source” on everything, you could follow it up with “p4 resolve -at” and see if that works any better than what you tried in P4V.

#25856 Syntax for P4IGNORE entry to ignore symbolic link folders

Posted by Sambwise on 20 January 2020 - 03:00 PM in General

Oh, I take it back -- it's apparently a Java limitation.  :(

C:\Perforce\workshop\guest\perforce_software\p4java>p4 grep -e "with this client program" ...
//guest/perforce_software/p4java/r14.1/src/main/java/com/perforce/p4java/impl/mapbased/rpc/func/client/ClientMessage.java#1:											"%type% file %file% can't be sync'd or created with this client program.",


Not sure what the workaround would be.

#25855 Syntax for P4IGNORE entry to ignore symbolic link folders

Posted by Sambwise on 20 January 2020 - 02:57 PM in General

That's not a Perforce error message:

C:\Perforce\workshop>p4 grep -e "or created with this client program" //guest/perforce_software/p4/2018-2/msgs/...


so it must be that (for whatever reason) the Jenkins plugin disallows symlink files.

#25854 Settings for perforce

Posted by Sambwise on 20 January 2020 - 02:54 PM in General

What's "BC"?

#25849 Syntax for P4IGNORE entry to ignore symbolic link folders

Posted by Sambwise on 17 January 2020 - 06:27 PM in General

The symlink's content should just be the link (i.e. the text of the path that the link points to).  By default, the filesystem will redirect file accesses to the path that the link points to (if you're on a platform that supports symlinks!), but if you tell Perforce that the file is a symlink, it should just operate on the link, not the underlying file, when you diff/edit/reconcile/sync/etc.

If you replace the symlink with the actual content of the file, it's not a symlink any more.  It sounds like you don't actually want to use the symlink type here (perhaps you're on a platform that doesn't support symlinks?), and should just reopen the file as text (p4 edit -t text includes.h) so that it can contain the actual content.

If you do want the file to be a symlink, you should revert your change (do "p4 clean includes.h" if you haven't opened the file, or "p4 revert" if you have).

#25847 Syntax for P4IGNORE entry to ignore symbolic link folders

Posted by Sambwise on 17 January 2020 - 03:41 PM in General

Does Perforce know that these are symlinks?  (Is the filetype "symlink" or "text"?)

C:\Perforce\test\types>p4 files ...
//stream/main/types/bar#1 - add change 126 (symlink)
//stream/main/types/foo#1 - add change 126 (text)

What does "p4 diff" at the command line show (just to rule out confusion in P4Merge -- I'm not sure if it's necessarily sensitive to filetypes)?

What if you do "p4 edit -t symlink (filename)" to explicitly open the file for edit as a symlink?

#25843 P4Api client.GetFileMappings not returns leading minus/dash sign for branched...

Posted by Sambwise on 16 January 2020 - 04:12 PM in APIs

View PostLubos Suk, on 16 January 2020 - 08:01 AM, said:

First but not last scenario is to Rename/Move Files.


This method accepts only FileSpec that first is current file and second is new filename/location.

Just pass the client path directly to the move command.

C:\Perforce\test\move>p4 edit foo
//stream/main/move/foo#1 - opened for edit

C:\Perforce\test\move>p4 move foo bar
//stream/main/move/bar#1 - moved from //stream/main/move/foo#1

I can't emphasize enough that in just about every situation Perforce will simply automatically translate client paths to depot paths.  That's the entire point of having a client view; it's not something that's there to make you do annoying extra steps.  :)  Any time you see a command that takes a "filespec", you SHOULD be able to pass either a depot path or a client path to it.

I think in the .NET context that looks something like this?  (I don't use C# at all so my syntax is probably wrong, I'm just looking at the docs)

MoveFiles(FileSpec(ClientPath("foo")), FileSpec(ClientPath("bar")))

i.e. when you build your FileSpec out of a PathSpec, use a ClientPath as the PathSpec instead of a DepotPath.

The same applies to, say, the "p4 add" command, where there is no depot file (so "p4 fstat" will return nothing); the server will use your client view to automatically translate the local path you give it into the appropriate depot path:

C:\Perforce\test\move>p4 add -n ola
//stream/main/move/ola#1 - opened for add

The only situations where you need to use fully qualified depot paths are the ones where there is no client mapping at all (in which case doing the stuff with "p4 where" will get you no result anyway) or where you need to override the client view (e.g. you have some tricky overlay mapping and you want to "p4 add" into an ambiguous non-default path -- your code doesn't handle that case either because it assumes that mappings are non-ambiguous).  Actually parsing client views is something that you should simply never need to do unless you're doing something VERY specialized (like trying to debug client views themselves, or write a UI for building client views, or something like that).

Note that when the server resolves a local path into a depot path, it will do things that your current approach isn't accounting for.  The client view might have changed since the file was synced (in which case the server will use the client sync records, aka the "have table", to figure out where the file came from), or there might be an overlay mapping (in which case there are multiple potential depot file mappings and the server will check all of them and return the first extant match).  It's much better to simply let the server handle this than to try to reverse-engineer it and slowly work your way through all the edge cases that Perforce's server engineers have already handled over the course of more than two decades.  :)

#25835 Cannot delete changelist with 'phantom' shelved files

Posted by Sambwise on 13 January 2020 - 04:18 PM in General

Worst case scenario, you could grep the checkpoint for that changelist number, find the db records that correspond to the shelved files, and apply a journal patch to zap 'em.

I've never heard of this "phantom shelf" scenario before, though.  I'd be interested in seeing the output from:

p4 describe CHANGE
p4 change -d CHANGE
p4 files @=CHANGE
p4 user -d -f USER

#25824 p4 submit should not chown

Posted by Sambwise on 09 January 2020 - 04:05 PM in General

I don't think there's an intentional chown happening; the files get edited by the +k expansion, which is automatically going to change the owner, right?  Perforce doesn't track individual file ownership (since it's not portable across clients) so there's no option to set a file as being owned by a specific user the way you can set "+w" or "+x".

If you have "+w" set I believe Perforce just won't touch the file permissions, so you could chmod the files to be owned by all users and then that'd stick.

The other option would be to have your script that manages these files grab the file ownership before it runs and then chown it back after it's done.

#25808 Locking files across streams

Posted by Sambwise on 07 January 2020 - 04:17 PM in Streams

If you've been hearing that suggestion a lot and your takeaway has been "just don't version those files", the people suggesting this approach haven't explained it correctly.  :)  Files on the mainline that are never branched still have version history, but the difference is that the history is linear (i.e. you don't have branches that can diverge from one another and require merging -- something you'd want to prevent, yeah?).  

In a situation where a single file is branched to lots of other points but you prohibit the branches from getting out of sync (i.e. by locking across branches and propagating changes to all other branches before the lock is released, which is the only way to prevent conflicting changes), you also have a linear history, but with a lot of extra bookkeeping; the branching mechanism adds nothing because for all practical purposes the history is not allowed to branch.  Rather than try to add complex mechanisms to make it easier for a branched file to act like a file that is not branched (each of which will probably generate its own unforeseeable problems that will need increasingly complex solutions), the KISS solution is to not branch the file in the first place.

In environments where branching isn't possible (or desirable) but you still need to have controls on when a version is released, considered stable, etc, a standard version control workflow is to use some sort of "tag" (in Perforce this would traditionally be a label, but I've heard of people also using a client's have list or a counter that references a changelist, and in the modern era of streams we also have the option of versioned import paths) to represent significant versions (e.g. particular releases, or to denote stability or approval).  The difference between a tag and a branch is that a tag references a point on a line, whereas a branch creates a whole new line (again, that's the thing we want to avoid).

For work in progress that needs review, rather than a task stream I'd use a shelf (which is pretty standard for code review in Perforce anyway).  Artist opens the file, locks it, does work, shelves for review.  Lead unshelves the file in their own workspace, reviews/tests, gives approval.  Artist submits, and the new version goes in at the same time the lock is released.

You could also do this with a label-based workflow (in the days before branching, shelving, and continuous integration, people did this by incrementing a "blessed" label once a given version had passed tests), but locking gets a lot more awkward; the artist would need to reopen the file and hold the lock while waiting for the lead to increment the label, releasing it only after they're sure the new version has been approved.

"Merge the latest mainline down" becomes "sync the head revision of the mainline".  Merging is to a branch what syncing is to a workspace.

If a particular stream needs to have an old version of the file and you don't want to deal with using a label to track which version goes with which stream, you can do that with a version specifier on the import path.  Versioned import paths also prohibit submits, which handles the "stale branch" problem that the "recursive lock" solution fails at.

FWIW, if I were implementing this workflow I'd skip the import+ thing entirely; it probably doesn't make sense for artists to be working in a bunch of different streams given that they can't merge their work between them anyway; why should they even have to think about other streams when everything they're doing is in a straight line?  Instead I'd have artists work solely in the mainline stream, using shelves and locks to manage reviews and prevent conflicts.  Cut all the other streams with a versioned import of those files.   Once work is submitted to the mainline, any other stream (release streams, feature streams, whatever) can get it by bumping their version number to match the new version of the art they want to code against -- or if they want to just "float" at the head revision at all times, don't put a version number on the import.

#25806 login fails using P4PASSWD env variable

Posted by Sambwise on 07 January 2020 - 03:49 PM in General

Authenticating via P4PASSWD is less secure than using a login ticket, so your server's security setting might not allow it.  What's the "security" counter set to?

#25805 P4Api client.GetFileMappings not returns leading minus/dash sign for branched...

Posted by Sambwise on 07 January 2020 - 03:44 PM in APIs

If your intent is to figure out which depot path a given client path maps to, why not just pass the client path directly to GetFileMetadata or whatever method you're ultimately trying to pass the depot path to?  Almost every Perforce command that operates on depot files will take a local path as an argument (and it will use the same mapping logic that you're trying to reverse-engineer to figure out what depot path it goes with, while also handling a bunch of edge cases that you haven't considered).

I don't have a .NET dev env set up, but I'm pretty sure if it works for "p4 fstat" it'll work for GetFileMetadata():

C:\Perforce\test>p4 fstat C:\Perforce\test\foo
... depotFile //stream/main/foo
... clientFile c:\Perforce\test\foo
... isMapped
... headAction edit
... headType text
... headTime 1568160303
... headRev 4
... headChange 119
... headModTime 1543246621
... haveRev 4

#25783 user-resolve failed

Posted by Sambwise on 01 January 2020 - 06:05 PM in Administration

If it's missing from the rev table rather than the working table, opening it for edit won't fix it.  I think the error in that case might be due to one of the source files being obliterated out from under you.

The best path forward is probably going to be to revert (use the -k flag if you want to preserve local changes) and then re-open the file for whatever you were doing.  If you were integrating and some revisions of the source (or the base) got obliterated while you were resolving, you'll want to re-run integrate so it can figure out what needs to be merged (if anything) out of whatever's left and what the new best base revision is.  If it was a normal edit and a revision you were resolving against got obliterated, you might not need to do anything at all other than re-open it (although you should double-check your diff to make sure it still makes sense and doesn't include any changes that were supposed to have been obliterated).

#25777 Best way to prevent use of external symlinks

Posted by Sambwise on 30 December 2019 - 08:45 PM in General

You could try to implement a command trigger on "add", but you're in for some really fun times trying to figure out if the file is an external symlink since it's not visible to the server at that point.  :)

#25775 Best way to prevent use of external symlinks

Posted by Sambwise on 30 December 2019 - 07:48 PM in General

Seems like the easiest thing would be a change-content trigger.  Pretty quick to run "p4 opened" and check for symlinks, and you could then inspect the contents of the symlink via "p4 print" if you want to allow relative paths but disallow absolute paths.

#25745 What's the suggested P4API for developing custom tool on MacOS

Posted by Sambwise on 17 December 2019 - 04:06 PM in APIs

It depends on the tool.  If it's running a simple p4 command, you don't need any API beyond the CLI tool.  If it's something more complex but nothing graphical, I'd write a Python script using P4Python.  If it's a graphical tool where I needed to use Cocoa, I'd write it in Obj C (ugh) and use the C++ API.