P4 Copy "Identical"?
Posted 29 January 2019 - 11:59 PM
"Target files that are identical to the source are not affected by the p4 copy command unless you use the -f option."
"-f Force the creation of extra revisions in order to explicitly record that files have been copied. Deleted source files are copied if they do not exist in the target, and files that are already identical are copied if they are not connected by existing integration records."
My question is what is Perforce's definition of "identical"? From what I see in practice, it seems to be file content only, not file meta data (i.e. modtime).
If my source is revision 1 and target is revision 3 such that the target has been changed and then changed back such that the content is the same as source I am not getting a revision record for the p4 copy even though the meta data is different (different modtimes). I know there is a -f option to force the record, and then meta data IS correct because it was forced... but this forces a record for everything which is a waste especially when you have 100k files like I do.
This seems like a bug to me and the -f a kludge catch all. Should not the term "Identical" mean content + meta data? Should it not create a revision record if either changes? Or am I really off base?
This is effecting my build system as it uses the modtimes of the files, if I do not -f force everything I get wrong modtime induced errors after a p4 copy.
Posted 05 February 2019 - 12:37 AM
Specific metadata fields are compared -- IIRC it's just the md5 hash and filetype. (cough DmrRev::Identical() cough cough) You wouldn't want to compare *all* of them since some will necessarily be different (e.g. file path, submitting user, revision action associated changelist...). Client modtime is really only valid to compare if the files have the +m type, since otherwise the modtime generally reflects the time the file was synced to the workspace rather than the time the file was actually touched by the user, and would create a lot of spurious extra revisions being created by copy -- that said, there's probably a good enhancement request there for comparing it in that specific +m case (though I dunno if it applies to your situation).
I think you actually do want the -f flag in this case. With the -f, copy still excludes files that have integration credit, so it shouldn't actually copy all 100k files (i.e. it will exclude those were no changes have been made since the last copy or merge operation), but it will catch those that have "revved" in the source with no net change in the contents (which is the exact thing that you want).
Posted 05 February 2019 - 03:01 AM
I see. md5 content hash + filetype = "identical".
Preserve original modification time
The file’s timestamp on the local file system is preserved upon submission and restored upon sync. Useful for third-party DLLs in Windows environments, because the operating system relies on the file’s timestamp. By default, the modification time is set to the time you synced the file.
So from the docs, I see the +m is the per file basis modtime preservation flag. I do not use that. I have the workspace option "modtime" checked on all workspaces. What I see is the modtime is preserved on submit and other workspaces on get see updated modtimes.
For files without the +m (modtime) file type modifier:
If modtime is set, the modification date (on the local filesystem) of a newly synced file is the datestamp on the file when the file was last modified.
If nomodtime is set, the modification date is the date and time of sync, regardless of version.
Files with the modtime (+m) type are primarily intended for use by developers who need to preserve original timestamps on files. The use of +m in a file type overrides the workspace's modtime or nomodtime setting. For a more complete discussion of the +m modifier, see “File Types”.
nomodtime (date and time of sync) for most files.
Ignored for files with the +m file type modifier.
I bolded the sections I think most relevant:
The first one seems to confirm what I am seeing with modtime preservation when only using "modtime" on the workspace instead of per file +m as expected.
The second is a bit more nuanced I think. It suggests the the +m flag will override the "modtime" workspace flag, but not exactly how. It seems to me that this is a one way override that +m will override only if checked and workspace "modtime" is not check, and will not override to no modtime if +m is unchecked. Correct?
If not then maybe I need both +m on all file and "modtime" checked on all workspaces?
"I think you actually do want the -f flag in this case. With the -f, copy still excludes files that have integration credit, so it shouldn't actually copy all 100k files (i.e. it will exclude those were no changes have been made since the last copy or merge operation), but it will catch those that have "revved" in the source with no net change in the contents (which is the exact thing that you want)."
This is what I thought and hoped, but not what I am seeing. When I -f the copy between streams it looks like there is an entry for every file, even ones that have not "revved". Perhaps this be because the filetype is not +m on any file in either stream and the copy is not respecting the "modtime" flag on the workspace? Or could this be because we are copying one stream to another and stream are a special situation?
Posted 05 February 2019 - 03:25 AM
Posted 05 February 2019 - 04:54 AM
C:\Perforce\test\copy>echo asdf > foo C:\Perforce\test\copy>p4 add foo //stream/main/copy/foo#1 - opened for add C:\Perforce\test\copy>p4 submit -d foo Submitting change 108. Locking 1 files ... add //stream/main/copy/foo#1 Change 108 submitted. C:\Perforce\test\copy>p4 copy foo bar //stream/main/copy/bar#1 - branch/sync from //stream/main/copy/foo#1 C:\Perforce\test\copy>p4 submit -d copy Submitting change 109. Locking 1 files ... branch //stream/main/copy/bar#1 Change 109 submitted. C:\Perforce\test\copy>p4 copy foo bar File(s) up-to-date. C:\Perforce\test\copy>p4 copy -f foo bar File(s) up-to-date.
Note that "copy -f" doesn't do the copy, because bar#1 was copied directly from foo#1.
Now I add another revision to foo (but without changing the content), and try the two copy commands again:
C:\Perforce\test\copy>p4 edit foo //stream/main/copy/foo#1 - opened for edit C:\Perforce\test\copy>p4 submit -f submitunchanged -d null Submitting change 111. Locking 1 files ... edit //stream/main/copy/foo#2 Change 111 submitted. C:\Perforce\test\copy>p4 copy foo bar File(s) up-to-date. C:\Perforce\test\copy>p4 copy -f foo bar //stream/main/copy/bar#1 - sync/integrate from //stream/main/copy/foo#2 C:\Perforce\test\copy>p4 submit -d force Submitting change 112. Locking 1 files ... integrate //stream/main/copy/bar#2 Change 112 submitted.
And now that there's a direct copy record, neither copy command will try to copy that same revision again:
C:\Perforce\test\copy>p4 copy foo bar File(s) up-to-date. C:\Perforce\test\copy>p4 copy -f foo bar File(s) up-to-date.
0 user(s) are reading this topic
0 members, 0 guests, 0 anonymous users