Jump to content


Justin's Content

There have been 10 items by Justin (Search limited from 28-May 19)


By content type

See this member's

Sort by                Order  

#17863 Perforce filetype limitations

Posted by Justin on 16 June 2015 - 03:37 PM in P4V

Hi Shimada,

Here is my information:

a] Server version: P4D/LINUX26X86_64/2012.2/756218 (2013/12/09)
b] Windows (Only a few clients run under OSX)
c] My server does not output '... unicode enabled' when running that command.

Sorry I didn't get back to you earlier, I forgot to follow this thread to receive notifications.



#17724 Perforce filetype limitations

Posted by Justin on 05 June 2015 - 09:52 AM in P4V

Here's the attached file

Attached Files




#17723 Perforce filetype limitations

Posted by Justin on 05 June 2015 - 09:47 AM in P4V

Hi all!

I'd like to know more about the impact of Perforce's filetypes out of technical interest, rather than needing to solve a particular problem. (But it may save me headaches later when diagnosing issues!) Not sure how to express it other than questions about hypothetical bad practices, so here goes:


1. How should I store UTF-8 text files in Perforce?

My understanding is that Perforce's UTF-8 provision is the 'unicode' filetype. This filetype isn't enabled on my server and I'll assume it's not going to change.

If I choose 'binary' then that will be fine, until a client syncs to it with different OS line ending expectations?


2. Why do (some?) UTF-8 text files 'survive' being stored under the 'text' filetype?

For example:
  • Download the attached utf8_as_text_filetype_text.txt (Content generated from http://generator.lor....info/_japanese)
  • Submit it with the 'text' filetype
  • Make a local copy
  • Force sync
  • Observe both the local and re-synced files are binary-identical

When will be the point when a UTF-8 text file is corrupted due to being stored as 'text'? Could I have a concrete example?


3. At what point does the 'text' filetype break down for arbitrary binary data?

I've experienced images corrupted because they were somehow ended up with the 'text' filetype, but I am curious to exactly why that happened.

I know that Perforce translates the line endings for us, but thought it's a moot point if the submitted and syncing clients are of the same OS.

If it's anything to do with deltas, then is 'text' guaranteed to work at least for just the initial 'add'?


Thanks very much!


Justin



#15902 P4VS on large solutions

Posted by Justin on 21 November 2014 - 10:43 AM in General

I haven't heard many complaints about the issue since so I figured people were getting by happily enough, but I just had a user who needed an alternative and started using P4EXP; it must still be a critical issue (for many) if someone felt forced to abandon P4VS entirely.

I've advised him to instead use custom commandline tools in VS which are basic but covers 95% of his needs - he's happy with it now (he sticks to the same workspace most of the time so it should keep working).

Tools > External Tools > Add

Title:
Command:
Arguments:

[P4] Open for Add
C:\Program Files\Perforce\p4.exe
add $(ItemPath)

[P4] Open for Edit
C:\Program Files\Perforce\p4.exe
edit $(ItemPath)

[P4] Open Directory for Edit
C:\Program Files\Perforce\p4.exe
edit $(ItemDir)*

[P4] Recursive Open Directory for Edit
C:\Windows\system32\cmd.exe
/c dir /s/b $(ItemDir) | p4 -x- edit



#15750 P4VS on large solutions

Posted by Justin on 05 November 2014 - 12:18 PM in General

Nice to know that I'm not alone in this matter, and that it's still being looked at. :)

Bill, have you been able to reproduce the issue at all? I think the biggest hurdle towards getting this sorted is providing solid reproduction steps (which I've found unwieldy to produce).



#15241 P4VS on large solutions

Posted by Justin on 12 September 2014 - 03:25 PM in General

Ah right... yeah in my case we store everything in the workspace; our .sln just happens to be at the workspace root for convenience, so you're right in it making things worse.

Look forward to the test build.

Thanks!



#15239 P4VS on large solutions

Posted by Justin on 12 September 2014 - 01:17 PM in General

Sorry for the delay in responding.

>> Are you saying that it previsouly worked better a few months back when there were less than 100K files checked in?
Yes definitely.

>> I can probably get you a test build sometime next week to expose that 100k limit.
That would be great to test out -- especially if it could help out others too.

>> Is your solution root the same as your workspace root? The way you describe the 'Treat...' option working (widening the scope), seems like a bug to me.
Yes they are the same; our workspace root is D:\foo\bar\ and the .sln file is D:\foo\bar\baz.sln



#14974 P4VS on large solutions

Posted by Justin on 26 August 2014 - 11:11 AM in General

Thanks Bill, hope my previous post sheds more light on what I'm experiencing.

Also, my gut feeling is that this issue is caused by having >100k files checked in. This makes sense in that we didn't have issues with the plugin until a few months ago, and since then the issue has progressively become worse.

It seems compelling to update the plugin so that the -max flag value of 100k is exposed, so I can increase it just to try out.

Thanks,

Justin



#14972 P4VS on large solutions

Posted by Justin on 26 August 2014 - 10:49 AM in General

With the 'Treat...' option checked:
  • There is no longer a long hang after loading the solution. However I do get a question mark '?' next to all items in the Solution Explorer.
  • When I try to make an edit, I get a popup: Managed Source Control Profider - The file xxx you are trying to edit is read only on disk. Do you want do make the file writable so you can continue editing it?

The server logs show fstat calls still being issued, for well over an hour now(!!) The calls are definitely being issued to folders that contain only by-products of compiling, which can only be making things worse.

2014/08/26 10:12:16 pid 327 <user>@<workspace> <IP> [P4VS/2014.1.85.4506] 'user-clients -u <user> -m 1'
2014/08/26 10:12:16 pid 327 <user>@<workspace> <IP> [P4VS/2014.1.85.4506] 'user-login -s'
2014/08/26 10:12:16 pid 327 <user>@<workspace> <IP> [P4VS/2014.1.85.4506] 'user-client -o <workspace>'
2014/08/26 10:12:16 pid 327 <user>@<workspace> <IP> [P4VS/2014.1.85.4506] 'user-opened -m 1'
2014/08/26 10:12:16 pid 327 <user>@<workspace> <IP> [P4VS/2014.1.85.4506] 'user-changes -l -c <workspace> -m 10 -s pending -u <user>'
2014/08/26 10:13:22 pid 304 <user>@<workspace> <IP> [P4VS/2014.1.85.4506] 'user-fstat -Olhp -m 100000 <workspace_root>\...'
2014/08/26 10:13:22 pid 304 <user>@<workspace> <IP> [P4VS/2014.1.85.4506] 'user-fstat -Olhp -m 100000 <workspace_root>\...'
2014/08/26 10:13:32 pid 308 <user>@<workspace> <IP> [P4VS/2014.1.85.4506] 'user-fstat <workspace_root>\<solution_file>'
2014/08/26 10:13:43 pid 315 <user>@<workspace> <IP> [P4VS/2014.1.85.4506] 'user-fstat -Olhp -m 100000 <workspace_root>\<intermediate_files>'
2014/08/26 10:13:56 pid 318 <user>@<workspace> <IP> [P4VS/2014.1.85.4506] 'user-depots'
2014/08/26 10:13:56 pid 319 <user>@<workspace> <IP> [P4VS/2014.1.85.4506] 'user-client -o <workspace>'
2014/08/26 10:17:31 pid 851 <user>@<workspace> <IP> [P4VS/2014.1.85.4506] 'user-fstat -Olhp -m 100000 <workspace_root>\<subfolder01>\*
2014/08/26 10:17:46 pid 855 <user>@<workspace> <IP> [P4VS/2014.1.85.4506] 'user-fstat -Olhp -m 100000 <workspace_root>\<subfolder02>\*
2014/08/26 10:17:56 pid 860 <user>@<workspace> <IP> [P4VS/2014.1.85.4506] 'user-fstat -Olhp -m 100000 <workspace_root>\<subfolder03>\*
2014/08/26 10:18:11 pid 869 <user>@<workspace> <IP> [P4VS/2014.1.85.4506] 'user-fstat -Olhp -m 100000 <workspace_root>\<subfolder04>\*
...
2014/08/26 11:25:58 pid 3936 <user>@<workspace> <IP> [P4VS/2014.1.85.4506] 'user-fstat -Olhp -m 100000 <workspace_root>\<subfolderXX>\*


What it looks like to me is:
  • The fstat calls are now asynchronous, so although the IDE is quicker to respond to user input, it's still churning through fstats.
  • P4VS is uninitialized until all those fstat calls are done. This is why the option seems to 'break' the plugin for me, since it takes over an hour to initialize.
  • P4VS churns through every local folder under the workspace root regardless of whether it's relevant to the solution or not.
  • My workspace root contains 107k files and 13k dirs, many of which are irrelevant to the VS solution.
  • 'Treat...' option widens the scope of the fstat calls to the entire workspace, instead of just the solution itself.



#14936 P4VS on large solutions

Posted by Justin on 22 August 2014 - 09:06 AM in General

Hi there


We're using P4VS in Visual Studio 2013. The issue is that upon loading our (large) solution, VS hangs for ~5 minutes and issues about 20k fstat commands.

This is similar to: http://forums.perfor...-p4vs-slowness/


Here's my theory to why this is happening:
  • Looking at the logs, P4VS gets its list of all files already submitted by using this command: fstat -Olhp -m 100000 d:\xxx\yyy\...
  • However, running that command without the -m flag returns around 120k results, which means P4VS thinks around ~20k files aren't submitted.
  • And that's the problem: It seems that P4VS will call fstat for each of those 20k files, and this causes the hang.

Notes:
In the P4VS options, I've tried:
  • "Automatically add new files to Perforce": When unticked, those fstats are still performed.
  • "Treat Solution/Projects as directories when selected": When ticked, the hang doesn't occur but this breaks things like automatically checking out a file when you start typing, which defeats the point of using the plugin in the first place.
Versions:
  • Microsoft Windows 7 Enterprise, Version 6.1 (Build 7601: Service Pack 1)
  • Microsoft Visual Studio Professional 2013, Version 12.0.30324.00 Update 2 RC
  • Microsoft .NET Framework, Version 4.5.50938
  • P4VS 2013.3.78.1524 and 2014.1.85.4506


Any help appreciated.

Thanks!


Justin