Jump to content


P4VS slowness

P4VS Slow Visual Studio

  • Please log in to reply
18 replies to this topic

#1 DaveCh

DaveCh

    Advanced Member

  • Members
  • PipPipPip
  • 40 posts

Posted 08 August 2014 - 01:47 PM

When the project I'm working on is loaded into Visual Studio (2010) P4VS appears to be the culprit of a major slowdown causing the 'Preparing Solution' dialog to be displayed for around 3 to 4 minutes (and also occasional slowdowns after that when it's refreshing its data).
It appears to be down to how the project is laid out. The project folder contains the solution/project file, then a subdirectory for source and a subdirectory for data. Over time the data directory has got huge (working on a big console game) with thousands of files and one or two that are up to 4GB in size.
The slowdown appears to be caused by P4VS updating its own internal cache, but it's not doing it for just the files in the solution - it's doing it for every single file in every subdirectory below the solution file, all those data files. I'm not even sure what it's doing, it drags in so much data over the network it's almost the equivalent of doing an offline reconcile.

Anyway, it'd be nice if P4VS could just stick to the files that are in the solution instead of what appears to be a wildcard search that includes other files.

(Based on what shows up in Task Manager/Resource Monitor and using PerfView to see that P4VS is spending all its time in some cache update function...can't remember the name of it)


Dave

#2 P4Shimada

P4Shimada

    Advanced Member

  • Members
  • PipPipPip
  • 831 posts

Posted 13 August 2014 - 05:03 PM

Hi Dave,

Sorry you are having slow performance with P4VS. May I have the following details, in order to better troubleshoot and also to share with our developers?

1. What is the version of P4VS you are using? In Visual Studio, go to:

    Tools -> Extension Manager.
    
2. What is the version of Visual Studio you are  using? In Visual Studio go to:

    Help -> About Microsoft Visual Studio‚Ä®

3. What version of Windows are you using?

#3 DaveCh

DaveCh

    Advanced Member

  • Members
  • PipPipPip
  • 40 posts

Posted 14 August 2014 - 12:23 PM

P4VS - 2014.1.85.4506
VS - 2010 SP1 -  10.0.40219.1
Windows - 8.0 Enterprise

#4 p4bill

p4bill

    Advanced Member

  • Members
  • PipPipPip
  • 217 posts

Posted 25 August 2014 - 03:20 PM

Do you have this preference checked?

Tools -> Options -> Source Control -> Perforce - General

[] Treat Solution/Projects as directories when selected

It sounds like you are seeing individual fstats for all of the files in the data directory. This preference would still get fstat data for those files, but with a more effificient path //<solution dir>/...

Is the data directory a directory of files that are not versioned in Perforce? If they are not, have you tried using an ignore file?

#5 DaveCh

DaveCh

    Advanced Member

  • Members
  • PipPipPip
  • 40 posts

Posted 26 August 2014 - 12:13 PM

I've tried with and without that preference - doesn't seem to make a huge amount of difference to the delays we see.

The data directory is probably 95% versioned in Perforce - it's a huge collection of things like 3DS MAX files and P4VS is spending a lot of time doing an fstat on those both when opening the solution and then again at intervals.

#6 p4bill

p4bill

    Advanced Member

  • Members
  • PipPipPip
  • 217 posts

Posted 29 August 2014 - 02:09 PM

To expand on the data directory question... do you need to update them in Perforce from your work in Visual Studio? That is, will you ever need to check them in and out, or do they just happen to be part of the solution that is loaded (or in that directory).

If they are not part of your Visual Studio work and ignore file might do what you want - eliminating fstats on those files. Are you familiar with the P4IGNORE feature?

#7 DaveCh

DaveCh

    Advanced Member

  • Members
  • PipPipPip
  • 40 posts

Posted 01 September 2014 - 09:51 AM

As I said in the opening post, there's a root directory containing the solution file and then one subdirectory containing source code and one subdirectory containing data files - the solution only references the source code directory.
The data files are all checked in or out by other tools outside of Visual Studio.
I'm aware of P4IGNORE but I can't see how it can be customised to only affect Visual Studio and not all the other tools...?
I guess it boils down to one question - is there a bug in P4VS that causes it to scan the entire data directory, or does it not know which files are in the solution and has to scan everything just to be sure?

#8 p4bill

p4bill

    Advanced Member

  • Members
  • PipPipPip
  • 217 posts

Posted 01 September 2014 - 08:36 PM

Sorry, Dave. It was not clear to me if the root directory you were referring to was a workspace root, or solution/project root. It is sounding like the later.

I don't know if it is necessarily a bug with P4VS, but it is certainly not optimal behavior. If you are getting similar performance with this setting checked and unchecked ([] Treat Solution/Projects as directories when selected), this is my guess at what is happening:

Unchecked: Visual Studio will notify the plugin "a file under source control is being loaded" for each and every file in the project/solution that is in Perforce. P4VS runs an fstat on each file. I do not see much of a delay with small projects (a few hundred files or less) and a decent connection to the server. A lot of files or a slow connection is the reason we added that option.

Checked: With a project/solution containing a lot of files this option will make the fstats run on <solution root>/... rather than the individual files under source control that Visual Studio notifys P4VS about. If that data directory is in the solution root, then it falls under that wildcard fstat, and is needlessly included.

Almost sounding like we need a feature request for customizing the data fetching for files in perforce. When we do the ... wildcard it is typically faster (but all inclusive) when we rely on Visual Studio we only know about one file at a time.

You are correct about P4IGNORE, it cannot be customized to only affect Visual Studio. Perhaps you could try unsing one temporarily, with the '[] Treat Solution/Projects as directories when selected' option set. If things are noticably improved, that could prove the feature request that I mentioned as a viable solution in the future.

#9 DaveCh

DaveCh

    Advanced Member

  • Members
  • PipPipPip
  • 40 posts

Posted 30 September 2014 - 01:38 PM

(Sorry for the delay, been away)
I have the option unchecked so VS should ask P4VS to fstat each file individually. When I enable the logging in P4VS I see this at the start:

[Info: P4API.NET] 30/09/2014 14:28:36.2082 : Connecting to Perforce server on [MYSERVER] as [MYNAME] using client [MYCLIENT]
[Info: P4API.NET] 30/09/2014 14:28:36.3018 : ->changes -l -c [MYCLIENT] -m 10 -s pending -u [MYNAME]
[Info: P4API.NET] 30/09/2014 14:28:38.1115 : ->fstat -Olhp -m 100000 [MYSOLUTIONDIRECTORY]\...

followed by individual fstat commands for project and source files.

It's that first fstat of [MYSOLUTIONDIRECTORY]\... which gets the status of many thousands of data files which causes the delay. According to your description of the "Treat Solutions/Projects..." option, this should not be happening.

#10 p4bill

p4bill

    Advanced Member

  • Members
  • PipPipPip
  • 217 posts

Posted 30 September 2014 - 01:48 PM

CORRECTION: The situation below should not happen on refresh of solutions, but does occur on initial solution load into Visual Studio regardless of what the preference is set to.

Correct, an fstat of [MYSOLUTIONDIRECTORY]\... should not occur with that unchecked. Could you contact support@perforce.com about this. We'll want to dig into this if that preference is not being applied correctly.

#11 jamesh

jamesh

    Member

  • Members
  • PipPip
  • 18 posts

Posted 19 January 2015 - 11:05 AM

Hi,

I am seeing the same issue with opening our solutions. There are a lot of files in the folder tree under the solution directory that are not part of the solution itself. I don't have the "Treat Solution/Projects as directories when selected" option checked, but the logs show a single fstat of [soluion directory]\... on opening a solution. Opening a solution for the first time in a day takes a good few minutes with the "Preparing Solution" dialog showing as described above. Whilst it's possible to live with it, it would be much nicer not having to wait for this to start editing, since usually the only thing I care about is the ability to auto-check out files when I edit them. I'd be quite happy if it did an fstat only when files are selected in the solution explorer. Alternatively, an fstat of the whole solution directory may be ok if we could have a Visual Studio-specific ignore file (though still not ideal).

I am using VS2013.4 on Windows 7 and P4VS 2014.2.97.6861.

As an aside (but related) I also tested the performance of refreshing the solution and closing and then reopening the connection, both of which were also very slow. Looking at the logs, the refreshing solution does the expected fstat on a per file basis (and takes a few minutes to do so), but the close and then reopen shows it doing both the fstat of the whole directory under the solution AND a per file fstat - consequently this was significantly slower than either of the other cases! Note that it is rare for us to do either of these operations in my team, so isn't really an issue.

#12 Creak

Creak

    Newbie

  • Members
  • Pip
  • 4 posts

Posted 26 October 2015 - 07:12 PM

I have the same problem here. Working in a big company with a huge solution (17000 code files). With or without the option "Treat Solution/Projects as directories when selected" checked, I have around the same performance (~5 min to open the solution). It takes 12 seconds to open the solution without the extension.

When enabling the log in visual, with or without the option checked, I've got ~16000 lines like these ones:

->fstat -Olhp o:\main\code\[directory]\[file1].vcxproj
->fstat -Olhp o:\main\code\[directory]\[file1].vcxproj.filters
->fstat -Olhp o:\main\code\[directory]\[file2]_PCH.cpp
->fstat -Olhp o:\main\code\[directory]\[file2]_PCH.h
->fstat -Olhp o:\main\code\[directory]\[file3].cpp
->fstat -Olhp o:\main\code\[directory]\[file3].h
...

The only difference is that I also have these kind of lines when the option is enabled:

->fstat -Olhp -m 100000 o:\main\code\[directory]\...

Versions:
  • Windows: Windows 7 SP1
  • P4VS: 2015.1.105.4164
  • Visual Studio: Visual Studio Pro 2012, Version 11.0.61219.00 Update 5
This is a very important problem for our team here. The only solution we have right now is to remove the extension and make our own shortcuts to check out the files we need.

#13 P4Shimada

P4Shimada

    Advanced Member

  • Members
  • PipPipPip
  • 831 posts

Posted 29 October 2015 - 02:16 PM

Hi James,

View Postjamesh, on 19 January 2015 - 11:05 AM, said:

I am using VS2013.4 on Windows 7 and P4VS 2014.2.97.6861.

As an aside (but related) I also tested the performance of refreshing the solution and closing and then reopening the connection, both of which were also very slow. Looking at the logs, the refreshing solution does the expected fstat on a per file basis (and takes a few minutes to do so), but the close and then reopen shows it doing both the fstat of the whole directory under the solution AND a per file fstat - consequently this was significantly slower than either of the other cases! Note that it is rare for us to do either of these operations in my team, so isn't really an issue.

Sorry that you were experiencing this issue in the 2014 version of P4VS. For the files that are not part of the solution itself, have you tried using the ignore feature? If they are not part of your Visual Studio work then the ignore file feature may do what you want - eliminating fstats on those files. (See 'P4VS: Ignoring "unwanted" files in Visual Studio' http://answers.perfo...rticles/KB/3674 .)

If you are experiencing freezing or hanging with your 2014 P4VS plug-in, please upgrade to the latest P4VS to take advantage of more solutions:

A] The new  'Lazy Load' option

This should alleviate the issue of many fstats on your project files. It was introduced in the version 2015.1 of P4VS. This feature is turned on/off via the menu Tools -> Options -> Source Control -> Perforce General.

The "Lazy Load" feature was designed to cut down load time and general performance on large projects and/or slow connections.  The doc also describes the feature here:

  http://ftp.perforce....s.general.files

The P4VS release notes mention it here under New functionality in 2015.1,
1020819 (Bug 77607) *:

  https://www.perforce...r/p4vsnotes.txt

B] Modify your refresh settings

From P4V, go to Edit --> Preferences --> Server Data and change Check server for updates every from 5 to 1440 minutes (i.e., 24 hours).  that's the largest value P4V will accept.

From Visual Studio, go to Tools --> Options --> Tools Source Control --> Perforce - Data retrieval, change Check server for updates every from 5 to 1440 minutes.

After trying these solutions, please let us know your results.

NOTE - The P4VS options for:

- Treat Solution/Projects as directories when selected
- Preload file state
- Lazy load file state options
- Update status on Selection Changed
- Refresh settings: Tools --> Options --> Tools Source Control --> Perforce - Data retrieval -> Check server for updates every
- - Also check in P4V: go to Edit --> Preferences --> Server Data -> Check server for updates every

are all used to help tune the performance of P4VS for your environment.

#14 P4Shimada

P4Shimada

    Advanced Member

  • Members
  • PipPipPip
  • 831 posts

Posted 29 October 2015 - 02:24 PM

Hello Romain,

View PostCreak, on 26 October 2015 - 07:12 PM, said:

This is a very important problem for our team here. The only solution we have right now is to remove the extension and make our own shortcuts to check out the files we need.

Thank you for sending your version details right away for VS, P4VS and Windows. Are these .h and .cpp files ones that you do not need? If so, have you tried using the ignore feature? If they are not part of your Visual Studio work then the ignore file feature may do what you want - eliminating fstats on those files. (See 'P4VS: Ignoring "unwanted" files in Visual Studio' http://answers.perfo...rticles/KB/3674 .)

If you are using this feature already and the issue persists, please let us know your settings for the following P4VS options:

- Treat Solution/Projects as directories when selected
- Preload file state
- Lazy load file state options
- Update status on Selection Changed
- Refresh settings: Tools --> Options --> Tools Source Control --> Perforce - Data retrieval -> Check server for updates every
- - Also check in P4V: go to Edit --> Preferences --> Server Data -> Check server for updates every

since are all used to tune the performance of P4VS for your environment.

#15 Creak

Creak

    Newbie

  • Members
  • Pip
  • 4 posts

Posted 02 November 2015 - 03:50 PM

View PostP4Shimada, on 29 October 2015 - 02:24 PM, said:

Are these .h and .cpp files ones that you do not need?

All the .cpp and .h files are needed.
What is particularly strange is that, not so long ago, I could open the same solution with a little overhead, but nothing unusable. The first time I had this same slowdown problem I managed to fix it I don't know how, but this time the problem seems to be persistent.

View PostP4Shimada, on 29 October 2015 - 02:24 PM, said:

If [...] the issue persists, please let us know your settings for the following P4VS options:

- Treat Solution/Projects as directories when selected: checked
- Preload file state: checked
- Lazy load file state options: grayed/unchecked
- Update status on Selection Changed: I don't have this checkbox in the P4VS preferences
- Refresh settings: Tools --> Options --> Tools Source Control --> Perforce - Data retrieval -> Check server for updates every: 5 minutes
- - Also check in P4V: go to Edit --> Preferences --> Server Data -> Check server for updates every: 5 minutes

#16 jamesh

jamesh

    Member

  • Members
  • PipPip
  • 18 posts

Posted 06 November 2015 - 03:14 PM

Hi,

My post was made back in January, and since then a combination of updating Visual Studio, updating P4VS and setting certain options seems to have broadly done the trick. The solution opens within a few seconds, and continues to initialize projects for the following ~20s, during which time I can work with any files already initialised, which is acceptable for me. We are now using VS2015, and the latest P4VS plug-in. My settings are as posted in another thread here: http://forums.perfor...les/#entry18923. In particular, I have "Do not optimize" selected, which actually turns out to be faster for our system than either of the other options, as far as I have been able to observe.

#17 P4Shimada

P4Shimada

    Advanced Member

  • Members
  • PipPipPip
  • 831 posts

Posted 11 November 2015 - 04:53 PM

View PostCreak, on 02 November 2015 - 03:50 PM, said:

- Treat Solution/Projects as directories when selected: checked
- Preload file state: checked
- Lazy load file state options: grayed/unchecked
- Update status on Selection Changed: I don't have this checkbox in the P4VS preferences
- Refresh settings: Tools --> Options --> Tools Source Control --> Perforce - Data retrieval -> Check server for updates every: 5 minutes
- - Also check in P4V: go to Edit --> Preferences --> Server Data -> Check server for updates every: 5 minutes

Thanks for sending your settings. Make sure you have the latest version updates for your P4VS and Visual Studio applications. Next, please select the Lazy Load option to help with your performance. You can also try 'Do not optimize' as user 'jamesh' suggested.

#18 Creak

Creak

    Newbie

  • Members
  • Pip
  • 4 posts

Posted 12 November 2015 - 10:29 PM

View PostP4Shimada, on 11 November 2015 - 04:53 PM, said:

Thanks for sending your settings. Make sure you have the latest version updates for your P4VS and Visual Studio applications. Next, please select the Lazy Load option to help with your performance. You can also try 'Do not optimize' as user 'jamesh' suggested.

The lazy load seems to work. Opening my solution now takes only a few seconds.
It seems to be a bit long when I want to auto-check out a file, but it's not as much of a problem than was the opening of a solution. I suppose it is because it has to check the file status before actually checking it out.

Thanks!

#19 P4Shimada

P4Shimada

    Advanced Member

  • Members
  • PipPipPip
  • 831 posts

Posted 12 November 2015 - 11:36 PM

View PostCreak, on 12 November 2015 - 10:29 PM, said:

The lazy load seems to work. Opening my solution now takes only a few seconds.
Thanks!

You're welcome!  Thanks for the update. Glad the option worked.  That is good news.





Also tagged with one or more of these keywords: P4VS, Slow, Visual Studio

0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users