Jump to content


Initialize depot with files


  • Please log in to reply
12 replies to this topic

#1 peetonn

peetonn

    Member

  • Members
  • PipPip
  • 12 posts

Posted 15 June 2018 - 11:21 PM

I'm completely new to Perforce and after two painful days of figuring out Perforce, I finally arrived at the 5th stage of my grief.
Followed this video where everything is simple and straightforward, however unrealistic (dreaming of a better world where tutorials and sample code are as down to earth as possible).

Why it's not realistic? Because existing/freshly initialized project will never consist just of files, but directories.
This is the structure I currently have of my initial depot:

```
.
├── assets
│   ├── authoring
│   │   └── asset.ma
│   └── runtime
│       └── asset.fbx
├── dev
│   └── main
│       ├── ARCoreQuickStart.Editor.csproj
│       ├── ARCoreQuickStart.Plugins.csproj
│       ├── ARCoreQuickStart.csproj
│       ├── ARCoreQuickStart.sln
│       ├── Assembly-CSharp-Editor.csproj
│       ├── Assembly-CSharp-firstpass.csproj
│       ├── Assembly-CSharp.csproj
│       ├── Assets
│       ├── Packages
│       ├── ProjectSettings
│       └── UnityPackageManager
└── docs
    └── README.md
```

As in the video above, I connected to the server and created a workspace `all_ws` as in the video, empty.
Then, I copied the above folder structure to my workspace folder and then tried to "Mark for Add..."
However it spits out many errors, like "file(s) not in client view" (it's 722 warnings, as Unity project has a lot of SDK files already).

How shall I initialize depot with the given folder structure?
Do I need to manually add every file out of those 722?

Thank you in advance for help.

#2 Sambwise

Sambwise

    Advanced Member

  • Members
  • PipPipPip
  • 548 posts

Posted 18 June 2018 - 07:18 PM

Like the error said, the problem is that the files aren't actually mapped in your workspace (the "client view" is the subset of your client machine that is mapped to the depot via your client spec, aka "workspace").  This kind of thing is really easy to fix and once you see an example of it the whole thing will hopefully make sense.  

Could you please post these four things?  (I promise that this will lead to a very quick end to your current frustration, but I especially need items 3 and 4 or my answer is going to include some handwaving that you'll need to spend a lot of time unraveling  on your own.  Items 1 and 2 are mostly about being able to explain what's wrong with what you currently have.)

1) Your client Root
2) Your client View
3) The absolute path of the file tree from your first post (i.e. the folder that /assets and /dev and /doc are immediately under)
4) Where in the depot you want these things to go (i.e. do you want //depot/main/[assets, dev, doc], or do you want everything rooted directly under //depot, or do you want them divided up some other way...)?

#3 peetonn

peetonn

    Member

  • Members
  • PipPip
  • 12 posts

Posted 18 June 2018 - 10:09 PM

Thank you for the reply. Please see below:

1) Not sure what do you refer to as a client root, I've found workspace root in "Connection/Edit current Workspace..." and it's this:

```
/Users/peetonn/Documents/Work/perforce/all
```

2) Is it the "Workspace Mappings" in the same window? Otherwise, I don't know where to get it from. Here's the mapping:

```
//master/... //all/master/...
```

3) Is the same as 2): `/Users/peetonn/Documents/Work/perforce/all`
4) So, this was supposed to be an initial test setup for Perfroce. I intend to have several workspaces, like:

* `all` -- to map everything in the depot (//depot/[assets, dev, docs])
* `dev` -- to map workspace for developers (//depot/[dev, assets/runtime])
* `dev_mobile` -- this will selectively map some folders that should be included for mobile client (under dev/main/Assets)
* `dev_desktop` -- this will selectively map some folders that should be included for desktop client (under dev/main/Assets)
* `docs` -- to map documentation (//depot/docs)
* `design` -- to map workspace for designers (//depot/[assets, docs])


Please, let me know if I got you wrong in any of these or you need more information, and thank you for helping!

Also, I noticed that in your reply you used path like "//depot/main/[assets, dev, doc]" which I believe correspond to stream depot. In my case, as I was following the video, it's local depot.
I was trying to understand how stream depot are different from local depot but the official description is just too vague ("Stream depots contain streams, a type of branch that includes hierarchy and policy.").

Anyhow, I repeated the process with stream depot and was able to add the whole folder structure. However, when I tried to define another workspace that is supposed to map selectively ("dev"), I couldn't do it, because in the settings, when assigning stream to a workspace, you are not able to modify the mapping anymore (it gets greyed out) and everything gets mapped as-is.

So, I'm still stuck and looking for help.
Thank you in advance!

#4 Sambwise

Sambwise

    Advanced Member

  • Members
  • PipPipPip
  • 548 posts

Posted 18 June 2018 - 11:06 PM

To get you over the immediate obstacle, change the client view (or "Workspace Mappings" -- I can't even keep up with the changing terminology in the UI, lol) from:

//master/... //all/master/...


to:

//master/... //all/...


When you're reading a client view, it helps if you mentally substitute "//clientname" (//all in your case) with the client root, which is what Perforce does whenever you operate on local files.  So your mapping was in effect:

//master/... <-> /Users/peetonn/Documents/Work/perforce/all/master/...

Note that this local directory is one that doesn't exist, which is why you weren't able to add anything.  :)  Taking the extra "/master" off the client side of the view mapping makes it:

//master/... <-> /Users/peetonn/Documents/Work/perforce/all/...

which matches your local directory structure (and includes all of the folders you're trying to add).

Hopefully that at least clears up how client views work!

Now, reading the rest of your post, one caution to throw your way: workspaces are not meant to be shared!  Each workspace should belong to one user (one user can have multiple workspaces, but one workspace should not as a general rule have multiple users).  Your description of your different workspaces suggests that you're envisioning having multiple people use one workspace, which is going to be a very bad time.  In general, users should be defining their own workspaces -- you might define "template" workspaces that they can base their own workspaces off of, but they should not directly use them.

This segues nicely into your question about streams -- the short answer of what streams are is that they're a tool to make it so that users don't need to define their own workspaces.  Streams are like workspace views that are designed to be shared -- each user still hast heir own workspace(s), but when defining a workspace they don't need to define a mapping, they just say what stream they want to use, and Perforce generates the mapping for them (that's why you couldn't mess with the mapping when you made a stream workspace; you define that in the stream, and whatever you set there is automatically reflected in all workspaces of that stream).


Note that if you use streams, the top level directory of the depot tends to be the codeline, so you'd want "main" to be the top level directory rather than having it under "dev".  When you define child streams (i.e. branched variants), you can specify that "dev" gets branched but everything else just automatically inherits from "main" (assuming you don't want to branch everything else as implied by your layout above -- but I'd challenge that assumption, since assets and docs usually have dependencies on code and should therefore be versioned alongside them).

#5 peetonn

peetonn

    Member

  • Members
  • PipPip
  • 12 posts

Posted 19 June 2018 - 04:48 PM

Thank you very much for the detailed answer! This clarifies a lot.
Especially, the fact that workspaces aren't meant to be shared by users.
You got it pretty accurately on what I'm trying to achieve. In this regard, do you think I shall proceed with stream depots? If that's the case, will it mean that I need to have one stream depot and then define several streams with mappings according to the workspaces from my previous message?

#6 Sambwise

Sambwise

    Advanced Member

  • Members
  • PipPipPip
  • 548 posts

Posted 19 June 2018 - 07:38 PM

Based on what you've described I think that'd make sense -- if you have chunks that can be split off from each other those might live in different depots, but it looks like everything has some overlap with everything else.

So the way I'd set this up would be to first eliminate the "main" directory in your local tree.  Then create a stream depot (I'm just going to call it "game" here, but a better name would be the actual name of the project/game you're working on), and inside it create a stream called "main".  I'm going to use command line commands for this because explaining how to do things in P4V always takes ten times as long, but once you have the general idea you can probably work out how to do it in P4V.  :)  I'm also going to just ignore the possible existence of any clients you've already created; you might want to delete those, or repurpose them, or just set a new P4CLIENT name so you can create new ones.

p4 depot -t stream game
p4 stream //game/main

(then fill the stream spec out with something like this:)
Stream: //game/main
Type: mainline
Paths:
share ...

This basically says "this stream consists of (and owns) all the files under the depot path '//game/main/...' and doesn't inherit anything from anywhere else".  Now you can create a workspace:

cd $HOME/Documents/Work/Perforce/all
p4 client -S //game/main

and you can add everything to this stream:

p4 add ...
p4 submit

After the submit all the files are in the depot; you can list them with "p4 files //...", you can see them in P4V's Depot Tree, etc.

Now, how does the dev stream look?

p4 stream //game/dev-main

Stream: //game/dev-main
Type: virtual
Parent: //game/main
Paths:
share dev/...
share assets/runtime/...

Try it out:

p4 switch dev-main

Look at your local workspace -- only those two folders should be there now.  (Everything else is still safe and sound in the depot, but it's not part of this virtual stream, and hence it's not part of your workspace.)  If you "p4 switch main", everything else comes back.  Anyone else can "p4 switch" to either stream in their own workspace to get the same effect.  

Follow the same model to create the other virtual streams that include other components.

At some point down the line when you want to have more branches (e.g. a release branch), you can choose which paths to branch and which to maintain as a single version in the mainline.  For example, suppose your developers need a "release1" branch to mark a certain milestone, and you want to make a new stream with a new variant of the "dev" files:

p4 stream //game/dev-release1

Stream: //game/dev-release1
Type: release
Parent: //game/main
Paths:
share dev/...
share assets/runtime/...

p4 populate -r -S //game/dev-release1

Now you have a "dev-release1" stream you can switch to; it's like the "dev-main" branch but it has its own copies of all the files.  Use the "p4 merge" command to selectively propagate changes from one to the other (the typical process for a release branch would be that you make bug fixes in the release branch and merge them back to the main branch, but you don't merge in the other direction because you don't want to mix new feature work from main into your older release(s)).

#7 peetonn

peetonn

    Member

  • Members
  • PipPip
  • 12 posts

Posted 19 June 2018 - 11:19 PM

Thank you, I'll be coming back to your posts again, as they are very valuable.

I like `p4 switch`, I think it acts similar to `git checkout`.
I followed your instructions in the terminal, however after I do `p4 switch dev-main`, I still see `docs` folder in the workspace.

Here's additional info:


$ p4 stream
Stream: //game/dev-main
Update: 2018/06/19 16:12:04
Access: 2018/06/19 15:41:20
Owner: peetonn
Name: dev-main
Parent: //game/main
Type: virtual
Description:
Created by peetonn.
Options: allsubmit unlocked notoparent nofromparent mergedown
Paths:
share dev/...
share assets/runtime/...

$ p4 switch -l
dev-main *
main

Another question - when switching, do I always stay in the same workspace? If that's the case, shall I just forget about workspaces and use streams together with `p4 switch`? As I see it now, I'll setup stream depot and then define virtual streams similarly to how I wanted to setup workspaces in my previous post.

#8 Sambwise

Sambwise

    Advanced Member

  • Members
  • PipPipPip
  • 548 posts

Posted 20 June 2018 - 12:36 AM

You might want to set the "rmdir" option in your client spec.  Run "p4 client" and then in the "Options" field change "normdir" to "rmdir" -- that will make it so that empty directories are cleaned up when you sync (or switch).  (I could be guessing wrong about why that folder is there -- if it's not empty you can try running commands like "p4 have" and "p4 where" and "p4 status" on it to see whether the files in there are depot files or if they're files that somehow didn't get added to the depot.)

"p4 switch" operates on the current workspace, so you don't need to have multiple workspaces with streams; it's designed to be very simple to "reuse" the same workspace with different streams.  However, you *can* have multiple independent workspaces if you want to; just make sure they all have a different Root!  And like I said earlier, each user will want to have their own workspace(s) -- what streams gets you is that setting them up is simpler since all anyone has to do is specify the name of the stream they want to use.

#9 Miles O'Neal

Miles O'Neal

    Advanced Member

  • Members
  • PipPipPip
  • 56 posts

Posted 20 June 2018 - 04:58 PM

For clarification, a client[1] is basically a 1:1 mapping of a set of depot data to disk data. (If you like, you can even restrict this mapping to a specific host's access.)
While multiple users *can* use the same client if you structure things right, this generally causes more problems than it solves, except in very specific cases.[2] It's a good idea not to rush into this, even if you think you have a valid case. It's like buying a house together after a first date.

[1]Perforce uses the terms "client" and "workspace" somewhat interchangeably, which frustrates me no end.
[2] For instance, a client used only to populate a production directory from the latest blessed files in a depot.

#10 peetonn

peetonn

    Member

  • Members
  • PipPip
  • 12 posts

Posted 20 June 2018 - 05:59 PM

Thanks, helps a lot!
I've added "rmdir" option and it did the trick.

Now, I have virtual child stream "dev-mobile" inherited from "dev-main" with this configuration:

Stream: //test2/dev-mobile
Update: 2018/06/20 10:48:04
Access: 2018/06/19 22:31:10
Owner:  peetonn
Name:   dev-mobile
Parent: //test2/dev-main
Type:   virtual
Description:
		stream for mobile development, excludes desktop client features
Options:		allsubmit unlocked notoparent nofromparent mergedown
Paths:
		share ...
		exclude dev/game/Assets/SDKs/Desktop/...
Remapped:
		assets/runtime/... dev/game/Assets/RuntimeAssets/...
Ignored:
		dev/game/Assets/SDKs/Desktop/...

My intent is to have "assets/runtime/" mapped onto "dev/game/Assets/RuntimeAssets/". I tried to symlink it at first, it worked, but then after another sync, it got broken and perforce just created an empty folder instead of symlink. So I decided to try remapping. However, it does not work as expected -- dev/game/Assets/RuntimeAssets/ is empty.

I also want to exclude "dev/game/Assets/SDKs/Desktop" from this stream and ignoring it doesn't help -- it is still present on disk, thus Unity picks it up and includes assets in the build.

Can you please point me to what I'm doing wrong here?

#11 Sambwise

Sambwise

    Advanced Member

  • Members
  • PipPipPip
  • 548 posts

Posted 20 June 2018 - 06:52 PM

That seems like it *ought* to work but maybe there's some complexity with inheriting from another virtual stream that I'm missing.  Does this work?  (I'm having it inherit directly from main, and also dropped the Ignored which I think is superfluous here).

Stream: //test2/dev-mobile
Parent: //test2/main
Type: virtual
Paths:
	share dev/...
	share assets/runtime/...
	exclude dev/game/Assets/SDKs/Desktop/...
Remapped:
	assets/runtime/... dev/game/Assets/RuntimeAssets/...

If that doesn't work, try running "p4 where //..." to see what the generated view looks like.

Also: is your server case-sensitive?  I notice that you have "assets" in some spots and "Assets" in others...

#12 peetonn

peetonn

    Member

  • Members
  • PipPip
  • 12 posts

Posted 20 June 2018 - 07:42 PM

I removed paths from "ignore" and it seemed to start working now.
"p4 where" is a useful command, thanks!

I think I more or less figured out different streams and the folder structure I want to support development (mobile/desktop), design and documentation.
Now, the next step would be to figure out development, staging and production branching. I suppose I should start branching up from main for production and branching down from main for development (per desktop/mobile).

#13 Sambwise

Sambwise

    Advanced Member

  • Members
  • PipPipPip
  • 548 posts

Posted 20 June 2018 - 08:42 PM

If everything is on the same release cycle, you might just want to have one stream that is a branch of everything in "main", and create your virtual streams off of that the same way you did for main.

Otherwise, you can do fun things with stream paths to "import" the paths that a given branch for a given component depends on (but does not modify) and "share" the ones that need to be modified locally in that branch.




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users