Jump to content


Is Perforce the right choice for my workflow?

decision Perforce choice workflow

  • Please log in to reply
4 replies to this topic

#1 Marcurion

Marcurion

    Newbie

  • Members
  • Pip
  • 3 posts

Posted 20 April 2019 - 08:30 AM

Hi,

I am trying to wrap my head around Perforce recently, maybe some more experienced users can help me decide whether it's the right tool for my
workflow :)

Our immediate goal is to improve our projects cost scalability (multiple clients) by reusing assets from a single source of truth, however our assets
usually need to live in a folder structure that is maintained by the game engine (Unity3d) for editability and meta file generation.
I already had a look at GIT submodules and deemed them unfit for the task as we are missing control over the folder hierarchy and included content
(there is sparse checkouts though), apart from that contributing back into submodules is a pain because of constant detached head state.

A usual Unity project contains the following folder structure in order to work with the engine:
/Assets
/Project Settings
/Packages

Under /Assets lives the content the project uses, its our main concern to reuse Assets (Art & Code) from these Project I call Producers. For sake of
reuse ability I am prepared to put the essential reuse artefacts into one folder e.g. /Assets/__Export if necessary.
An Producer's folder hierarchy below /Assets might look like this:
/__Export
/_3D
/_UI
/_Prefabs
/_Code
/3rdPartyOptional1
/3rdPartyOptional2
With the folders except /__Export  possibly existing under /__Export as well to keep things organized there  e.g. /__Export/_3D

So these  Producer projects contain Assets that should be reused by other projects as well which I call Consumers.
A Consumer folder structure below /Assets might look like this:
/_3D
/_UI
/_Prefabs
/_Code
/`REUSED1
/3rdPartyOptional1
/3rdPartyOptional2


In theory this should be no problem with Perforce's Workplace View or Stream Mapping, however I have some questions whether the following things
will work at the same time:

A) Can I keep the revision of content in /'REUSED fixed until I manually choose to allow this workplace newer revisions, I saw there is a revision
feature in the form of @1000 in one example but I am confused since Perforce gives Assets individual revisions and some in the same folder might be
at Rev 5 while others are still at Rev 2 which might make it hard to give a common upper limit.

B ) Can I keep some path mappings readonly while other can be contributed to (e.g. write to the Consumer Path and to /Assets/`Reuse1 but not to /
Assets/`Reuse2)? I read about Include and Include+ but it seems to be a stream-only feature? So maybe the actual question is whether Streams are
the way to go and work with these other points...

C) Give collaborators a nice way to check out one Consumer Project Folder Mapping? Streams seem to do exactly that and Workspace Views seem
to be shareable.

D) I can see things get complicated with Perforce when one of our Producer Projects has essential dependencies without the /__Export would not
work but we would like to keep centralized and not dublicated e.g. In a Producer:
/__Export
/`UITemplate
Which is basically a Subrepo in Subrepo problem, for this Perforce seems to have no simple solution. Streams in Streams are not possible right? So I
would need to keep track of the dependencies of my Producers manually and include them manually in every Consumer?

E) Will a still have solid merging with all points considered, even without Streams?


Thank you very much to those who have taken the time to read to this point, it took also some time to write this post and I hope you can see my
honest interest in trying to understand Perforce, I would be very glad if you could show me a way so this could work out :)

Kind reagrds,
Marc

#2 Sambwise

Sambwise

    Advanced Member

  • Members
  • PipPipPip
  • 894 posts

Posted 22 April 2019 - 08:17 PM

Quote

A) Can I keep the revision of content in /'REUSED fixed until I manually choose to allow this workplace newer revisions, I saw there is a revision
feature in the form of @1000 in one example but I am confused since Perforce gives Assets individual revisions and some in the same folder might be
at Rev 5 while others are still at Rev 2 which might make it hard to give a common upper limit.

A pretty fundamental part of Perforce's versioning power is the fact that you can easily refer to a particular version a bunch of different ways (some of which are more convenient in different contexts).  Each individual file has its own individually incrementing sequence of revision numbers (#N), but the repository as a whole has a globally incrementing sequence of changelists (@NNN).  Whenever you see "@N", it refers to the global changelist number, NOT an individual file's revision number!  You can also use date/time stamps in the same way; since changelists increase according to the time they were submitted; dates and change numbers have a fairly straightforward mapping between each other and you can use whichever is more convenient.

Keep in mind also that because Perforce gives branches different file paths (aka "inter-file branching"), the depot path is a whole other axis of the versioning space, such that "//depot/main/...@1000" and "//depot/rel1/...@1000" refer to different versions of your code that exist independently at a single point in time (based on when rel1 was branched, which changes have been merged between the two branches, etc).

Quote

B ) Can I keep some path mappings readonly while other can be contributed to (e.g. write to the Consumer Path and to /Assets/`Reuse1 but not to /
Assets/`Reuse2)? I read about Include and Include+ but it seems to be a stream-only feature? So maybe the actual question is whether Streams are
the way to go and work with these other points...

Yes.  There are two ways to make a path read-only -- one is access control at the admin level ("p4 protect"), and the other is via individual streams/workspaces (i.e. opt-in by the user rather than enforced by the admin).  In a streams context, "import" paths are always read-only within a workspace of that stream.  In a manually defined workspace, you can use the "ChangeView" to get similar functionality to an "import" in terms of locking a particular path to a particular version and making it read-only.

Quote

C) Give collaborators a nice way to check out one Consumer Project Folder Mapping? Streams seem to do exactly that and Workspace Views seem
to be shareable.

DO NOT SHARE WORKSPACES.  It will make you sad.  If you want to share a view, make sure that each collaborator creates their own unique workspace, using the original workspace's view as a template.  As you've observed, streams make this easy (a stream is essentially a dynamic template for workspace views, and multiple people share a stream while each having their own workspace).

Quote

Streams in Streams are not possible right? So I
would need to keep track of the dependencies of my Producers manually and include them manually in every Consumer?


Alas, this is so; the basic problem is that when you "import" something into your stream (which would otherwise be ideal for this use case), you must specify the source of the import as a single depot path, not as a stream (which might itself be composed of its own set of imported depot paths).

One potential solution is to have each consumer branch its dependencies into its own namespace.  From a workflow perspective, the main drawback of this is that as a consumer you don't get new versions automatically until you merge them in.  Depending on the scale, the fact that the repository has a bunch of extra branched copies could also have negative implications for performance (the size of the assets doesn't matter, but once you get over a million revisions or so you need to either get more careful with your usage or more extravagant with your server hardware provisioning).

Quote

E) Will a still have solid merging with all points considered, even without Streams?

You can definitely still do merging without streams, but you will need to track the relationships yourself.  This isn't actually that hard; you can store the merge mappings between classic branches as "branch views", similar to how you define "client views" for classic workspaces instead of having them dynamically generated from stream specs.

#3 Marcurion

Marcurion

    Newbie

  • Members
  • Pip
  • 3 posts

Posted 23 April 2019 - 04:06 PM

Hi Sambwise,

thank you so much for your extensive answer, it gives me hope to keep going and trying with Perforce :)

Could you elaborate a bit more on "branching the dependencies into their own namespace" or maybe direct me to a documentation page, tutorial or example?

Also I found this example on Component Based Development (CBD) with Perforce sounds very similar to what I am trying to achieve, but could not try it out yet since I dont have access to a Linux system currently. What do you think?
https://swarm.worksh...ce-software-cbd


Kind reagrds,
Marc

#4 Sambwise

Sambwise

    Advanced Member

  • Members
  • PipPipPip
  • 894 posts

Posted 23 April 2019 - 04:56 PM

View PostMarcurion, on 23 April 2019 - 04:06 PM, said:

Could you elaborate a bit more on "branching the dependencies into their own namespace" or maybe direct me to a documentation page, tutorial or example?

In other words:

p4 integrate //Producer1/main/Assets/_Export/... //Consumer1/main/Assets/_Import/Producer1/...

or whatever reflects what you want to import and where you want to import it to.  Make the copy in the depot rather than remapping it in the client.  You can "integrate" from any path to any other path so there are no limitations whatsoever on how many copies of each folder you make, going between projects, etc.

Again, if you do this, you may need to integrate again each time the original asset changes in the "producer" (assuming you want to get the new version in the consumer), and you'll need to do this for each asset!

Quote

Also I found this example on Component Based Development (CBD) with Perforce sounds very similar to what I am trying to achieve, but could not try it out yet since I dont have access to a Linux system currently. What do you think?
https://swarm.worksh...ce-software-cbd

I haven't used the CBD scripts myself, but if you contact Perforce I'd guess they can hook you up with a consultant who could assist you with deploying them.  It might be that they're able to automate some of the workflows I've talked here about doing manually.

#5 Marcurion

Marcurion

    Newbie

  • Members
  • Pip
  • 3 posts

Posted 23 April 2019 - 05:21 PM

View PostSambwise, on 23 April 2019 - 04:56 PM, said:

In other words:

p4 integrate //Producer1/main/Assets/_Export/... //Consumer1/main/Assets/_Import/Producer1/...

or whatever reflects what you want to import and where you want to import it to.  Make the copy in the depot rather than remapping it in the client.  You can "integrate" from any path to any other path so there are no limitations whatsoever on how many copies of each folder you make, going between projects, etc.


Thanks a lot again, I get it now :)
Will try around with all your tips for the next few weeks ^_^





Also tagged with one or more of these keywords: decision, Perforce, choice, workflow

0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users