Jump to content


Using a Broker to route traffic based on depot paths?

broker

  • Please log in to reply
7 replies to this topic

#1 mcru

mcru

    Advanced Member

  • Members
  • PipPipPip
  • 67 posts

Posted 06 November 2019 - 02:20 AM

I'm planning on moving a few depots out of our central perforce instance and to their own instances. I'll use a central auth service and a central change service.

What I was hoping to do is use a p4 broker with a filter program to direct any requests to the moved depots to their appropriate instances, but am somewhat lost as it seems like a *ton* of checks to be able to reliably do this, not to mention I think it will break P4V.

Has anyone used a broker to filter on path and redirect all commands for a matching path to another instance?

#2 Miles O'Neal

Miles O'Neal

    Advanced Member

  • Members
  • PipPipPip
  • 169 posts

Posted 06 November 2019 - 05:04 PM

The broker isn't really set up for that. It would, indeed, require a filter progran, and unless you have a very narrow set of constraints to switch on, would end up being complex, slow, and probably not 100% reliable.

Why do you want to do split these depots out?

#3 Sambwise

Sambwise

    Advanced Member

  • Members
  • PipPipPip
  • 1004 posts

Posted 06 November 2019 - 05:54 PM

Redirecting every command in such a way as to make usage seamless seems like a lot more difficulty than it can possibly be worth.  I think the strategy you want to pursue is education-based rather than trying to hide the change -- make sure users who are looking for these depots know where to find them.  This could be as simple as putting empty depots in those spots with a README inside them.  If you want to get really fancy, maybe a broker filter on specific commands like "p4 fstat //old_depot/*" that will produce an error message (with the new server info) which will display as soon as a user tries to navigate to that depot?

View PostMiles O, on 06 November 2019 - 05:04 PM, said:

Why do you want to do split these depots out?

Actually, yes, this too.  If you're splitting them out for performance reasons but you want to hide that from the user, might something like commit/edge fit the bill?

#4 mcru

mcru

    Advanced Member

  • Members
  • PipPipPip
  • 67 posts

Posted 06 November 2019 - 10:32 PM

Quote

Why do you want to do split these depots out?
Politics mostly. We also have insane ACL requirements, including a port of a home grown ACL model to p4 which generates a *lot* of protect lines.

#5 mcru

mcru

    Advanced Member

  • Members
  • PipPipPip
  • 67 posts

Posted 06 November 2019 - 10:43 PM

BTW - I came to the same conclusion about the Broker. What I ended up doing is splitting the depot out to a stand alone instance that uses a centralized auth and change server that is the old 'master' instance. I'll probably add an empty instance to the architecture to act purely as a centralized auth and change server for any additional instances we add. Each instance will also have a read only replica for failover and read only requests. I'll likely add a broker to each cluster to handle failover and to direct reads and writes to their appropriate servers.

I think Perforce would prefer a single server + commit edges, but this seems to work nicely. My major concern is large DVCS pushes locking up instances if anyone does one that pushes all RCS objects and locks the server while the central change server generates CLs for every RCS/commit on the target instance.

#6 Matt Janulewicz

Matt Janulewicz

    Advanced Member

  • Members
  • PipPipPip
  • 208 posts
  • LocationSan Francisco, CA

Posted 12 November 2019 - 08:47 AM

View Postmcru, on 06 November 2019 - 10:32 PM, said:

Politics mostly. We also have insane ACL requirements, including a port of a home grown ACL model to p4 which generates a *lot* of protect lines.

How many is a lot? Our protections table is a few thousand lines long and I know others that are orders of magnitude larger. Are you running into perceived performance issues or is it just ugly to read? I sorted our protections table alphabetically by depot path a few years ago, with admin/special cases at the bottom, and it was one of the best thing I've ever done. :)

What you're attempting to do would scare me. You're adding a bunch of points of failure that don't really need to be there, honestly. If you have the hardware to make another standalone server, it may as well be an edge. If you want to hide stuff from people connecting to that edge, you can easily handle that through the protections table (and the often neglected IP column/field.)

Future p4 admins will not be happy trying to re-assemble backups in the event those are needed.

In a commit/edge architecture, the commit _is_ your central/master (change and auth) and everything else hangs off of it one way or another. Gun to my head, I think I could make a pretty compelling argument that any p4 installation (given the hardware budget) should automatically be commit/edge, or at the very least commit/replica. The caveat being storage limitations for extra-large projects. But then again, the commit server is the only one that absolutely needs to be 100% populated with content (and any backup/failovers), edge servers don't need to initially be populated at all. And if you are lucky enough to have a SAN you don't need to deck all your local servers out with tons of storage.

One last note is that the latest p4d has a built-in failover mechanism. I haven't played with it but if you want a hot spare that automatically fails over into, say, a commit server, I think there is a facility to do that now.
-Matt Janulewicz
Staff SCM Engineer, Perforce Administrator
Dolby Laboratories, Inc.
1275 Market St.
San Francisco, CA 94103, USA
majanu@dolby.com

#7 mcru

mcru

    Advanced Member

  • Members
  • PipPipPip
  • 67 posts

Posted 13 November 2019 - 12:22 AM

We're upwards of 10x that, currently at ~120k lines, and will likely at minimum double it as we continue migrating to Perforce. I can definitely reduce the protect line count with smart use of groups, but did a 1:1 port of our old ACL model to Perforce that explicitly adds and removes access for individual users. We're very ACL heavy, and need to let users control the ACLs themselves via an overlay that generates protect table lines under delegated paths. We did run into protects table related performance issues with git-fusion, and patched git-fusion to reduce the number of checks it runs that process the protect table on each iteration. I suspect we'll run into it on p4d itself eventually as well.

A big reason we're moving this depot to a new instance is because we need to allow access to the triggers table to a group of admins, which will be editing the triggers table as needed for automation surrounding change.

Originally, this was going to be a stand-alone instance, but global changelists and not replicating/reconciling users across both instances won out. I'll have to read more about a commit/edge architecture to see if it's a decent fit. If it is, we can probably migrate to it. If for any reason the current architecture causes issues, we're going to break the two instances fully apart and keep the new instance completely discrete.

For p4 administration - we're currently provisioned with the SDP, which was a great time saver to get up and running, but I'm finding has some major issues (straight up bugs) when getting creative with the architecture. I'm in the process of adding everything to ansible so we can easily reproduce the architecture, check for config drift, and have defined state vs. trusting the SDP generated scripts. Though, I will be leveraging the SDP and templating it so we can still use the built in scripts.  

The plan is to upgrade to the latest p4d for the automatic failover - it looks awesome, with the only limitation being how p4 port is re-pointed. I'll let you know how it goes once I get to it.

#8 Matt Janulewicz

Matt Janulewicz

    Advanced Member

  • Members
  • PipPipPip
  • 208 posts
  • LocationSan Francisco, CA

Posted 19 November 2019 - 10:58 PM

View Postmcru, on 13 November 2019 - 12:22 AM, said:

We're very ACL heavy, and need to let users control the ACLs themselves via an overlay that generates protect table lines under delegated paths.

Are you using the 'owner' permission for that? If not, you should look into it. Saves a lot of overhead/process. Not sure when that was introduced but yeah, I have nightmares about trying to do that before 'owner' was a thing.

Aside from the sorting thing, some time ago we (slowly) migrated to a group-centric protections table. Not sure how big it was prior to that, but probably not 100k+ lines. I would guess that it cut the size at least in half. I couldn't imagine trying to manage each user individually. Yikes! I hope you have an appropriate hobby to work out the aggression that surely accumulates. :)

Giving non-hardcore-admins access to the triggers would freak me out. :) Too easy to wreak havoc in there. Another new feature in 2019.1 that I haven't had the chance to play with are extensions. The look to be meant as an alternative for triggers that you can delegate management too. This bullet point might be especially useful to your situation:

users that the superuser has authorized to configure extensions, can do so without super-user involvement within the repo or depot that user owns

If you ended up messing with extensions, I'd like to hear how that went, too!
-Matt Janulewicz
Staff SCM Engineer, Perforce Administrator
Dolby Laboratories, Inc.
1275 Market St.
San Francisco, CA 94103, USA
majanu@dolby.com





Also tagged with one or more of these keywords: broker

0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users