Jump to content


Anyone have a way to preview p4 protect updates outside of the P4V admin tool?

protects p4 protects

  • Please log in to reply
13 replies to this topic

#1 mcru

mcru

    Advanced Member

  • Members
  • PipPipPip
  • 63 posts

Posted 01 April 2018 - 09:44 PM

Does anyone know of any existing tooling that lets a user preview changes to the p4 protects table? Ideally, you feed the tool/script the relevant protect lines and it generates a list of all paths and files a user can view. I know the P4V admin tool has this functionality, but I need to allow users who own delegated repo paths to be able to get a preview of what changes would make to any users permissions to their delegated path(s).

I'm pretty sure this would be easy to do using the same logic listed in this article - https://www.perforce...nistration-tool

The hard part (in my mind anyway) is actually applying the list of rules to a set of files outputted from p4 files or any other list of files. I don't fully understand how the p4 protects table checks actually work... multiple times I've had lines I've applied end up not working or having unintended consequences. So, I guess clarity on this would really be key.

Any ideas and/or input is greatly appreciated.

#2 Sambwise

Sambwise

    Advanced Member

  • Members
  • PipPipPip
  • 917 posts

Posted 02 April 2018 - 04:08 AM

View Postmcru, on 01 April 2018 - 09:44 PM, said:

I don't fully understand how the p4 protects table checks actually work... multiple times I've had lines I've applied end up not working or having unintended consequences. So, I guess clarity on this would really be key.

Could you give an example of a case where the protections behavior has been surprising?

The mapping API would be a good building block to develop a tool like this, since that implements the logic for combining and overriding lines in a complex map, but you'd still need a good understanding of the different permissions levels to be able to build the map correctly.

#3 mcru

mcru

    Advanced Member

  • Members
  • PipPipPip
  • 63 posts

Posted 02 April 2018 - 02:25 PM

Sure, a good example of where I biff'd the permissions was when I was trying to make the mainline stream writable to only a few people, read only to others, and make every other stream writable to everyone with permissions on that path.

I first tried something like this:

write user bob * //depot/project/*/...
write user bill * //depot/project/*/...
write user carl * //depot/project/*/...
write user joe * //depot/project/*/...
write user sally * //depot/project/*/...
read user carl * //depot/project/main/...
read user joe * //depot/project/main/...
read user sally * //depot/project/main/...

And eventually with help from support figured out taking away write and adding read isn't good enough, you need to explicitly revoke the =write permission. So, this was correct...

write user bob * //depot/project/*/...
write user bill * //depot/project/*/...
write user carl * //depot/project/*/...
write user joe * //depot/project/*/...
write user sally * //depot/project/*/...
=write user carl * -//depot/project/main/...
=write user joe * -//depot/project/main/...
=write user sally * -//depot/project/main/...

Stuff like this I understand after the explanation from support, but I'm unclear on how the protects algorithm actually does it's checks. From what I read in the admin guide, I would think my original stab at the permissions met the checks it makes.

I'm totally fine understanding the permissions levels, and probably should if I'm going to build something like this, but I was hoping there was something existing that exposed the algorithm that determines permissions based off of protect lines so I don't miss things like I did in the above.

write user carl * //depot/project/main/...
write user joe * //depot/project/main/...
write user sally * //depot/project/main/...

#4 Sambwise

Sambwise

    Advanced Member

  • Members
  • PipPipPip
  • 917 posts

Posted 02 April 2018 - 03:07 PM

The main thing is that additive protections lines are additive -- if you grant someone "write" and "read" to the same file, the "read" does not override the "write" even if it occurs later.

The only way to override previously-granted protections is with a subtractive line.  If you do not specify an exact permission level (with the = syntax), a subtractive line removes ALL permission levels (so removing "list" and removing "super" are equivalent and leave you with zero access).

So in your example, if you want to grant write permissions as a general rule but read permissions to one branch, in addition to the way you did it with =write, you could do:

write user carl //depot/project/...
write user carl -//depot/project/main/...
read user carl //depot/project/main/...

If you want to model a protection table using the mapping API, figure out the permission level you're looking for, and then read down it in order to build a map containing the file specs for lines where:
  • The permission level includes the one you're looking for (e.g. "write" includes "read", and "=read" includes "read", but "=write" does not include "read")

  • The user or group specified matches the one you're querying for

  • The IP specified matches the IP you're querying for
At the end of it you have a map that includes some files and excludes some files.  Doing a map check with a file path will tell you whether the given user@IP has the specified level of permission to that file.

In the above example I did with Carl's protection table, Carl's map for "read" access would look like this (because every line in the table includes "read" access):

//depot/project/...
-//depot/project/main/...
//depot/project/main/...

which in effect matches all files under //depot/project/... since the third line completely overrides the second line.  Carl is therefore allowed to do "read" level operations on everything in these path paths.

Carl's map for "write" access however looks like:

//depot/project/...
-//depot/project/main/...

since when building a map for "write" we ignore the last line (since it does not include the "write" level of permission).  This map of course excludes the mainline, and so Carl does not have "write" access to the mainline.

With the "=write" version:

write user carl //depot/project/...
=write user carl -//depot/project/main/...

our read map is:

//depot/project/...

and our write map is:

//depot/project/...
-//depot/project/main/...

These maps are semantically identical to the ones in the previous example even though they're constructed a little differently.

#5 mcru

mcru

    Advanced Member

  • Members
  • PipPipPip
  • 63 posts

Posted 02 April 2018 - 03:35 PM

Ok, thanks, this makes sense and is probably the approach I'll have to take. I'm going to send something to support to see if they know of any existing tools or scripts I can leverage - ideally, they have something that exposes the same checks the p4 server makes on protect lines so I don't have to rely on a recreation of the rules. I'll update this thread with the response.

#6 Matt Janulewicz

Matt Janulewicz

    Advanced Member

  • Members
  • PipPipPip
  • 176 posts
  • LocationSan Francisco, CA

Posted 02 April 2018 - 05:40 PM

This might be an incredibly insane idea. Or ingenious. Not sure yet.

If you don't already have a spec depot, create one. Spec depots save the history of spec changes so you can refer back to them later (or undo things that turn out to be not so hot without having to remember what you just did.) The spec depot, importantly, includes the history of the protections table and any sub-tables.

There's a new-ish feature in p4 where you can run 'p4 protects' on an older version of the protections table from the spec depot.

So maybe such a tool could make changes in the protections table then immediately undo them. Then //spec/protects.p4s#<head-1> could be fed through 'p4 protects' to see what the results would be for a specific protections table change.

It would clutter up //spec/protects.p4s quite a bit so maybe the tool could take the insanity one step further and create a sub-table of the depot in question (using the 'owner' permission) and use that for all the 'math', then an approved change could be applied to the main protections table (sub-tables using 'owner' permission are saved in //spec/protects/<depot/path>.p4s), then all your undos/redos would be in a sub-spec file and the real //spec/protects.p4s file/history would be clean.

Just spitballin' but something crazy like that might get you where you want to go. Or it might crash and burn in spectacular fashion. Let us know!
-Matt Janulewicz
Staff SCM Engineer, Perforce Administrator
Dolby Laboratories, Inc.
1275 Market St.
San Francisco, CA 94103, USA
majanu@dolby.com

#7 Sambwise

Sambwise

    Advanced Member

  • Members
  • PipPipPip
  • 917 posts

Posted 02 April 2018 - 06:48 PM

I'm voting "insane" on the spec depot thing, personally.  As if spec edit races weren't scary enough...

The idea I thought of but didn't say because it seemed a little overboard was to pipe the protection table into an unlicensed sidetrack server and run "p4 protects" against it.

#8 Matt Janulewicz

Matt Janulewicz

    Advanced Member

  • Members
  • PipPipPip
  • 176 posts
  • LocationSan Francisco, CA

Posted 02 April 2018 - 06:53 PM

View PostSambwise, on 02 April 2018 - 06:48 PM, said:

I'm voting "insane" on the spec depot thing, personally.  As if spec edit races weren't scary enough...

True enough, and it fits most of my suggestions. :)

An enhancement request coming out of this discussion might be to add the semi-ubiquitous '-i' switch to 'p4 protects' to accept input from a file or stdin, as well as the existing spec ...
-Matt Janulewicz
Staff SCM Engineer, Perforce Administrator
Dolby Laboratories, Inc.
1275 Market St.
San Francisco, CA 94103, USA
majanu@dolby.com

#9 mcru

mcru

    Advanced Member

  • Members
  • PipPipPip
  • 63 posts

Posted 02 April 2018 - 07:59 PM

That's actually not the worlds most terrible idea. What I'm even doing in the first place is somewhat nuts. To give you guys a little background...

I'm placing a new ACL model on top of p4 protect. We need to ACL our code down to the file level, with multiple delegated and sub-delegated paths. We also need to let the 'project' owners apply and delegate their own ACLs, without having any access to any other projects. I also need to migrate a non-trivial number of SVN repositories (which aren't even actual repositories, they are individual directories under a single repo) to p4, while maintaining the existing ACLs from SVN. The only real way to do this is to port the ACL model from our existing SVN setup over to p4.

What I'm doing is creating an acl repo that is a big tree of all p4 paths to protect, with each 'repo' owner having write access on the path to the files that will maintain the acl for their project(s). Any sub-delegated paths will write a file under this same path that is writable to the repo owner, and the users selected for sub-delegation. The sub-delegated path owners can further sub-delegate if necessary.

Each repo that is delegated to an owner or group of owners is given the owner permission in the base protects table, owned by a service user, to keep all acl lines for that path outside of the base protect table.

The new ACL model is much more simple than the p4 protects model, and much more flexible. It allows the depot or sub-delegate owners to define their own groups of users or groups of files to protect, and apply permissions in a way that are additive, but automatically get restrictive as finer grained paths are defined. These rules are written following a specified format that, when checked back into perforce, is automatically parsed/verified via pre-submit trigger. If all checks pass the file is submitted, and a post submit trigger expands the rules across all acl files related to the path to acl, plus any delegate or sub delegate files, and generates p4 protect lines. The service user that was given ownership on the path to this repo then takes the generated protect lines and updates the delegated p4 protects table file.

So far, this is working great. And it actually wasn't that bad to write with p4 python. What I need to do now is to give users a way to preview what their changes will do before they submit them. For example, a repo owner or a delegate/sub-delegate should be able to run a script against the new ACL model file they own that will combine all related files, generate the protect lines that would be applied, and then display all paths a specified user can read/write to under that path using the generated protect lines. It would be a breeze if I had access to the same algorithm that p4 uses to apply protections based on the protect table, but I can guarantee I will F this up if rolling my own check, miss some edge case, and lead someone to apply incorrect permissions. This is a huge no-no for us.

Matt - you've got me thinking now... what if I applied the protects to a local DVCS clone of the path to protect and ran the check against that? I think it would work, but I'm not a fan of writing the clone to users's disk and cleaning that up. Also, not even sure if I can apply protections to a local DVCS instance or not.

So far, I'm thinking my best bet is to test a few specific scenarios of p4 protect and force my generated protect lines to match those scenarios and those scenarios only. That way, I can write reliable tests and generate reliable previews using those tests. The only downside is this might be really limiting in what I can support protections wise.

#10 Matt Janulewicz

Matt Janulewicz

    Advanced Member

  • Members
  • PipPipPip
  • 176 posts
  • LocationSan Francisco, CA

Posted 02 April 2018 - 08:21 PM

I don't think the protections table is cloned into a DVCS instance, and a very cursory test seems to not allow cloning of the spec depot, so I'm not sure using DVCS would work. I don't have a lot of experience with it, though, mostly just moving servers from one place to another.

I think your best bet at this point is probably the direction you're going, to only support a few specific scenarios and to be terse about what a project manager can do.

Another outside horse in this race might be the p4broker. You could theoretically use the broker to do most permissions checking, even based on your own system, without invoking the actual protections table much. This would have to be a more command-centric way of doing it, though. You could open the entire depot to all users and give them write access, then use the broker to intercept things like submit, sync, etc., running your own custom script to decide whether or not the user should be able to do that. Probably all kinds of danger in that idea, too, but it could (again, theoretically) eliminate the need to translate the tool's idea of permissions into Perforce's idea of the protections table. Hard to tell off the top of my had how much worse of an idea that is, but in any case the p4broker is a fun tool to play with. :)
-Matt Janulewicz
Staff SCM Engineer, Perforce Administrator
Dolby Laboratories, Inc.
1275 Market St.
San Francisco, CA 94103, USA
majanu@dolby.com

#11 Sambwise

Sambwise

    Advanced Member

  • Members
  • PipPipPip
  • 917 posts

Posted 02 April 2018 - 09:18 PM

View PostMatt Janulewicz, on 02 April 2018 - 08:21 PM, said:

I don't think the protections table is cloned into a DVCS instance, and a very cursory test seems to not allow cloning of the spec depot

"Cloning" any spec from one server to another is trivial:

p4 -p server1:1666 protect -o | p4 -p server2:1666 protect -i

I wrote more than one script that did this before DVCS was a thing.  :)  That's pretty much what I was envisioning when I was talking about running the check on a sidetrack server.  Take your new protection table, pipe it into a local server instance, do whatever you want, delete the local server.

Though really, emulating the protection table (using the Spec and Map APIs) seems easier the more I think about it.  Most of the complexity is in parsing the spec and combining the mappings, and those two APIs do that work for you.

#12 mcru

mcru

    Advanced Member

  • Members
  • PipPipPip
  • 63 posts

Posted 03 April 2018 - 01:13 AM

View PostSambwise, on 02 April 2018 - 09:18 PM, said:

Though really, emulating the protection table (using the Spec and Map APIs) seems easier the more I think about it.  Most of the complexity is in parsing the spec and combining the mappings, and those two APIs do that work for you.

I'm actually already using the spec API, and this route seems much less intrusive vs. spinning up a temporary instance on a user machine for every ACL update.

Cool, thanks for the input guys. I'll follow up with whatever I end up actually building ;)

#13 Sambwise

Sambwise

    Advanced Member

  • Members
  • PipPipPip
  • 917 posts

Posted 03 April 2018 - 05:07 AM

import argparse
import enum
import shlex
import P4

argp = argparse.ArgumentParser(description='Run protection table checks.')
argp.add_argument('-i', 
  dest='input', help='Input filename for protect table', required=True)
argp.add_argument('-u', 
  dest='user', help='User to check', required=True)
argp.add_argument('-p', 
  dest='perm', help='Permission level to check', required=True)
argp.add_argument('-g',
  # This is so you can pass a list of groups the user is a member of.
  dest='groups', help='Group(s) to include', action='append')
argp.add_argument('path',
  help='Depot path to check')
args = argp.parse_args()

# Build up dict of desired permission to levels we see in the table.
perms = {}  # type: Dict[str, Set[str]]

for perm in ['read', 'branch', 'open', 'write']:
  perms['='+perm] = set([perm])

perms['list'] = set(['list'])
perms['read'] = perms['list'] | perms['=branch'] | perms['=read']
perms['open'] = perms['read'] | perms['=open']
perms['write'] = perms['open'] | perms['=write']
perms['admin'] = perms['write'] | set(['admin'])
perms['super'] = perms['admin'] | set(['super'])

perms['review'] = perms['read'] | set(['review'])
perms['owner'] = set(['owner'])

# Parse the file and build a map.
f = open(args.input, 'r')
spec = P4.P4().parse_protect(f.read())
map = P4.Map()

for line in spec._Protections:
  perm, type, name, _, path = shlex.split(line)
  # Check to see if this line is applicable to our protection map.
  if args.perm not in perms[perm] and \
		(not path.startswith('-') or perm.startswith('=')):
	continue
  if type == 'user' and name != args.user:
	continue
  if type == 'group' and (not args.groups or name not in args.groups):
	continue
  map.insert('"'+path+'"')

print("%s's %s protections mapping:" % (args.user, args.perm))
print(map)
print("includes %s? %s" % (args.path, map.includes(args.path)))

Example:

Protections:
super user Samwise unknown //...
super user Samwise 127.0.0.1 //...
write user bob * //depot/...
read user bob * //...
list user bob * "//depot/a path with spaces/..."
read user bob * "-//depot/a secret file"


C:\Perforce\test>python protect_preview.py -i input.txt -u bob -p read "//depot/a secret file"
bob's read protections mapping:
//depot/... //depot/...
//... //...
"-//depot/a secret file" "//depot/a secret file"

includes //depot/a secret file? False


#14 Sambwise

Sambwise

    Advanced Member

  • Members
  • PipPipPip
  • 917 posts

Posted 03 April 2018 - 03:55 PM

^ Only took a little longer than I thought it'd take (about an hour, and half of that was figuring out how to use argparse).  :)  Definitely easier than something involving actual server state -- note that it never even connects to the server, so if you want groups to be taken into account you need to pass them in explicitly (e.g. from "p4 groups -i USER"), but that'd be easy to add into the script.  I just wanted to proof-of-concept that you could implement all of the logic client-side if you had to.

(edit) Checked it in as https://swarm.worksh...s/p4_protect.py and set it up so you can do this in a script:

from p4_protect import build_protection_map

form_data = open(form_file, 'r').read()
map = build_protection_map(form_data, 'write', user)
has_write_access_to_some_path = map.includes('//depot/some/path')






Also tagged with one or more of these keywords: protects, p4 protects

0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users