Jump to content


list of command:metadata mapping

metadata command

  • Please log in to reply
8 replies to this topic

#1 Miles O'Neal

Miles O'Neal

    Advanced Member

  • Members
  • PipPipPip
  • 214 posts
  • LocationAustin. Texas. Y'all.

Posted 14 April 2020 - 04:52 PM

I could have sworn that at one point I found a document listing which metadata tables were affected by which commands, but I can't find such a document today. Has anyone seen such a beast, or am I misremembering?

Thanks!

#2 Sambwise

Sambwise

    Advanced Member

  • Members
  • PipPipPip
  • 1192 posts

Posted 14 April 2020 - 05:46 PM

I'd be skeptical of the accuracy of such a document if it ever existed, personally.  Almost every command can touch almost every table depending on what arguments you pass to it, how your workspace is configured, etc, and it's going to change from version to version (in corner cases it might even change depending on the client version as well as the server version, since there are backward-compatibility paths for a lot of commands and some of them diverge significantly).  I can't imagine anyone putting in the hours needed to keep that document up to date.

If you need to figure out which tables a given command affects, I'd run the command in the actual environment (or as close to it as you can get) with the -Zdbstat flag so you can see exactly what it's doing to the db.

#3 Miles O'Neal

Miles O'Neal

    Advanced Member

  • Members
  • PipPipPip
  • 214 posts
  • LocationAustin. Texas. Y'all.

Posted 14 April 2020 - 08:30 PM

I'm specifically interested right now in which files are written to, hence "affected by".
Actually, it's exactly the sort of documentation I have had to keep up to date in the past.
But the -Zdbstat flag is great for my needs. I had forgotten about that; I think I used it once at support's behest shortly after the Helix ownership was dropped on me, many moons ago.
Thanks!

#4 Sambwise

Sambwise

    Advanced Member

  • Members
  • PipPipPip
  • 1192 posts

Posted 14 April 2020 - 08:45 PM

To start you off: almost every command updates db.user and/or db.domain, but only once every five minutes.  ;)  Most of the rest of the db writes are pretty obviously deterministic, but that one usually throws people.

#5 Miles O'Neal

Miles O'Neal

    Advanced Member

  • Members
  • PipPipPip
  • 214 posts
  • LocationAustin. Texas. Y'all.

Posted 14 April 2020 - 10:33 PM

I'm trying to understand why a couple of commands have to go to the master rather than a r/o replica (per https://community.pe.../s/article/1253 . The chief offender is "print -o". That seemed weird to me. And running with -Zdbstat, the only db write I see is to db.monitor, which is fine.

%  p4 -Zdbstat  print -o /tmp/foom.out //eit/README
//eit/README#2063 - edit change 1816882 (text)
--- db.counters
---   pages in+out+cached 3+0+2
---   locks read/write 0/0 rows get+pos+scan put+del 1+0+0 0+0
---   peek count 1 wait+held total/max 0ms+0ms/0ms+0ms
--- db.user.rp
---   pages in+out+cached 3+0+2
---   locks read/write 1/0 rows get+pos+scan put+del 1+0+0 0+0
--- db.group
---   pages in+out+cached 39+0+38
---   locks read/write 1/0 rows get+pos+scan put+del 0+251+522 0+0
--- db.domain
---   pages in+out+cached 5+0+4
---   locks read/write 1/0 rows get+pos+scan put+del 1+0+0 0+0
--- db.view
---   pages in+out+cached 3+0+7
---   locks read/write 1/0 rows get+pos+scan put+del 0+1+2 0+0
--- db.rev
---   pages in+out+cached 39+0+38
---   locks read/write 0/0 rows get+pos+scan put+del 0+1+2064 0+0
---   peek count 1 wait+held total/max 0ms+1ms/0ms+1ms
--- db.trigger
---   pages in+out+cached 3+0+2
---   locks read/write 1/0 rows get+pos+scan put+del 0+1+27 0+0
--- db.repo
---   pages in+out+cached 3+0+2
---   locks read/write 1/0 rows get+pos+scan put+del 0+1+1 0+0
--- db.graphperm
---   pages in+out+cached 3+0+2
---   locks read/write 1/0 rows get+pos+scan put+del 0+1+1 0+0
--- db.protect
---   pages in+out+cached 42+0+41
---   locks read/write 1/0 rows get+pos+scan put+del 0+1+2400 0+0
--- db.monitor
---   pages in+out+cached 6+4+2
---   locks read/write 0/2 rows get+pos+scan put+del 0+0+0 1+1

#6 Miles O'Neal

Miles O'Neal

    Advanced Member

  • Members
  • PipPipPip
  • 214 posts
  • LocationAustin. Texas. Y'all.

Posted 14 April 2020 - 10:35 PM

And what is this "every five minutes" bit?

#7 Sambwise

Sambwise

    Advanced Member

  • Members
  • PipPipPip
  • 1192 posts

Posted 14 April 2020 - 11:19 PM

View PostMiles O, on 14 April 2020 - 10:35 PM, said:

And what is this "every five minutes" bit?

That's the "access" timestamps.  To cut down on write lock contention, they only get refreshed if they're more than five minutes out of date.  So the first time you run a command (any command) after not having done anything for a few minutes, you'll see a write refreshing the timestamp, but if you run the same command again immediately it won't do the same thing.

#8 Miles O'Neal

Miles O'Neal

    Advanced Member

  • Members
  • PipPipPip
  • 214 posts
  • LocationAustin. Texas. Y'all.

Posted 15 April 2020 - 04:37 PM

Interesting. But it doesn't appear that every command does that, especially if it's not directly naming a client: https://community.pe...s/article/10837 .

That document also states, "The access time for client (and label and branch) specs is only modified when commands update metadata and the interval since the last spec usage exceeded the update timer value (specified by dm.domain.accessupdate)." Looking through the -Zdbstat output above, I don't see how "p4 print -o" writes to the metadata (again, excluding db.monitor", which is necessarily writable on a read-only replica).

Thanks for the heads-up on the access time issue; that's good to know.


#9 Sambwise

Sambwise

    Advanced Member

  • Members
  • PipPipPip
  • 1192 posts

Posted 15 April 2020 - 04:48 PM

It varies in non-obvious ways.  For example, neither "p4 users" nor "p4 clients" itself updates any metadata, but "p4 clients" updates the user's access timestamp, while "p4 users" does not:

C:\Perforce\test>p4 -Zdbstat users
Samwise <Samwise@host> (Samwise) accessed 2020/04/15
--- db.user
--- pages in+out+cached 3+0+2
--- locks read/write 0/0 rows get+pos+scan put+del 0+1+2 0+0
--- peek count 1 wait+held total/max 0ms+0ms/0ms+0ms
...

C:\Perforce\test>p4 -Zdbstat clients
Client Samwise-dvcs-1509687817 2020/01/19 root c:\Perforce\test 'Created by Samwise. '
...
--- db.user
--- pages in+out+cached 5+3+2
--- locks read/write 1/1 rows get+pos+scan put+del 1+0+0 1+0
...

which I think is something to do with the two commands having subtly different access requirements.  

There are also cases where commands that don't update any metadata still update the domain access time, e.g.:

C:\Perforce\test\python>p4 -Zdbstat sync -n
...
--- db.domain
--- pages in+out+cached 5+3+2
--- locks read/write 1/1 rows get+pos+scan put+del 1+0+0 1+0
...

which is probably an exception because the "sync" command generally updates metadata?  Which implies that there might be exceptions in the opposite direction as well (i.e. a command that generally does not update metadata, but does in a particular circumstance or because you passed a particular flag, might not update the access time when the general rule would suggest that it should).

I remember spending some time many years ago trying to pin down comprehensive rules for exactly when the update/access timestamps get updated (to help answer this exact question because it was relevant to a script the user was writing) and I was able to find an exception to every rule I tried to formulate, so my answer ended up being "usually but not always" and to not place any absolute trust in any more specific rule.

And that's only one very tiny specific subset of the much larger "what are the rules for what db tables each command alters" question...





Also tagged with one or more of these keywords: metadata, command

0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users