Jump to content


Automated checkpoints

checkpoint server

  • Please log in to reply
10 replies to this topic

#1 argon

argon

    Newbie

  • Members
  • Pip
  • 1 posts

Posted 02 January 2015 - 05:24 PM

Hello,

I would like to set up a periodic job on my server that (on a daily or weekly basis) performs a
p4 verify -q //...
and, if successful,
p4 admin checkpoint -z

logging the output and emailing any issues.

However, while p4 verify can be performed by any user, p4 admin checkpoint can only be executed by users with "super" access level.

The thing is, I'm not sure I want to create a Perforce superuser just for this task. Am I being too paranoid?
Sure, I can grant access to that user from localhost only, using p4 protect.

But what other best practices can you suggest, for implementing this in a secure way?
i.e. do you recommend having the shell script log the user in by using a ticket that does not expire, generated once by another process? Or is 'echo my_password | p4 login" safe enough as long as the shell script itself is not readable by others?

Thanks!

#2 Bruce Mc

Bruce Mc

    Advanced Member

  • Members
  • PipPipPip
  • 84 posts
  • LocationSeattle Area

Posted 06 January 2015 - 03:23 AM

I'm not a Perforce support person, but these are my thoughts, your mileage may vary.

You could create an user of the type operator to run "p4 admin checkpoint". It won't be counted against your licensed user count and it has a limitted set of commands it can do.

I favor using a non-expiring ticket. I prefer to create these tickets by hand so I'm not keeping the password around in clear text. I can see that this approach wouldn't scale well so a scripted solution may be prefered.

Regards,

Bruce

#3 Matt Janulewicz

Matt Janulewicz

    Advanced Member

  • Members
  • PipPipPip
  • 223 posts
  • LocationSan Diego, CA

Posted 06 January 2015 - 05:53 PM

Do you not already have an admin account? Save for creating an operator account, seems like an admin account should already be available. I echo the sentiment that you should never risk putting an admin password in plain text, anywhere. Put the admin or operator in a group that has a long expiry on the ticket, generate it, then run the script in the context of that user.

Traditionally, p4 admins (myself included) are in the habit of running checkpoints directly on the server using p4d, which you can do if you have physical access to the db's. No p4 account required.

You're right on the edge of a deep rabbit hole, you're about to fall in, so I'll be the first to suggest you look into replicating your db to a different server/path, or perhaps replaying your journals into a second copy of your db files, then running the checkpoint offline from there. This way you don't have to contend with database locking (downtime) for the duration of the checkpoint.

If you think your installation is too small and checkpoints take a very short amount of time, think to the future. You never know how big your db is going to get so it's a great idea to set up the processes to handle it now while you might still be small.

My personal preference (even though I don't do it right now, we're in the midst of a commit->edge conversion) would be to set up a db-only replica. This way you get the experience of setting up a replica before you might need it. It also replicates your database in pretty much real time, so your emergency backup database(s) are as new as they can be at any given time, they aren't beholden to the journal rotation schedule.
-Matt Janulewicz
Currently unemployed, looking for work in Boise, ID!

#4 Robert Cowham

Robert Cowham

    Advanced Member

  • PCP
  • 271 posts
  • LocationLondon, UK

Posted 06 January 2015 - 08:05 PM

You might want to read up on the SDP (Server Deployment Package) which is available on the Workshop (Public Depot).

https://swarm.worksh...ain/README.html

Also the docs:

https://swarm.worksh.../files/main/doc

It might be overkill, but it might prompt some thoughts :)
Co-Author of "Learning Perforce SCM", PACKT Publishing, 25 September 2013, ISBN 9781849687645

"It's wonderful to see a new book about Perforce, especially one written by Robert Cowham and Neal Firth. No one can teach Perforce better than these seasoned subject matter experts"
  • Laura Wingerd, author of Practical Perforce, former VP of Product Technology at Perforce

#5 P4Shimada

P4Shimada

    Advanced Member

  • Members
  • PipPipPip
  • 831 posts

Posted 06 January 2015 - 09:46 PM

Hi Alberto,

When it comes to how to handle your Perforce user login within a script we have suggested examples here:

    http://answers.perfo...n-interactively

You can use your super user account in the script to run all the commands.  I am not sure which operating system you use but if on Unix, you can use a cronjob that runs the script when you please.

What type of security concerns do you have?

Let us know if you have any additional questions.

#6 obunmi

obunmi

    Newbie

  • Members
  • Pip
  • 3 posts

Posted 24 February 2016 - 09:52 AM

Hi Argon,

I know this is an old topic but I am in the same shows as you- trying to do exactly what you are doing and very much new to Perforce.

I would like to set up a periodic job on my server that (on a daily or weekly basis) performs a
p4 verify -q //...
and, if successful,
p4 admin checkpoint -z

logging the output and emailing any issue

Is it possible to have your script for this so I can adapt and where do you run this script- Powershell?
And is it done periodically through the task scheduler?

Thank you in advance.

#7 P4Shimada

P4Shimada

    Advanced Member

  • Members
  • PipPipPip
  • 831 posts

Posted 24 February 2016 - 07:33 PM

View Postobunmi, on 24 February 2016 - 09:52 AM, said:

I would like to set up a periodic job on my server that (on a daily or weekly basis) performs a
p4 verify -q //...
and, if successful,
p4 admin checkpoint -z

logging the output and emailing any issue

Is it possible to have your script for this so I can adapt and where do you run this script- Powershell?
And is it done periodically through the task scheduler?

There are no specific tools we recommend for automating backups but we do recommend that you do not directly backup the 'db.*' files from the P4ROOT directory.  This is because many backup tools will lock the file while backing them up. Instead you should take checkpoints and back them up along with the versioned files. This is discussed in the following article:

  https://www.perforce...ter.backup.html

Here are some example backup scripts that were contributed by users to the Public Depot Workshop.

  https://swarm.worksh.../utils/p4backup
  https://swarm.worksh...p4checkpoint.pl

NOTE: These are not officially Perforce supported scripts, just examples that work. Feel free to use and modify as you would like.

For example, you could create a simple batch file (backup.bat) in Windows. If your Perforce server was located at 'C:\Program Files\Perforce', a script could start off like:

  @echo off
  REM.-- Run the verify command
  C:\Program Files\Perforce\p4.exe verify -q //...

  REM.-- Do the checkpoint
  C:\Program Files\Perforce\p4d.exe -r C:\Program Files\Perforce -z -jc

for your automated backup and then schedule a Windows task that points to your backup script. See 'Schedule a task':

    http://windows.micro...k#1TC=windows-7

* NOTE: You mentioned the '-z' flag.  If you are using the -z (compress) option to compress your checkpoints, you will have to restore the uncompressed journal file separately from the compressed checkpoint.  These steps are given in the Perforce Admin Guide.

REFERENCES

- Server Backup in P4V (See section: Using the "p4d -jc" Command in Backup Scripts)
http://answers.perfo...rticles/KB/3346

- Speeding Up the "p4 verify" Command
http://answers.perfo...rticles/KB/2422

- Checkpoints and Journals
http://answers.perfo...rticles/KB/2408

- Offline Checkpoints
http://answers.perfo...rticles/KB/2419

#8 obunmi

obunmi

    Newbie

  • Members
  • Pip
  • 3 posts

Posted 29 February 2016 - 10:05 AM

Thank you so much P4 Shimada, I will d this and get back to you.

#9 P4Shimada

P4Shimada

    Advanced Member

  • Members
  • PipPipPip
  • 831 posts

Posted 01 March 2016 - 06:58 PM

You're welcome!

Also to note, you can speed up your verify command in the backup script and work around checking all files and their revisions on the Perforce server by adding a date and time range to the verify command. You can verify from the last good checkpoint until the current time (now), instead of all revisions that were already verified in the last good checkpoint.

For example, if your backup script was run at 1PM on 2016/02/24, you can run the command:

  p4 verify -q //...@2016/02/24:12,@now


#10 obunmi

obunmi

    Newbie

  • Members
  • Pip
  • 3 posts

Posted 03 March 2016 - 04:49 PM

Thanks again P4Shimada, now looks like I am good backing up...how do I move the files to a different directory? where my server root is E:\Perforce\Server ?

#11 P4Shimada

P4Shimada

    Advanced Member

  • Members
  • PipPipPip
  • 831 posts

Posted 03 March 2016 - 08:31 PM

View Postobunmi, on 03 March 2016 - 04:49 PM, said:

Thanks again P4Shimada, now looks like I am good backing up...how do I move the files to a different directory? where my server root is E:\Perforce\Server ?

Glad you are good with the backups now. Which files are you trying to move? Is this within the backup script? Since you are using Windows, I would think you would use the 'copy' command:

http://www.computerh...com/copyhlp.htm





Also tagged with one or more of these keywords: checkpoint, server

0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users