Jump to content


Ran out of space and can't log into workspace!


  • Please log in to reply
11 replies to this topic

#1 EDarkness

EDarkness

    Newbie

  • Members
  • Pip
  • 6 posts

Posted 22 January 2019 - 02:39 AM

Hello!  I've been using P4V on my Mac for a long time and I never considered what happens when there are too many files in the workspace.  Well, unfortunately, I hit that limit and now I can't log in.  Whenever I try, it says it failed because it couldn't write anything.  I need to get in there and free up some space and maybe change how many revisions it keeps.  But first I need to be able to log in.  Anyone know how I can remove some files so I can actually change how many revisions it keeps?

I've been trying to research this for about a week now and so far nothing has worked, because I don't know how to change typemaps in P4V.  Everything is given in command line commands and I have no idea how to actually use them in the visual client.  I also don't know if obliterating old files is a good idea.  I really don't want to start over from the beginning, so I'm holding that as a last resort.

Any help would be greatly appreciated.

#2 P4Jen

P4Jen

    Advanced Member

  • Staff Moderators
  • 141 posts

Posted 22 January 2019 - 04:08 PM

Hi,

From your post I assume that you have run out of disk space on the Helix Server.

P4V itself does not store multiple revisions of the file, but if you have a personal server (DVCS) this
will store multiple revisions of files on your computer.

It will also need to create temporary files.

It sounds like the '/tmp' directory may be full.

If this is the case then you will have to either remove some data for it, or put '/tmp' on a larger volume.

You can use the command 'df -h' to check how much space is left on the disk, or use the 'Disk Utility' Mac
application.

Also, this KB article provides some suggestions for what you can do if you run out of space:
  https://community.pe...s/article/15322

Hope this helps,
Jen.


#3 EDarkness

EDarkness

    Newbie

  • Members
  • Pip
  • 6 posts

Posted 22 January 2019 - 07:05 PM

I'm using AWS servers for my repository and have 30GB that is completely used up.  When I checked it literally says it has 0 space.  Which is a problem, because I can't really do anything and can't expand what I have without messing with my server instance.  Which is why I was wondering what could be deleted or removed so I could at least log in and maybe obliterate some old files that aren't necessary anymore.  I made the mistake of not putting a limit on how many revisions and didn't think about it when I first started using Perforce.

#4 Sambwise

Sambwise

    Advanced Member

  • Members
  • PipPipPip
  • 727 posts

Posted 22 January 2019 - 09:57 PM

Do you have any checkpoint.N and journal.N files in your server root?  If so, move those out of there.  They're only for backup purposes and don't need to live in the active server root.  That'll probably free up enough space that you can at least start up the server.

The log file is another good candidate for removal if you need a bit of space in a pinch.

I also suggest that now is a really good time to get over your fear of the command line -- I would not wish the experience of trying to figure this out via P4V on my worst enemy.  :)

#5 EDarkness

EDarkness

    Newbie

  • Members
  • Pip
  • 6 posts

Posted 23 January 2019 - 02:27 AM

View PostSambwise, on 22 January 2019 - 09:57 PM, said:

Do you have any checkpoint.N and journal.N files in your server root?  If so, move those out of there.  They're only for backup purposes and don't need to live in the active server root.  That'll probably free up enough space that you can at least start up the server.

The log file is another good candidate for removal if you need a bit of space in a pinch.

I also suggest that now is a really good time to get over your fear of the command line -- I would not wish the experience of trying to figure this out via P4V on my worst enemy.  :)

You know where those files would be generally?  In my Perforce folder, I only see a ton of db.X files and a file named Journal.

Also, I don't use Linux.  I use my Mac to do my game development on.  So the visual client is the best way to do things.  I assume that the visual client is supposed to do everything, however.  Even so, I'm not even sure where I would even use the command line stuff in the first place.

#6 Sambwise

Sambwise

    Advanced Member

  • Members
  • PipPipPip
  • 727 posts

Posted 23 January 2019 - 05:02 AM

View PostEDarkness, on 23 January 2019 - 02:27 AM, said:

You know where those files would be generally?  In my Perforce folder, I only see a ton of db.X files and a file named Journal.

Generally they'd be in the same folder.  Do you have shell access to this server?  (I haven't used AWS so not familiar with how tightly it's locked down.)  If so you could truncate the journal with "p4d -jj" and then archive it off.

Quote

Also, I don't use Linux.  I use my Mac... I'm not even sure where I would even use the command line stuff in the first place.

Cmd-Space -> "Terminal".  Welcome to the dark side!

#7 EDarkness

EDarkness

    Newbie

  • Members
  • Pip
  • 6 posts

Posted 23 January 2019 - 04:06 PM

When I look at the Linux files on the server, I just see p4d, a bunch of db files, a file named "journal", and another file named "server.locks".  I know that p4d is an executable file and it runs the server.  Not sure if there are supposed to be other files.

I looked around in P4V and I don't see anywhere to change into a command line.  Is there another program I should have that does this?

#8 Sambwise

Sambwise

    Advanced Member

  • Members
  • PipPipPip
  • 727 posts

Posted 23 January 2019 - 05:28 PM

Hold down the key on your keyboard that looks like a clover (aka the "Cmd" key) and hit the space bar -- a little box you can type into will pop up.  Type "Terminal" into it and hit the Enter key.

You should now have a larger box you can type into -- that's the command line terminal.  Type:

p4 set P4PORT=your.server.address:port
p4 set P4USER=your.user.name
p4 set P4CLIENT=your.workspace.name

(but type your real server address, username, workspace, etc instead of the placeholders I used there)

Now you're all set to use Perforce command line commands, which will be useful the next time you need to do something that you can't do in P4V.  Unfortunately, most of those commands won't work while your server is jammed up due to zero disk space.  Double-check your settings by running "p4 info" -- that should give you something indicating it can at least talk to the server (either you'll get valid connection info output or you'll get a disk full message -- if you get a "connection failed" message you set the wrong P4PORT, so fix that and try again).

If you don't have shell access to the AWS instance (you didn't answer this question so I assume the answer is no -- if the answer is yes or if you didn't understand the question, stop and ask more questions), time to do something a little unsafe -- delete the journal file.  Make a safe copy of it somewhere else if you're able to do that.  (If you can't log in to the AWS instance to run normal admin commands your next option is going to be to just abandon it and spin up a new one, so I guess this is a better option.)

NOW, go back to the terminal and you should be able to run real commands.  Start with:

p4 admin checkpoint

If you just deleted the journal and you've been running the server for a while, the checkpoint should be a good bit smaller than the journal was, so I assume this will work fine without running out of space.  If you can, copy the new checkpoint off of the AWS instance (put it somewhere safe where storage space is less tight) and then delete it from AWS.  Now you're back in a safe state where you can easily recover if something bad happens to the db.

Your next step in freeing up space is going to be careful use of "p4 obliterate".  You can identify which files are eating the most space with the "p4 sizes" command:

p4 sizes -a //...

Is that too much data to sift through?  This is where the command line is awesome.  Change the command a bit:

p4 sizes -a -b 1048576 //...

and now you see an extra "N blocks" on each line of output -- we gave a block size of 1 MB so anything that's under 1MB is going to say "1 blocks".  Let's filter those out with our friend grep:

p4 sizes -a -b 1048576 //... | grep -v "1 blocks"

oh snap, now we're only seeing revisions that are more than a megabyte!  (Is that still too many?  Nudge the block size up -- maybe you only care about things that are over 10MB, or 100MB, I dunno.)

So now maybe you see some output like:

//depot/main/foo.mov#4 16383731 bytes 15 blocks
//depot/main/foo.mov#3 16389331 bytes 15 blocks
//depot/main/foo.mov#2 16319731 bytes 15 blocks
//depot/main/foo.mov#1 16389731 bytes 15 blocks

and you're thinking "man, I really don't need those first two revisions of foo.mov".  Here's how you get rid of them:

p4 obliterate -y "//depot/main/foo.mov#1,#2"

Buh-bye forever!  (I hope you were really certain you didn't need those revisions, because there is no way to get them back now.  Be careful with obliterate!)  And if you want to never keep more than two revisions of foo.mov from here on:

p4 retype -t +S2 "//depot/main/foo.mov"

The "+S2" means "store 2 revisions" -- when you submit a new revision in excess of 2, the oldest one will get "purged" from the archive automatically.

Now that you've mastered the command line, you'll be able to follow directions in other KB articles etc.

Good luck!  :)

#9 Sambwise

Sambwise

    Advanced Member

  • Members
  • PipPipPip
  • 727 posts

Posted 23 January 2019 - 06:46 PM

In reflecting on how overwhelming the above must be to someone who's never done any system administration before, I'd like to take this time to give a shout-out to all the hardworking Perforce admins at various game companies who handle these things magically behind the scenes so their users can check in giant level files from their GUI clients without every having to even think about things like server disk space.  :wub:

#10 EDarkness

EDarkness

    Newbie

  • Members
  • Pip
  • 6 posts

Posted 23 January 2019 - 07:45 PM

Honestly, I thought you meant a terminal in P4V, not the basic Terminal that comes with Mac OS.  Heh, heh.

Anyway, I think I'm missing something fundamental.  Whenever I try to do any P4 stuff anywhere, it always says:

-bash: p4: command not found

There's no way to do any P4 anything.  If by shell access you mean "can I log into the AWS server and do stuff via ssh?"  Then, yes.  I put the P4D file on my AWS server and ran it.  I opened up a screen so that it runs even if I'm not logged into AWS server.  On my local machine, I don't have anything going on with Perforce.  I just do everything via P4V.  It accesses the server and I do what I can from there.  I'd much rather do it this way than messing with command line stuff.  If the P4V program doesn't do everything (and it SHOULD do everything), then they should really go back and add the stuff it's missing.  What's the point of the visual client if it doesn't do what you want it to do?  Anyway, I apologize if this comes off as a little rough.  I'm just a little frustrated since I just need to delete some stuff to make space (it's a simple thing), but it's taken me over a week to get it done.  I just don't think this should be that hard.

I also want to add that I REALLY appreciate the help.  This has been driving me crazy.

#11 Sambwise

Sambwise

    Advanced Member

  • Members
  • PipPipPip
  • 727 posts

Posted 23 January 2019 - 10:26 PM

View PostEDarkness, on 23 January 2019 - 07:45 PM, said:

-bash: p4: command not found

Ah!  Install the p4 command line client.  (I would have thought it'd come with P4V, but I guess Mac installers don't work like that.)  If you look on the download site for "P4 command line" or something like that I expect you'll find a binary file that you can drop into /usr/bin.

Quote

If by shell access you mean "can I log into the AWS server and do stuff via ssh?"  Then, yes.  I put the P4D file on my AWS server and ran it.  

Cool!  Okay, so see if you can do:

p4d -jj

That should rotate off the current journal and bump the counter -- I don't think this actually needs any free disk space so it should be okay, and it's better than just grabbing the active journal because it'll make sure the counter is consistent with the journal file(s).  Once the journal has been rotated (it'll be renamed to "journal.0" or something), get it off AWS, and then proceed with the obliterate/retype stuff (once you've installed the p4 CLI).

Quote

If the P4V program doesn't do everything (and it SHOULD do everything), then they should really go back and add the stuff it's missing.


It's possible to use P4V for everything as an end user (IMO this is still somewhat painful but it's at least *possible*), but it's never really been practical to administer a server via P4V alone, unless you have the good fortune of being able to run the server on a machine that is both 100% reliable and has unlimited disk space (which is probably possible in the modern cloud era, now that I think of it, but it's not free).  

The typical way to use Perforce is to have a single server that's operated by one knowledgeable admin and used by a bunch of end users, so the vast majority of people can use P4V for everything, but if you're administering your own server you should expect to get your hands dirty now and then (especially if you let it go for a while without monitoring disk usage, doing backups, etc -- think of it like owning a car and expecting that you'll never have to pop the hood, check the tires, OR have a mechanic who'll do those things for you.  P4V is everything you can reach from the driver's seat, the server is everything else).

#12 EDarkness

EDarkness

    Newbie

  • Members
  • Pip
  • 6 posts

Posted 07 February 2019 - 08:04 PM

I apologize for not getting back to this sooner, but I've been extremely busy with other things and so this took a bit of a back seat.  Even though, it's is by far my most pressing matter with my game.

After messing around with this, I still haven't found any files that can be deleted, so I'm in the same boat.  I haven't installed the P4 command line yet.  I don't want to do this unless it's absolutely mandatory.

Tried p4d -jj and it says "command not found".  I assume that it's either an older file or the server is missing something for that to work.  The journal file is HUGE, though.  Removing that would free up a bunch of space.  Just not sure how to delete it.  I tried moving it to my local machine, but it won't move and can't be copied.  Due to the size of it, I won't be able to make a copy of it on the server, either.  I may have to take a leap of faith and just delete it without a backup.  Not sure what will happen if that happens.  Is this file necessary?




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users