Jump to content


Which version of a file resides in Depot data

depot metadata

  • Please log in to reply
5 replies to this topic

#1 funnymonk

funnymonk

    Newbie

  • Members
  • Pip
  • 6 posts

Posted 30 October 2013 - 06:11 AM

Hi,

Perforce sores information about files in depot and one as metadata.

Metadata contains the control information of the files (changelists, branching integration info)

Depot contains actual file data. But my question was how is this data stored.

1) Which version of the file resides in depot data? Is it the the first version or the last version?
2) Is all file data for all versions stored in the depot or is the delta information stored in metadata?

Is there a detailed write-up on what goes to depot data and what goes to metadata?

Thanks.

#2 P4JohnW

P4JohnW

    Advanced Member

  • Members
  • PipPip
  • 17 posts
  • LocationVictoria, BC, Canada

Posted 30 October 2013 - 03:24 PM

Hi,

Text files are stored in RCS format using reverse deltas.  That means we store the head revision of a file with diffs going backwards all the way back to revision #1.

Binary files are stored as gzipped files (one gzip per revision).

All file contents are stored in these archive files.  No file content is stored in our database.  The database contains only file information (changelist numbers, revisions, authors, dates, integration history, etc.).  Of course there are pointers in the database to where the file archive is stored.

I can't seem to find any documentation that explains the two parts of the Perforce server (the database and the archives) and what goes into each and how they interact with each other.

John

#3 Robert Cowham

Robert Cowham

    Advanced Member

  • PCP
  • 271 posts
  • LocationLondon, UK

Posted 30 October 2013 - 04:59 PM

The best overview I have found in official docs is:

http://www.perforce....ro.html#1093183

Quote

The Perforce versioning service manages shared file repositories, or depots. Depots contain every revision of every file under Perforce control. Perforce organizes files in depots into directory trees, like a large hard drive. Files in a depot are referred to as depot files or versioned files The service maintains a database to track change logs, user permissions, and which users have which files checked out at any time. The information stored in this database is referred to as metadata.

Personally I always refer to a single repository which is managed by a single server. The repository consists of one or more depots. Otherwise it is a pretty good description :)
Co-Author of "Learning Perforce SCM", PACKT Publishing, 25 September 2013, ISBN 9781849687645

"It's wonderful to see a new book about Perforce, especially one written by Robert Cowham and Neal Firth. No one can teach Perforce better than these seasoned subject matter experts"
  • Laura Wingerd, author of Practical Perforce, former VP of Product Technology at Perforce

#4 funnymonk

funnymonk

    Newbie

  • Members
  • Pip
  • 6 posts

Posted 04 November 2013 - 06:20 AM

Thanks a lot John and Robert.

By your replies, i understand that actual file content is stored in the Perforce depot(Archives). If its a text file, the latest version of the file (head revision) will reside on to the server with previous revisions upto the first revision stored as diff's. If its a binary file, each and every revision will be stored as zip file. Thats why i guess, binary files should be cautiously used in Perforce.

Secondly, all information other than the actual file content will be stored in Perforce metadata (Database) server. This database will contain just the pointers as to where the file content resides.

Is this understanding right?

Thanks.

#5 Robert Cowham

Robert Cowham

    Advanced Member

  • PCP
  • 271 posts
  • LocationLondon, UK

Posted 04 November 2013 - 04:25 PM

View Postfunnymonk, on 04 November 2013 - 06:20 AM, said:

By your replies, i understand that actual file content is stored in the Perforce depot(Archives). If its a text file, the latest version of the file (head revision) will reside on to the server with previous revisions upto the first revision stored as diff's. If its a binary file, each and every revision will be stored as zip file. Thats why i guess, binary files should be cautiously used in Perforce.
You have the basics correct, read up on the finer details here:

http://www.perforce....file.types.html

I would also question the "caution with binary files" comment - Perforce is as good or better at storing those as any other system, including the most common which is typically a fileshare with various "randomly" named directories! Perforce has a very strong presence in games development companies due to its abilities to deal with terabytes of art, video and other binary assets that typically form part of the history of the development of any modern game these days.

View Postfunnymonk, on 04 November 2013 - 06:20 AM, said:

Secondly, all information other than the actual file content will be stored in Perforce metadata (Database) server. This database will contain just the pointers as to where the file content resides.

Is this understanding right?

Yes
Co-Author of "Learning Perforce SCM", PACKT Publishing, 25 September 2013, ISBN 9781849687645

"It's wonderful to see a new book about Perforce, especially one written by Robert Cowham and Neal Firth. No one can teach Perforce better than these seasoned subject matter experts"
  • Laura Wingerd, author of Practical Perforce, former VP of Product Technology at Perforce

#6 funnymonk

funnymonk

    Newbie

  • Members
  • Pip
  • 6 posts

Posted 05 November 2013 - 06:16 AM

Thanks for the confirmation Robert. My comment about binary files files was regarding the space requirement which increases at a much higher rate than text files. What you said is right. Perforce is capable and excellent for even binary files. But its just that as a user, we should keep the memory requirement for binary files in mind.





Also tagged with one or more of these keywords: depot, metadata

0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users