Discussion: Common file format for magazine data?

cccmagfan · February 13th, 2008, 06:44 PM

Hi,

Here are my thoughts on a unified file format for data exchange. I already have some experience, and my current files use a similar structure. But I'm sure that there are important things that I don't think about.

It should be a text only format, preferred 8-bit western encoding, but I don't know how importand eastern-europe characters could be. The data format should be tab-separated, CSV is a mess regarding commas and quotes within values. Any relational information must be flattened, so each entry must carry a record type. I don't know how fussy you folks want to be, breaking down the content to photosets would be OK, but in case somebody wants to collect information on individual photos, I include this option in my suggestion. I try not to into details like numebr and date formatting.

Record types
M=Magazine
X=Extra Content (related to M)
S=Photoset (related to M)
P=Photo (related to S)
C=Cast (can be related to S and P)
...

Magazine
-Short reference (like CCC-BC-50)
-Publisher
-Distrubutor
-Series Name
-Magazine Title
-Magazine topic
-Seq. Number
-Major Number
-Minor Number
-Date: YYYY-MM-DD / YYYY-MM
-EAN (or other product codes)
-Pages
-Number of Photoset

Photoset
-Title
-Pagerange
-Original appearance
-Photoset topic

Model
-Gender F/M/X
-Name
-Alternate names
-Comments

Photo
-Page(s)
-Location on page (top/bottom/left/right/double/centerfold...)
-Content (e.g. what is inserted where)

Example, for file structure
M ... (magazine date)
S Photoset 1 ...
C F...
C M...
X Magazine index
S Photoset 2 ...
C F...
C F...
S Photoset 3 ...
M ... (next magazine)

This is just what I have in mind. Any comments and additions, especially regarding properties for the records is welcome. Just keep in mind that we might not succeed, so don't take the discussion too serious.

Right now I am using text files, sometimes fixed format, sometimes with "|" as field separator, and I use indentation instead of record types. Makes it easier for human reading, but harder to parse. So I'm not trying to "sell" you something I already have. A lot of conversion can be done with PERL Scripts, editing can be done with spreadsheet applications that can import/export TAB-Separated text. Question to the data collectors: What tools and skills do you have/use?

In this very special forum context, I would say: My *mind* is open for anything...

**9876543210** · February 14th, 2008, 03:48 AM

cccmagfan,

I'm not exactly positive what your asking here, but, if your asking the best way to transfer files between different types of databases, then your probably best to use comma delimited text files in Notepad. Any PC database can read and import/export them. Even going back to an old DOS database app. Even Apple and Linux databases should be able to read them although I don't think Apple will read Notepad.

Hope that helps.

**flateric** · February 14th, 2008, 07:43 AM

No - I'm not exactly clear what is being discussed here - are you proposing a way of standardising *lists* of pictures to make it easy for people to find what they are looking for? or a way of naming the files themselves.

You original message shows a good deal of careful thought so I'd be keen to see whether your ideas might be useful to us all

cccmagfan · February 14th, 2008, 08:30 AM

What I have in mind is to suggest a standard format to describe the content of a magazine. This can have several advantages: Current "data owners" can export and exchange their existing data, and users can post entries, and the data collectors can import this data. Maybe even from a plain text forum message. I'll try to create a working example for one magazine.

With this in mind, TAB as "invisible" character might be a bad separator. But I'm not that happy with comma, because it is a common character. That's why I also want to know about the skills and tools of the users. In MS Excel you can just paste text in a sheet and choose "data - text to columns" and enter a custom delimiter. Then my favourite would be the pipe / vertical bar "|". With too many values, it's just a bit hard to keep track of the columns, but spaces around the sepchar should be OK.

The end-of line mark is a technical detail (CR for Mac, LF for UNIX, CR/LF for Windows).

The intention of the discussion is to collect information, what properties are important. Each magazine series has its little special features. The vintage CC have some of these, like the T* Galleries. But also advertizing content might be interesting, such as video announcements where you can sometimes read the runtime from the cover, or collection of back issues cover shots, or caompany history like in the CC150ff issues.

There are several approaches: Page based and story based. My latest own apporach is to mix both: Entries for photosets with page range including TOC and cover appearance like: "3,4-21,100" for TOC, Story and backcover. Then add page based data for all extra content like: "2: Model request", "62-63: Magazine Index '95/96".

February 13th, 2008, 06:44 PM	#1
cccmagfan Senior Member Join Date: Feb 2008 Location: Germany Posts: 161 Thanks: 98 Thanked 1,062 Times in 160 Posts	Discussion: Common file format for magazine data? Hi, Here are my thoughts on a unified file format for data exchange. I already have some experience, and my current files use a similar structure. But I'm sure that there are important things that I don't think about. It should be a text only format, preferred 8-bit western encoding, but I don't know how importand eastern-europe characters could be. The data format should be tab-separated, CSV is a mess regarding commas and quotes within values. Any relational information must be flattened, so each entry must carry a record type. I don't know how fussy you folks want to be, breaking down the content to photosets would be OK, but in case somebody wants to collect information on individual photos, I include this option in my suggestion. I try not to into details like numebr and date formatting. Record types M=Magazine X=Extra Content (related to M) S=Photoset (related to M) P=Photo (related to S) C=Cast (can be related to S and P) ... Magazine -Short reference (like CCC-BC-50) -Publisher -Distrubutor -Series Name -Magazine Title -Magazine topic -Seq. Number -Major Number -Minor Number -Date: YYYY-MM-DD / YYYY-MM -EAN (or other product codes) -Pages -Number of Photoset Photoset -Title -Pagerange -Original appearance -Photoset topic Model -Gender F/M/X -Name -Alternate names -Comments Photo -Page(s) -Location on page (top/bottom/left/right/double/centerfold...) -Content (e.g. what is inserted where) Example, for file structure M ... (magazine date) S Photoset 1 ... C F... C M... X Magazine index S Photoset 2 ... C F... C F... S Photoset 3 ... M ... (next magazine) This is just what I have in mind. Any comments and additions, especially regarding properties for the records is welcome. Just keep in mind that we might not succeed, so don't take the discussion too serious. Right now I am using text files, sometimes fixed format, sometimes with "\|" as field separator, and I use indentation instead of record types. Makes it easier for human reading, but harder to parse. So I'm not trying to "sell" you something I already have. A lot of conversion can be done with PERL Scripts, editing can be done with spreadsheet applications that can import/export TAB-Separated text. Question to the data collectors: What tools and skills do you have/use? In this very special forum context, I would say: My mind is open for anything...

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

February 14th, 2008, 03:48 AM	#2
9876543210 Veteran Member Join Date: Nov 2007 Posts: 4,044 Thanks: 24,638 Thanked 34,288 Times in 4,008 Posts	cccmagfan, I'm not exactly positive what your asking here, but, if your asking the best way to transfer files between different types of databases, then your probably best to use comma delimited text files in Notepad. Any PC database can read and import/export them. Even going back to an old DOS database app. Even Apple and Linux databases should be able to read them although I don't think Apple will read Notepad. Hope that helps.

February 14th, 2008, 07:43 AM	#3
flateric Vintage Member Join Date: Apr 2006 Posts: 373 Thanks: 571 Thanked 12,028 Times in 343 Posts	No - I'm not exactly clear what is being discussed here - are you proposing a way of standardising lists of pictures to make it easy for people to find what they are looking for? or a way of naming the files themselves. You original message shows a good deal of careful thought so I'd be keen to see whether your ideas might be useful to us all

February 14th, 2008, 08:30 AM	#4
cccmagfan Senior Member Join Date: Feb 2008 Location: Germany Posts: 161 Thanks: 98 Thanked 1,062 Times in 160 Posts	What I have in mind is to suggest a standard format to describe the content of a magazine. This can have several advantages: Current "data owners" can export and exchange their existing data, and users can post entries, and the data collectors can import this data. Maybe even from a plain text forum message. I'll try to create a working example for one magazine. With this in mind, TAB as "invisible" character might be a bad separator. But I'm not that happy with comma, because it is a common character. That's why I also want to know about the skills and tools of the users. In MS Excel you can just paste text in a sheet and choose "data - text to columns" and enter a custom delimiter. Then my favourite would be the pipe / vertical bar "\|". With too many values, it's just a bit hard to keep track of the columns, but spaces around the sepchar should be OK. The end-of line mark is a technical detail (CR for Mac, LF for UNIX, CR/LF for Windows). The intention of the discussion is to collect information, what properties are important. Each magazine series has its little special features. The vintage CC have some of these, like the T* Galleries. But also advertizing content might be interesting, such as video announcements where you can sometimes read the runtime from the cover, or collection of back issues cover shots, or caompany history like in the CC150ff issues. There are several approaches: Page based and story based. My latest own apporach is to mix both: Entries for photosets with page range including TOC and cover appearance like: "3,4-21,100" for TOC, Story and backcover. Then add page based data for all extra content like: "2: Model request", "62-63: Magazine Index '95/96".