Rabbit's Warren

I’m working on a new web page that will serve as an introduction to the JFK Assassination Record Collection (ARC) at the National Archives and Record Administration (NARA). My guess is that I can have it up sometime next week. I had a fair number of misconceptions about the Collection, so I hope writing an introduction will improve my understanding of this unusual historical resource. In the meantime, I will resume writing about NARA’s 2017 releases from the ARC, and the additional releases which I believe will happen this year.

[Text revised and postscript added 2018-03-13]
I’ve been doing some record counting at the Mary Ferrell website (MF). This is possible through a tool added to MF in 2016, called the JFK Database Explorer. According to the FAQ for the Explorer, it is based on a copy of NARA’s on-line finding-aid database for the JFK Assassination Record Collection.

The copy was made by Ramon Herrera in the summer of 2015 and has 319,106 records.¹ Surprisingly, MF quotes NARA’s Martha Murphy, as saying that NARA’s finding aid database has only 318,866 records, 240 fewer than Herrera’s copy. MF then observes that ‘the reason for this discrepancy is as yet unknown.’

Total collection size is of course a very useful thing to know, and on this page of the Explorer, MF gives a helpful break-down of total records counts in the Explorer by prefix. NARA assigned numbers to each agency providing records to the Collection, and it uses these as prefixes to the record numbers for each item in the collection. There are 37 prefixes, from 104 (CIA) to 208 (Senate Permanent Subcommittee on Investigations).

Adding up the number of records MF gives for each of these prefixes, however, the total is 318,733, which matches neither the Explorer total given by the FAQ, nor the NARA total given by Murphy.

I would be curious to know the story on this. I went through the sub-pages for each prefix on MF; these provide a further break-down ‘disk-by-disk’, using record counts on each floppy disk that NARA used to get record information from each agency. This gave me the same figure: 318,733. (I’d rather not count the individual records one by one.) Perhaps I should write to Mary Ferrell as well as NARA.

Postscript:

Regarding the alleged discrepancy between the number of records that Herrera scraped and Murphy’s estimate quoted by MF, NARA now states that there are 319,106 records in its on-line finding-aids database.² This number matches the count MF gives for Herrera’s copy. I checked the Wayback machine, and NARA’s new estimate was apparently added at the end of October 2017.³ The NARA page also now says that the finding-aid database was last updated May 12, 2008. This is puzzling, and I will comment on it in the not too distant future.

I am concerned that I have gotten several things wrong in recent posts, so for now I am going to stop posting about the JFKRA releases at NARA. It will probably take a while to figure out what’s what. This post is a brief note on my concerns.

Catalog issues

In my last post on the the NARA releases, I was looking at NARA 18. This list shows the records in the JFK Assassination Record Collection (JFK ARC) that are not yet released in full.

I checked these against the copy of the Collection’s on-line catalog at the Mary Ferrell website, and found that 298 records which NARA 2018 listed as available in redacted form were listed as “Postponed in full”. Mary Ferrell’s copy of the catalog, which they call the JFK Explorer, is over two years old, so I checked these records again using the current on-line catalog at NARA, and the on-line catalog gave the same result.

I am reasonably certain that all of this just means that Mary Ferrell’s JFK Explorer and the current version of the catalog on line at NARA are out of date, and that redacted versions of these documents are actually available at NARA. To really find out, of course, one has to order some of the documents in question. I’m willing to do that, as long as it doesn’t involve hocking my left, er, leg, but it will take a while for the results to come in, so a pause is in order.

Other problems

I’m also concerned that my post before that, where I discussed document fragments in NARA’s 2017 releases, may have some misconceptions. In particular, I attributed the RIF numbers on these documents to NARA’s cataloging. I now think this is quite wrong.

As I understand things now, these numbers, and the RIF sheets that include the various document metadata that NARA provides, were not all done by NARA. Many of them were done by the US government agencies that produced, or had custody of, the documents when NARA tracked them all down.

In particular, as NARA archivist James Mathis told Mary Ferrell, “the Federal Bureau of Investigation did not list page numbers for documents in the Database that were declared Not Believed Relevant (NBR) and indeed often used one entry for multiple documents.” (see here) I interpret “used one entry for multiple documents” to mean ‘used one record number for multiple documents.’

If it is not NARA putting these duplicate RIF numbers on documents, it makes a difference in how one treats them. For now, please put a star on that particular post; the files which I indicated are pieces of a single document still go together, but credit or blame for this situation remains to be assigned.

In addition to being out of date, I am also now sure that the current ARC catalog online at NARA is not complete. More documents are available from the JFK Assassination Records Collection at NARA than the on-line catalog indicates. One example: the on-line catalog lists no FBI documents with the prefix 124-10203, yet there are several hundred of these on line at Mary Ferrell.

I wrote to NARA about this, and would like to thank them here for their courteous and prompt responses. The short story: 124-10203 was the number of a floppy disk containing electronic file(s) listing up to 500 FBI documents. This disk was sent to NARA along with printed versions of the listings (the RIF sheets), and the documents themselves. The documents and the rif sheets arrived, but the disk was bad (corrupt). Because of this, the metadata on these documents were never added to the on-line catalog.

In addition to 124-10203, this also happened to disks 124-10204 and 124-10223, so as many as 1500 records do not have metadata in NARA’s on-line catalog. I checked at Mary Ferrell, and they list 1412 records with these three prefixes. I believed something like this happened in a few other cases as well, though probably not as many records are involved.

All of this has affected my figuring, and my estimates of what I can do with the information available on line. So for now a pause, until I can make sure I have a proper understanding of the ARC and its reference tools.

[See the postscript added on 3/25/2018]

I have finished looking at the 3082 records which are listed in NARA 2018, but not in NARA 2017, as discussed by John Greenewald (see earlier post). Unfortunately, I have not been able to access NARA’s JFK Records Collection catalog for over 5 days now. For whatever reason, I have usually found access to the NARA catalog on again off again, but this is really too much; for almost a week my session is killed or the search results are 0 for every query, using different networks, computers, and browsers. Just frustrating as all hell.

As a result, I have relied for this count primarily on the copy of the JFK Collection catalog at Mary Ferrell. Mary Ferrell is always reliable, and my subscription was a truly worthwhile investment. Unfortunately, the MF copy of the catalog is stale; it was done over two and a half years ago by Ramon Herrera, according to the MF FAQ. This means my result will almost certainly change, but at least I can shrink the amount of queries I have to make to the unreliable NARA system to a minimum.

I have found 298 records which are listed as ‘redact’ on NARA 2018 but as ‘Postponed in full’ on the MF copy of the Collection catalog. 136 are HSCA records, and one group, 180-10068, accounts for 66 of these. Looking at the description, these 66 are all payroll records for the HSCA staff. Similar records for the ARRB were also withheld for privacy reasons as I noted in a previous post, so perhaps these really are withheld. In any case, I would like to be able to check on the current NARA catalog.

There are other anomalies as well; 9 records are listed as ‘OPEN’ on the MF copy of the catalog; I thought this meant no deletions; if so, this also contradicts NARA 2018, which is supposed to be a list of records with deletions.

On the other side of the coin, 3 of the records which NARA 2018 lists as ‘Withheld’ are actually present on NARA 2017, and were indeed released last year.

My conclusion on this whole thing is that NARA 2018 is not the final answer to whats up and what’s not.

Postscript (3/25/2018):
I have a later post up that gives different numbers. This is in part because the new post looks at NARA 2018 after removing duplicate records

This is another post on the JFK Records Act Release 6 at NARA. To get up to speed on the subject, I suggest looking at the other posts in this category (JFK ARCA).

As I noted at the end of a recent post, NARA’s spreadsheet for JFK record act release 6 lists a number of records where there are two different files having the same record number. Some of these are different versions of the same file, but in a few cases, the two files appear to be simply different documents.

As I said, this seems to defeat the purpose of the record numbers, which are supposed to be unique for each document in the collection. Is there really a reasonable explanation? Yes, Virginia, there is!

Bit and pieces of files

This link has a list of six cases where NARA seems to do this, assigning the same record number to two files with large differences between them. But a closer look shows that these documents are closely related.

Take the case of 124-10328-10025. This number is assigned to two separate pdf files: one from release 5 with 4 pages, and one from release 6 with 20 pages. When you go through them carefully, however, the two are actually one document: a 1958 report by FBI agent Malcolm Carr. In the report, Carr details his surveillance of Rafael Medan, an assistant press director for the Israeli delegation to the United Nations.

The report is captioned ‘Espionage Case’, but there’s not really enough information to tell us what is going on. (I’ll try to explain later on why this report is in the collection.) The 20 page document (19 minus the cover sheet) includes the original report and a carbon copy. The carbon copy is complete, but the original is missing 3 pages. The 4 page document (3 minus the RIF sheet) is the missing 3 pages of the original report.

Why was the document released in two pieces? Perhaps this is just how NARA got it from the HSCA. No need to ask too many questions; by statute, all documents from the HSCA are automatically part of the collection, so into the collection it goes. The only real question is how do we handle these two chunks of one document? NARA’s answer is to release it in two pieces, but to index it as one document. Index here means to give it the same document number.

124-10173-10382 is also one FBI document in two pieces: one pdf from release 5 with 7 pages, and one pdf from release 6 with 11 pages. The 7 page pdf includes a RIF sheet, a JFK ARCA “cross-reference” sheet, an administrative memo (I think) that gives the name of informants and is mostly blacked out, and a letterhead memo which is completely blacked out except for the date. Then a postponement sheet that says pages 2 and 3 are not here. Then a page marked ‘3’ at the top and blacked out except for the middle, which says that (name blacked out) claimed that Oswald worked for the FBI. Then another postponement sheet that says 8 more pages are not here. Notice that only one page, page 6, actually has any content. Cheez.

The 11 page pdf from release 6 is a cover sheet (105-NY-66954 sec 11), then a ten page 302 form, the record of an interview with Walter George Sikora. This is the part that the pages in the 7 page pdf postponed (no 2 pages, no 8 pages). It does not have the administrative sheet, the LHM, or the page mentioning the claim about Oswald, but clearly this is something that must have come up in the course of interviewing Sikora. Like the two chunks of the report about the surveillance of Medan, NARA gives these two files the same record number.

The other documents in the list are even more complicated. For example, 124-10193-10031 and 124-10193-10032 are two related documents put together from 4 chunks, with bits of each document distributed across all four chunks, and all the pages out of order. 124-10164-10276 and 124-10164-10277 are two documents put together out of three chunks, one of them a humongo 571 pages.

Some thoughts

I take all this as evidence that NARA sometimes uses one record number (RIF#) for multiple files when it determines that all the files are fragments of a single record. This is very distinct from the other cases we looked at before, when the same number was assigned to two files that were trivially distinct from each other (different case nos, with RIF without RIF etc).

In fact, this multiplicity of files with the same record number occurs in material on Mary Ferrell as well. In what must be an extreme case, I have found one record number, 124-10289-10035, which is assigned to 10 different files. Can these all be pieced together into one large file? Possible, I think, but a tough job; there are 848 pages in these 10 files.

On the other hand, the fact that these different files have the same RIF# tells us that NARA believes they belong together. This is an essential aid. If we non-adepts had to do this unassisted, we are in the same spot as the Chinese archaeologists trying to reassemble the tortoise shells used to record royal divinations, after they have been broken up and sold to herbal doctors as dragon bones.

Another more general thought after going through this material is how hard it is to understand these documents without context. The Carr report on Medan is a typical example of this problem. Why is this an assassination record? The answer is to look up the record number on Mary Ferrell and see where it came from. The Carr report, in a very mutilated condition, in the HSCA’s FBI subject files, under the name of Joseph Shimon. Shimon is indeed mentioned in the report, in a couple of places, but without checking I would never have suspected that his mention in this 1958 report is what made it an assassination record.

On similar lines, it is also hard to avoid the thought that for those interested in the JFK assassination, the amount of dross, as opposed to silver, in these materials is very very high. The 10 page interview with Walter George Sikora that omits the remark about Oswald is marked NAR: not assassination related. This FBI annotation is not arbitrary; it is apt.

I am still looking at NARA 2018, the new list of JFK records from NARA which The Black Vault website posted January 29. It is interesting and helpful information, but as I said in my last post, I don’t think the list indicates there are still over 3000 JFK records that have never been released.

Jefferson Morley, owner of the JFKFacts blog, posted a note I sent to him on the problem (here), and noted that Rex Bradford at the Mary Ferrell Foundation also does not agree with the figure of over 3000 unreleased records.

The Black Vault’s John Greenewald has since written more on the subject, in a comment posted at JFKFacts (link here).

I will post again when I have finished going over the 3082 records in NARA 2018 which did not appear in NARA 2017, the spreadsheet for the 2017 releases of JFK records. The 798 ‘withheld’ records in NARA 2018 are also worth a separate post.

Another list from NARA has been posted, this time at The Black Vault. The link can be found here.

According to the link, the list is the result of an FOIA case (#NARA-NGC-2018-000072), and the BV was nice enough to provide both a pdf and excel version of the list, very handy indeed. The list seems to be NARA’s latest assessment of the state of the JFK Assassination Records Collection. BV cites a letter from NARA which states:

We conducted a search and were able to locate an EXCEL spreadsheet that lists everything that has not been released since December 15th, 2017 (the last release date). We are releasing this document in full with no redactions. The spreadsheet lists the JFK record number, the decision, the file number, document date, number of pages, and the origination agency.

The list totals 22933 rows (excluding the field list at row 1). Counting up, 798 rows are listed as ‘Withheld’, the remaining 22135 are listed as ‘Redact’.

I interpret this as follows: ‘Withheld’ means the document has not yet been released. ‘Redact’ means the document has been released, but not in full. (As the 2017 releases show, these redactions can be anywhere from 2 letters to whole pages.) I also think the list is intended to be comprehensive. This is a description of the entire JFK ARC collection.

I thought this was a straightforward explanation, but BV has a different interpretation. They write:

However, upon investigation, NARA also listed the entire set of PARTIALLY released records, along with those completely withheld. Digging deeper, and with the help and verification of Jimmy Falls of the news agency WhoWhatWhy we came up with the same numbers, using two entirely different methods. It confirms there are 3,082 Documents, totaling 217,114 Pages that are not yet released to the public.

I was quite puzzled by this and left a note on the BV forum asking about this. I haven’t yet seen their response, but will post if they do. In the meantime, I have probably figured out what they did. They compared the new list, (hereafter NARA 2018) to the release 6 spreadsheet posted at NARA here (hereafter NARA 2017). There are indeed 3082 records on NARA 2018 which do not appear on NARA 2017, and using the number of pages listed in NARA 2018, these add up to 217,114.

The 3082 figure includes all of the 798 records that are ‘Withheld’, and these are indeed not yet released, I believe. The remaining 2284 records were not posted at archives.gov in 2017, but most, if not all, of these records were released long ago, as indicated on the NARA on-line catalog of the JFK Assassination Record Collection. Many of them are available at Mary Ferrell. As an example, all of the records in the group ‘124-90104’ which appear in NARA 2018 but not in NARA 2017 (a total of 70) are up at the MFF.

Of the total of 2284 records, all the ones I have checked so far are listed at either Mary Ferrell or on the NARA catalog as ‘Released with deletions’. It would be nice to see the deleted material, but one cannot say that these records are not yet released.