Rabbit's Warren

I have done a couple of posts about the agencies whose records are missing from the Assassination Collection Reference System (ACRS), the on-line database of finding-aids for the JfK Assassination Records Collection at NARA.¹ I emailed NARA about the problem a while ago and have recently heard back from them.

In my email, I specifically asked about the omission from the ACRS of all records from the following agencies:

• the Investigative Reports Repository (agency prefix 194)
• the National Security Council (agency prefix 145)
• the National Security Agency (agency prefix 144)
• the Immigration and Naturalization Service (agency prefix 136)
• the United States Secret Service with the agency prefix 154
• the Department of Justice with the agency prefix 117

I heard back from Gene Morris at Archives II Textual Reference Branch (whom I wrote to in Feb 2018 about the missing FBI records from disks 124-10203, 10204, and 10223). Mr. Morris replied:

I personally spot checked the records for each of the agencies you cited and they are all here and open and available for public review and they have been for some time.

So the records are indeed available. Unfortunately, it looks like the only way you can find out what’s there is to visit NARA’s College Park facility in person. According to Mr. Morris:

There are no plans to add any of the missing records to the database as we do not have the resources or funding to do so. They do appear in the Collection Register, so their existence has been made known. Actually the National Security Council records you cited are not listed there. The entry must have fallen off during an update and not been noticed. I’ve arranged to have it fixed and should be corrected next week.

As for why these records didn’t make it into the ACRS, Mr. Morris explained:

The records arrived with either the RIFs attached or at least printed out. The metadata, the information contained within the individual RIFs was placed on discs and transferred to us at the same time. As with the FBI RIFs we discussed previously, the information on the discs with the metadata was corrupted and could not be merged with our database. Given the technology of the time and the rather cumbersome process involved, it was decided to simply work around the issue. Since we had the RIFs and the paper records were open for public review, we could just treat those files like the other files not in the database.

As I said in an earlier post,² the ARC is one of the most important documentary collections for cold war history, and the ACRS is a tremendous aid for anyone who wants to do research in it. To leave the ACRS unfinished not only hobbles serious researchers, it leaves the door open to false and misleading claims about the content of the ARC. Such claims have already appeared more than once.

Having gone through tons of ARC release trivia, I’m finally looking at what it all adds to our knowledge of the JFK assassination. This post begins with a look at a December 2017 article by Max Holland, who makes many important points on this subject.

Background

The JFK Assassination Records Collection Act of 1992 (ARCA) created the JFK Assassination Records Collection (ARC), now the largest existing archival collection of government documents on the 1963 assassination of President John F. Kennedy and the federal government’s investigation of the assassination. The Collection is stored and administered by the National Archives and Record Administration (NARA).

Many of the records in the ARC were created by federal intelligence and law enforcement agencies, and as such were protected from public release. The ARCA, however, mandated that the vast majority of the Kennedy assassination records would be published in full, and set a deadline of 25 years from the passage of the Act for complete release of all records in the Collection.¹

Not content to let the public wait for 25 years to see much of this material, the ARCA also established an indepdent federal agency, the Assassination Record Review Board (ARRB), to begin immediately processing the records for public release, and to seek out additional assassination related materials in both public and private hands. From 1995 to 1998, a large percentage of the restricted materials in the Collection were in fact released in full through the ARRB’s efforts.

When the 25 year limit set by the ARCA was reached on October 26th, 2017, hitherto unreleased materials were in fact made public, amid widespread press coverage. Unfortunately, most of this coverage, including stories in major U.S. newspapers such as the New York Times and Washington Post, was of surprisingly low quality.

The reasons for this almost unrelieved journalistic confusion and error were the subject of Much Ado About Nothing, an article by Max Holland published in the now defunct news magazine, the Weekly Standard, on December 8, 2017 (available here). Holland, a contributing editor of The Nation magazine, has written extensively not just on the JFK assassination, but on the Cold War era as a whole.

Holland’s work is particularly notable for his effective use of ARC documents, often in areas where they are important and relevant, yet have been ignored or neglected.² Holland thus offers some of the best-informed commentary available on the ARC, and his article is well worth reading.

Holland on the ARRB review of the ARC

One of Holland’s most important points is how news coverage of the releases either ignored or misunderstood the ARRB’s review of the ARC documents.

Looking back at the news stories of October 2017, this is especially baffling. With the exception of a few local stories in Minnesota interviewing former ARRB chair John Tunheim, who is still serving as Chief Judge in in Minnesota federal district court, I can’t recall a single article that interviewed members or staff of the ARRB.

Holland did not make this mistake. He interviewed Board member William Joyce and ARRB executive director David Marwell. He read the Final Report of the ARRB (and himself testified before the Board in 1995 on the subject of defining an assassination record). He spoke to NARA staff working with the ARC.

The long and short of the story: the ARRB released all assassination information in the ARC documents by 1998. The ARRB only left text redacted when it was deemed to be irrelevant to the facts of the assassination (or fit into one of the VERY narrow exemptions mentioned in footnote 1 below).

This has a rather devastating implication for the news stories which attempted to describe what was new in the ARC releases.

Cold rice fried again

This Chinese slang for newspaper stories that have passed their best if used by date is an apt description of Holland’s vivisection of the newspaper coverage of the October releases.

Holland’s discussion of ARC document 180-10110-10104 (the “Jenkins memo”) is a fascinating example of the bogus news that arose from the failure to recognize something that had been previously released. This document is a memo of a phone conversation between Johnson’s top aid Walter Jenkins and FBI head J. Edgar Hoover on 11/24/1963, immediately after JFK assassin Oswald had been shot by Dallas club owner Jack Ruby. As Holland notes, this memo featured prominently in testimony before the HSCA³ and was discussed in Holland’s own 2004 book The Kennedy Assassination Tapes (89-90).

The memo was released in part by the HSCA in 1978, and in full by the ARRB in 1998, but in spite of this, NARA released the Jenkins memo with the annotation that it was previously withheld in full. Rather than note the error, newspapers, journalists and commentators cited it as an example of important information withheld for 25 years. One could argue that these people were simply unlucky enough to have missed Holland’s book, but anyone familiar with the HSCA investigation should have spotted NARA’s error.

In this example, the newspaper coverage at least has the excuse that NARA presented the document with inaccurate information, leading those who were not familiar with the HSCA investigation to draw erroneous conclusions. In fact, document after document presented in news coverage of the releases was the same story, without any NARA error to explain why they were writing news articles about decades old material.⁴

Experts with no expertise

While Holland has harsh things to say about newspaper coverage of the ARC releases in general, his strongest criticism is reserved for people who he thinks ought to know better, professional historians and specialists in the Cold War.

Holland’s criticism of professional historians like Robert Dallek and Michael Beschloss is, I think, well taken. Both men were clearly not familiar with the material in the ARC releases and their comments were substantially uninformed. This was surprising and disappointing: the ARC documents are certainly important for the period they have specialized in, yet Dallek and Beschloss were not able to offer useful or perceptive comment on them.

Nor does Holland spare journalists. He is particularly critical of two reporters: Jefferson Morley and Philip Shenon, whom he compares to two earlier figures who wrote influential works on the JFK assassination: Mark Lane and Edward Epstein.

I have read only a limited amount by Lane, Epstein, Shenon, and Morley, so I should perhaps hold my tongue on the appropriateness of the comparisons. However, it seems to me that it would take extremely dubious behavior to match Lane. While Morley is certainly firmly conspiratorial in his views, I haven’t yet seen anything from him that would put him in the same class with Lane.

I know a little more about Epstein and Shenon, having read both Epstein’s book Inquest and Shenon’s book A Cruel and Shocking Act. These are currently the two primary accounts of the work of the Warren Commission, and Shenon’s book in particular was a disappointment. But again, I’m not sure I see the basis for the comparison between Epstein and Shenon, except that people interviewed by them later had serious complaints about the accuracy of what Epstein and Shenon wrote.

Comparing the flack Epstein took for bad quotes, and the complaints directed against Shenon, it seems Epstein is way up on the non-existent and twisted word scales. On the other hand, some of the Commission staff that Shenon criticizes are dead, and thus have said little to defend themselves.

Future of ARC research

The ARC is surely one of the most important collections for Cold War history available. As Holland says early on in his article, “Dozens of dissertations are waiting to be written based on the records [in the ARC].” Why then have professional historians and academics failed to make effective use of it? Holland does not directly answer this question. The failure of historians and specialists to appreciate its value for areas outside the narrow issue of JFK’s assassination is, however, implicit in Holland’s critique. And even on the narrow issue of the JFK assassination, the quality of research done using the ARC is often low indeed. The lure of appearing a pundit and the interest in making books more saleable are also implicit in Holland’s critique. Overall, it is a depressing picture.

In my recent posts on the JFK Assassination Records Collection (ARC), I have been looking at some of the fine (even trivial) details of the most recent release of redacted texts in the Collection. This post steps back to look at totals for all seven releases of documents in 2017-2018. Surprisingly, NARA’s totals for released in full documents and documents still redacted show significant discrepancies.

This post starts from NARA 18, a spreadsheet NARA posted with its 26 April releases of ARC documents. NARA 18 is a cumulative list of all ARC documents released from July 2017 to April 2018. Documents are identified on the spreadsheet by a column listing their RIF numbers, a unique fifteen-character number given on the Reader Information Form attached to each document. Each row on the spreadsheet is supposed to be linked to a document posted as a file on NARA’s public server, and since there are 54,637 rows (the first row is headers for each column), one could say there are 54,636 documents released.

For reasons discussed in earlier posts on the ARC, there are many, many documents listed multiple times on NARA 18. Adjusting for these, there were a total of 36576 unique RIF numbers listed on NARA 18.¹

The 2017-2018 releases were mandated by the JFK Assassination Records Collection Act of 1992, which required all documents with content withheld to be released in full by October 27, 2017. This deadline was pushed back six months by an executive order from President Trump, so that 26 April 2018 was the deadline for releasing this material. However, to make a long story short, President Trump agreed to continue postpone release of some withheld text in a number of documents, subject to further review by October 26, 2021.

The question for researchers now is which documents still contain redacted material and which have been released in full. Unfortunately, this has turned out to be a very difficult question to answer. NARA should be able to answer it, but the information it has given so far has been surprisingly contradictory.

The ARPP figures

One answer is given on NARA’s JFK Assassination Records Processing Project webpage (ARPP), a general FAQ posted following the 26 April release. The ARPP gives much valuable information, but its document totals have serious problems.

According to the ARPP, “Since July 2017, NARA has released in full 13,371 documents.” Note that this a cumulative total. In the same section, the ARPP also notes that “15,834 documents are still redacted.” The problem with this accounting is that the two figures add up to 29,205, 8731 short of the number number of documents listed on NARA 18.

Unless NARA released a large number of documents that were already released in full, I don’t understand why NARA’s total of released in full and remaining redacted documents should be thousands less than the total number of documents released.

The IG report figures

Another way to see the problem is to go back to an earlier report from the NARA Inspector-General (previously discussed here). The IG report predates the 26 April release, so it is no longer current, but it offers totals that come much closer to the actual releases than the ARPP does. Before we look at these, however, we should note a problem with the IG figures. The IG report gave a total of 34,873 ARC documents released in 2017. The actual figure should be 35,436.²

Despite this error, the IG report gives more consistent figures for released in full documents and redacted documents than the ARPP webpage does. As discussed in an earlier post on the IG report, the IG relies on NF18, a January 2018 list of redacted documents in the ARC, to provide a figure of 18,980 redacted documents remaining. It also gives a total figure of approximately 16,000 documents released in full during 2017. It does not give a source for this figure, but by comparing NF18 against the list of releases in 2017, I was able to get a similar result.

Explaining the IG and ARPP differences

The difference between the IG report total for documents released in full and the ARPP total is striking. Despite the fact that hundreds, more likely thousands of documents were released in full on 26 April 2018, ARPP claims a cumulative total of only 13,371 records released in full. In contrast, the IG report gives a figure of 16,000 for the 2017 releases alone.

One of these figures is clearly wrong, and after running through the 2017 releases, the ARPP figure seems much more likely to have a large error in it somewhere. The total number of ARC documents released in full should now be far in excess of 16,000. perhaps as much 20,000. An update from NARA on this odd problem is needed.

This post discusses NARA 18, a spreadsheet posted at the National Archives and Record Administration on 26 April 2018. NARA 18 gives a cumulative list of documents released from the JFK Assassination Records Collection from July 2017 to April 2018.

After a recount, I have to revise the file count I gave for NARA 18 on 5 May 2018 (here). There are ten bad filenames on the spreadsheet, rather than 5 as I previously stated. A table of the bad filenames is here.¹

Bad filename means that no files with the names given on these ten rows exist on the NARA server. In my original 5 May post, I caught only five of these, and as a result counted 19,045 rows linked to files on the NARA server. This correction means that there are actually only 19,040 rows linked to files. This does not mean, however, that the records cited on each of these rows have not been released by NARA.

In fact, all of the record numbers listed in the ten rows appear multiple times in NARA 18. This is true of many records in the 4/26 release (and earlier releases as well). As a result, there is no document cited in these ten rows that does not have an associated file elsewhere in the release. There is therefore no reason to assume that a document scheduled to be released in this set has been omitted.

As I’ve noted previously, the reason for these duplicate listings in NARA’s ARC releases is unclear. The ten rows here all refer to FBI documents, and for many FBI records, NARA released a single pdf file which included multiple ARC documents, so that NARA 18 links to the same pdf file multiple times. Perhaps this practice caused a problem in the case of these ten rows. This is of course just a guess on my part.

[Revised 01/10/2018 for clarity]

NF18 is a list of redacted documents in the John F. Kennedy Assassination Records Collection (the ARC) held at the National Archives and Records Administration (NARA). This is an important document for those who are interested in how much material in the Collection remains unreleased.¹

This post looks at how NF18 fits into the latest ARC release on 26 April 2018. The results of this comparison show that NF18 was by no means a complete list of all redacted documents in the ARC.

The status of NF18

NARA released NF18 in excel format on 29 January 2018 to John Greenewald, owner of “The Black Vault” website, in response to his FOIA request. Apparently the spreadsheet was accompanied by a note or letter, which Greenewald quotes as saying: “We conducted a search and were able to locate an EXCEL spreadsheet that lists everything that has not been released since December 15th, 2017 (the last release date).”²

While not very elegantly stated, this simply means that the NF18 spreadsheet lists all redacted documents remaining in the ARC after the six 2017 releases, which began in July 2017 and ended in December 2017.

This is confirmed (indirectly) in the March 2018 report from the office of NARA’s Inspector-General, James Springs (recently discussed here). That report states: “Currently, 21,890 documents [in the ARC] have not been fully released which represents about 7 percent of the collection.” The 21,890 record figure is consistent with NF18, which lists 21,890 unique document numbers.³

In other words, NF18 was NARA’s best effort at a comprehensive list of redacted documents in the ARC as of January 2018. As of March 29th, this was still the IG’s understanding.

NF18 in the 26 April 2018 releases

Following a six month review requested by President Trump, on 26 April 2018 more records were released and a new, cumulative spreadsheet was published for all of the now seven releases of ARC records. NARA’s press release on this set of records stated that 19,045 documents were in the 26 April release.⁴

If NF18 is indeed a complete list of all the ARC records with redactions, all of the new releases should be from records listed on NF18. ⁵

This turns out, however, not to be the case. To see that this is so requires two steps. First, one must determine how many records are unique to the April release: i.e., were not released in any of the 2017 sets. There are 1078 such records. Second, one must compare THESE files to NF18. Any records not in NF18 are then records that had redactions, but were not listed in NF18. Somewhat to my surprise, there are 320 such documents. A spreadsheet of these files is posted here.

In terms of the size of the ARC, this is not a large number: it is only one one thousandth of the 319,000 plus ARC documents. It also indicates that after coming up with its January 2018 total of redacted documents, NARA continued looking for more redactions all the way up to deadline. In addition, most of these newly discovered redactions were released in full and all of them were posted online at NARA.

In terms of finding out how many redacted ARC documents remain at NARA, however, it must remind us that tracking down every last one of these has been a difficult task, and we should not be surprised if more turn up in the months and years ahead.

Do discoveries of more redacted documents have any significance for those interested in researching the JFK assassination? This is a subject I’m slowly working on. Given the extremely broad scope of the material in the ARC, more documents released in full doesn’t automatically mean more information on the JFK assassination is available. Nor need it automatically diminish our confidence in previous studies, done when this material was unavailable.

This post will discuss the second of two recent reports on the status of the JFK Assassination Records Collection (ARC). My post on the first of the two is here. For those new to this subject, see here for an introduction to the ARC.

The report covered in this post, dated June 18, 2018, is by Rex Bradford, president of the Mary Ferrell Foundation. It can be found here.

The Mary Ferrell Foundation provides on-line access to a large percentage of the ARC records, as well as other important historical government records from the 1960s and 1970s. In terms of ARC documents, and the access it provides to them, it is of very high quality indeed.

Bradford is of course strongly conspiracy oriented, as are all of the MFF directors, but he is also well-qualified to provide an overview of the problems and work remaining for the ARC, so I had high expectations of his review of the current status of the ARC.

Unfortunately, his critique was a disappointment overall. I agree with Bradford on a number of points, but this post will focus on what I feel is his most problematic claim: that the ARC still contains records “withheld in full.”

Summary of Bradford

Bradford first offers a brief overview of the 2017-2018 releases, then summarizes what he sees as problems in these releases under four headings:

1) excessive and undocumented redactions;
2) errors, anomalies, and mysteries in online data;
3) missing withheld in full files
4) lack of accountability for the releases and the full collection

He closes with several recommendations, which mostly echo an open letter that the MF Board of Directors sent to National Archivist David Ferrigno in March 2018.¹

Bradford’s doubts on NARA’s accounting

A number of my posts on the ARC documents have been responses to various claims that there are still large numbers of records withheld in full at NARA. All such claims that I have checked have been based on misunderstandings or error.

Yet Bradford too believes there are still entire documents in the ARC which are eligible for release under the ARCA, but which NARA has not yet released. Bradford does not make the mistake of claiming “thousands” of ARC records still withheld in full; in the end he claims only 13. Even this claim, however, is in error.

The basis for Bradford’s claim

What is the basis for Bradford’s claim that there are still releasable records withheld in full at NARA? It comes from one FOIA document and Bradford’s reading of a NARA webpage. The FOIA document is a list of records that still had redactions as of January 2018. I now call this important document NF18 (see here for a description of where it came from and what is in it). The webpage Bradford cites is NARA’s JFK Assassination Records Processing Project. This is also an important source for the final ARC releases.

NF18 has been misunderstood by several writers. It is basically a list of ARC document numbers and their current status: redacted or withheld. The list is NARA’s accounting of all the ARC documents with redactions as of January 2018.² Bradford has clearly understood this, and matches NF18 against the information on the Records Processing Project page to check whether everything on NF18 that should be released has been released.

This is an appropriate method, and I tried to do the same thing in a post on 2018 May 5 (here). My results, however, were different from Bradford’s results, so a closer look is in order.

At the time of NF18, there were 798 ARC records withheld in full. It is important to remember, however, that the law creating the ARC, the John F. Kennedy Assassination Records Collection Act (ARCA), did NOT mandate that all records in the Collection would be made public. Sections 10 and 11 of the Act specifically exempted three types of records from public release: 1) sealed court documents and federal grand jury information; 2) deeded gifts to the federal government, 3) “records held under section 6103 of the Internal Revenue Code.”³

According to NARA’s webpage on “The JFK assassination records processing project,” there are 520 records which are withheld in full are withheld under Sections 10 and 11, and will therefore not be released.

But how do these 520 files fit into the 798 records withheld files listed in NF18? This is where some of Bradford’s doubts arise.

There are two other issues that come up as well. In addition to the section 10 and 11 records, there were 10 audio tapes in the Gerald Ford Presidential Library that were “not recoverable.” There were also 79 Record Information Forms (RIFs) that could not be matched to records in the collection. All 89 of these records are therefore also unavailable online or in NARA’s archives, adding to Bradford’s doubts.

A final set of 180 documents is also listed in NF18. This document set is a microfilm copy of the CIA’s 201 file for Lee Harvey Oswald.⁴ The microfilm 201 file turned out to be a 100% duplicate of the original file, released all the way back in 1992. The “Project page” thus states that the microfiled files “were not processed for release or posted.”⁵

Bradford’s list

Based on all these items, Bradford begins by trying to specify which files are exempt from release under Sections 10 and 11 of the ARCA. This attempt is marred by what appear to be count errors and a math error.⁶ In addition, he has overlooked sealed court documents and federal grand jury information, of which NARA’s JFK Project page says there are 5.

He then lists 19 files which he says have not been released but which he suggests may actually be releasable. Six of these are FBI files which Bradford concedes “may be tax records,” so that in his list it is only 13 files which are “at issue.” Here are the files Bradford lists:

Problems with the list

The first problem with this list is that although Bradford claims none of these documents have been released, in fact five of the documents already have been, as the links I have added show. Note that 180-10131-10326, a document Bradford was particularly concerned about, is one of these.

The second problem with this list is that Bradford apparently worked directly from NF18, and did not go back and check the RIF sheets for these documents. NF18 only gives a few data fields from the RIF sheets for these documents. It omits, among other things, the restrictions field for these records. I have added this back in because it shows the basis for restricting access to these records.

Using this, we can see that there are five records on Bradford’s list withheld on Section 10 grounds. Section 10 refers to legal documents under seal of court [10(a)1], or grand jury information [10(a)2]. Another 6 records are withheld on Section 11 grounds. Section 11 refers to “records held under section 6103 of the Internal Revenue Code.” Two of the CIA records which were released in April 2018 have information withheld on 1B grounds. This is the ARCA exemption for “intelligence sources or methods.” Several records are also listed as “referred.” This means that agencies which provided information in the record have been asked to clear release of the record. Several of these have indeed been released.

The RIF “restrictions” field does not resolve all questions, but it certainly answers some of Bradford’s doubts: the six FBI documents that Bradford says “appear to be IRS documents” are withheld under section 11; there is no ambiguity here. Five of the documents he lists as questionable are withheld on Section 10 grounds. This is consistent with NARA’s Project webpage, which says five documents were withheld under Section 10.

It is a pity that Bradford did not check the full RIFs for these documents before compiling his list. It is strange that he lists documents as unreleased that are in fact released.

Conclusions

The 798 withheld records listed in NF18 and the 520 records withheld under Sections 10 and 11 are largely resolvable. Here are the numbers I have:

This is all 798 withheld records on NF18. Of these records, I can only identify 515 as Section 10 or 11 documents. There are four documents remaining which have not been released, but I have no idea why. Even if the four unreleased documents are all Section 11 records (very unlikely), we are still one document short of 520. Perhaps one unreleased Section 10 or 11 record has indeed been omitted from NF18. (I will have more to say about this in the near future.) Or perhaps 520 is a miscount. Everyone makes this kind of mistake; a look at my past posts will show lots of places where I have had to go back and correct my numbers.

Following the trail down to the very end, I am also uncertain of the status of the typewriter ribbons. Bradford seems to think that these two records represent two pieces of one object, but I think they are separate: the cartridge is not just a box, it has more ribbon inside it that was used to type classified documents. I have no idea why these two ribbons were kept. It seems unlikely that two only ribbons were required to type all the classified documents that passed through the HSCA, but these are the only two listed in the ARC.

Overall, this is almost the same accounting I gave in my May 5th post, as the excel sheet posted there shows. I have changed my mind on one item: the document 124-10286-10391. I originally thought this was NOT a withheld document because Mary Ferrell has a document with this record number. That left me with only four Section 10 records.

In fact the 124-10286-10391 document on Mary Ferrell is an error; the RIF sheet for the withheld 124-10286-10391 document has been erroneously attached to another document on a similar subject.

How did this happen? Federal agencies were required to produce record information forms (RIFs) for all their assassination related records, and during this process, it happened more than once that the wrong RIF was attached to the wrong document. This happened to several of the FBI documents in the record group that 124-10286-10391 belongs to. This group is a set of FBI files relating to William Waldman, a vice president of Klein’s Sporting Goods. Klein’s was the company which sold Lee Harvey Oswald the rifle he used to shoot President Kennedy.

I am now sure that the Mary Ferrell 124-10286-10391 RIF sheet is mis-attached because the RIF sheet indicates the document date is 1953, while the Mary Ferrell document attached to this RIF sheet is from 1965.

Summing up, the document count NARA gives is off by only one file. Although the remaining four unreleased files may have other issues, I conclude that Bradford’s claim of over a dozen missing withheld in full files is simply wrong.

For those not familiar with the subject of this post, I have a new page introducing the JFK Assassination Records Collection (a link is at the top of the blog page). This should help make the discussion a little more comprehensible.

I’ve now had a look at the two reports I mentioned in my last post. These reports cover the final releases of material from the JFK Assassination Records Collection (ARC), which is held at the National Archives and Record Administration (NARA). Neither report is as useful as I had hoped.

The first of the two reports is from the office of NARA Inspector-General James Springs (here). It was issued 29 March 2018, a little less than a month before NARA released what is supposed to be the final set of ARC documents. The IG report therefore only covers the 2017 ARC releases.

I should note that I am not certain whether the document I cite here is the report itself, or simply a summary of the report. ¹

Much of the report is simply a recitation of the basic history of the JFK assassination; no need to go over that here.

The remaining part of the report recounts various aspects of the 1992 ARCA, the legislation which created the ARC. It notes that, in the IG’s view, the ARCA is still in effect and imposes a “continuing obligation” on federal agencies to release assassination records.

It also spells out the records which the ARCA exempted from public release, including sealed court documents, federal grand jury information, deeded gifts to the federal government, and “records held under section 6103 of the Internal Revenue Code.”

There is then a brief description of NARA work on the ARC releases, beginning in 2014 and ending with work current as of the report date (March 2018).

For those who have not followed this long story, in 2014 the JFK Assassination Record Collection still had numerous records that were not available to the public, or available only in part.

The ARCA set a 2017 deadline to release most of this information, so after lengthy preparation and consultation, described in the IG report, NARA released six sets of records from the ARC during July to December 2017. According to the IG report, during this period 34,873 documents were ‘released.’

A caveat is now in order here for readers who may be confused by the word released.

Many readers have taken the word “released” in this context to mean that a document was previously “withheld,” i.e. unavailable to the public. This is not how these terms are used in describing ARC releases.

There are three possible states in which an ARC record may appear: withheld in full, withheld in part, and released in full. Documents withheld in full are not available to the public at all. Documents released in full are publicly available, with no text removed at all. Documents withheld in part are publicly available, but have part of their text removed. These removals (usually called ‘redactions’) range from a single excision of as few as two letters (a digraph), to several pages of text.²

When a redaction is “removed” from a document, this means the blanked out or blacked out text is restored. For ARC documents, however, the restored text is said to be ‘released.’

Why is this confusing? Consider a hypothetical ten page document, published ten years ago with only one redaction, two letters from a CIA cryptonym. When the two letters are restored, NARA counts the entire document as a “release.” Moreover, if redaction x in a document is ‘released’, but redactions y and z are not, the same document may later be referred to as ‘released’ again and again, until it finally reaches the stage of ‘released in full.’

This counter-intuitive terminology led some writers last year to believe that ALL of the documents ‘released’ during 2017 were previously withheld in full! Since a total of 34,873 documents were ‘released’ in 2017, it is easy to get the mistaken impression that a vast amount of material was previously unavailable to the public, based on the false assumption that all of these 34,873 documents were withheld in full.

But this was far from the case. Pursuant to an FOIA request in 2016, NARA attempted to compile a list of all the withheld in full documents. This list, which I refer to as NF16, counted 3598 documents withheld in full.³

The vast majority of the 34,873 documents released in 2017 were thus NOT withheld in full. Many of them had only one or two redactions, and after the 2017 ‘release’ were actually ‘released in full.’

Ah, but which ones? Unfortunately, NARA did not provide a count of these. This lack is partially filled by the IG report. For instance, the report states that all the documents released in July and October 2017 were released in full.

In addition, the IG report also indicates a publicly available source for further checking.

This source is another list which NARA released in January 2018. The list, which I call NF18, included ALL documents still containing redactions after the 2017 releases. The list, which was also released pursuant to an FOIA request, consisted of an excel spreadsheet with 22,933 rows. For reasons that are unclear to me, many of these rows were duplicates. The total number of unique documents in NF18 was 21,890. ⁴

This list is the basis for the claim on pages 3 and 6 of the IG report that 21,890 ARC documents still contain one or more redactions after the 2017 releases.

NF18 can thus be a tool for testing the IG report statement that the documents released in July and October 2017 were all released in full. Do any of the July/October documents appear on NF18? The answer is no. But is NF18, and the IG, correct in asserting that there is nothing left redacted in these documents?

Well, the documents were posted on the internet. Just check if a July/October document still has redactions. Alas, this is a task that none of the internet commentators on the ARC have yet attempted.

Going further, one can also check the November/December 2017 releases using the same method. For the count of November/December releases not included in NF18, the results I got were as follows. Of a total of 24,626 unique records released in November 2017, 8856 of these do NOT appear in NF18. Of a total of 4217 unique records released in December 2017, 986 do NOT appear in NF18.

These documents are also available for checking to see if they are indeed released in full. No one has bothered checking these either.

This being the case, and now having some spare time, I have attempted this feat for all of the 2017 releases, and will report on the results in a future post.

Meanwhile, adding these figures all together, I get 16,536 documents “released” in 2017 that are not on NF18, meaning that they were released in full.⁵ Returning one more time to the IG report, this is consistent with the claim in the second paragraph of the report that “approximately 16,000” of the 2017 documents were released in full.

These figures are the main contribution of the IG report to our understanding of the current state of the ARC. Since the IG is using NF18 to track releases, we can also do so for the 16 April 2018 release. This will also be the subject of a future post.

This post is a continuation of my April 2018 post on the “old RIF numbers” formerly used in the JFK Assassination Record Collection (ARC) at the National Archives and Records Administration (NARA).

This ID system is no longer used at NARA, but there are still many documents using it in the important on-line version of the ARC at the Mary Ferrell website.

For those who want to get maximum use out of the MF collection, matching up the old IDs with the current IDs can therefore be very, very helpful, as this post explains.

For others, however, this post will probably have little of interest. Caveat lector.

RIFs old and new

The act which created the ARC, the 1992 JFK Assassination Records Act (ARCA), required NARA to “prepare and make available to all Government offices a standard form of identification or finding aid for use with each assassination record subject to review under this Act.”¹ It further required the National Archivist to “ensure that the identification aid program is established in such a manner as to result in the creation of a uniform system of electronic records by Government offices that are compatible with each other.”²

In compliance with this requirement, NARA created the RIF, short for “Reader Information Form”. The RIF has 17 fields for metadata concerning the record it covers, and a unique 13 digit identifying number, the “RIF number”.³ It is usually attached on top of an ARC document as a cover sheet. As a convenient shorthand, I’ll call all these documents “RIF docs.”

The ARCA requirement to create finding aids was limited, however, to federal government records which were not in the possession of NARA when the ARCA went into effect, or which had text redacted and were therefore not open in full for public inspection. The most obvious example of records which do NOT meet these criteria is the documents from the Warren Commission’s investigation of the JFK assassination. These have been in NARA’s possesion since the 1960s, and most of them have been open to the public for over 50 years. This is why most most records from the Warren Commission do not have RIF finding aids, despite being an integral part of the ARC.⁴

Other examples of ARC records without RIFs include documents from Jim Garrison’s investigation of the assassination, which resulted in the trial of New Orleans businessman Clay Shaw for conspiracy to assassinate President Kennedy. None of these documents have RIF forms either. Overall, however, documents without finding aids make up only a small percentage of the total JFK collection; they are far exceeded by documents for which the ARCA required a finding aid.

The Mary Ferrell collection of ARC documents includes a huge number of both RIF docs, such as the CIA 201 file on Oswald, and non-RIF docs, such as records from the Warren Commission and Garrison investigations. The MF website states it has 101,113 documents from the ARC collection available on line.⁵ I originally took this to refer to the total number of documents on the site, but after taking a hard look at documents counts at MF, I believe that the figure of 101,113 documents refers specifically to RIF docs, and does not include any of the thousands of non-RIF docs in the MF collection.

In addition to both RIF and non-RIF docs, as my previous post explained, the MF collection also includes a huge number of documents which use a different form, labeled “NARA Identification Aid.” This form, by my count, has 32 fields, and uses a different form of identifying number: a 35 character string which consists of a date-time string followed by a 6 digit number. (See my previous post for several examples of these.) These forms are only used for CIA documents.

This is not a third set of distinct documents. Instead, it turns out that this form was produced by the CIA when it began processing its records for the ARC, all the way back in 1992. Based on the date-time strings, the forms in the MF collection were created between May 1993 and January 1995. This CIA form was later replaced by NARA’s RIF; the old forms were then removed from documents in the ARC, and the new RIF forms were added in their place.

Why do so many of these forms remain in the MF collection? I believe this is because they were still in use at the time these documents were acquired by groups such as the Assassination Archive Research Centter (AARC), who in turn provided them to MF.

In my previous post I called these older forms “old RIFs” and the id numbers on them “old RIF numbers”. Since the function of the form was the same as the RIF, I will continue to use the name “old RIF” to refer to this earlier type of finding aid.

Old RIF doc count

Getting a count of old RIF docs in the MF collection is challenging; my current count is about 23,300. This is a much larger number than I estimated in my previous post; recall that the MF Foundation’s estimate of their RIF doc total was 101,113, so this potentially increases the size of the MF collection by 20 percent.

These documents are distributed in the MF collection as follows:
HSCA segregated collection: 18,819
Oswald 201: 3653
HSCA microfilm collection: 880

The 23,300 figure includes duplicate numbers; accounting for these, the total number of unique old RIF numbers in the MF collection is about 22,600. There were a fair number of typos in the old RIF numbers which I have corrected to the best of my ability.

Matching old RIFs to new RIFs

As the previous post noted, it is possible to match the old RIFs to the new RIFs. The 35 character old RIF number for these documents was not just dropped. Instead, it was moved to the Comment field of the RIF form, together with the “box number” and “vol/folder number” on the old RIF. I have now found some documentation for this move.

ARC record 104-10331-10342 (dated 27 June 1994) is a discussion of changes NARA requested to the “JFK Database Extract.” Item 6 on the list notes “NARA wants the date time user stamp removed from the ID aid. This will require a modification to the application.” Item 8 notes “NARA would like the box number and folder information prefixed to the comments field. This will require a modification to the application.”

ARC record 104-10331-10343 (dated 28 July 1994) notes that as a result of a 25 July meeting, it was decided to “Move date/time ID number to Comment Section along with box numbers and folder information (#8).”

Based on this, I thought it would be possible to match all of the old RIF docs to the new RIF docs. It turns out, however, that this is not possible. I was able to match only about 21,300 old RIF docs to their new RIF counterparts.

One reason for this is that some of the CIA documents in the ARC have redone the date-time string in the comment field. An example of this is MF docid 95842. This old RIF doc is titled “RPTS ON ACTIVITIES TRAVEL OF THE OSWALDS”, and its id number ‘1993.05.17.17:29:39:000065’ This can be identified as ARC record 104-10001-10015. The current version of this record was released on 26 April 2018, and the comment field is OSW7 : V32 : 20031124-1016188.

How do we know that the these two files represent the same record in the ARC? Merely eyeballing them is NOT adequate. There are far too many duplicates in the ARC to safely claim that two documents have the same RIF number simply because they are identical in appearance. The reason we can say these two documents ARE the same is that there is an earlier release of 104-10001-10015 in the MF collection (docid 49995). In this earlier release, the comment field is OSW7:V32 1993.05.17.17:29:39:000065, exactly matching the ID on the old RIF doc 95842.

Why did the date-time string change in the later release of this record? Apparently, this was one of a set of records that were reprocessed by the CIA in the period 2003-2005,⁶ and these new date-time strings reflect the date of this reprocessing.

Trying to match up old RIF and new RIF records where the date-time string has changed is a mind-boggling task. I have managed to match about 500 of them, and in theory it may be possible to do it for all of them, but at this point I don’t really believe it is worth further time and effort.

New totals for CIA records in the Mary Ferrell Collection

Even without the unmatched records, 21,500 is a lot of records. The MF website’s JFK Database Explorer, which is based on the NARA’s Assassination Collection Reference System (ACRS), gives a total of 85492 CIA records in the ARC. It gives the total number of CIA records in the MF collection as 52314.

The figure of 52314 does not include ANY of the old RIF docs, as far as I can tell. We cannot, of course, simply claim that all 21,500 of the old RIF docs that I was able to match up as new additions to the MF collection. The old RIF docs include a large number of documents that duplicate new RIF documents also on Mary Ferrell, as the examples in the this post and the previous post show.

But, as the excel file linked to in my last post also showed, there are a huge number of old RIF docs that are NOT available as new RIF docs on MF. Thus, the old RIF docs significantly expand the total number of CIA documents available from the MF collection.

At this point I can only provide very rough counts on exactly how many more CIA records the old RIF docs add to the collection. An estimate of 14,000 more records is probably close. This would represent an increase of about 20% from the 52,314 Mary Ferrell has already indexed, to around 66,000. I sent a note to MF on this a while ago, but haven’t heard back from them. Looks like another DIY project. Converting my current database tabulation to an excel sheet is not high among my priorities now, but if you are interested in such a list, leave a comment.