ARRB Electronic Records at NARA

Posted on December 11, 2017 by Robert

In all the news coverage about the release of documents under the JFK Records Act, one release has attracted almost no attention. As the NARA press release on the October 26 release noted (here), the electronic records of the Assassination Record Review Board, the agency which determined what government records would be released under the JFKRA, were also released.

The ARRB records are themselves a noteworthy release. Previously few records from the ARRB were available on line, except for their Final Report, and transcripts of the public hearings they held from 1994 to 1998. The Mary Ferrell Foundation website, maryferrell.org, has only one set of files from an ARRB staffer (here), and a series of memos from ARRB staffer Douglas Horne (here). The newly released electronic ARRB records are therefore useful indeed for anyone who wants to know not just what the ARRB did, but how and why they did it. The rest of this post provides a summary of the content of the files, as I did earlier for the JFKRA releases (more boring numbers, little on file content).

Number of Files

The ARRB electronic records were released in 83 files. These consist of 13 email archives in csv format, and 70 zip files (all available here). This post will cover only the zip files. The follow-up post will cover the email archives.

In the October release, there are 58 zip files associated with specific ARRB staff members, and another 12 zip files which are general board files. Each zip file is divided into two main directories, Electronic Records and Technical Documentation. The TD directories all have the same components: a readme file explaining a dating discrepancy in some files, and a csv file of metadata for the original files in the ER directory. The ER directory has all the documents from each staff account, converted from their original formats into pdf files, and arranged under sub-directories such as wp-docs, excel, etc.

The pdf files were all created between March and September 2017, the majority in June and July. These of course were not the dates the original files were created, but fortunately the TD directory csv files have the ‘date last modified’ for the original files. All you have to do is match up the directory and file names in the csv file to the pdfs in the zip file.

Oddly, this is not entirely possible, even though the csv files were mostly made in August 2017, after the pdf files were created. There are several dozen files in the zip archives that do not match up with the listings in the csv files. Some of these mismatches are due to things like missing file extensions, files in different directories, file extensions missing or misshapen, misspellings, and some cases that defy explanations, such as rec.id becoming non-rec.id. There are also pdfs in the zip archives that clearly were not registered in the csv file, and files listed in the csv data that are not present in the zip archives. The numbers are small; files from the zip archives that I have not been able to match back to the csv file lists total 31. pdfs listed in the csv file that do not match to pdfs from the zip archives number only 49. So not a tremendous deal, but I don’t understand how these differences came about.

In addition, a significant number of files from the zip archives are not pdfs of the original file, but instead have been replaced by forms that state the original files were “withdrawn.” These replacement forms are clearly marked by adding the string “_wd_NNNNNNNN” to the file name (wd standing for ‘withdrawn’ I presume). Instead of the original file, these pdf forms consist of a one page document explaining the reasons for the withdrawal.

There are two reasons for file withdrawal: 1) The file “contains electronic characters that are unintelligible and therefore cannot be authoritatively reviewed according to the John F. Kennedy Records Collection Act of 1992.” These files are almost all binary files, with extensions such as .bmp, .dll, .exe, etc. and are thus not worth looking at anyway. 2) The file “has been withdrawn according to (one of the exemptions of) the John F. Kennedy Records Collection Act of 1992.” The file then lists which of the five exemption categories the document falls into (some of them fall into multiple categories). The binary junk files I will call WDI files, the exempt files I will call WDE files. The totals for the different types of files in the zip files is as follows:

File type No. files
Non pdf files 142
pdf files (not withdrawn) 14167
withdrawn pdf files (unintelligible) 2063
withdrawn pdf files (exempt) 396
Total 16768

The non-pdf files are all from the Technical Documentation directories except for two miscellaneous items. The NARA press release states that the ARRB files consisted of “16,627 files from the ARRB drives.” This must refer to the total number of pdf files (and exclude the 140 csv and readme.txt files in the Technical Documentation directories). My count is one more than theirs, who knows what the extra file is (perhaps the novelty mpg file staffer Carrie Fletcher had on her drive).

The withdrawal of files must have been done by NARA, though why they did it according to the JFKRA rules is not clear to me (the JFKRA does not seem to say that it applies to ARRB records), and why the NARA is entitled to carry out such a withdrawal is also unclear. Unlike documents released under the JKFRA, the ARRB withdrawal sheets state that one may apply for the release of the withheld materials under FOIA provisions.

The zip with by far the largest number of files is s-adm-g_pub-20171001.zip, which is general “administrative” matters and has 5838 files. After this jumbo zip, the top three zips are from ARRB staff members T. Jeremy Gunn, Laura Denk, and Tracy Shycoff, each of whom has a zip file with from 1000 to 800 plus files. As we should expect, the zips with the smallest number of files are from the interns, and the Board members, who have only two files apiece, except for Board Chairman John Tunheim, who has nine.

I have put up a list of the 58 ARRB personnel with zip files here.

Pages in files

The number of pages in each file ranges from a 2982 page monster (S-ADM-G/PRG/FBIFIX.TXT.pdf; this seems to be a database table dump), to one page scraps. Here is a table of number of pages per file (this does not include non-pdf files or the WDI and WDE files):

Page range No. of Docs
> 100 78
51 – 100 188
31 – 50 168
21 – 30 141
16 – 20 177
11 – 15 297
6 – 10 751
4 – 5 1076
3 1312
2 3978
1 6002

Based on this count, the total number of pages released from ARRB files is 79962.

Dates of files

After matching up the pdfs in the zip files with the files in the csv lists, I can also summarize when the files were last modified:

Year Number of documents
< 1994 63
1994 235
1996 3769
1996 3609
1997 3648
1998 2841

Because the csv lists include the date last modified for the withdrawn files, I could have put these into the summary, but I have chosen to omit withdrawn files from this count, for consistency with the counts above. I have also included in the pre-1994 count unmatched files from the csv lists.

Other than withdrawn files and unmatched files, the pre-1994 records are all text files from various software applications, such as database programs or printer drivers. None of these materials were created for or by the ARRB.

The 1994 files also include a number of such files. The earliest files that clearly were created by ARRB staffers are from September 1994 (by then executive director David Marwell), so that the actual figure for 1994 is more like 189 files.

Most of the staff accounts were set up in January 1995, as dated by the creation of an EXCEL subdirectory with a text file at this time.

The ARRB closed on September 30, 1998, and the last files in this release are some thank you notes by the ARRB’s final executive director Laura Denk, dated September 26.

(There are also three files dated from 2015 whose date stamps I cannot explain. Two of these are excerpts from a 1998 article by David Mantik, the other what looks like a version of a 1995 staff memo)

Withdrawal of files

As the withdrawal notifications provided in the ARRB files indicate, files were reviewed for withdrawal from December 2016 to July 2017, with most of the reviews done in June and July 2017.

There were two reasons for withdrawal of files, as noted above: files had unintelligible (non-ASCII) characters that made review impossible (WDI), or they fell under one of the ‘five exemptions’ of the JFKRA (WDE). In fact, as the withdrawal sheets show, there is one more exemption, 11(A), which exempts tax returns from the JFKRA. This exemption was invoked in four records.

Exemption 6(5), which exempts records revealing Secret Service protection measures, is in fact never invoked at all for the ARRB file withdrawals. The most commonly invoked exemption was 6(3), which was invoked 293 times. This exemption covers records whose release would constitute an unwarranted invasion of privacy. Judging from the title of these records, many were documents dealing with ARRB budget information, and probably revealed Board and staff salaries.

Final notes

There are some quite interesting memos in this release, and overall one gets a good idea of the ARRB’s sometimes contradictory goals and attitudes. My own interest at this point is in comparing what ARRB hoped to release, and what has come out so far. This will be the focus of later posts.

Posted in History, JFK ARC | Comments Off on ARRB Electronic Records at NARA

Review of Reporting the Chinese Revolution: The Letters of Rayna Prohme

Posted on December 9, 2017 by Robert

Letters from a fading past

This book consists of a narrative by two editors, Baruch Hirson and Arthur Knodel, written around a few dozen letters by Rayna Prohme. Rayna was an American who, together with her husband Bill Prohme, ran the People’s Tribune, the English language newspaper of the Nationalist Party (the KMT) in China from 1926-27. You have never heard of her unless you read Vincent Sheean’s 1934 autobiography, Personal History. Personal History was a best seller, one of those books that convinced people that the most adventurous thing to do with your life was become a reporter. Rayna is a central figure in Sheean’s book, where she is portrayed as the very spirit of Revolution, an event which “Jimmy” Sheean thought was just around the corner.

Apparently quite a few men who read the book decided that the most romantic thing a reporter could do was fall in love with this radical spirit, and went around for years searching for someone like her. The book did not have that effect on me, but it certainly made me wonder what she was really like and how she had got into a very unusual situation. This book (partly) answers both questions, giving a moving description of Rayna and her husband Bill through letters she wrote from 1926 to 1927. The letters are to her sister, Grace Simons, her friend in Berkeley, Helen Freedland, and her husband Bill. They end a few days before she died in Moscow, and are supplemented by a few more letters written by Jimmy Sheean and Bill Prohme, describing the aftermath of her death from some type of meningitis or encephalitis.

In the early letters, Rayna is excited to be in Canton and Hankow, working for a cause with people like Eugene Chen, foreign minister of the KMT regime in Hankow, Michael Borodin, chief Russian advisor to the KMT, and Soong Ching-ling, the widow of Sun Yat-sen. In the end though, the KMT expelled both the Chinese Communists and their former Russian advisors, for reasons sketchily explained in the preface to the book. Rayna strongly identified with the Communists (it is not clear whether she was a party member), so she and Bill quit the paper (or were fired) and returned to Shanghai.

Mrs. Sun chose to go to Moscow rather than remain in China, perhaps to express her rejection of the KMT’s change of direction (or perhaps not). For reasons quite unclear, Rayna was invited to accompany Mrs. Sun, and Bill was asked to stay in Shanghai. Mrs. Sun and her fellow travelers arrived in Moscow in early September, at the climax of the struggle between Stalin and Trotsky. The failure of the Russian efforts in China played an important part in the struggle, and as an inconvenient witness, Rayna was very unwelcome. She was still looking for a regular job and a place to stay, when she became ill and died suddenly on Nov. 23, 1927. The last few letters she wrote to Bill, which unknowingly describe the onset and progress of the disease that killed her, are truly heartbreaking.

That these letters survived after all those decades is simply eerie. When Helen Freeland died in 1956, Helen’s sister Nancy gave the letters to Marian Parry, a friend of Rayna from Berkeley in the 1920s. Marian (who died in 1986) then gave letters to Knodel, who had been a fan of Sheean’s book since the 1940s. Knodel began preparing a manuscript based on the letters, and apparently had a typescript by the early 1980s (according to the C Frank Glass papers in the Hoover Library), but eventually he must have put it aside.

The letters to Bill Prohme had an especially turbulent passage. Bill was tubercular, and after some very difficult times he killed himself in 1935, on the anniversary of Rayna’s death. He destroyed all his papers except for the letters Rayna sent him from Moscow, which he gave to Rayna’s sister Grace. The letters were found in Grace’s papers after she died in 1985, and finally passed into the hands of Baruch Hirson, who was interested in writing a biography of Grace’s husband, C. Frank Glass. Hirson was unaware that there were more letters from Rayna until two years later, when he was shown Knodel’s manuscript of Rayna’s letters to Helen. The two then collaborated on this book, but it is hard to say when. In any case, their collective work must have lain in yet another box for many years: Hirson died in 1999 and Knodel in 2001. How it was rescued from this final oblivion is also hard to say; Gregor Benton, who wrote the introduction, does not explain. Perhaps a letter to Benton might be in order before he too passes away! Truly a haunted book.

For those interested in the period, this book is fascinating. If you have read Andre Malraux’s book Man’s Fate, read this to find out about real radicals in China in the 1920s. If you have read Sheean’s book, read this to find out what kind of person Rayna really was. Read this even if you haven’t read Sheean. Despite an extremely difficult situation, she comes across as a talented, resilient, and loving woman.

Posted in Book reviews | Comments Off on Review of Reporting the Chinese Revolution: The Letters of Rayna Prohme

An index to Hsiao-hsueh k’ao

Posted on December 5, 2017 by Robert

Today I’m posting another recycled paper, an index to a book called 小學考 (Hsiao-hsueh k’ao, pinyin Xiaoxue kao). The link to download the index is here.

I compiled this index in June 1991, when I was still a graduate student. As the preface indicates, I thought I was going to do a revised version later that year, but here it is 26 years later and the revision never happened. Off to the Internet bitbucket with you! The pdf is scanned from the original printout. The files and database I used are either lost or in wordperfect 4.2 format with Chinese added, which cannot be easily reconstituted into any readable format. Apparently OCR doesn’t work very well on these older Chinese fonts, so I have just posted the scan without doing anything more. The author of the Hsiao-hsueh k’ao, Hsieh Ch’i-k’un (謝啟昆, pinyin Xie Qikun) has a short biography in Hummel’s Eminent Chinese of the Ch’ing Period, but it is not under his name, and thus not easy to find. If I have the time, I may do another post on Hsieh and his book.

Posted in Linguistics, Recycled papers | Comments Off on An index to Hsiao-hsueh k’ao

How to select/display all rows with duplicate values in MySQL

Posted on December 3, 2017 by Robert

(This should be a fub, but the description is too long.)

One of the most frequent uses of database tables is to identify duplicate values in your data. Assume you have a table with a list of names, and you want to know how many of the names appear in the table more than one time. Most books on MySQL will tell you how to do this. Here is a recipe from Paul Dubois (MySQL Cookbook 3rd ed., O’Reilly p. 556):
SELECT COUNT(*), last_name, first_name
-> FROM catalog_list
-> GROUP BY last_name, first_name
-> HAVING COUNT(*) > 1
Suppose, however, that you want more information about the duplicates, for example their birthdates, which is contained in another field in each row. Just adding the field birth_date to this sql command will only show you the birthdate for one of each of the duplicates. Suppose you want to show data for all of the duplicates? Dubois actually suggests how to do this (p. 560), but his solution involves making a temporary table and joining it against the main table. Is it possible to accomplish the same task without an intermediate table? The answer is yes: use a self-join. I have great difficult understanding and using self-joins, so it took me quite a while to get it. Even worse, I’m pretty sure I’ve spent considerable time figuring this out more than once. So here is the solution, for permanent reference:

SELECT t1.last_name,t1.first_name,t1.birth_date
  FROM catalog_list as t1 
    INNER JOIN (
    SELECT last_name,first_name,count(*) 
      FROM catalog_list 
      GROUP BY last_name,first_name 
      HAVING COUNT(*)>1
    ) as t2
  ON t1.last_name = t2.last_name and t1.first_name = t2.first_name
ORDER BY t1.last_name, t1.first_name, t1.birth_date
Posted in Programming, Software | Comments Off on How to select/display all rows with duplicate values in MySQL

Review of Ronald Radosh, Red Star Over Hollywood: The Film Colony’s Long Romance with the Left

Posted on December 2, 2017 by Robert

Blacklist back-story

Author Ronald Radosh has a back story himself, for those interested: once a Communist Party member himself, he quit, moved to the new “left”, then moved to the new “right” in the 1980s. I knew of him because of his book on the Rosenberg case (The Rosenberg File: A Search for Truth, written with Joyce Milton), perhaps the best available to date.

I came to this book looking for more information on Otto Katz, a Comintern agent who was in Hollywood 1935-36. Radosh has a very interesting section on Katz, who apparently had a massive FBI file stuffed with letters from famous Hollywood people, most notably Fritz Lang. Radosh gives a clear summary of the correspondence of Katz and Lang, with an interesting take on Katz’s appeal to people like Lang. He even comes up with juicy second hand Hollywood gossip about Katz (Marty Peretz told Radosh that Lillian Hellman told him that she had a fling with Katz in Paris).

I hadn’t really expected to read the whole book, since I just don’t care that much about Hollywood people, even when they’re having flings with Comintern agents. Moreover, the Hollywood blacklist, the center of the book, has provoked more controversy than I’m prepared to wade through. However, it’s a short book, and turned out to be very entertaining, at least for people who mumble ‘whoa’ when they read how Budd Schulberg recruited Dorothy Parker into the Communist Party.

A lot of the book goes over old ground, but there were several stories that I had never heard before. These were the result of Radosh’s reading in a long list of archives. For example, there is the story of how Howard Koch hired Jay Leyda as a script “adviser” on the notorious “Mission to Moscow,” which Dwight McDonald called “the first totalitarian film to come out of Hollywood.” This film has been discussed before, but Radosh found interesting new material in the papers of Koch and Leyda.

Then there is the story of Melvyn Douglas and the Motion Picture Democratic Committee. Douglas and his friends put much time and energy into the Committee, but the Popular Front of the 1930s meant the Committee also had many Communist Party members. After the Soviet-German Non-Aggression Pact, Douglas tried to get the Committee to denounce the Pact and reaffirm its commitment to Roosevelt’s anti-Nazi policy. How he fought and lost goes a long way to explaining where anti-communist liberals came from.

Perhaps the most interesting story was the conflict between the figures in the Hollywood Ten who had quit the Communist Party (Edward Dmytryk and Adrian Scott), and those who were still in it (everyone else). This story is mostly based on Dmytryk’s book, but the daughter of their lawyer, Bartley Crum, was also present at many discussions and wrote about the ten at length in her 1997 autobiography, a book which I had never heard of.

Radosh’s point is that the blacklist had a long back-story, with fall-out over many incidents, from “Mission to Moscow” to the Non-Agression Pact, having a deep effect on people’s attitudes. That plus the militant nose-thumbing of Hollywood Ten members like John Lawson certainly irritated people like Harry Warner, about whom Radosh tells an anecdote I’m sure is apocryphal. A script writer is blacklisted and Warner promptly fires him. The writer complains, “This is a mistake. The plain fact is that I’m an anti-Communist.” Warner replies angrily “I don’t give a shit what kind of a Communist you are, get out of here!”

Posted in Book reviews | Comments Off on Review of Ronald Radosh, Red Star Over Hollywood: The Film Colony’s Long Romance with the Left

JFK Records Act Releases: A comparison with NARA 2016

Posted on November 30, 2017 by Robert

In a recent series of posts here, I have been looking at the records released by the National Archives and Records Administration (NARA) under the JFK Records Act. These are the JFK assassination records that were originally mandated to be released by October 26, 2017. Despite the newspaper coverage, there are still many records that have not yet been released to the public. This post looks at what we know about these as yet unseen records.

As noted in a previous post, the records that NARA has been releasing since July of this year are of two kinds: records “previously withheld in full” (WIF records), and records “previously withheld in part” (WIP records). WIF records have never been released in any form by any governmental agency or investigation. WIP records have been released at some point in the past, but with redactions which range from a single two letter prefix (cryptonym prefix) on a single page, to multiple pages which have been completely blanked out.

The WIF records, however, have received the majority of the attention in the run up to October 26. These records are by no means terra incognita. In response to an FOIA request, NARA released a detailed list of these documents in January/February 2016. I have tracked down some of the details about this list from the following sources:

a 2/4/2016 post at WhoWhatWhy.org (here)
a post at the Mary Ferrell Foundation website, apparently put up February 10, 2016 (here)
A file posted at the official FOIA online website (here)

The FOIA website shows that a request for this information was submitted to NARA by Michael Raznitsky on Nov. 30, 2015, and a “partial” response was provided January 21, 2016. “Partial” seems to mean that parts of the list were redacted (as the annotations on the list show).

The file posted was originally cited by both WWW and MFF as listing 3603 records. It does not. John Newman’s website 2017JFK.org (here) attributes the figure of 3603 WIF records remaining to an April 15, 2015 presentation by NARA’s Martha Murphy (available on youtube here). I can’t find where Murphy gives this specific number, but she does make the important point that NARA’s online catalogue of records subject to the JKFRA had, or have, many inaccuracies concerning which records remain WIF, and which are only WIP.

In any case, as MFF later acknowledged (here), the list NARA provided to Raznitsky in 2016 (hereafter simplified to “NARA 2016”), had data for 3598 records. The file posted online omitted one page, which NARA provided to MFF (available here). With this correction, all the data for these records is publicly available. Many of these records list 0 pages; this means that the number of pages has not yet been entered into the NARA catalog. As a result of this, we do not yet know the total number of pages remaining to be released. We do have all the RIF numbers, which are the record numbering system devised by NARA for the JFK record collection, and we do know which agencies provided the records. MFF gives a breakdown of records per agency for NARA 2016 (here), but it is slightly off. The corrected numbers are as follows:

RIF prefix Agency No. Records
202
173
198
111
177
157
178
181
176
119
180
137
179
104
124		 JCS
ONI
Army
DIA
LBJ Library
Church Cmte
Rock Comm
NARA
NARA-C
DoS
HSCA
IRS
DoJ
CIA
FBI		 2
2
6
14
19
26
31
39
45
94
164
178
549
1171
1258
Totals 3598

(Note that 176 records are not from the US Secret Service, but from several collections held by NARA, in this case the papers of C. Douglas Dillon, Sec. of the Treasury and head of the Secret Service at the time of the Kennedy assassination. 181 records, on the other hand, are records produced by NARA itself)

The RIF numbers also allow us to check which of these records have been released in 2017 (as of 29 Nov). As noted in my previous post, NARA has provided spreadsheets which list all the data for the records released so far, but there are some ambiguities in the spreadsheets. As a result, MFF lists 1211 WIF records released as of 17 Nov, but I count 1179 records, 32 fewer than MFF counts.

Comparing the RIFs of the released records to the RIFs listed in NARA 2016, we can see that WIP vs WIF numbers are still a problem. First, there are 31 released records which are listed in NARA 2016, but the NARA release spreadsheet gives their status as WIP. These records are:

104-10413-10305
124-90046-10009
157-10002-10151
157-10002-10179
177-10002-10109
181-10002-10005
181-10002-10017
181-10002-10022		 181-10002-10033
181-10002-10036
181-10002-10056
181-10002-10094
181-10002-10113
181-10002-10115
181-10002-10125
181-10002-10126		 181-10002-10131
181-10002-10132
181-10002-10135
181-10002-10137
181-10002-10141
181-10002-10142
181-10002-10211
181-10002-10212		 181-10002-10213
181-10002-10219
181-10002-10221
181-10002-10224
181-10002-10225
181-10002-10245
181-10002-10248

Second, there are 19 released records where the NARA release spreadsheet gives their status as WIF, but they are NOT listed in NARA 2016. These records are:

124-90100-10464
144-10001-10103
144-10001-10110
177-10001-10437
179-40007-10244		 180-10097-10488
194-10001-10392
194-10010-10397
194-10010-10421
194-10012-10037		 194-10012-10139
194-10013-10328
194-10013-10341
194-10013-10342
194-10013-10343		 194-10013-10344
194-10013-10345
194-10013-10346
194-10013-10448

I believe that the discrepancies between NARA 2016 and the data in the release spreadsheets is due to NARA reviewing previous records which were not correctly accounted for. The RIF sheets that accompany the majority of the new releases support this suggestion.

For example, of the 31 records which NARA 2016 lists as WIF and the release spreadsheet lists as WIP, the RIF sheets for 27 of these are stamped “Released under the John F. Kennedy Assassination Records Collection Act of 1992 ($$ USC 2107 Note). Case#: NW 54545 Date: 10-12-2017”, with the Current Status marking on the sheet all originally reading ‘Withhold’, but in two cases crossed out and hand marked ‘Open.’

I take this to mean that these records were processed as a set on 10/12, at which point the incorrect status was noted and corrected. (Also, it is interesting that most of these records were from NARA itself, i.e. prefix 181.)

In the case of the records that were not listed in NARA 2016, but are marked on the release spreadsheets WIF, this was in at least some cases the reverse process. For example, on record # 194-10013-10328 the Current Status marking was “Released with Deletions”. This would explain why it was not listed in NARA 2016. As for its release now listed as WIF, I assume that this was also due to review, though there is no marking on the RIF sheet to support this.

The end result: 1191 records in NARA 2016 have now been released, with 2407 to go. Here is table of these records by agency:

RIF prefix Agency No. Records
202
173
198
181
111
177
157
178
176
119
180
137
104
179
124		 JCS
ONI
Army
NARA
DIA
LBJ Library
Church Cmte
Rock Comm
NARA-C
DOS
HSCA
IRS
CIA
DOJ
FBI		 2
2
6
13
14
15
23
26
45
94
101
178
234
549
1103
Totals 2405

(Table revised 12/20/2017)

Posted in History, JFK ARC | Comments Off on JFK Records Act Releases: A comparison with NARA 2016

JFK Records Act Releases: Audit

Posted on November 28, 2017 by Robert

This post is a continuation of my previous post on the JFKRA releases. It is more number checking, no comments here on the content of the releases. This is probably boring to most readers (if there are any), but it’s always good to know the exact extent of the material you’re dealing with, otherwise you may wind up drawing conclusions based on partial or inaccurate information and numbers.

In this post I am especially interested in the issue of records “previously withheld in full” (WIF), as opposed to records “previously withheld in part” (WIP). WIF records are records designated by the ARRB as assassination records which have never been released in any form, either by the ARRB or any other government body investigating the JFK assassination. WIP records have already been released, with redactions, by one or more government agencies or investigations.

How many WIF JFK records does NARA still have? There is a NARA document, released in 2016, which lists them. This list, after correction, had details on 3,598 records. The details of the list, its release and its correction, is a subject I’ll take up in a separate post. Here I am just interested in how many of these 3,598 records have been released in 2017.

Why focus on WIF records? Most of the websites commenting on the releases have focused on the “withheld in full” records because they view them as a metric for “how much more is still to come.” This is not necessarily the right way to look at things, but it is true that 3,598 is now a more or less set number, which will allow us to say when this part of the release process is done. The number of records “withheld in part” is much harder to estimate, as is the point at which we can say the release process for these is also done.

I am also interested in comparing my numbers with the numbers posted by the Mary Ferrell Foundation (MFF), which are now online here. MFF is an authoritative source for JFK assassination record information, but I think in this case they have missed a couple of the problems in the spreadsheets posted by the NARA for each release. These problems I can summarize as follows:

1) There are a number of duplicates among the records (29) released so far. I am not sure of the reason for this, but if you want to know how many of the WIF records have been released, you can’t count these duplicates as if they were distinct files.

2) There are cases where it is not clear how to count the material that the NARA spreadsheets list because two items are listed under one record.

3) There are cases where records are listed inconsistently on the release spreadsheet, sometimes as WIF, other times as WIP. If we are relying on a list of WIF records, this introduces uncertainty into how many files are going to be released in the future.

4) There are cases where there are inconsistencies between the NARA 2016 WIF list, and the spreadsheets for the 2017 releases. In these cases, records on the NARA 2016 WIF list are listed in the release spreadsheets as WIP.

5) Finally, there are records listed in the release spreadsheet as WIF that do not appear in the NARA 2016 list.

These last two cases I will discuss in my next post, when I look at the list of WIF records NARA released in 2016. The rest of this post explains the differences between my counts and the NARA/MFF counts. Here is the main table (numbers in parentheses are where I differ from the NARA spreadsheets):

Release date WIF records WIP records Notes
24 Jul 2017 441 (425) 3369 I omit 16 pdfs for audio files
26 Oct 2017 52 2839 MFF adjusts WIF/WIP numbers to 39 – 2852
3 Nov 2017 583 (561) 93 I omit 22 dup. recs.
9 Nov 2017 4 13209
17 Nov 2017 144 (137) 10600 I omit 7 dup. recs.
Totals 1224 (1179) 30110 MFF adjusts WIF/WIP totals to 1211 – 30123

Following are more detailed explanations of the differences in counts for each release

Release 1 (24 Jul 2017): This release includes 17 audio files of interviews with Yuri Nosenko. With the exception of the first audio file, the spreadsheet for this release includes two items under each of these records: an mp3 audio file, and a pdf file. According to the NARA spreadsheets, the audio files are WIF records. The 16 pdfs listed in the NARA spreadsheet, however, are each one page documents that merely list the tape number and first names of Nosenko’s interrogators.

Note that there are not separate RIFs for the pdf and audio file, and the 2016 NARA file only lists the 17 audio records. Because of this, I do not think the pdfs should be counted, at least not as separate WIF records.

When MFF says that this release had 402 CIA records previously withheld in full, however, they are counting the 16 pdfs. My count therefore has 16 fewer WIF records than theirs.

Release 2 (26 Oct 2017): The spreadsheet for this release lists 52 WIF records. The spreadsheet for the November 09 release, however, changed the status of 13 records marked as WIF to WIP. The spreadsheet for the November 17 release then changed these back to the original status. My count follows the original October figures. MFF uses the 11/9 correction.

Release 3 (3 Nov 2017): This release includes 22 duplicate records, as discussed in my previous post. These 22 duplicates all originally appeared in the 7/24 release. All 22 duplicates were CIA records, previously WIF, and were listed in the 2016 NARA list. It makes no sense to count these records twice, so I subtract them from the total of 583 WIF records for this release to get 561.

Release 4 (9 Nov 2017): I get the same figures as MFF.

Release 5 (17 Nov 2017): This release includes 7 duplicate records, as discussed in my previous post. These 7 duplicates all originally appeared in the 11/3 release. All 7 duplicates were FBI records, previous WIF, and were listed in the NARA 2016 list. It makes no sense to count these twice, so I subtract them from the total of 144 WIF records in this release to get 137.

Totals: I count 45 fewer records than MFF does: 32 fewer WIF records (1179 vs 1211). and 13 fewer WIP records (30110 vs 30123) The reason for the difference between my count and the MFF count is that I didn’t count the 16 pdfs listed under the audio files, subtracted 29 duplicate records, and kept the original WIF status of 13 records. Other than these differences, I have the same count as MFF.

Post script on agencies

There are several records in the releases whose RIFs begin with 179-. MFF lists these as DoJ records, but the actual agencies listed on the RIFs of these records are quite miscellaneous: (WC, NARA, FBI, etc.) For the sake of consistency, in the future I will also label these DoJ records. In addition, there are many records whose RIFs begin with 178-. MFF lists these as Rockefeller Commission records, but the actual agencies listed on the RIFs of these records are even more miscellaneous, including separate collections in the Gerald R. Ford Presidential Library. For the sake of consistency and simplicity, in the future I will follow MFF in labeling all of these as ROCKCOM records. I have also checked the agency counts for all releases in the NARA spreadsheet, and using the DOJ-179 and ROCKCOM-178, I have the same count as MFF, except for the Nov 3 release, where MFF’s agency count for WIF records is in error. [2018/1/24 note: The MFF figures have now been corrected]

Posted in History, JFK ARC | Comments Off on JFK Records Act Releases: Audit

Review of Robert Middlekauff, The Glorious Cause: The American Revolution, 1763-1789

Posted on November 23, 2017 by Robert

Robert Middlekauff has read deeply in the history of the American revolution and the early republic. Moreover, he is interested in more than just a simple narrative; he is interested in causes and motives, as he shows in chaps. 20 and 21 of this book, which discuss why soldiers fought instead of ran.

Unfortunately, the narrative in this book has holes, and Middlekauff often fails to put people and personalities into context, making the reading less interesting than it should be. He also makes high demands on readers’ attention; this, plus the holes, made the book heavy going at times.

Here are some examples of holes: 1) In his discussion of the Intolerable Acts, Middlekauff fails to say what the Quebec Act was, yet on pp. 239 and 280 he assumes you know. 2) On p. 471 he writes: “They all knew what happened to Buford’s men at Waxhaws when they tried to run away.” This is the only time “Buford” and “Waxhaws” are mentioned in the book. 3) On p. 340 Middlekauff says: “June also brought William Howe back to New York.” I can’t find where it says Howe had been in New York before. 4) On p. 462 it reads: “Some hint of what was coming was given …when the victors, shouting ‘Tarleton’s Quarter,’ shot and stabbed the wounded…” There is no explanation of this anywhere in the book. On p. 478 we are told: “… Lee’s Legion rode in. Greene once more had his army in one piece.” This is the first time that “Lee’s Legion” is mentioned. I had to look in the index to find out that “Lee” was Henry Lee. It never explains how he got a legion. The last time we saw him, on 417, he was foraging in Delaware.

No context for people and personalities: Isaac Barre gives a speech supporting the colonies in parliament (74-75), but Middlekauf never tells us who he is or why he speaks so strongly. Directly below, the American who thinks Barre’s speech is “noble” is never identified. Apparently it was Jared Ingersoll, who appears in a very different light in other parts of the book.

Demands on reader’s attention: Pp. 406-7 says that “Amherst told the king…” This is Jeffrey Amherst. The last time we met him, also identified only as “Amherst”, was page 276, where he was fighting Montcalm in Quebec for all of one sentence. Look up Amherst in the index, see where he appears, and see how easy it is to connect these references. This is very tough, demanding writing.

Middlekauff knows the period, is a very intelligent writer, has interesting views and judgments which he backs up effectively. However, if you want to understand what is going on, you will have to go to other books in addition to this one, and you will have to pay very close attention to Middlekauff, with pencil in hand and constant reference to the index.

As an example of a book which brings people and personalities strongly into context, I recommend Barbara Tuchman’s “March of Folly” which has an outstanding chapter called “The British Lose America.” This will tell you who Barre was, why they were drinking toasts to John Wilkes in South Carolina in 1768, and what the Quebec Act was. It’s only a tiny fragment of the history Middlekauff tries to cover, and occasionally Tuchman falls down as well (she mangles the text of Barre’s speech), but it is a great example of fascinating historical writing which historians would do well to study.

Posted in Book reviews | Comments Off on Review of Robert Middlekauff, The Glorious Cause: The American Revolution, 1763-1789

Book reviews coming

Posted on November 22, 2017 by Robert

If you haven’t recently looked at Amazon’s book reviews, these have undergone a massive change. The reviews are still there, but greatly diminished in utility. Perhaps the main value of the old book review system was that it allowed you to look at reviews ranked by “helpfulness”; in this familiar scheme, people vote reviews up or down, and you can then use the voting results to pick out the reviews that are not junk. This was so useful that I even saw Brian Lamb use Amazon reviews to grill authors on his Book Notes show, way back in the 1990s.

This is no longer possible however, for reasons I don’t understand in full. Part of the reason is Amazon’s new “Verified Purchase” campaign. If you are not registered as purchasing the product you review, FROM AMAZON, the default ranking system no longer counts your reviews. (Note that this system also excludes books you might have bought from an ‘Amazon seller’.) This campaign only began in recent years, yet it has now been applied to all previous reviews, across the board. As a result, all reviews from before ‘verified purchases’ began to be counted are now essentially kicked out of the rankings. In addition, the metric for ranking seems to have changed, even for those who have made a ‘verified purchase’, so that a ‘verified purchase’ review with dozens of ‘upvotes’ will not necessarily outrank a ‘verified purchase’ review with only one or two upvotes, or even no upvotes. Age of the review seems to play some part in the new rankings, but other than that I can’t see rhyme or reason for these rankings.

In any case, it is a waste of time to post reviews under this new system, which was intended for consumer goods, not books. I didn’t have many reviews up, but I did spend time on the ones I posted, so in a feeble attempt to recoup some of that effort, I will repost my Amazon reviews here, slightly modified in some cases when I left off interesting stuff to fit into Amazon review limitations. In the future I’ll post all new reviews here.

Posted in Book reviews | Comments Off on Book reviews coming

JFK Records Act Releases: 11-17 Update

Posted on November 19, 2017 by Robert

[Corrected 1/19/2018]

That was fast.

NARA released another set of JFKRA documents on November 17. According to NARA’s press release, there 10744 documents in this set (144 previously withheld in full, 10600 withheld in part), all from the FBI. Once again there are duplicates in this set, this time 7 documents that have the same RIF numbers as do documents in the November 3 release. This puzzled me enough that I went back and checked. Here are the results in a table:

Document RIF 11-03 doc 11-17 doc
124-10211-10454 11-03 11-17
124-90143-10043 11-03 11-17
124-90143-10044 11-03 11-17
124-90143-10430 11-03 11-17
124-90143-10432 11-03 11-17
124-90144-10004 11-03 11-17
124-90144-10078 11-03 11-17

There are actually two pdfs on NARA for each of these documents. Although they are the same documents, with identical RIFs, the pdfs released on 11/17 have removed most of the redactions that were in the 11/03 pdfs. This “double” release seems unnecessary. Why release redacted versions then two weeks later release unredacted versions? Why not just release the unredacted versions on 11/03? Perhaps the FBI had agreed to release the unredacted versions at the last minute and the corrected list didn’t get through to the NARA team posting the documents. Or perhaps not.

Not that big a deal, of course, but it does make one wonder how to do a real document count. I don’t see any point in counting these documents twice; subtract 7 from the official count of 10744 to make 10737 documents released this time. So what about the duplicate documents I noted in my last post? There were all listed twice, first under the 7/24 release, then again under the 10/26 11/03 release. Some of these were real duplicates, i.e. only one document is available from NARA. Here is a list of these:

Identical documents in 7/24 and 10/26 11/03 Releases
104-10196-10330
104-10210-10036
104-10211-10112
104-10230-10078
104-10230-10088
104-10230-10104
104-10230-10139
104-10519-10108
104-10533-10013

In the remaining 13 cases, however, there were two distinct files, listed again below:

Document RIF 7-24 doc 10-26 11-03 doc
104-10268-10001 7-24 10-26 11-03
104-10268-10003 7-24 10-26 11-03
104-10268-10005 7-24 10-26 11-03
104-10268-10007 7-24 10-26 11-03
104-10268-10009 7-24 10-26 11-03
104-10268-10011 7-24 10-26 11-03
104-10268-10013 7-24 10-26 11-03
104-10268-10015 7-24 10-26 11-03
104-10439-10114 7-24 10-26 11-03
104-10512-10245 7-24 10-26 11-03
104-10512-10245 7-24 10-26 11-03
104-10433-10097 7-24 10-26 11-03
104-10433-10097 7-24 10-26 11-03

None of the 7/24 files have RIF sheets attached, while all the 11-03 files do.

[1/19/2018 Correction: This is exactly backwards, all of the 7/24 files have RIF sheets, none of the 11/03 files do.]

All of the RIF sheets [for the 07/24 files] refer to a ‘case NW 53216’, for which I have not yet found any useful references. Other than the RIF sheets, the two sets of files are identical; no redactions seem to appear in any of them, though some of the files are quite long and I can’t 100% guarantee it. Like the 7 duplicate files listed above, I don’t see the point in counting what are actually the same documents twice. Thus in future counts, I will subtract all these 29 duplicates from the total.

There are other glitches in NARA’s spreadsheet of releases worth noting. As one might expect, the guys at the Mary Ferrell Foundation have been keeping an eye on the releases, and they noted that the 11/09 spreadsheet had changed the status of 13 documents released on 10/26, from previously “withheld in full’ to ‘withheld in part’ (see here). As a result, the count of documents released on 10-26 that were ‘previously withheld in full’ went down from 52 to 39.

I checked my tables and found the same thing. The newest spreadsheet, however, has now reverted this change of status back to what it was before. In other words, the 10/26 count of documents previously withheld in full has gone back up to 52. Having spent some time on the counts in my last post, I’m frustrated by this sort of flip-flop, but since they’re posting so much faster than I expected, I’ll hold back on revising my status counts for at least another week or two. Some of my other counts also seem to be different from Mary Ferrell’s as well, so a careful audit is in order.

Posted in History, JFK ARC | Comments Off on JFK Records Act Releases: 11-17 Update