“Those who take oaths to politically powerful secret societies cannot be depended on for loyalty to a democratic republic.”
— John Quincy Adams
Read the Reference Tables, Period analysis, Introduction and Prologue below!
Also available on the kindle page.
Also available on the kindle page.
The Secrecy Web: The Black-Box Republic and the Death of Democracy (1865–2025)
A 160-year autopsy of how secrecy became the real constitution.
The Secrecy Web starts from a simple refusal: the idea that “national security” scandals are accidents, overreaches, or temporary deviations from an otherwise healthy democracy. Instead, it treats the last century and a half as one continuous project—capital building tools in the dark, the state moving in alongside it, and both learning how to protect themselves from scrutiny while talking endlessly about “oversight” and “the rule of law.”
The book starts where most histories of secrecy don’t: not in Washington war rooms, but in the offices of railroad barons and mine owners after the American Civil War. The first web is private. Pinkerton detectives infiltrate unions. Company spies draw up blacklists. Railroads and industrial firms quietly build the earliest worker databases in ledgers and card files. Long before there’s a CIA or NSA, information about workers flows up to bosses and decisions about punishment and exclusion flow down. The state doesn’t invent secrecy; it studies what capital is already doing and takes notes.
From there, The Secrecy Web moves through the 20th century in six long parts and five “phases,” tracking how each crisis supposedly demanding more secrecy actually locks in what the previous phase built.
World War I brings the Espionage Act—still on the books more than a century later—criminalising disclosure to the public under the same heading as spying for a foreign power. Wartime censorship mutates into a permanent classification culture. The interwar period experiments with overlapping agencies and red-baiting raids. Then World War II and the Manhattan Project show something new: you can build whole cities and industrial systems in secret if you fuse state money, private contractors and university labs and put them behind fences. When the dust of Hiroshima settles, what remains isn’t just nuclear weapons—it’s a blueprint for how to run a permanent security structure out of sight.
The early Cold War chapters follow that blueprint as it solidifies. The National Security Act of 1947 creates the modern “national security state”; classification explodes; clearances become tickets to entire career paths. Covert operations overthrow governments in Iran, Guatemala, Chile. Domestic programmes like COINTELPRO target civil rights leaders and the American left. The point isn’t to stack scandal on scandal, but to show the underlying pattern: once secrecy is installed as normal infrastructure, anything that passes through it can be framed as “necessary,” no matter how obviously illegal it would look in daylight.
When we finally get the supposed moment of reckoning in the 1970s—the Church Committee, the new oversight committees, FISA, executive orders limiting assassination—The Secrecy Web leans in on an uncomfortable argument. These reforms don’t tame the system; they teach it how to survive exposure. Oversight becomes theatre. Senators are briefed into silence. Courts approve surveillance they can’t realistically test. Inspectors General publish damning reports that change almost nothing. On the surface, the republic learns its lesson. Underneath, the machine learns to route around damage.
From the late 1970s through the 1990s, the book traces the big structural pivot: functions sliding from agencies into a contractor class that lives in office parks around Washington. Budget pressure to “shrink government” and reform pressure to constrain government operations combine into one answer: move the work to entities that look private, obey trade-secret law instead of freedom-of-information law, and can be shielded with NDAs, state secrets claims, and retroactive immunity. By the time the 21st century begins, thousands of cleared analysts, engineers and operators sit in buildings with corporate logos, doing core intelligence work behind corporate secrecy that never expires.
The post-9/11 section follows what happens when that contractor architecture collides with endless war and the platform era. Intelligence budgets double and redouble. Mass surveillance programmes that were already gesturing toward total capture are suddenly legalised and expanded. Telecommunications firms, banks, data brokers and tech platforms become part of the same security–commercial mesh, feeding data in all directions and relying on the same offshore circuits and legal shields. Snowden’s leaks, the Panama and Paradise Papers, torture memos, targeted killing, “emergency” content takedowns, continuity-of-government exercises, private security firms, NGO front work—it’s all mapped as one structure rather than a pile of disconnected outrages.
By the time we reach the present, The Secrecy Web argues that the United States functions less as a constitutional republic and more as a black-box republic: a system where the real decisions move through code, contracts, classified annexes and private networks that cannot be meaningfully examined or voted on. Elections still happen. Courts still sit. Committees still hold hearings. But the most consequential flows—money, data, targeting, immunity—run across lines the public can’t cross.
The final part of the book doesn’t offer an easy fix. There’s no “ten-point reform plan” or fantasy of a good oversight committee finally doing its job. Instead, it lays out seven structural contradictions at the heart of the secrecy system: protection that doubles as extraction; oversight that legitimates unaccountability; knowledge produced collectively but enclosed as private property; legal order used as a shield for operations that would collapse outside the dark. It’s explicit about the limits of analysis: understanding the web doesn’t dissolve it. What it does is strip away alibis—so that when real social forces move, they hit the right targets.
The whole thing is written in plain language, from an openly Marxist position, for readers who are tired of being told that every new leak is an “aberration” and that the answer is always more training, more ethics guidelines, more “transparency initiatives” managed by the same people who built the problem. It’s for workers, students, public servants, librarians, geeks, and anyone who has ever watched a national security scandal vanish into a classified report and wanted to know what actually happened to all that power.
HOW TO READ THE WEB TABLE
This table maps how modern state secrecy evolved from 1865 to the present. It shows not what officials believed or what reformers tried, but who controlled access to what, how the money and power flowed, and how structural critique was systematically blocked.
Two Ways to Read
Read ACROSS a row to see how one element evolved over time. What controlled access in 1865 (employers, Pinkertons, blacklists) became something entirely different by 2025 (algorithmic flagging, platform terms of service, risk scores). The system didn't simply grow—it transformed while preserving its core function.
Read DOWN a column to see how all interests aligned at one historical moment. In any given phase, the system's promises, its actual functions, its gatekeeping mechanisms, and its suppression of structural critique all interlocked. The table shows how coordination worked—not just what individuals wanted.
The Contradiction Row (→)
This is the engine of the table. Each phase's solution to the previous crisis creates the next crisis. Private infrastructure worked—until the state needed to sort at distance. Legal criminalisation worked—until emergency powers had no sunset. Oversight theatre worked—until contractors became the system itself.
The system doesn't fail because people made mistakes. It transforms because each solution contains the seeds of its own obsolescence.
Master Mechanisms
Each phase has one coordination mechanism that dominates all others. Pinkerton/blacklist (1865–1914) controlled who got sorted. Espionage Act/fusion model (1914–1947) controlled who got criminalised. Classification/permanent agency (1947–1974) controlled who got cleared. Privatisation/oversight theatre (1975–2001) controlled who got immunity. Platform/biometric (2001–present) now controls the infrastructure that absorbs all previous forms.
The previous mechanism doesn't vanish—it becomes subordinate to the new master.
The Suppression Thread
Row 7 tracks how structural critique was blocked in each period—not through a single conspiracy, but through evolving mechanisms. Labour organisers were framed as anarchists. Wartime patriotism silenced dissent. McCarthyism purged left academics. Postmodernism and identity politics displaced class analysis. Platform metrics replaced political consciousness. The suppression didn't require coordination—the structure produced it.
The Invariant Mechanism
Beneath all five phases, one engine persists: three asymmetries plus proof-blocking. Concentrated beneficiaries have career and profit incentives to maintain secrecy; fragmented publics have diffuse, long-term harms that don't mobilise. Beneficiaries know what's secret; publics don't know what they don't know. Beneficiaries are already organised; publics must build coordination from scratch against active resistance. Classification, destruction, immunity, and sealed proceedings block the proof that would enable public coordination.
Self-reinforcing: more secrecy → more power to secret-holders → more capacity to expand secrecy → more secrecy.
Phase IV: The Braided Crisis
Phase IV braids War on Terror expansion, platform capitalism, and biometric/algorithmic systems into a single formation. It has no arrow—this is the current crisis, not yet resolved. All prior contradictions accumulate here: private infrastructure, emergency powers, bureaucratic reproduction, and accountability gaps all persist within the platform-state fusion.
The Core Function
What does the secrecy system actually do? It produces, maintains, and leverages asymmetric access to consequential knowledge while blocking proof of its own operations. Three functions make this work: classification (sorting who knows what for institutions), legitimation (producing 'national security' ideology that naturalises unaccountability), and extraction (generating capabilities, then deploying them for accumulation). The promise changes. The function endures.
Follow the money. Follow the coordination. Follow the suppression.
In 1975, the Church Committee walked into the vaults and discovered that the conspiracy theorists had been lowballing it.
On television, the committee looked like civics class: senators in heavy chairs, wood-panelled rooms, choreographed outrage, cameras panning across legal pads and nameplates. Underneath that stage, staff were working in windowless rooms under fluorescent light, sitting at metal desks while cartons of classified files were wheeled in from CIA, FBI, NSA and Army intelligence stores.
They opened CIA cables about assassination plots against Patrice Lumumba, Fidel Castro, and other foreign leaders. They read FBI memoranda describing the surveillance of Martin Luther King Jr.—including the letter urging him toward suicide. They went through NSA files that showed international telegrams from Americans being copied as a matter of routine. They read Army intelligence reports on infiltrating campus groups, church meetings, and antiwar coalitions.
Over two years, the committee produced fourteen reports. The record was built from documents agencies had fought to bury and from testimony officials had done everything to avoid giving.¹ The hearings sold themselves as a last-chance reckoning. The files told a simpler story: this was how the system worked when nobody was looking. The evidence was overwhelming. The abuses were systematic. The reforms were promised, with all the solemnity of a warranty card.
Twelve years later, in the late 1980s, another set of investigators followed a trail that was never supposed to exist. The Iran–Contra committees dug through safes in the Old Executive Office Building, through memos that had been shredded and then Scotch-taped back together, through flight logs for planes that officially weren’t going anywhere important.
They found a shadow foreign policy run out of the National Security Council and deliberately kept beyond congressional reach. NSC staff—for whom the job description supposedly ended at “coordination”—were running covert operations. Arms sales to Iran—illegal under the embargo—were being used to fund the Nicaraguan Contras—illegal under the Boland Amendments. The scheme ran through private operatives, foreign intermediaries, Swiss bank accounts, and shell companies with names that only made sense if you already knew the joke.
When it blew up, the paper trail filled warehouses: pallets of seized files, stacks of legal pads, shipping records, bank transfers. Fourteen Reagan administration officials were charged. Eleven were convicted. On Christmas Eve 1992, President George H.W. Bush signed pardons for six of them, including people whose trials might have dragged him into the witness box.² A conspiracy that officially didn’t exist ended in a televised ritual of absolution from the man who helped run it.
In 2004, the evidence arrived as photographs. First on a late-night news segment, then on front pages, then as images burned into the public memory: naked prisoners stacked into pyramids; a hooded figure in a poncho, standing on a box with arms outstretched, wires taped to his fingers; soldiers smiling and giving thumbs-up beside bruised bodies. There was the dog baring its teeth at a cowering detainee. There was the woman with the leash, cigarette in hand, posing over a man lying on the floor.
The Army opened investigations and convened courts-martial. Eleven low-ranking soldiers were convicted. The highest-ranking officer punished—a brigadier general—was reprimanded and demoted. The photos were turned into the story of “a few bad apples” in one “out-of-control” unit. The lawyers who drafted the memos authorising “enhanced interrogation,” the senior officials who signed the programme, the CIA officers who designed and ran it, and the contractors who supplied interrogators went on with their careers. No criminal consequences.³ Apparently the rule of law has a height requirement.
In June 2013, The Guardian published a grainy scan of a secret court order and detonated a different kind of scandal. The document, stamped TOP SECRET and signed by a Foreign Intelligence Surveillance Court judge, ordered Verizon to hand over telephone metadata on virtually every call its U.S. customers made. For years, Congress and the public had been told that NSA surveillance “targeted” foreign terrorists. The order described a domestic dragnet.
Then the rest of Edward Snowden’s archive began to surface. PRISM: direct collection from major internet companies. XKeyscore: an interface that let analysts pull up emails, chats, and browsing histories like a search engine for private lives, without individual warrants. Upstream collection: interception at the fibre-optic backbone of the internet. Each leak sketched another section of an architecture that had been quietly normalised while officials lied or misled under oath.⁴
The government confirmed the programmes, in the careful language of lawyers who know perjury is a crime but public credulity is not. Newspapers on three continents printed the documents. The USA FREEDOM Act of 2015 followed—renaming programmes, shuffling legal authorities, producing diagrams for PowerPoint decks. The basic logic of bulk collection survived just fine.
In December 2014, after years of trench warfare with the CIA, the Senate Intelligence Committee released the executive summary of its study of the agency’s detention and interrogation programme. Staff had spent years in a secure facility reading cables, medical files, internal emails, and contractor invoices. The full report runs to roughly 6,700 pages.⁵ Only a 525-page summary was released. It was more than enough.
The summary described detainees strapped to boards and waterboarded again and again; “rectal feeding” and “rectal rehydration”—medical euphemisms for rape; sleep deprivation that lasted more than a week; detainees left naked and shackled in rooms chilled close to freezing; confinement in coffin-sized boxes. It documented the CIA’s false claims that torture had produced crucial intelligence, and its efforts to obstruct the committee’s investigation, including monitoring computers the staff were using to review CIA documents.
Even this partial account only emerged after a pitched battle between a Senate committee and the agency it was supposed to oversee. The full report remains locked in Senate vaults, waiting for a future staffer or a future shredder. No-one was prosecuted. A few careers cooled; some consulting fees went up.
The evidence exists. It has existed for a long time. The question is why, after all that, the machine that produced it is still running on schedule.
This is not a book about intelligence scandals.
Scandal is the lens that keeps the structure out of view. Each exposure is treated as exceptional: a deviation from normal practice, an “abuse” by individuals, an “intelligence failure” that better procedures will fix. Church leads to new oversight committees and forms. Iran–Contra leads to stern editorials, a handful of indictments, and Christmas pardons. Abu Ghraib leads to some courts-martial, updated training slides, and a fresh round of statements about “who we really are.” Snowden leads to rebranded surveillance and a privacy office with a logo. The torture report lands with a thud and is quietly filed away.
None of it alters the basic machine. It absorbs the blow, adds another line item for “lessons learned,” and continues.
Across half a century of investigations, leaks, and declassifications, the pattern repeats in almost insulting detail. Abuse, investigation, reform, closure. Then the next abuse, the next investigation, the next promise that “this time” the safeguards will work. The scandal frame breaks the continuity and invites amnesia.
This book refuses that invitation. Institutions are not treated here as fundamentally sound with a few psychopaths sprinkled through the org chart. Collapses are not lined up and presented as if the resulting rule changes were genuine repairs. The worst operations are not written off as the work of lone rogues who somehow found the keys to the most sensitive systems on earth.
The reality is structural. The congressional inquiries, the court cases, the declassified files, the whistleblower archives, the foreign commissions—they belong to one continuous history. Each opens a window onto the same machine. That machine reliably generates the outcomes we see. It also reliably generates its own “reforms,” which function as armour rather than constraint.
The chapters that follow track that machine and offer a framework that makes the fragments legible as system.
The Evidence Inventory
The evidence is not hidden in a single sealed room. It’s worse than that: it’s lying all over the place in pieces—across archives, in bound congressional volumes, in badly scanned PDFs on activist websites, in foreign court records nobody in Washington reads, in depositions from lawsuits that were terminated a decade ago. The problem is not a shortage of information. The problem is the way it has been sliced up and mislabelled.
Congressional investigations provided much of the spine. The La Follette Committee, sitting from 1936 to 1941, dragged corporate executives, private detectives and union organisers into hearing rooms and forced them to describe, under oath, how industrial America actually policed itself. Its reports documented how corporations hired private detective agencies to surveil workers, infiltrate unions, and coordinate strikebreaking on an industrial scale: Pinkerton and Burns detectives on retainer; armouries of rifles, machine guns and tear gas stored for “labour disputes”; networks of labour spies planted inside organising campaigns.⁶
The Nye Committee, from 1934 to 1936, went looking for the “merchants of death.” It traced the munitions firms that had pushed for U.S. entry into World War I and then fed on contracts; it followed bank loans that only made sense if the Allies won. For a moment, under the lights, it was possible to see how decisions about war and peace were being made with ledgers in hand.
Four decades later, the Church Committee assembled the most comprehensive record of U.S. intelligence abuses in history: fourteen public reports; thousands of pages of hearings; staff studies on everything from assassination plotting to opening mail in bulk. The Pike Committee’s parallel House inquiry was formally suppressed by a vote, then leaked to journalist Daniel Schorr and published anyway. The Iran–Contra committees added their own mountain of documents and testimony—cables, diaries, legal opinions, presidential findings. The 9/11 Commission mapped the “intelligence failures” and inter-agency knife fights that were sold as tragic accidents and looked, under the microscope, like the predictable result of secrecy, compartmentalisation and bureaucratic self-interest.
Court records captured parts of the machine that Congress never managed to pry loose. Decades of Freedom of Information Act litigation—much of it driven by the National Security Archive at George Washington University—forced agencies to disgorge material they had every intention of hiding indefinitely. Torture victim lawsuits opened narrow discovery windows before the state secrets privilege slammed shut.
When Maher Arar, a Canadian citizen, sued over his rendition to Syria, the U.S. government intervened to kill the case on “national security” grounds. Before the door closed, affidavits, filings and limited discovery set out parts of the system that had grabbed him off a New York layover and sent him to a cell in Damascus.⁷ Contractor fraud prosecutions, brought under qui tam provisions that let whistleblowers sue on the government’s behalf, exposed billing practices and performance data that no oversight committee had thought to ask for. You learn a lot about a system when it has to itemise what it charged you to commit the crime.
Declassified files accumulated slowly, through mandatory review schedules, relentless FOIA requests, clerical accidents and the occasional desire by a retiring official to clean their conscience on the way out. The MKULTRA records survived because they were misfiled in budget archives when CIA Director Richard Helms ordered the programme’s destruction in 1973. A filing error preserved about 20,000 pages of documents, which later surfaced through a FOIA request in 1977 and revealed years of drug and “behaviour modification” experiments on unwitting subjects.⁸
COINTELPRO files came to light after activists broke into an FBI office in Media, Pennsylvania in 1971, stole documents, and mailed them to reporters. The papers laid out a campaign aimed at “disrupting, discrediting and neutralising” civil rights organisations, Black liberation movements and the New Left. Internal memos described tactics that would not look out of place in a manual on psychological warfare. NSA collection programmes were formally acknowledged through declassification only after Snowden made continued denial unworkable.
Whistleblowers supplied what classification was designed to conceal and what “oversight” chose not to touch. Daniel Ellsberg photocopied the Pentagon Papers in 1969, dropping thousands of pages of internal analysis into public view and demonstrating that the government had been lying about the Vietnam War for years. Mark Felt—“Deep Throat”—met Woodward and Bernstein in parking garages and gave them the confirmation they needed to keep the Watergate investigation alive when editors and owners started to feel nervous.
Thomas Drake, William Binney and J. Kirk Wiebe tried to expose NSA illegality and waste through internal channels and inspector general complaints. They discovered that those channels were better at identifying dissidents than at correcting abuses. Snowden watched that history from inside the same institution and drew the obvious conclusion. Chelsea Manning released battlefield reports and diplomatic cables that showed what the occupations of Iraq and Afghanistan looked like from the ground up: civilians killed and then erased in euphemism, detainees abused, double-talk written down in plain English. Daniel Hale disclosed documents proving that drone strikes killed far more civilians than official figures admitted. He went to prison for it. The people who ran the programme did not.
Foreign archives provided evidence that American secrecy rules were specifically designed to block. The Chilcot Inquiry in the United Kingdom, sitting for seven years, reconstructed how Britain ended up in the Iraq War. Its documents—including correspondence with the Bush administration that American courts would never have ordered released—showed how a supposedly agonised decision had been treated as inevitable long before the public debate.
In Germany, the Bundestag’s inquiry into NSA surveillance forced intelligence officials to testify under oath about programmes and data-sharing arrangements that had never been described in U.S. hearings. In Italy, prosecutors tried and won criminal convictions against twenty-three Americans—twenty-two CIA officers and a U.S. Air Force colonel—for the kidnapping of Hassan Mustafa Osama Nasr (Abu Omar) from a Milan street, in a rendition case the United States would never have brought itself.⁹
In Latin America, truth commissions sifted through decaying military archives and live testimony to document Operation Condor: a cross-border system of kidnapping, torture and murder coordinated among South American dictatorships with U.S. support and knowledge. The archive of the machine was rebuilt out of the very paperwork it had produced to keep itself running.
The evidence is not thin. It is overflowing. What has been missing is a framework that connects those fragments into a single, visible structure—and shows how a system sold as “security” has, over and over, functioned as a protected channel for accumulation and control.
The Framework
Secrecy is not just a pile of classified documents. It is a system: spread across agencies and corporations, held together by rules and incentives, staffed by people in specific roles, driven by contradictions it cannot resolve, and constantly reproducing itself through the operations it hides.
It works through a set of institutional sites that straddle the public–private line. On the government side: the formal U.S. Intelligence Community, with seventeen named agencies and more units tucked into classified annexes; specialised military commands; law-enforcement intelligence arms and “fusion centres”; regulators who police money and sanctions. On the private side: defence and intelligence contractors ranging from behemoths with hundred-billion-dollar market caps to boutique companies that effectively are one contract; law firms that act as rented shields and attack dogs; lobbying outfits that protect budgets and kill reform bills in committee; financial institutions that move money through offshore routes; platform companies whose cables, clouds and apps now handle everyday communication while quietly partnering with state collection.
Coordination does not require a single mastermind. Classification rules decide what can be known and by whom. The career logic pushes toward over-classification, because stamping something SECRET never ruins you, while leaving something unclassified that later becomes controversial can. Clearance systems filter who is allowed near sensitive programmes, selecting people whose livelihoods depend on not rocking the boat. Contracts funnel profit, creating private constituencies whose business model depends on secrecy budgets rising, not falling.
Legal shields—sovereign immunity, qualified immunity, the state secrets privilege, prosecutorial discretion—define who is effectively untouchable. Employment conditions enforce silence through security agreements, non-disclosure obligations, and the implicit threat that a clearance can be pulled like a rug. Oversight architecture—committees, courts, inspectors general—provides a performance of scrutiny which, by existing, reassures the public that someone is “keeping an eye on things.”
Inside that structure, people occupy specific roles. Operators carry out work that cannot be admitted in public: Pinkerton detectives infiltrating unions and breaking strikes; CIA officers running coups and covert wars; Blackwater contractors guarding convoys and interrogating detainees; platform trust-and-safety staff deleting posts and suspending accounts in line with opaque “community guidelines” that always seem to line up neatly with state priorities.
Overseers look in and, by looking, bless the process. Congressional committee members sit in sealed rooms for briefings curated by the very agencies they are meant to supervise. FISA Court judges approve 99.97 percent of surveillance requests brought before them.¹⁰ Inspectors general know their job is to “restore confidence,” not burn the building down. The ones who forget can be fired, sidelined, or starved of resources.
Contractors profit from opacity—first as vendors, then as embedded workforce, eventually as infrastructure officials insist the state “cannot do without.” Classifiers reproduce secrecy through habit: the mid-level bureaucrat whose safest move is always one more stamp. Whistleblowers interrupt the system and are turned into examples. Increasingly, they are charged under the Espionage Act—a law written for spies—even when their disclosures go to journalists and the public rather than foreign governments. Targets live inside the consequences: workers surveilled and blacklisted; dissidents prosecuted; foreign populations subjected to coups, raids and drone strikes; platform users harvested, ranked and nudged. Journalists mediate between machine and public, their access dependent on sources whose careers and loyalties are entangled with the system they leak about. Lawyers and judges provide the script, writing opinions and injunctions that turn violence and fraud into “policy,” “operations,” and “tools.”
The machine runs on contradictions it cannot resolve. At its centre sits a simple one: it has to function as an engine of private accumulation while presenting itself as an instrument of collective security. “National security” is the story told to justify the budgets, the secrecy and the casualties. Accumulation is the underlying content—who gets paid, who is protected, which firms live off which programmes.
The contractor billing $500 an hour for work a government employee could do for a fraction of that rate is not an embarrassing outlier. That invoice is the point: a channel through which public funds become private profit, insulated from scrutiny by classification and by the taboo against questioning anything wrapped in the right acronyms. Around that core sit recurring tensions: protection versus extraction; oversight versus theatre; democracy versus unaccountability; knowledge versus enclosure; care versus coercion; legality versus operation. A further internal tension slices through them all: secrecy demands compartmentalisation, while complex operations demand coordination. The result is a system permanently at war with itself—divided into silos and factions, stitched together by informal networks and personal deals, and periodically blindsided when the left hand literally cannot be told what the right hand is doing.
The system reproduces itself through its own motion. Successful concealment strengthens the argument that similar operations must remain secret. Failed exposure attempts justify tightening internal controls and punishing the people who tried to speak. Captured overseers—committee chairs, judges, inspector general offices that learn to flinch—become stamps of approval on the next programme. Profitable contracts fund lobbying for more of the same. Legal victories for immunity and state secrets doctrine protect the next round of violations before they happen. “Reform” generates new layers of oversight theatre that absorb public anger and stabilise the underlying machinery.
Exposure, handled correctly from the system’s point of view, is not a threat. It is an opportunity: identify a few expendable figures, rearrange the flowchart, update the talking points, and explain that the problem has been solved.
The Method
The pages ahead are not hunting for the next leak. They are reorganising what has already been put on the record.
The method is straightforward. Take what has been proved—often in official settings—and read it as evidence of a single system rather than as a pile of unrelated scandals. Once that step is taken, inferences about what is happening now follow from pattern and incentive, not from fantasy.
Doing this properly means being strict about what counts as what. Not every claim rests on the same kind of support. Not every unsettling allegation belongs next to a declassified cable or a sworn admission. Evidence is sorted into three categories, and the text signals which kind of material underpins which kind of statement.
Proved refers to claims grounded in documentary record, sworn testimony, or official admission. Congressional findings backed by primary documents. Court records, including discovery obtained before the state secrets privilege cut a case short. Whistleblower disclosures that have been independently corroborated. Government acknowledgments, however grudging. Foreign legal proceedings that meet ordinary evidentiary standards. Assertions presented here as factual draw on this category.
Plausible covers claims that fit the proved material, have not been disproved, and are supported by pattern and circumstance but still lack direct documentation. These are inferences from what we already know and implications of structures that have been mapped. When a plausible reading is advanced, it is marked as such.
Indeterminate is the space where there is not yet enough information. Ongoing operations whose documentary trail remains locked. Internal decision-making that may exist only in classified archives. When the evidence is too thin to support a judgment, the question is left open.
This discipline is what separates structural analysis from free-floating conspiracy talk. Conspiracies in the literal sense—plans hatched by specific groups and carried out in secret—are common. The historical record is full of them: the assassination plots catalogued by the Church Committee, the parallel networks exposed in Iran–Contra, the documented lies about mass surveillance before and after Snowden. Treating that coordination as unthinkable is as unserious as treating it as the answer to every question.
The point is to anchor any claim of conspiracy in evidence and to show how those episodes sit inside a larger system that does not need a single mastermind to keep operating.
The proved evidence is already devastating. Speculation is less a contribution than a temptation. It gives defenders an easy way out: attack the weakest allegation and pretend the rest falls with it. The analysis stays as close as possible to what can be shown. When it moves beyond that into inference, it labels the step.
The Five Phases
The secrecy system did not arrive fully formed. Across roughly 160 years it moved through five phases. Each phase reworked the same basic tension—security as story, accumulation as content—under new historical conditions. Each claimed to fix problems inherited from the last. Each generated its own reforms, which then became the next round of workarounds.
Phase 0 (1865–1914): Private Coercion + State Partnership. After the Civil War, corporate capital built the first modern enforcement system long before the federal government created its own. Pinkerton’s National Detective Agency, founded in 1850, grew to employ more agents than the U.S. Army had soldiers. Pinkerton and its rivals—Burns, Thiel, Baldwin–Felts—sold labour surveillance, strikebreaking, and industrial espionage to mine owners, steel magnates, and rail barons.
The state provided law and muscle: injunctions against strikes; conspiracy prosecutions aimed at organisers; National Guard and federal troop deployments to break picket lines. Personnel moved back and forth between private and public roles. Secrecy served accumulation directly. There was no need to dress it up in national security language.
Phase I (1914–1947): Legal Criminalisation + Fusion Model. World War I rewired the landscape. The Espionage Act of 1917 and the Sedition Act of 1918 criminalised political dissent and remained on the books long after the war. The Committee on Public Information ran propaganda at home and abroad. The wartime state absorbed private coercive capacity and scaled up its own. The American Protective League—a volunteer organisation with Justice Department blessing—conducted surveillance and raids while wearing badges that looked official enough to most of the people they stopped.
When the armistice came, the machinery did not pack up. The Palmer Raids, the first Red Scare, and the Bureau of Investigation’s expansion under J. Edgar Hoover extended wartime methods into peacetime permanence, now aimed at “subversives” and “radicals” rather than enemy spies.
Phase II (1947–1974): Classification Bureaucracy + Permanent Agencies. The National Security Act of 1947 created permanent institutions—the CIA, the National Security Council, a unified Defence Department—and hardened the executive-branch classification system. Cold War rivalry justified permanent mobilisation and permanent secrecy. Contractor ties deepened into structural dependency: private firms designed weapons systems, built listening posts, ran logistics.
Covert action became an everyday instrument: coups in Iran and Guatemala; assassination plotting in the Congo, Cuba, and elsewhere; MKULTRA’s experiments on unwitting subjects; COINTELPRO’s campaigns against civil rights groups, Black liberation movements, and the New Left. The structure expanded until it collided with its own contradictions: the Pentagon Papers, which showed systematic lying about Vietnam; Watergate, which exposed a White House using intelligence tools at home; Seymour Hersh’s reporting on CIA domestic spying; the Church and Pike investigations, which dragged some of this into the open. The exposures were real. The answer was to build a new layer of oversight machinery.
Phase III (1975–2001): Privatisation + Oversight Theatre. The reforms that followed Church produced committees, reporting requirements, inspectors general, and the Foreign Intelligence Surveillance Court. Secret operations did not stop; they changed shape. Privatisation accelerated—for deniability, for profit, and because permanent mobilisation demanded more labour than the civil service could supply. The contractor workforce grew, then overtook the official one.
Iran–Contra revealed the new configuration: operations run by NSC staff, funded outside normal appropriations through arms deals and third-country money, organised through private channels, carried out by contractors and foreign proxies, kept away from the committees nominally watching. When the scandal broke, a handful of prosecutions followed. Presidential pardons cleaned the slate. The structure remained. Oliver North went from the witness table to television hosting. The oversight machinery continued to sit, listen, and nod.
Phase IV (2001–2025): Data Infrastructure + Platform Fusion. The attacks of September 11, 2001 unlocked tools that had been in development through the 1990s and licensed their expansion beyond anything previously attempted. The USA PATRIOT Act passed six weeks after the attacks. Legal limits dissolved, budget lines exploded, and suspicion became policy.
The contractor boom turned into an ecosystem. By 2010, 1,931 private companies were working on counterterrorism, homeland security, and intelligence programmes at more than 10,000 locations across the United States.¹¹ Mass collection turned into infrastructure: the default setting rather than a special exception requiring specific justification. Platform companies—telecoms, internet giants, cloud providers—became partners and co-architects, supplying data, building systems, and knitting their services into state collection and analysis. Snowden’s leaks in 2013 exposed much of this architecture. Hearings were held. Programmes were renamed. Compliance officers updated training slides. The core machine stayed where it was.
The Thesis
Under capitalism, secrecy is not a temporary malfunction that better ethics training or a new round of regulations will fix. It is a core feature that allows accumulation on a scale open scrutiny would not tolerate.
Reform is blocked from the inside. The people and institutions that benefit from the secrecy system control the mechanisms that would have to change it. Clearance rules decide who is even allowed to see the evidence. Classification hides the most revealing material from the public and from most of Congress. Contractor dependency creates a constituency—millions of employees, shareholders, and pension funds—for ever-expanding secret budgets. The revolving door moves people between oversight posts and the industries they were meant to police. Oversight committees are briefed by the agencies they supervise, in sealed rooms, on terms the agencies set.
Seen from this angle, oversight is not constantly “failing.” It is doing the work it has been selected and shaped to do. The function is to confer legitimacy, not to impose real constraint. Congressional committees receive briefings they cannot share. Their members are bound by secrecy rules that prevent them from telling the public what they know. The FISA Court hears only the government’s side, approves almost every request, and issues opinions the public cannot read. Inspectors general who dig too aggressively can be fired, sidelined, or quietly starved of resources. The existence of these bodies is then cited as proof that “checks and balances” are in good health.
The machine is visible. Its existence is not a theory. The record has been piling up, in plain sight and in official documents, for half a century. What has been missing is the willingness to see system where we are trained to see scandal; structure where we are invited to see aberration; design where we are encouraged to see failure.
The work here is to supply that frame.
The immediate question is not how to fine-tune the machine with a better reform package. The history of the five phases suggests that reform is one of the machine’s outputs—a way of stabilising itself when pressure builds. The deeper question is whether the conditions that make such a machine necessary for the current order can themselves be changed.
The evidence follows.
The alarm sounds at 5:47 a.m. Thirteen minutes before six, because the drive takes forty-five minutes without traffic and an hour fifteen with, and the badge has to scan at the outer perimeter before 7:30 or the system flags it, and a flag means a conversation with Security, and a conversation with Security is never just a conversation.
She lies in the dark for a moment. The house is quiet. Her husband won’t wake for another hour; he works from home now, a small software company, no clearance required, no polygraph, no need to account for every foreign contact or explain the gap year in college when she backpacked through Eastern Europe and once shared a hostel room with a Russian exchange student whose last name she cannot remember and had to estimate on the SF-86.
The SF-86. Standard Form 86, Questionnaire for National Security Positions. One hundred twenty-seven pages when she first filled it out, eleven years ago. Every address for the past decade. Every foreign trip. Every financial account. Every relative, their addresses, their citizenship status. The names of people who had known her at each address, who could verify she had lived there, who could confirm she was who she claimed to be. References who would be interviewed by investigators asking about her drinking, her politics, her sex life, her debts.
She filled it out again three years ago, for the periodic reinvestigation. The same questions. New addresses to document, new foreign contacts to disclose—a work trip to a conference in London, a vacation in Mexico, the Brazilian colleague she emailed about a technical problem. Each contact a potential flag. Each omission a potential felony.
She gets out of bed.
The shower is hot and the coffee is already made, programmed the night before. She dresses in the dark, business casual, nothing memorable. The badge goes around her neck on the lanyard with the logo of her employer—not the government, but a contractor, one of the large ones, a name on office parks across Northern Virginia and on earnings reports that mention the percentage of revenue from classified contracts without ever saying what those contracts involve.
The commute is muscle memory. Neighbourhood streets to the parkway, parkway to the interstate, interstate to the exit that leads to a road that isn’t marked on commercial maps but shows up on the traffic app as a grey strip with no name, only an estimated arrival time. The road ends at a cluster of buildings that could belong to any business: mirrored glass, beige concrete, landscaping that looks the same in every season because it’s plastic. The sign at the entrance names the contractor, not the client.
The first badge scan is at the outer perimeter. A booth, a guard, a reader. The badge carries a chip linked to a database that confirms she is authorised to enter today. Yesterday’s authorisation does not guarantee today’s. The system checks every time. The gate lifts. She drives through.
The parking structure is half full at 7:15. Early risers. People who have learned that the later you arrive, the longer Security takes. People whose projects live on deadlines that exist in a world they cannot describe to spouses, parents, children, or the college roommate who has stopped asking what she actually does all day.
Second badge scan at the parking structure entrance. Third at the building door. Fourth at the turnstile in the lobby. The security officer behind the desk nods, a face she recognises but whose name she has never learned. Her phone—photos of the kids, texts from her husband, a running app, the debris of a normal life—goes into Locker 247. She has had the same locker for six years. The key joins the badge on the lanyard.
No personal electronics past this point.
Fifth badge scan at the inner door. The door is heavier than it looks, soundproofed and shielded. On the other side, the SCIF: Sensitive Compartmented Information Facility. Windowless. Climate-controlled. The air is a constant low hum from the HVAC and the white-noise generators that mask conversations from the next row.
Her workstation is one in a grid of identical workstations. Ergonomic chair, twin monitors, government-issue keyboard. She logs in with her badge and PIN—the PIN she has to change every ninety days and cannot write down and cannot reuse for twenty cycles, which means she is always one morning away from locking herself out and having to phone the help desk and explain why she can remember IPv6 addressing schemes but not eight digits she invented last quarter.
The desktop loads. Email first. Always email first. The classified system looks like Outlook and behaves like Outlook and isn’t Outlook. Messages live on servers she will never see, in facilities she will never visit. The inbox has the usual overnight sediment: status updates, meeting invitations, distribution list spam. She scans for anything that screams. Nothing screams. A normal morning.
Then she sees it.
The notification sits in the corner of the screen. Not an email. A system message. The kind that comes from IT, Security, or HR. The kind that almost never appears and never carries good news.
PLEASE CONTACT THE SECURITY OFFICE AT YOUR EARLIEST CONVENIENCE.
Not “immediately.” Not “urgent.” Just “at your earliest convenience”—the phrase Security uses when they don’t want a scene in the aisle, when they want the employee to walk down the hall on their own rather than be walked, when things are serious enough for a talk but not serious enough—yet—for an escort.
She stares at the message. The cursor blinks. The office around her keeps its quiet rhythm: keyboards, chairs, the soft click of a stapler, the white hiss of the vents. No one is looking at her. No one knows.
She runs through the list.
The foreign contacts are all disclosed. Finances are clean—no debt beyond the mortgage, no gambling, no odd deposits. No calls to journalists. No files on unauthorised drives. No photos of screens. No notes carried out. No offhand classified remarks at a restaurant because she has internalised the rule that the world is an “uncontrolled environment.” She has played by every rule, initialled every page, sat through every annual training.
She also knows that innocence and safety are different categories. The system does not care about her conscience. It cares about patterns.
The algorithm that monitors her communications does not know why she emailed a particular colleague more often last month; it knows only that the frequency spiked. The badge readers that log her entries and exits do not know why she stayed late three Thursdays in a row; they know only that her curve changed. The financial monitoring that pulls her credit report twice a year does not know why the card balance jumped in December; it knows only that it did.
Now the system has decided that something belongs in a file.
She should go straight away. “At your earliest convenience” means now. Waiting will be logged. Delay becomes another data point in a pattern she will never see.
Instead, she sits for a moment, watching the cursor blink, and thinks about the collar.
She has used that word silently for years. Never aloud. Never in writing. The clearance is the collar. It fits so well that most days she forgets she is wearing it. It is the credential that opens doors, the line on a résumé that justifies a salary forty percent higher than the unclassified world would pay, the membership card for a community of people who know things and do things and matter in ways that cannot be discussed over dinner. The clearance is the identity.
The clearance is also the leash.
Leaving is possible in theory and expensive in practice. The skills she has built over eleven years are skills she cannot fully describe to a civilian employer. The projects are classified; the work product is locked in vaults. The expertise is real and undocumented. References who know what she can do cannot talk about how they know it. They can say “she’s reliable” and “she works hard” and then run out of unclassified synonyms.
Even if she quits, the clearance does not really go away. The non-disclosure agreements she has signed—the SF-312, the programme-level forms, the contractor’s own paperwork—bind her for life. What she has seen, she cannot describe. What she has done, she cannot narrate. The collar is not a workplace rule; it is a condition of biography.
She thinks about the people she knows who have lost theirs. The colleague whose divorce turned toxic, whose ex invented allegations that were never proved but were enough to trigger an investigation, then a suspension, then a year of limbo that ended with a quiet resignation and a job that pays half as much. The analyst who sought treatment for depression, disclosed it as required, and discovered that “doing the right thing” meant eighteen months of questions about her stability. The project manager who stepped on a data tripwire because he had once rented a flat in a building where someone on a watchlist also lived. Wrong building, wrong time, wrong database join.
None of them betrayed secrets. All of them lost years. Some lost careers, marriages, houses.
She logs out of the workstation. She stands. She walks toward Security.
The Security office is at the end of the corridor, past the conference rooms and the break area with coffee that tastes as if it was brewed yesterday and reheated by policy. The door is unmarked except for a room number. She knocks.
“Come in.”
The room is small. Desk, two chairs, a computer, a filing cabinet that presumably contains files she will never see. The security officer is a man she recognises, middle-aged, practiced friendliness, the kind of easy tone that invites people to talk and keeps them forgetting that “anything you say” is taken down and filed.
“Thanks for coming by,” he says. “Have a seat.”
She sits. The chair is uncomfortable. The chairs in Security are always uncomfortable. She has wondered whether this is a subtle technique or just the result of whatever bid came in lowest.
“This is just a routine matter,” he says. “Nothing to worry about.”
She files that in the drawer marked “phrases that never mean what they say” and waits.
“We’ve noticed some anomalies in your access patterns,” he says. “Nothing serious. Just some things we need to clarify.”
Anomalies. The word the system uses when ordinary behaviour has been recast as data.
“What kind of anomalies?” she asks.
“You’ve been accessing some files outside your normal project scope. Probably nothing—people get curious, look at things they don’t need for their immediate work. But we need to document why.”
She tries to reconstruct her own movements. The system she works on is sprawling and tangled; any non-trivial task involves following links from one database to another. Hundreds of files in a month, thousands over a quarter. She has no idea which ones the algorithm has tagged.
“I’d have to see specifically what you mean,” she says. “I access a lot of material for my work.”
“Of course.” He turns his monitor toward her.
On the screen: time stamps, file names, identifiers. She recognises some. Others mean nothing. A few are clearly related to her main project, just not labelled that way. A few might be mis-clicks. A few were probably opened because a colleague mentioned them in a meeting and she wanted context. All of them live on systems she is cleared to use. She has the access. She has the need-to-know, at least in the loose sense in which work in this building ever happens.
Work does not care about how a model classifies it.
“Some of these are project-related,” she says. “Some are likely accidental. I don’t remember every click.”
“That’s fine,” he says. Still friendly. “We just need to document it. I’ll write up a summary, you sign it, and we’re done.”
She knows the choreography. Conversation. Summary. Signature. The piece of paper that confirms the talk took place and staples her name to the anomaly forever. At the next reinvestigation, an adjudicator will see it. If there is ever a second incident, this one becomes “a pattern.”
She signs.
“Thanks,” he says. “Like I said, nothing to worry about.”
She walks back to her workstation. The cursor is still blinking. New emails have arrived. The day resumes its shape.
Nothing to worry about.
This person is not unusual.
As of 2019, more than four million Americans held security clearances. Over 1.3 million held clearances at the Top Secret level or above.¹ Those numbers have climbed steadily since September 11, 2001, as the security system expanded and the contractor workforce that feeds it ballooned. The intelligence budget—the part the government admits to—now runs above ninety billion dollars a year.² The part it does not admit to is, by design, unknowable.
Most of the people behind those numbers look nothing like the pop-culture stock characters. They are not Jason Bourne in disguise, not slick operators with passports in five names, not lone geniuses spotting threats in a blizzard of feeds. They are IT support and facilities management, HR and accounting, logistics and procurement. They keep the lights on, the servers patched, the clearances processed, the paperwork filed. Their jobs put them near classified information, so they must be cleared. Their presence in classified spaces requires investigation, so they are investigated. Once they are inside, their behaviour is monitored because the system monitors everyone inside it.
Most will never be summoned to Security except to replace a lost badge or sit through a briefing. Their movements through checkpoints and turnstiles will blur into routine. Their anomalies will be minor enough to sink into the statistical background. They will retire the way they entered: processed, documented, and managed, the control so ordinary that it barely registers as control.
All of them wear the collar.
The clearance system is sold as protection. The investigations filter out people who might sell or leak secrets. The monitoring catches insider threats before they act. The compartmentalisation limits damage when something goes wrong. Protect the secrets, protect the nation.
There is some truth in this. Some secrets do need protecting. Some insider threats are real. Some background checks catch problems that would otherwise go unseen.
If that were the whole story, the cleared population would be as small as possible, the monitoring would be narrowly focused on real security indicators, and the contractors who profit from the system would be kept at arm’s length from its gatekeeping.
That is not how it works.
The clearance system also disciplines labour. It shapes who can organise and what they can say. It constrains exit options. It punishes complaint.
The cleared employee cannot easily organise. Union activity requires meetings, messaging, lists of names. In the classified world, communication systems are monitored and external organising tools are barred at the door with the phones. The cleared employee cannot easily leave. Their specific skills are tied to classified projects; their best work is trapped behind access rules; the employers who can use that experience are the same small cluster of firms and agencies that already hold their files.
The cleared employee cannot easily fight back. A grievance generates documents. Documents go into the security file. The security file is opened at the next review by someone whose job is to look for reasons not to renew access. The rational move is to keep your head down and hope the next anomaly belongs to someone else.
The collar disciplines without visible coercion. No batons, no tear gas, no riot police. The mortgage depends on the salary. The salary depends on the job. The job depends on the clearance. The clearance depends on a continuous demonstration of “reliability” whose content is defined by the same system that writes your performance review.
The contractor, meanwhile, profits twice. The contract work itself is billed at rates that would be shredded in any market with real competition; the classified market is protected by clearance requirements that incumbents help administer. On top of that, the discipline produced by the system is a bonus. A workforce that cannot easily leak, organise, or walk away is a very valuable asset.
No one had to sit in a room and design it for this purpose. The thing grew, step by step, through decisions that made sense in isolation: more vetting after a scandal, more monitoring after a breach, more contractors after a surge in workload, more rules after an embarrassing leak. Each move addressed an immediate “security problem” on its own terms. The result, over time, is a system that reliably serves accumulation while presenting as protection.
That result is not accidental enough to be dismissed as an unfortunate side effect. It is what happens when every “pragmatic” decision is made inside the same set of incentives.
And because the system has grown this way, it does not have a single weak point that can be fixed by design. There is no villain you can remove to change its nature; no single policy lever that, pulled hard enough, will turn the collar into a necklace. The system is the function. The cleared workforce is one of its main products.
The woman in this prologue is real in one sense and unreal in another. Her situation is real. Her morning is drawn from the lives of millions. Her experience matches accounts you can find in congressional testimony, investigative reporting, and the quieter parts of office gossip. She is not a single identifiable person whose file can be requested and whose badge number can be looked up. She is a composite, an archetype, a structural role given a name and a commute.
This book is not about her as an individual.
It is about the system that makes her routine make sense: the system that built the checkpoints and badge readers and workstations and monitoring tools; the system that created the clearance process, the investigation machinery, the contractor dependence, and the security framing that justifies all of it; the system that has evolved through five phases over 160 years, absorbing shocks, digesting exposures and producing “reforms” that mostly serve to protect its core.
From where she sits, she cannot see that structure. She sees the badge scan, the notification, the conversation with Security. She does not see the appropriations that pay her employer, the lobbying that protects those appropriations, the revolving door that shuttles officials between the agencies that monitor her and the companies that pay her, the classification rules that keep the whole arrangement outside public view.
She cannot see it because she is inside it. The collar fits so well it feels like skin.
This book is for everyone on the outside—and for the people inside who have started to feel that something around their neck is beginning to tighten.
What follows is the history of how that system was built.
The story begins before there was any official intelligence community, before the CIA or NSA or FBI, before there were SCIFs or SF-86s. It begins with private capital—with corporations that needed secrets kept, workers controlled, and competitors watched—and with the private detective agencies they hired to do the work the state had not yet learned to do for itself.
The state did not invent secrecy. The state inherited secrecy from capital and scaled it. The machine that watches the cleared employee in Northern Virginia in 2025 descends from the forces that watched and broke the steelworkers at Homestead in 1892. The technologies changed. The language changed. The basic relationship did not.
The chapters that follow track that evolution: Pinkerton, the Espionage Act, the birth of the CIA, the Church Committee, Iran–Contra, the post-9/11 explosion, the Snowden disclosures. Each phase resolves some of the tensions of the last and generates new ones. Each crisis produces reforms that promise control and deliver continuity.
By the end, the reader will understand why the woman in the prologue received that message from Security. Not the specific trigger in her log files, but the structural reason she lives in a world where such triggers exist. Why a system that claims to protect the nation’s secrets has become a system that quietly monitors millions of workers. Why a system that advertises accountability has become a machine for evasion. Why a system draped in democratic language operates so far beyond democratic reach.
The machine is visible. The rest of this book is devoted to making it impossible to unsee.