Brian Christian: Scientific Knowledge Should Be Structured as "Literature"

In my view, what's most outmoded within science, most badly in need of retirement, is the way we structure and organize scientific knowledge itself. Academic literature, even as it moves online, is a relic of the era of typesetting, modeled on static, irrevocable, toothpaste-out-of-the-tube publication. Just as the software industry has moved from a "waterfall" process to an "agile" process—from monolithic releases shipped from warehouses of mass-produced disks to over-the-air differential updates—so must academic publishing move from its current read-only model and embrace a process as dynamic, up-to-date, and collaborative as science itself.

[...]

What academic literature needs goes deeper than the view of citations as kudos and shout-outs. It needs what software engineers have used for decades: dependency management.

A dependency graph would tell us, at a click, which of the pillars of scientific theory are truly load-bearing. And it would tell us, at a click, which other ideas are likely to get swept away with the rubble of a particular theory. An academic publisher worth their salt would, for instance, not only be able to flag articles that have been retracted—that this is not currently standard practice is, again, inexcusable—but would be able to flag articles that depend in some meaningful way on the results of retracted work.

An academic publisher worth their salt would also accommodate another pillar of modern software development: revision control. Code repositories, like wikis, are living documents, open not only for scrutiny, censure and approbation, but for modification.

In a revision control system like Git (and its wildly successful open-source community on GitHub), users can create "issues" that flag problems and require the author's response, they can create "pull requests" that propose answers and alterations, and they can "fork" a repository if they want to steward their own version of the project and take it in a different direction. (Sometimes forked repositories serve a niche audience; sometimes they wither from neglect or disuse; sometimes they fully steal the audience and userbase from the original; sometimes the two continue to exist in parallel or continue to diverge; and sometimes they are reconciled and reunited downstream.) A Git repository is the best of top-down and bottom-up, of dictatorship and democracy: its leaders set the purpose and vision, have ultimate control and final say—yet any citizen has an equal right to complain, propose reform, start a revolt, or simply pack their bags and found a new nation next door.

The "Accept," "Reject," and "Revise and Resubmit" ternary is anachronistic, a relic of the era of metal type. Even peer review itself, with its anonymity and bureaucracy, may be ripe for reimagining. The behind-closed-doors, anonymous review process might be replaced, for instance, with something closer to a "beta" period. The article need not be held up for months—at least, not from other researchers—while it is considered by a select few. One's critics need not be able to clandestinely delay one's work by months. Authors need not thank "anonymous readers who spotted errors and provided critical feedback" when those readers' corrections are directly incorporated (with attribution) as differential edits. Those readers need not offer their suggestions as an act of obligation or charity, and they need not go unknown.

Some current rumblings of revolution seem promising. Wide circulation among academics of "working papers" challenges the embargo and lag in the peer review process. PLOS ONE insists on top-down quality assurance, but lets importance emerge from the bottom-up. Cornell's arXiv project offers a promising alternative to more traditional journal models, including versioning (and its "endorsement" system has since 2004 suggested a possible alternative to traditional peer reviews). However, its interface by design limits its participatory and collaborative potential.

http://edge.org/response-detail/25514