The NESE project is currently in the “Launch Phase,” lead by Scott Yockel and his team at Harvard FASRC. This means that our starter batch of NESE hardware (approximately 15 PB raw, with more than 50% buy-in) is being set up, configured, integrated into Harvard operations and tested for initial applications. We think that the easiest initial applications are going to be Globus endpoints (with various back end storage types), and CephFS configured to be a storage endpoint for the BU/Harvard Northeast Tier 2 Center (which is really an international facility which will be used by the worldwide ATLAS collaboration as well as by local uses).

That’s supposed to be the relatively easy stuff. Harder storage applications are POSIX file systems that get used interactively. Because this is more challenging and to keep latency to Boston sites low, there is actually a second deployment of NESE at the Markley Data Center at 1 Summer Street (known as 1SS) downtown Boston. There, Justin Riley leads the team including Austin Smith (recent hire from RedHat) and Milan Kupcevic. These guys are testing different Ceph configurations and various ways of deploying CephFS, NFS and Samba to sites on the Harvard campus.

The main applications that Justin and his team have in mind is home directories (including Odyssey home directories) and especially “lab space”. A reference example of this is Jeffrey Lichtman’s lab where his parallel electron microscopes are writing 2-3 GB of brain imaging data every hour of every day.

One of the more exciting things about NESE is that if work at 1SS goes well, we could end up with a reliable, flexible easy to manage way of delivering high performance file systems to any of our Boston area campuses!