Data Storage & Edge Computing for Astronomy & Observatories 2026 — Petabyte Archives, Radio Telescopes & Remote-Site Hardware

Modern astronomy is a data firehose attached to a telescope on a remote mountain top. A wide-field survey camera reads out billions of pixels every night; a radio interferometer correlates dozens to hundreds of antennas into a stream that has to be reduced before it can be stored; and the resulting science archive must remain readable for decades, long after the instrument that produced it is decommissioned. The awkward part is where all of this happens: thin air and a generator at 2,400–4,200 m, a thin crew, and no data centre within hundreds of kilometres. This guide maps hardware to each layer of the pipeline — instrument edge, on-site reduction, and the petabyte archive — so an observatory or survey team can size the platform from the data rate, not from a brochure.

Astronomy Is a Storage Problem Before a Compute Problem

Most instruments are built to collect photons, and the volume of raw data is set by the detector and the cadence, not by the science goal. A wide-field optical survey may dump tens of terabytes per night; a radio correlator can emit hundreds of gigabits per second before any averaging. The first sizing question is therefore not "which GPU" but "where does the raw stream land, and how fast can it be drained".

The lesson repeats at every facility: raw and intermediate data are an order of magnitude larger than the science-ready product, so the buffer and scratch tiers dominate the hardware bill long before the archive does.

Three Layers, One Pipeline

Treating "observatory computing" as one requirement is the mistake that produces an over-specified instrument edge and an under-specified archive. There are three distinct layers, and they pull the spec in different directions.

The field edge is the layer that cannot be redone: a survey cannot repeat a night that was lost to a full buffer. Everything downstream can be reprocessed; the raw capture cannot.

Edge Compute at the Telescope

A remote observatory is a power- and cooling-constrained site. Generating capacity is finite and expensive, and cooling efficiency drops sharply with altitude because the air is thinner and carries less heat per unit volume — a fan curve that works at sea level can leave a rack running 10–15 °C hotter at 4,200 m. Ingress protection matters less on a clean mountain top than altitude de-rating, power quality and vibration, but optical enclosures and drives exposed to wind-blown grit and nightly freeze–thaw still argue for industrial-grade, wide-temperature parts. Edge compute earns its place by doing two jobs locally: real-time radio-frequency-interference (RFI) flagging and transient detection, where the data is discarded or an alert is raised before the volume is written, and protocol-level data reduction, where a correlator's raw stream is averaged down to a storable size.

Data-Rate Arithmetic Before You Buy

Do the arithmetic on one exposure or one second of correlator output and the storage tier follows. A 3.2-gigapixel camera reading out in 16-bit words produces about 6.4 GB per frame; at a 15-second cadence that is roughly 0.43 GB/s, or about 37 TB per clear night once overheads are included. A radio array with 100 antennas and 1 GHz of bandwidth digitised at 8 bits generates on the order of 100 GB/s at the correlator input, reduced by integration to a few TB of visibilities per day. The buffer between the sensor and the disk must absorb every spike, so a truncated power-loss window is not a theoretical risk — it is a data-loss event waiting for the next generator glitch.

Storage: Three Tiers, Sized Separately

Plan three tiers rather than one pool, and size each against its own workload. The hot tier absorbs detector readout and holds the raw data until reduction is complete; it needs power-loss protection, RAID with genuine hot-spare behaviour, and sustained write endurance, not the largest capacity. The warm tier holds nightly reduction products and the live project store; capacity and throughput matter more than endurance. The cold tier holds the science archive, where the governing metric is cost per terabyte and the ability to verify integrity over decades, because the value of a twenty-year-old observation is that it can be reprocessed with tomorrow's algorithms — which requires the bits to still be there.

The rule that saves the most money is to size the buffer and the archive separately: buying archive capacity at buffer-grade endurance prices, or buffer endurance at archive volumes, is the most common way an observatory hardware budget blows out.

What Actually Breaks in the Field

Deployment failures at remote sites cluster around a short list, and each has a hardware answer rather than a software one.

• Altitude cooling. Specify fans and heat sinks rated for the site's air density, and de-rate the thermal budget; do not carry a sea-level cooling design up the mountain.
• Power quality. Generators and long feeders bring surges and brownouts — use wide-input industrial DC supplies with a UPS or supercap hold-up on the buffer writes.
• Temperature swing. A nightly freeze–thaw cycle is harsher than a steady cold soak; choose wide-temperature drives and enclosures with condensation control.
• Vibration. Telescope drives and dome machinery inject low-frequency vibration that a desktop-class SSD's mount is not tested against.

Procurement Spec Matrix

Selection Rules

• Rank the raw ingest rate first; the detector and correlator set the buffer, not the science goal.
• Split the pipeline before you split the budget — the instrument edge, the reduction room and the archive are three different purchases.
• Protect raw capture above all: PLP NVMe, RAID, hold-up power and wide-temperature parts.
• Push RFI flagging and transient detection to the edge so you discard data before you store it.
• De-rate the thermal design for altitude and put wide-temperature drives wherever the site freezes at night.
• Size buffer and archive separately; intermediates run many times the raw volume and archives last decades.
• Treat the parallel filesystem and the network as part of the purchase, not a later upgrade.
• Match environmental compliance to the site — altitude, freeze–thaw and vibration certificates, not just the temperature range.