Big Bang, bigger questions, biggest data

The big questions are the oldest. Where are we? How did we get here? Where are we going?

A newer question is one of the biggest of all: is the ΛCDM (Lambda-CDM, or Lambda cold dark matter) model of physical cosmology correct in its three postulates that, 1] the universe is the same everywhere and is expanding; 2] lines in spacetime intersect at only one point, and time along each line can be synchronized; and, 3] general relativity accurately correlates the geometry of spacetime to the distribution of matter and energy?

Simple, right?

Scientists and engineers at the European Space Agency (ESA) seek answers to these questions every day, just as they have since its founding in 1975. Each generation of insights inspires even more curiosity.

When ESA was created in 1975, it was more than a scientific milestone—it was Europe’s decision to bring together fragmented national programs under a single convention and speak with one voice in space. That unifying step laid the foundation for the agency’s achievements over the past five decades. From that foundation, ESA has built a wide-ranging program—from cutting-edge science to exploration, Earth observation and human spaceflight.

Impressive as it is to participate in the International Space Station’s (ISS) human spaceflight program, ESA also manages crewless exploration of other planets and moons in our galaxy and numerous manmade satellite arrays in near-earth orbit. In total the agency currently oversees around 20 missions, studying everything from Earth’s solar winds and changing weather patterns in the Arctic, to plasma physics (plasma is the most disordered and ubiquitous state of matter in our universe) and the knock-on effects of the Big Bang some 14B years ago (the singular cosmic event remains the mother of all the big-bigger-biggest mysteries known to humankind).

ESA’s mission to answer the ΛCDM question is named for Euclid of Alexandria, the ancient Greek mathematician who organized the properties of points, lines, planes, and angles into the traditional form of geometry. By definition, the Euclid space telescope is designed to observe the geometry of billions of galaxies out to 10 billion light-years away, using the classic properties of points, lines, planes, and angles.

Putting aside (for most of us) hard math and many zeros, the data that the Euclid mission creates effectively maps more than one-third of the observable sky. But mind-blowingly, the Euclid team’s research centers on observing only the invisible—gravity, and so-called dark energy and dark matter. This is because, even more mind-blowingly, only 5% of the cosmos is visible (e.g., atoms, light, your go-to Starbucks order). The other 95% of the cosmos, hiding in plain sight until now, holds the answers Euclid seeks.

Those answers will soon be found in the data on NetApp servers. Coming soon to a Milky Way near you.

Ruben Alvarez achieved his MSc in Astrophysics at Complutense University of Madrid, founded in 1293. As a postgraduate alumna of one of the world’s oldest universities, it's fitting then that Ruben is helping answer some of the world’s oldest questions. Years later he achieved a master's degree in IT management. As ESA’s Science IT infrastructure coordinator and cybersecurity manager, Ruben applies his multi-disciplinary craft at a rare crossroads. A scientist and researcher in his early career, he now bridges the intangible world of cosmological theory, and the very tangible world of hardware and software required to capture and manage the data that proves or disproves the big questions in space-based science. In plain language, Ruben provides the IT infrastructure and cybersecurity needed by the ESA’s Science Operations Centers (SOCs). It’s here that highly specialized professionals deftly and simultaneously orchestrate the demands of multiple space-based and astronomical missions.

They have their hands full, and no two days are the same. Their work ranges from post-launch calibration of onboard instruments to architecting storage environments across the data lifecycle. This is achieved as they process and analyze raw space data from multiple spacecraft and astronomical satellites. Theirs is an always-on duty of care that ultimately shapes our understanding of the universe’s physical existence—and our own. The datasets created by ESA across the past half century result in a body of work duly recognized as the Digital Library of the Universe. And on a shelf in that library is a growing volume of data representing one of the world’s newest books, billions of years in its writing.

The Euclid dataset is our digital Atlas of the Cosmos.

The Euclid wide-angle telescope today follows a halo trajectory around the sun, one million miles from where it launches at Cape Canaveral, Florida, in July 2023. Euclid is an ESA-led mission with contributions from NASA, including infrared detectors, ground support, and U.S. scientists in the Euclid Consortium. The launch was carried out on a Falcon 9 provided by SpaceX. The satellite entered stable orbit about a month later and scientists held their collective breath. But Euclid’s first images transmitted back to Earth are tainted by light pollution. Not to worry. Calibration of the craft by a few degrees after coordination between the SOC team in Madrid and engineers in Toulouse, France and Turin, Italy soon sets things right, and the mission continues apace.

Core responsibilities of the European Space Astronomy Center (ESAC) Science Operations Centers (SOCs)

• Space mission planning—Including upwards of 15 missions at any given time, most continuing for 5-15 years
• Space instrument calibration—Assisting engineers as they adjust post-launch spacecraft parameters until stable orbit is attained and a mission’s reliable data transmission officially commences
• Space data processing—Processing, sorting, cataloging, analyzing, and generally scrubbing data in preparation for ‘publishing’ (it can take years)
• Space data archiving—Providing and safeguarding 24 / 7 / 365 access to the Library of the Universe by 30,000 researchers and academics anytime, from anywhere

Euclid is initially proposed as two missions with adjacent goals. In combining them, ESA today reaps the benefits of doing more with less. It’s also a fantastic example of how the distributed nature of ESA benefits us all: more than 2,500+ engineers and scientists across 15+ countries are part of the Euclid Scientific Consortium. But Euclid is also a perfect example of the complexity that Ruben and his team face. Many space missions have a singular focus—studying one distant moon, for instance, or landing on an asteroid transiting near Earth. Conversely, Euclid’s agenda is among the broadest imaginable: investigating billions of extragalactic objects and stars outside the Milky Way, and at 4X the resolution of the world’s most advanced ground-based telescopes.

Fortunately, there is a precedent.

Even as this article is posted online in 2025, one of the European’s Space Agency’s most remarkable missions reaches its end of life, because as the seasons inevitably change on its home planet, the Gaia spacecraft inevitably enters ‘passivation’ at which point its remaining internal energy (e.g., untapped battery power and unspent propellant) is intentionally depleted. The instrument enters an orbit determined by international agreement to pose no threat to future space missions as dangerous space debris. This final stage of spacecraft design and mission scope officially designates Gaia as ‘one for the record books’. It also means that after more than a decade dedicated to create the largest and most accurate three-dimensional map of the Milky Way galaxy, Gaia will no longer be an active voice in our intergalactic conversation with the cosmos. But what a voice Gaia proves to be in its mission to enlighten and inspire.

Ruben Alvarez and NetApp, despite both being earthbound, are metaphorically along for the entire ride.

When it launched in 2013, Gaia was designed for disciplined study across various dimensions of astrometry: measuring attributes of stars, exoplanets, and their neighbors in our galaxy, based on their relative positions, sizes, and colors. It’s far more complicated than this, of course, but the result is an unprecedented catalog of astronomical objects—stars, planets, comets, asteroids, quasars, more. And all of it in 3D.

At its outset, Gaia is expected to produce 100X more data than previous missions, but even at that rarified altitude of expectations Gaia proves to be an over-achiever. Targeting just 1% of the Milky Way—that is, 1 billion of the 100 billion stars in the galaxy we call home—Gaia ultimately mapped nearly 2 billion celestial objects and logged some 200TB of data through which researchers and academics will one day have a romp. (It takes a few years for the Gaia SOC in Madrid to work its magic on coalescing mission data on a project as massive Gaia into usable condition. The final Gaia catalog is expected to be delivered by 2030.)

It joins other datasets from the agency’s past in the virtual library residing on 7 PiBs of intelligent infrastructure from NetApp. Specifically, the SOCs in Spain rely on a mix of all-flash (AFF) and fabric-attached storage (FAS). To achieve the lowest data lifecycle costs Ruben uses FAS for about 90% of the European Space Agency’s science data footprint.

Benefits of the NetApp AFF A150 all-flash array  

• Enterprise-grade performance at an entry-level price 
• Low latency and high throughput 
• High-speed data access and improved scalability 
• More performant, reliable, and energy efficient than other HDDs  

Benefits of NetApp FAS storage  

• Lowest data lifecycle cost for tiering of cold data and cyber vault workflows 
• Efficient backup for fast-growing SAN, NAS, and object datasets 
• Seamless scale of capacity and performance 

Roberto Prieto, Space Data Archive Software Engineer says, “Our NetApp AFF and FAS devices, together, create a single cluster that allows us to, in real time, migrate data volumes from one system to another.” The multi-protocol FAS service means the SOCs can offer network file system (NFS) capabilities, block-level access via iSCSI (Internet Small Computer System Interface), Fibre Channel Protocol (FCP), or S3 object storage to the scientific community they serve. Roberto continues, “NetApp is one of the most efficient, scalable, and flexible systems we’ve ever had. In 20 years we’ve never lost a single file.” 

 Best of all (aside from being economical, simple, and secure) is that FAS scales easily. And that means something when you’re investigating the Big Bang. As Gaia enters passivation after mapping nearly 2% of the stars in the Milky Way, Euclid is already mapping upwards of 36% of the wider cosmos. One era ends as another begins, and ESA’s data storage from NetApp keeps each science mission’s data securely stored, with confidence built in. Ruben puts it this way: "A critical point to make a mission successful is that the data infrastructure is flexible, and the data can be distributed in an agile way. NetApp systems provide us with that flexibility and agility.” 

A closer look at the SOCs’ IT strategy reveals the complexities inherent in managing a literal universe of information, even as its volume increases with every transmission from far-flung instruments zooming overhead. The SOCs keep cyber resilience front and center, especially considering the ‘open door’ that the Library of the Universe must maintain. Ruben’s vigilance on this front can’t be overstated. He says, “We must offer the public the data that our missions create. At the same time, we must protect ourselves and the data. It’s a race that never ends.” Data backups and redundancy for disaster recovery are also part of his purview. Here, it gets highly technical, very quickly, but among other NetApp solutions in his toolbox are:

• NetApp^® Snapshot^™ for instant copies of valuable data while applications run
• NetApp^® FlexClone^® for writable, point-in-time copies of parent volumes
• NetApp^® ONTAP^® FlexGroup for near-infinite capacity with predictable, low-latency performance in metadata-heavy workloads

All this technology comes together through NetApp^® ONTAP^®, the leading operating system for unified storage, delivering optimized capacity and performance across the Science Operations Centers’ ecosystem. It’s this seamless integration and ease of management that Ruben highlights, saying, “We value NetApp’s reliability and efficiency. They really delivered on the promises they made.” He also recognizes that there is more than hardware and software behind his long-standing relationship with NetApp, continuing, “This combination of factors—the technology, the people, the support—these have all been key to us during the past 20+ years.” It's a relationship NetApp is proud to be a part of.

Like the universe itself, data from the European Space Agency's many science missions is both expanding and accelerating. 

• BepiColombo, a duo-satellite mission launched in 2018, is set to enter Mercury's orbit in 2026
• Jupiter Icy Moons Explorer (Juice) is on track for orbital insertion around the Gas Giant in 2031

More than 15 additional science missions round out the list and that is the (literally!) astronomical task Ruben and his team will continue to do on behalf of all humankind today, tomorrow, and in the decades to come. Along the way, they'll raise a glass from time to time and celebrate key achievements in space exploration and astronomy as one mission begins and another ends.

Some of those celebrations will be to cheer the entry of new catalogs of space data as it's published for the world to use. These 'book release' events are particularly busy times for the librarians with oversight of the Library of the Universe, because before being shelved, each new volume of knowledge is rigorously tagged using a unique equivalent of the Dewey Decimal Classification system. The main purpose of ESA's 'data releases' is to maximize the scientific return of their work, and to enable discoveries well beyond their original missions' aims by letting the global community analyze the data.

It's a systematic and scientific approach to help answer the big questions. The ones all of us have, even if we don't yet have the words when we first begin to wonder, where are we? How did we get here? And where are we going? Ruben says, "Ever since I was a child, I was always attracted by space. I would see every night the wonderful spectacle of a night full of stars. That made me wonder...how distant they were, the journey of the light from the stars to my eyes. So, I am happy doing what I'm doing—being witness to the progress we are making across different missions, and how they are contributing to our understanding of the universe."

When asked if he sees himself in space one day, Ruben mentions the Artemis project, a multi-faceted initiative to return humans to our moon. ESA is a key partner in NASA's Artemis program, providing the European Service Module and lunar Gateway module, each playing critical roles in supporting four astronauts during their three-week missions aboard the Orion spacecraft. The missions are set to include European astronauts, marking a new chapter in Europe's exploration of the Moon.

As the question hangs in the air, Ruben pauses and smiles, then says, "I think interesting times are ahead. Only 12 people have stepped on the Moon. So, yeah, let's see what happens in a few years..."