Data
Warning
The nwLH latin-hypercube, containing 2,000 simulations varying \(\Omega_{\rm m}\), \(\Omega_{\rm b}\), \(h\), \(n_s\), \(\sigma_8\), \(M_\nu\), \(w\) is now publicly available! Check Latin-hypercubes for more details.
Description
The data is organized among different folders:
Snapshots. This folder contains the snapshots of the simulations
Halos. This folder contains the halo catalogues
Voids. This folder contains the void catalogues
Linear_Pk. This folder contains the linear power spectra of each cosmological model
Pk. This folder contains the non-linear power spectra
Marked_Pk. This folder contains the marked power spectra
Bk. This folder contains the bispectra
CF. This folder contains the correlation functions
PDF. This folder contains the pdfs
3D_cubes. This folder contains the 3D density fields
Inside each of the above folders there is the data for the different cosmologies, e.g. h_p, fiducial, Om_m. A brief description of the different cosmologies is provided in the below table. The standard and paired fixed snapshots or data products will be located inside the same folder. The paired fixed (or fixed) will be located inside folders starting with NCV (from No Cosmic Variance). Further details can be found in the Quijote paper.
Name |
\(\Omega_m\) |
\(\Omega_b\) |
\(h\) |
\(n_s\) |
\(\sigma_8\) |
\(M_\nu\) |
\(w\) |
\(\delta_b\) |
\(f_{\rm NL}^{\rm loc}\) |
\(f_{\rm NL}^{\rm equ}\) |
\(f_{\rm NL}^{\rm ort1}\) |
\(f_{\rm NL}^{\rm ort2}\) |
realizations |
simulations |
ICs |
\(N_c^{1/3}\) |
\(N_\nu^{1/3}\) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
fiducial |
0.3175 |
0.049 |
0.6711 |
0.9624 |
0.834 |
0 |
-1 |
0 |
0 |
0 |
0 |
0 |
15,000 |
standard |
2LPT |
512 |
0 |
fiducial |
0.3175 |
0.049 |
0.6711 |
0.9624 |
0.834 |
0 |
-1 |
0 |
0 |
0 |
0 |
0 |
500 |
paired fixed |
2LPT |
512 |
0 |
fiducial_ZA |
0.3175 |
0.049 |
0.6711 |
0.9624 |
0.834 |
0 |
-1 |
0 |
0 |
0 |
0 |
0 |
500 |
standard |
Zeldovich |
512 |
0 |
fiducial_LR |
0.3175 |
0.049 |
0.6711 |
0.9624 |
0.834 |
0 |
-1 |
0 |
0 |
0 |
0 |
0 |
1,000 |
standard |
2LPT |
512 |
0 |
fiducial_HR |
0.3175 |
0.049 |
0.6711 |
0.9624 |
0.834 |
0 |
-1 |
0 |
0 |
0 |
0 |
0 |
500 |
standard |
2LPT |
1,024 |
0 |
Om_p |
0.3275 |
0.049 |
0.6711 |
0.9624 |
0.834 |
0 |
-1 |
0 |
0 |
0 |
0 |
0 |
500 |
standard |
2LPT |
512 |
0 |
Om_p |
0.3275 |
0.049 |
0.6711 |
0.9624 |
0.834 |
0 |
-1 |
0 |
0 |
0 |
0 |
0 |
500 |
paired fixed |
2LPT |
512 |
0 |
Om_m |
0.3075 |
0.049 |
0.6711 |
0.9624 |
0.834 |
0 |
-1 |
0 |
0 |
0 |
0 |
0 |
500 |
standard |
2LPT |
512 |
0 |
Om_m |
0.3075 |
0.049 |
0.6711 |
0.9624 |
0.834 |
0 |
-1 |
0 |
0 |
0 |
0 |
0 |
500 |
paired fixed |
2LPT |
512 |
0 |
Ob2_p |
0.3175 |
0.051 |
0.6711 |
0.9624 |
0.834 |
0 |
-1 |
0 |
0 |
0 |
0 |
0 |
500 |
standard |
2LPT |
512 |
0 |
Ob2_p |
0.3175 |
0.051 |
0.6711 |
0.9624 |
0.834 |
0 |
-1 |
0 |
0 |
0 |
0 |
0 |
500 |
paired fixed |
2LPT |
512 |
0 |
Ob2_m |
0.3175 |
0.047 |
0.6711 |
0.9624 |
0.834 |
0 |
-1 |
0 |
0 |
0 |
0 |
0 |
500 |
standard |
2LPT |
512 |
0 |
Ob2_m |
0.3175 |
0.047 |
0.6711 |
0.9624 |
0.834 |
0 |
-1 |
0 |
0 |
0 |
0 |
0 |
500 |
paired fixed |
2LPT |
512 |
0 |
Ob_p |
0.3175 |
0.050 |
0.6711 |
0.9624 |
0.834 |
0 |
-1 |
0 |
0 |
0 |
0 |
0 |
500 |
standard |
2LPT |
512 |
0 |
Ob_m |
0.3175 |
0.048 |
0.6711 |
0.9624 |
0.834 |
0 |
-1 |
0 |
0 |
0 |
0 |
0 |
500 |
standard |
2LPT |
512 |
0 |
h_p |
0.3175 |
0.049 |
0.6911 |
0.9624 |
0.834 |
0 |
-1 |
0 |
0 |
0 |
0 |
0 |
500 |
standard |
2LPT |
512 |
0 |
h_p |
0.3175 |
0.049 |
0.6911 |
0.9624 |
0.834 |
0 |
-1 |
0 |
0 |
0 |
0 |
0 |
500 |
paired fixed |
2LPT |
512 |
0 |
h_m |
0.3175 |
0.049 |
0.6511 |
0.9624 |
0.834 |
0 |
-1 |
0 |
0 |
0 |
0 |
0 |
500 |
standard |
2LPT |
512 |
0 |
h_m |
0.3175 |
0.049 |
0.6511 |
0.9624 |
0.834 |
0 |
-1 |
0 |
0 |
0 |
0 |
0 |
500 |
paired fixed |
2LPT |
512 |
0 |
ns_p |
0.3175 |
0.049 |
0.6711 |
0.9824 |
0.834 |
0 |
-1 |
0 |
0 |
0 |
0 |
0 |
500 |
standard |
2LPT |
512 |
0 |
ns_p |
0.3175 |
0.049 |
0.6711 |
0.9824 |
0.834 |
0 |
-1 |
0 |
0 |
0 |
0 |
0 |
500 |
paired fixed |
2LPT |
512 |
0 |
ns_m |
0.3175 |
0.049 |
0.6711 |
0.9424 |
0.834 |
0 |
-1 |
0 |
0 |
0 |
0 |
0 |
500 |
standard |
2LPT |
512 |
0 |
ns_m |
0.3175 |
0.049 |
0.6711 |
0.9424 |
0.834 |
0 |
-1 |
0 |
0 |
0 |
0 |
0 |
500 |
paired fixed |
2LPT |
512 |
0 |
s8_p |
0.3175 |
0.049 |
0.6711 |
0.9624 |
0.849 |
0 |
-1 |
0 |
0 |
0 |
0 |
0 |
500 |
standard |
2LPT |
512 |
0 |
s8_p |
0.3175 |
0.049 |
0.6711 |
0.9624 |
0.849 |
0 |
-1 |
0 |
0 |
0 |
0 |
0 |
500 |
paired fixed |
2LPT |
512 |
0 |
s8_m |
0.3175 |
0.049 |
0.6711 |
0.9624 |
0.819 |
0 |
-1 |
0 |
0 |
0 |
0 |
0 |
500 |
standard |
2LPT |
512 |
0 |
s8_m |
0.3175 |
0.049 |
0.6711 |
0.9624 |
0.819 |
0 |
-1 |
0 |
0 |
0 |
0 |
0 |
500 |
paired fixed |
2LPT |
512 |
0 |
Mnu_p |
0.3175 |
0.049 |
0.6711 |
0.9624 |
0.834 |
0.1 |
-1 |
0 |
0 |
0 |
0 |
0 |
500 |
standard |
Zeldovich |
512 |
512 |
Mnu_p |
0.3175 |
0.049 |
0.6711 |
0.9624 |
0.834 |
0.1 |
-1 |
0 |
0 |
0 |
0 |
0 |
500 |
paired fixed |
Zeldovich |
512 |
512 |
Mnu_pp |
0.3175 |
0.049 |
0.6711 |
0.9624 |
0.834 |
0.2 |
-1 |
0 |
0 |
0 |
0 |
0 |
500 |
standard |
Zeldovich |
512 |
512 |
Mnu_pp |
0.3175 |
0.049 |
0.6711 |
0.9624 |
0.834 |
0.2 |
-1 |
0 |
0 |
0 |
0 |
0 |
500 |
paired fixed |
Zeldovich |
512 |
512 |
Mnu_ppp |
0.3175 |
0.049 |
0.6711 |
0.9624 |
0.834 |
0.4 |
-1 |
0 |
0 |
0 |
0 |
0 |
500 |
standard |
Zeldovich |
512 |
512 |
Mnu_ppp |
0.3175 |
0.049 |
0.6711 |
0.9624 |
0.834 |
0.4 |
-1 |
0 |
0 |
0 |
0 |
0 |
500 |
paired fixed |
Zeldovich |
512 |
512 |
w_p |
0.3175 |
0.049 |
0.6711 |
0.9624 |
0.834 |
0 |
-1.05 |
0 |
0 |
0 |
0 |
0 |
500 |
standard |
Zeldovich |
512 |
0 |
w_m |
0.3175 |
0.049 |
0.6711 |
0.9624 |
0.834 |
0 |
-0.95 |
0 |
0 |
0 |
0 |
0 |
500 |
standard |
Zeldovich |
512 |
0 |
DC_p |
0.3175 |
0.049 |
0.6711 |
0.9624 |
0.834 |
0 |
-1 |
0.035 |
0 |
0 |
0 |
0 |
500 |
standard |
Zeldovich |
512 |
0 |
DC_m |
0.3175 |
0.049 |
0.6711 |
0.9624 |
0.834 |
0 |
-1 |
-0.035 |
0 |
0 |
0 |
0 |
500 |
standard |
Zeldovich |
512 |
0 |
LC_p |
0.3175 |
0.049 |
0.6711 |
0.9624 |
0.834 |
0 |
-1 |
0 |
100 |
0 |
0 |
0 |
500 |
standard |
2LPT |
512 |
0 |
LC_m |
0.3175 |
0.049 |
0.6711 |
0.9624 |
0.834 |
0 |
-1 |
0 |
-100 |
0 |
0 |
0 |
500 |
standard |
2LPT |
512 |
0 |
EQ_p |
0.3175 |
0.049 |
0.6711 |
0.9624 |
0.834 |
0 |
-1 |
0 |
0 |
100 |
0 |
0 |
500 |
standard |
2LPT |
512 |
0 |
EQ_m |
0.3175 |
0.049 |
0.6711 |
0.9624 |
0.834 |
0 |
-1 |
0 |
0 |
-100 |
0 |
0 |
500 |
standard |
2LPT |
512 |
0 |
OR_CMB_p |
0.3175 |
0.049 |
0.6711 |
0.9624 |
0.834 |
0 |
-1 |
0 |
0 |
0 |
100 |
0 |
500 |
standard |
2LPT |
512 |
0 |
OR_CMB_m |
0.3175 |
0.049 |
0.6711 |
0.9624 |
0.834 |
0 |
-1 |
0 |
0 |
0 |
-100 |
0 |
500 |
standard |
2LPT |
512 |
0 |
OR_LSS_p |
0.3175 |
0.049 |
0.6711 |
0.9624 |
0.834 |
0 |
-1 |
0 |
0 |
0 |
0 |
100 |
500 |
standard |
2LPT |
512 |
0 |
OR_LSS_m |
0.3175 |
0.049 |
0.6711 |
0.9624 |
0.834 |
0 |
-1 |
0 |
0 |
0 |
0 |
-100 |
500 |
standard |
2LPT |
512 |
0 |
latin_hypercube |
[0.1 - 0.5] |
[0.03 - 0.07] |
[0.5 - 0.9] |
[0.8 - 1.2] |
[0.6 - 1.0] |
0 |
-1 |
0 |
0 |
0 |
0 |
0 |
2,000 |
standard |
2LPT |
512 |
0 |
latin_hypercube |
[0.1 - 0.5] |
[0.03 - 0.07] |
[0.5 - 0.9] |
[0.8 - 1.2] |
[0.6 - 1.0] |
0 |
-1 |
0 |
0 |
0 |
0 |
0 |
2,000 |
standard |
2LPT |
512 |
0 |
latin_hypercube |
[0.1 - 0.5] |
[0.03 - 0.07] |
[0.5 - 0.9] |
[0.8 - 1.2] |
[0.6 - 1.0] |
0 |
-1 |
0 |
0 |
0 |
0 |
0 |
2,000 |
standard |
2LPT |
1,024 |
0 |
nwLH |
[0.1 - 0.5] |
[0.03 - 0.07] |
[0.5 - 0.9] |
[0.8 - 1.2] |
[0.6 - 1.0] |
[0.01 - 1.0] |
[-1.3 - -0.7] |
0 |
0 |
0 |
0 |
0 |
2,000 |
standard |
Zeldovich |
512 |
512 |
See Latin-hypercubes and Primordial non-Gaussianities for more information about the latin-hypercubes and the simulations with primordial non-Gaussianities.
Data access
The data is stored in the three different supercomputers in San Diego, New York, and Princeton. Each cluster contains the following data:
San Diego: The realizations 8,000 to 14,999 of the fiducial set. The standard and fixed LH latin hypercube. The halos, voids, power spectra, bispectra, correlation functions, and pdfs. 235 Tb
New York: The snapshots of high-resolution latin-hypercube (latin_hypercube_HR), the nwLH latin-hypercube, the primordial non-Gaussianities, and the 3D density fields. 536 Tb
Princeton: All the other snapshots. 620 Tb
The data can be accessed through globus:
Log in into globus (create an account if you dont have one).
To access the data in San Diego type: Quijote_simulations (or with this link1)
To access the data in New York type: Quijote_simulation2 (or with this link2)
To access the data in Princeton type: Princeton TIGRESS QUIJOTE Snapshots (or with this link3)
Note that to download the data to your local machine (e.g. laptop) you will need to install the globus connect personal. For further details see here.
Globus
The simplest way to transfer data is to use the globus graphical environment. Just type the above names in collection (e.g. Quijote_simulations for the data in San Diego) or click the associated link. You will need to choose where the data is being moved in the other collection (e.g. your laptop or another supercomputer). Once the collection points are set, select the data you want to transfer and destiny folder and click in Start.
In some cases, there are so many files in a given directory, that globus may not be able to list them all and return an error. If this is the case, it is advisable to use the path line. For instance, if by clicking in Snapshots you get a time out error, you may want to just type in the path line: /Snapshots/ or /~/Snapshots/. This may show you the different content of the data and allow you to navigate it. You can also go to a given directory directly from there. E.g. to access the first realization of the fiducial cosmology, type in path: /Snapshots/fiducial/0/ or /~/Snapshots/fiducial/0/.
In some cases, the above option may not be desirable. For instance, imagine that you want to download all linear matter power spectra of the high-resolution latin-hypercube simulations. Those files are located in, e.g. for realization 45,
/Snapshots/latin_hypercube_HR/45/ICs/Pk_mm_z=0.000.txt
Thus, to download all those files, without involving downloading the full snapshots, will require that you access each simulation folder, then the ICs folder and then transfer the file individually. For 2000 files this is unpractical. For these situations, we recommend the usage of Command Line Interface (CLI). The first step is to install the CLI package, if you don’t have it. Next, login into globus by typing in a terminal
globus login
Then, the following command allow you to determine the associated endpoint of the Quijote simulations:
globus endpoint search "Quijote_simulations"
ID | Owner | Display Name
------------------------------------ | ------------------------- | -------------------
c42757fe-d570-11e9-98e2-0a63aa6b37da | fvillaescusa@globusid.org | Quijote_simulations
You should do the same to know the endpoint of the machine where you are transfering the data to. You can then explore the filesystem of the Quijote simulations (or your machine) as:
ep1=c42757fe-d570-11e9-98e2-0a63aa6b37da
globus ls $ep1:/Snapshots/latin_hypercube_HR/45/ICs/
The above command will list the content of the /Snapshots/latin_hypercube_HR/45/ICs/ directory. A single file can be transfered as:
ep1=c42757fe-d570-11e9-98e2-0a63aa6b37da
ep2=ddb59af0-6d04-11e5-ba46-22000b92c6ec
globus transfer $ep1:/Snapshots/latin_hypercube_HR/45/ICs/Pk_mm_z=0.000.txt $ep2:/Quijote_simulations/linear_Pk/45/Pk_mm_z=0.000.txt --label "single file transfer"
Where ep2 should be the endpoint of the machine where you are transfering the data. Entire folders can be moved as follows:
ep1=c42757fe-d570-11e9-98e2-0a63aa6b37da
ep2=ddb59af0-6d04-11e5-ba46-22000b92c6ec
globus transfer $ep1:/Snapshots/latin_hypercube_HR/45/ICs $ep2:/Quijote_simulations/45/ICs --recursive --label "single folder transfer"
Many folders can be moved with a single command as
ep1=c42757fe-d570-11e9-98e2-0a63aa6b37da
ep2=ddb59af0-6d04-11e5-ba46-22000b92c6ec
globus transfer $ep1:/Snapshots/fiducial/ $ep2:/Quijote_simulations/fiducial/ --batch --label "CLI 10 folders" < folders.txt
where folders.txt is a text file containing
--recursive 0 0
--recursive 1 1
--recursive 2 2
--recursive 3 3
--recursive 4 4
--recursive 5 5
--recursive 6 6
--recursive 7 7
--recursive 8 8
--recursive 9 9
For more options and details see Command Line Interface (CLI).