Data access

IceCube has a bunch of public datasets available at https://icecube.wisc.edu/science/data-releases/. icecube_tools provides an easy interface to this repository so that you can download and organise your data through python.

[1]:

from icecube_tools.utils.data import IceCubeData

The IceCubeData class provides this functionality. Upon initialisation, IceCubeData queries the website using HTTP requests to check what datasets are currently available. By default, this request is cached to avoid spamming the IceCube website. However, you can use the keyword argument update to override this.

[2]:

my_data = IceCubeData(update=True)

# The available datasets
my_data.datasets

[2]:

['2021_03-all_data_releases.zip',
 '20080911_AMANDA_7_Year_Data.zip',
 '20090521_IceCube-22_Solar_WIMP_Data.zip',
 '20110905_IceCube-40_String_Data.zip',
 '20131121_Search_for_contained_neutrino_events_at_energies_above_30_TeV_in_2_years_of_data.zip',
 '20150127_IceCube_Oscillations%20_3_years_muon_neutrino_disappearance_data.zip',
 '20150219_Search_for_contained_neutrino_events_at_energies_greater_than_1_TeV_in_2_years_of_data.zip',
 '20150619_IceCube-59%20_Search_for_point_sources_using_muon_events.zip',
 '20150820_Astrophysical_muon_neutrino_flux_in_the_northern_sky_with_2_years_of_IceCube_data.zip',
 '20151021_Observation_of_Astrophysical_Neutrinos_in_Four_Years_of_IceCube_Data.zip',
 '20160105_The_79-string_IceCube_search_for_dark_matter.zip',
 '20160624_Search_for_sterile_neutrinos_with_one_year_of_IceCube_data.zip',
 '20161101_Search_for_point_sources_with_first_year_of_IC86_data.zip',
 '20161115_A_combined_maximum-likelihood_analysis_of_the_astrophysical_neutrino_flux.zip',
 '20180213_Measurement_of_atmospheric_neutrino_oscillations_with_three_years_of_data_from_the_full_sky.zip',
 '20180712_IceCube_catalog_of_alert_events_up_through_IceCube-170922A.zip',
 '20180712_IceCube_data_from_2008_to_2017_related_to_analysis_of_TXS_0506+056.zip',
 '20181018_All-sky_point-source_IceCube_data%20_years_2010-2012.zip',
 '20190515_Three-year_high-statistics_neutrino_oscillation_samples.zip',
 '20190904_Bayesian_posterior_for_IceCube_7-year_point-source_search_with_neutrino-count_statistics.zip',
 '20200227_All-sky_point-source_IceCube_data%20_years_2012-2015.zip',
 '20200421_IceCube_Upgrade_Neutrino_Monte_Carlo_Simulation.zip',
 '20200514_South_Pole_ice_temperature.zip',
 '20210126_PS-IC40-IC86_VII.zip',
 '20210310_IceCube_data_for_the_first_Glashow_resonance_candidate.zip',
 '20211217_HESE-7-5-year-data.zip',
 '20220201_Density_of_GeV_muons_in_air_showers_measured_with_IceTop.zip',
 '20220902_Evidence_for_neutrino_emission_from_the_nearby_active_galaxy_NGC_1068_data.zip',
 '20220913_Evidence_for_neutrino_emission_from_the_nearby_active_galaxy_NGC_1068_data.zip',
 '20221028_Observation_of_High-Energy_Neutrinos_from_the_Galactic_Plane.zip',
 '20221208_Observation_of_High-Energy_Neutrinos_from_the_Galactic_Plane.zip',
 '20230424_Observation_of_High-Energy_Neutrinos_from_the_Galactic_Plane.zip',
 'ic22-solar-wimp-histograms.zip']

You can use the find method to pick out datasets you are interested in.

[3]:

found_dataset = my_data.find("20181018")
found_dataset

[3]:

['20181018_All-sky_point-source_IceCube_data%20_years_2010-2012.zip']

The my_data object has been inititalised to store data in the package’s default location (“~/.icecube_data”). This is where other icecube_tools modules will look for stored data.

[4]:

my_data.data_directory

[4]:

'/home/runner/.icecube_data'

The fetch method will download this dataset to this default location for later use by icecube_tools. This method takes a list of names, so can also be used to download multiple datasets. fetch has a built in delay of a few seconds between HTTP requests to avoid spamming the website. fetch will not overwrite files by default, but this can be forced with overwrite=True.

[5]:

my_data.fetch(found_dataset)

20181018_All-sky_point-source_IceCube_da…: 0%| | 0/11730609 [00:00<?, ?it/s]: </pre>
20181018_All-sky_point-source_IceCube_da{ldots}: 0%| | 0/11730609 [00:00<?, ?it/s]: end{sphinxVerbatim}

20181018_All-sky_point-source_IceCube_da…: 0%| | 0/11730609 [00:00<?, ?it/s]

20181018_All-sky_point-source_IceCube_da…: 9%|▉ | 1048576/11730609 [00:00<00:09, 1070870.92it/s]: </pre>
20181018_All-sky_point-source_IceCube_da{ldots}: 9%|▉ | 1048576/11730609 [00:00<00:09, 1070870.92it/s]: end{sphinxVerbatim}

20181018_All-sky_point-source_IceCube_da…: 9%|▉ | 1048576/11730609 [00:00<00:09, 1070870.92it/s]

20181018_All-sky_point-source_IceCube_da…: 18%|█▊ | 2097152/11730609 [00:01<00:05, 1812901.08it/s]: </pre>
20181018_All-sky_point-source_IceCube_da{ldots}: 18%|█▊ | 2097152/11730609 [00:01<00:05, 1812901.08it/s]: end{sphinxVerbatim}

20181018_All-sky_point-source_IceCube_da…: 18%|█▊ | 2097152/11730609 [00:01<00:05, 1812901.08it/s]

20181018_All-sky_point-source_IceCube_da…: 27%|██▋ | 3145728/11730609 [00:01<00:03, 2730729.14it/s]: </pre>
20181018_All-sky_point-source_IceCube_da{ldots}: 27%|██▋ | 3145728/11730609 [00:01<00:03, 2730729.14it/s]: end{sphinxVerbatim}

20181018_All-sky_point-source_IceCube_da…: 27%|██▋ | 3145728/11730609 [00:01<00:03, 2730729.14it/s]

20181018_All-sky_point-source_IceCube_da…: 36%|███▌ | 4194304/11730609 [00:01<00:02, 3660828.45it/s]: </pre>
20181018_All-sky_point-source_IceCube_da{ldots}: 36%|███▌ | 4194304/11730609 [00:01<00:02, 3660828.45it/s]: end{sphinxVerbatim}

20181018_All-sky_point-source_IceCube_da…: 36%|███▌ | 4194304/11730609 [00:01<00:02, 3660828.45it/s]

20181018_All-sky_point-source_IceCube_da…: 54%|█████▎ | 6291456/11730609 [00:01<00:00, 5972742.93it/s]: </pre>
20181018_All-sky_point-source_IceCube_da{ldots}: 54%|█████▎ | 6291456/11730609 [00:01<00:00, 5972742.93it/s]: end{sphinxVerbatim}

20181018_All-sky_point-source_IceCube_da…: 54%|█████▎ | 6291456/11730609 [00:01<00:00, 5972742.93it/s]

20181018_All-sky_point-source_IceCube_da…: 72%|███████▏ | 8388608/11730609 [00:01<00:00, 8041849.70it/s]: </pre>
20181018_All-sky_point-source_IceCube_da{ldots}: 72%|███████▏ | 8388608/11730609 [00:01<00:00, 8041849.70it/s]: end{sphinxVerbatim}

20181018_All-sky_point-source_IceCube_da…: 72%|███████▏ | 8388608/11730609 [00:01<00:00, 8041849.70it/s]

20181018_All-sky_point-source_IceCube_da…: 98%|█████████▊| 11534336/11730609 [00:02<00:00, 11332662.63it/s]: </pre>
20181018_All-sky_point-source_IceCube_da{ldots}: 98%|█████████▊| 11534336/11730609 [00:02<00:00, 11332662.63it/s]: end{sphinxVerbatim}

20181018_All-sky_point-source_IceCube_da…: 98%|█████████▊| 11534336/11730609 [00:02<00:00, 11332662.63it/s]

20181018_All-sky_point-source_IceCube_da…: 100%|██████████| 11730609/11730609 [00:02<00:00, 5811211.78it/s]: </pre>
20181018_All-sky_point-source_IceCube_da{ldots}: 100%|██████████| 11730609/11730609 [00:02<00:00, 5811211.78it/s]: end{sphinxVerbatim}

20181018_All-sky_point-source_IceCube_da…: 100%|██████████| 11730609/11730609 [00:02<00:00, 5811211.78it/s]

You may not want to use icecube_tools for other stuff, so you can also fetch to a specificed location with the keyword write_to.

[6]:

my_data.fetch(found_dataset, write_to="data", overwrite=True)

20181018_All-sky_point-source_IceCube_da…: 0%| | 0/11730609 [00:00<?, ?it/s]: </pre>
20181018_All-sky_point-source_IceCube_da{ldots}: 0%| | 0/11730609 [00:00<?, ?it/s]: end{sphinxVerbatim}

20181018_All-sky_point-source_IceCube_da…: 0%| | 0/11730609 [00:00<?, ?it/s]

20181018_All-sky_point-source_IceCube_da…: 9%|▉ | 1048576/11730609 [00:01<00:11, 940985.25it/s]: </pre>
20181018_All-sky_point-source_IceCube_da{ldots}: 9%|▉ | 1048576/11730609 [00:01<00:11, 940985.25it/s]: end{sphinxVerbatim}

20181018_All-sky_point-source_IceCube_da…: 9%|▉ | 1048576/11730609 [00:01<00:11, 940985.25it/s]

20181018_All-sky_point-source_IceCube_da…: 18%|█▊ | 2097152/11730609 [00:01<00:05, 1752336.33it/s]: </pre>
20181018_All-sky_point-source_IceCube_da{ldots}: 18%|█▊ | 2097152/11730609 [00:01<00:05, 1752336.33it/s]: end{sphinxVerbatim}

20181018_All-sky_point-source_IceCube_da…: 18%|█▊ | 2097152/11730609 [00:01<00:05, 1752336.33it/s]

20181018_All-sky_point-source_IceCube_da…: 27%|██▋ | 3145728/11730609 [00:01<00:03, 2459346.64it/s]: </pre>
20181018_All-sky_point-source_IceCube_da{ldots}: 27%|██▋ | 3145728/11730609 [00:01<00:03, 2459346.64it/s]: end{sphinxVerbatim}

20181018_All-sky_point-source_IceCube_da…: 27%|██▋ | 3145728/11730609 [00:01<00:03, 2459346.64it/s]

20181018_All-sky_point-source_IceCube_da…: 45%|████▍ | 5242880/11730609 [00:01<00:01, 4174728.68it/s]: </pre>
20181018_All-sky_point-source_IceCube_da{ldots}: 45%|████▍ | 5242880/11730609 [00:01<00:01, 4174728.68it/s]: end{sphinxVerbatim}

20181018_All-sky_point-source_IceCube_da…: 45%|████▍ | 5242880/11730609 [00:01<00:01, 4174728.68it/s]

20181018_All-sky_point-source_IceCube_da…: 63%|██████▎ | 7340032/11730609 [00:01<00:00, 6072432.45it/s]: </pre>
20181018_All-sky_point-source_IceCube_da{ldots}: 63%|██████▎ | 7340032/11730609 [00:01<00:00, 6072432.45it/s]: end{sphinxVerbatim}

20181018_All-sky_point-source_IceCube_da…: 63%|██████▎ | 7340032/11730609 [00:01<00:00, 6072432.45it/s]

20181018_All-sky_point-source_IceCube_da…: 80%|████████ | 9437184/11730609 [00:02<00:00, 7979942.29it/s]: </pre>
20181018_All-sky_point-source_IceCube_da{ldots}: 80%|████████ | 9437184/11730609 [00:02<00:00, 7979942.29it/s]: end{sphinxVerbatim}

20181018_All-sky_point-source_IceCube_da…: 80%|████████ | 9437184/11730609 [00:02<00:00, 7979942.29it/s]

20181018_All-sky_point-source_IceCube_da…: 100%|██████████| 11730609/11730609 [00:02<00:00, 5410751.14it/s]: </pre>
20181018_All-sky_point-source_IceCube_da{ldots}: 100%|██████████| 11730609/11730609 [00:02<00:00, 5410751.14it/s]: end{sphinxVerbatim}

20181018_All-sky_point-source_IceCube_da…: 100%|██████████| 11730609/11730609 [00:02<00:00, 5410751.14it/s]

For convenience, there is also the fetch_all_to method to download all the available data to a specified location. We comment this here as it can take a while to execute.

[7]:

# my_data.fetch_all_to("data")