{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Accessing SnowEx Data at the NSIDC DAAC\n",
"\n",
"2021 SnowEx Hackweek\n",
"\n",
"Author: Amy Steiker, NSIDC DAAC, CIRES, University of Colorado\n",
"\n",
"\n",
"This tutorial provides a brief overview of the resources provided by the NASA National Snow and Ice Data Center Distributed Active Archive Center, or NSIDC DAAC, and demonstrates how to discovery and access SnowEx data programmatically. \n",
"\n",
"## Learning Objectives:\n",
"\n",
"1. Identify resources available at the NSIDC DAAC to help you work with SnowEx data. \n",
"2. Search and discover file size and coverage of SnowEx data over a time and geojson region of interest.\n",
"3. Set up Earthdata Login authentication needed to access NASA Earthdata.\n",
"4. Download SnowEx data programmatically using the NSIDC DAAC Application Programming Interface (API)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"___\n",
"\n",
"## Explore snow products and resources\n",
"\n",
"\n",
"### NSIDC introduction\n",
"\n",
"[The National Snow and Ice Data Center](https://nsidc.org), located in Boulder, Colorado, provides over 1100 data sets covering the Earth's cryosphere and more, all of which are available to the public free of charge. Beyond providing these data, NSIDC creates tools for data access, supports data users, performs scientific research, and educates the public about the cryosphere. \n",
"\n",
"\n",
"\n",
"The [NASA National Snow and Ice Data Center Distributed Active Archive Center (NSIDC DAAC)](https://nsidc.org/daac) provides access to over 700 data products (over 2 PB of data and growing!) in support of cryospheric research, global change detection, and water resource management. NSIDC is one of 12 DAACs as part of NASA’s EOSDIS, or Earth Observing System Data and Information System, and is also the largest program within the National Snow and Ice Data Center, as part of the University of Colorado’s Cooperative Institute for Research in Environmental Sciences. \n",
"\n",
"\n",
"#### Select Data Resources\n",
"\n",
"* [NSIDC Data Tutorials](https://github.com/nsidc/NSIDC-Data-Tutorials)\n",
" * Jupyter Notebook tutorials providing guidance on working with various NSIDC DAAC data products, including how to access, subset, transform, and visualize data.\n",
"* [NSIDC Data Search](https://nsidc.org/data/search/#keywords=snow)\n",
" * Search NSIDC snow data\n",
"* [NSIDC Data Update Announcements](https://nsidc.org/the-drift/data-update/) \n",
" * News and tips for data users\n",
"* [NASA Earthdata Search](http://search.earthdata.nasa.gov/)\n",
" * Search and access data across the NASA Earthdata\n",
"* [NASA Worldview](https://worldview.earthdata.nasa.gov/)\n",
" * Interactive interface for browsing full-resolution, global, daily satellite images\n",
" \n",
"\n",
"\n",
"#### Snow Today\n",
"\n",
"[Snow Today](https://nsidc.org/snow-today), a collaboration with the University of Colorado's Institute of Alpine and Arctic Research (INSTAAR), provides near-real-time snow analysis for the western United States and regular reports on conditions during the winter season. Snow Today is funded by NASA Hydrological Sciences Program and utilizes data from the Moderate Resolution Imaging Spectroradiometer (MODIS) instrument and snow station data from the Snow Telemetry (SNOTEL) network by the Natural Resources Conservation Service (NRCS), United States Department of Agriculture (USDA) and the California Department of Water Resources: www.wcc.nrcs.usda.gov/snow.\n",
"\n",
"#### NSIDC's SnowEx pages\n",
"\n",
"* [SnowEx](https://nsidc.org/data/snowex)\n",
" * We will be using the 'SnowEx17 Ground Penetrating Radar, Version 2' dataset in the following steps: https://doi.org/10.5067/G21LGCNLFSC5\n",
"\n",
"#### Other relevant snow products:\n",
"\n",
"* [ASO Snow Data](https://nsidc.org/data/search/#keywords=aso/sortKeys=score,,desc/facetFilters=%257B%2522facet_parameter%2522%253A%255B%2522SNOW%2520DEPTH%2522%252C%2522SNOW%2520WATER%2520EQUIVALENT%2522%255D%257D/pageNumber=1/itemsPerPage=25)\n",
"\n",
"* [MODIS Snow Data](https://nsidc.org/data/search/#sortKeys=score,,desc/facetFilters=%257B%2522facet_parameter%2522%253A%255B%2522SNOW%2520COVER%2522%252C%2522Snow%2520Cover%2522%252C%2522SNOW%2520WATER%2520EQUIVALENT%2522%252C%2522SNOW%2520DEPTH%2522%252C%2522SNOW%2520MELT%2522%252C%2522SNOW%2520VOLUME%2520BUDGET%2522%252C%2522SNOW%252FICE%2522%255D%252C%2522facet_sensor%2522%253A%255B%2522%2520%257C%2520MODIS%2522%252C%2522Moderate-Resolution%2520Imaging%2520Spectroradiometer%2520%257C%2520MODIS%2522%255D%252C%2522facet_sponsored_program%2522%253A%255B%2522NASA%2520National%2520Snow%2520and%2520Ice%2520Data%2520Center%2520Distributed%2520Active%2520Archive%2520Center%2520%257C%2520NASA%2520NSIDC%2520DAAC%2522%255D%257D/pageNumber=1/itemsPerPage=25)\n",
"\n",
"* [VIIRS Snow Data](http://nsidc.org/data/search/#sortKeys=score,,desc/facetFilters=%257B%2522facet_sensor%2522%253A%255B%2522Visible-Infrared%2520Imager-Radiometer%2520Suite%2520%257C%2520VIIRS%2522%255D%252C%2522facet_parameter%2522%253A%255B%2522SNOW%2520COVER%2522%252C%2522Snow%2520Cover%2522%255D%257D/pageNumber=1/itemsPerPage=25)\n",
"\n",
"* [AMSR-E and AMSR-E/AMSR2 Unified Snow Data](http://nsidc.org/data/search/#sortKeys=score,,desc/facetFilters=%257B%2522facet_sensor%2522%253A%255B%2522Advanced%2520Microwave%2520Scanning%2520Radiometer-EOS%2520%257C%2520AMSR-E%2522%252C%2522Advanced%2520Microwave%2520Scanning%2520Radiometer%25202%2520%257C%2520AMSR2%2522%255D%252C%2522facet_parameter%2522%253A%255B%2522SNOW%2520WATER%2520EQUIVALENT%2522%252C%2522Snow%2520Water%2520Equivalent%2522%255D%257D/pageNumber=1/itemsPerPage=25)\n",
"\n",
"* [MEaSUREs Snow Data](http://nsidc.org/data/search/#keywords=measures/sortKeys=score,,desc/facetFilters=%257B%2522facet_parameter%2522%253A%255B%2522SNOW%2520COVER%2522%255D%252C%2522facet_sponsored_program%2522%253A%255B%2522NASA%2520National%2520Snow%2520and%2520Ice%2520Data%2520Center%2520Distributed%2520Active%2520Archive%2520Center%2520%257C%2520NASA%2520NSIDC%2520DAAC%2522%255D%252C%2522facet_format%2522%253A%255B%2522NetCDF%2522%255D%252C%2522facet_temporal_duration%2522%253A%255B%252210%252B%2520years%2522%255D%257D/pageNumber=1/itemsPerPage=25)\n",
" \n",
"* Near-Real-Time SSM/I-SSMIS EASE-Grid Daily Global Ice Concentration and Snow Extent (NISE), Version 5: https://doi.org/10.5067/3KB2JPLFPK3R"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"___\n",
"## Import Packages\n",
"\n",
"Get started by importing packages needed to run the following code blocks, including the `tutorial_helper_functions` module provided within this repository."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"import geopandas as gpd\n",
"from shapely.geometry import Polygon, mapping\n",
"from shapely.geometry.polygon import orient\n",
"import pandas as pd \n",
"import requests\n",
"import json\n",
"import pprint\n",
"import getpass\n",
"import netrc\n",
"from platform import system\n",
"from getpass import getpass\n",
"from urllib import request\n",
"from http.cookiejar import CookieJar\n",
"from os.path import join, expanduser\n",
"import requests\n",
"from xml.etree import ElementTree as ET\n",
"import time\n",
"import zipfile\n",
"import io\n",
"import shutil"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"___\n",
"\n",
"\n",
"## Data Discovery\n",
"\n",
"Start by identifying your study area and exploring coincident data over the same time and area. "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Identify area and time of interest\n",
"\n",
"Since our focus is on the Grand Mesa study site of the NASA SnowEx campaign, we'll use that area to search for coincident data across other data products. From the [SnowEx17 Ground Penetrating Radar Version 2](https://doi.org/10.5067/G21LGCNLFSC5) landing page, you can find the rectangular spatial coverage under the Overview tab, or you can draw a polygon over your area of interest in the map under the Download Data tab and export the shape as a geojson file using the Export Polygon icon shown below. An example polygon geojson file is provided in the /Data folder of this repository. \n",
"\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Create polygon coordinate string\n",
"\n",
"Read in the geojson file as a GeoDataFrame object and simplify and reorder using the shapely package. This will be converted back to a dictionary to be applied as our polygon search parameter. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"polygon_filepath = str(os.getcwd() + '/Data/nsidc-polygon.json') # Note: A shapefile or other vector-based spatial data format could be substituted here.\n",
"\n",
"gdf = gpd.read_file(polygon_filepath) #Return a GeoDataFrame object\n",
"\n",
"# Simplify polygon for complex shapes in order to pass a reasonable request length to CMR. The larger the tolerance value, the more simplified the polygon.\n",
"# Orient counter-clockwise: CMR polygon points need to be provided in counter-clockwise order. The last point should match the first point to close the polygon.\n",
"poly = orient(gdf.simplify(0.05, preserve_topology=False).loc[0],sign=1.0)\n",
"\n",
"#Format dictionary to polygon coordinate pairs for CMR polygon filtering\n",
"polygon = ','.join([str(c) for xy in zip(*poly.exterior.coords.xy) for c in xy])\n",
"print('Polygon coordinates to be used in search:', polygon)\n",
"poly"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Alternatively: Specify bounding box region of interest\n",
"\n",
"Instead of using a vector shape file to specify a region of interest, you can simply use a bounding box. The following cell is commented out below, which can be used instead of the `polygon` search parameter."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# bounds = poly.bounds # Get polygon bounds to be used as bounding box input\n",
"# bounding_box = ','.join(map(str, list(bounds))) # Bounding Box spatial parameter in decimal degree 'W,S,E,N' format. \n",
"# print(bounding_box)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Set time range\n",
"\n",
"This is an optional parameter; set this to specify a time range of interest. In this case we'll just select all of 2017 to ensure that we receive all files within this data set campaign."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"temporal = '2017-01-01T00:00:00Z,2017-12-31T23:59:59Z' # Set temporal range"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Create data parameter dictionary \n",
"\n",
"Create a dictionary with the data set shortname and version, as well as the temporal range and polygonal area of interest. Data set shortnames, or IDs, as well as version numbers, are located at the top of every NSIDC landing page."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"param_dict = {\n",
" 'short_name': 'SNEX17_GPR',\n",
" 'version': '2',\n",
" 'polygon': polygon,\n",
"# 'bounding_box': bounding_box, #optional alternative to polygon search parameter; if using, remove or comment out polygon search parameter\n",
" 'temporal':temporal,\n",
"}"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Determine how many files exist over this time and area of interest, as well as the average size and total volume of those files\n",
"\n",
"We will use the `granule_info` function to query metadata about each data set and associated files using the [Common Metadata Repository (CMR)](https://cmr.earthdata.nasa.gov/search/site/docs/search/api.html), which is a high-performance, high-quality, continuously evolving metadata system that catalogs Earth Science data and associated service metadata records. Note that not all NSIDC data can be searched at the file level using CMR, particularly those outside of the NASA DAAC program. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"def search_granules(search_parameters, geojson=None, output_format=\"json\"):\n",
" \"\"\"\n",
" Performs a granule search with token authentication for restricted results\n",
" \n",
" :search_parameters: dictionary of CMR search parameters\n",
" :token: CMR token needed for restricted search\n",
" :geojson: filepath to GeoJSON file for spatial search\n",
" :output_format: select format for results https://cmr.earthdata.nasa.gov/search/site/docs/search/api.html#supported-result-formats\n",
" \n",
" :returns: if hits is greater than 0, search results are returned in chosen output_format, otherwise returns None.\n",
" \"\"\"\n",
" search_url = \"https://cmr.earthdata.nasa.gov/search/granules\"\n",
"\n",
" \n",
" if geojson:\n",
" files = {\"shapefile\": (geojson, open(geojson, \"r\"), \"application/geo+json\")}\n",
" else:\n",
" files = None\n",
" \n",
" \n",
" parameters = {\n",
" \"scroll\": \"true\",\n",
" \"page_size\": 100,\n",
" }\n",
" \n",
" try:\n",
" response = requests.post(f\"{search_url}.{output_format}\", params=parameters, data=search_parameters, files=files)\n",
" response.raise_for_status()\n",
" except HTTPError as http_err:\n",
" print(f\"HTTP Error: {http_error}\")\n",
" except Exception as err:\n",
" print(f\"Error: {err}\")\n",
" \n",
" hits = int(response.headers['CMR-Hits'])\n",
" if hits > 0:\n",
" print(f\"Found {hits} files\")\n",
" results = json.loads(response.content)\n",
" granules = []\n",
" granules.extend(results['feed']['entry'])\n",
" granule_sizes = [float(granule['granule_size']) for granule in granules]\n",
" print(f\"The total size of all files is {sum(granule_sizes):.2f} MB\")\n",
" return response.json()\n",
" else:\n",
" print(\"Found no hits\")\n",
" return\n",
"\n",
"search_granules(param_dict)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"___\n",
"\n",
"## Data Access\n",
"\n",
"### Option 1: Python script from the data set landing page\n",
"The [Depths and Snow Pit Data Package Contents](https://snowex-hackweek.github.io/website/tutorials/core-datasets/02_data-package.html) tutorial demonstrates how to access data using the script provided under the \"Download Data\" tab of each NSIDC DAAC data set landing page. \n",
"\n",
"#### Earthdata Login authentication setup notes\n",
"\n",
"1. All data with the NASA Earthdata system are freely available to the public, but requires an Earthdata Login profile to access. If you have not already done so, visit http://urs.earthdata.nasa.gov to register (it just takes a minute to set up). \n",
"\n",
"2. Create a `.netrc` file within the JupyterHub environment. This is a hidden file typically stored in your home directory that contains your Earthdata login username and password. This is a secure and easy way to ensure that any data download requests are authenticated against your profile. You can set up your `.netrc` within the JupyterHub environment as copied from the preliminary work article [here](https://snowex-hackweek.github.io/website/preliminary/earthdata.html#configure-programmatic-access-to-nasa-servers):\n",
"\n",
"Run the following commands on the JupyterHub in a terminal replacing your Earthdata login username and password:\n",
"\n",
"```\n",
"echo \"machine urs.earthdata.nasa.gov login EARTHDATA_LOGIN password EARTHDATA_PASSWORD\" > ~/.netrc\n",
"chmod 0600 .netrc\n",
"```\n",
"\n",
"Note that the script below should prompt you with your Earthdata Login username and password if a `.netrc` file does not exist. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"os.chmod('/home/jovyan/.netrc', 0o600) #only necessary on snowex hackweek jupyterhub"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"%run './scripts/nsidc-download_SNEX20_SD.001.py' \n",
"print('Grand Mesa 2020 Snow Depth data download complete') "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Option 2: Additional data access services: API Access\n",
"\n",
"Data can be accessed directly from our HTTPS file system as described in the aforementioned tutorial, or through the NSIDC DAAC's Application Programming Interface (API). \n",
"\n",
"What is an API? You can think of an API as a middle man between an application or end-use (in this case, us) and a data provider. Here, the data provider is both the Common Metadata Repository (CMR) housing data information, and NSIDC as the data distributor. These APIs are generally structured as a URL with a base plus individual key-value-pairs separated by ‘&’. This option is beneficial for those of you who want to incorporate data access directly into your visualization and analysis coding workflow, without the need to utilize the NSIDC website. This method is also reproducible and documented to ensure data provenance.\n",
"\n",
"\n",
"This API offers you the ability to order data using specific temporal and spatial filters. These options can be requested in a single access command without the need to script against our data directory structure. See the [programmatic access guide](https://nsidc.org/support/how/how-do-i-programmatically-request-data-services) for more information on API options. "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Create the data request API endpoint\n",
"Programmatic API requests are formatted as HTTPS URLs that contain key-value-pairs specifying the service operations that we specified above. \n",
"\n",
"The cell below sets up the API request URL using our data search parameters as well as a few other configuration parameters. We will first create a string of key-value-pairs from our data dictionary and we'll feed those into our API endpoint. This API endpoint can be executed via command line, a web browser, or in Python below."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"#Set NSIDC data access base URL\n",
"base_url = 'https://n5eil02u.ecs.nsidc.org/egi/request'\n",
"\n",
"#Set the request mode to asynchronous, \"no\" processing agent (no subsetting or reformatting services available), and optionally removing metadata delivery\n",
"\n",
"param_dict['request_mode'] = 'async'\n",
"param_dict['agent'] = 'NO'\n",
"param_dict['INCLUDE_META'] ='N' #optional if you do not wish to receive the associated metadata files with each science file. \n",
"\n",
"param_string = '&'.join(\"{!s}={!r}\".format(k,v) for (k,v) in param_dict.items()) # Convert param_dict to string\n",
"param_string = param_string.replace(\"'\",\"\") # Remove quotes\n",
"\n",
"api_list = [f'{base_url}?{param_string}']\n",
"api_request = api_list[0]\n",
"print(api_request) # Print API base URL + request parameters"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Input Earthdata Login credentials\n",
"\n",
"For our API access option, An Earthdata Login account is required to access data from the NSIDC DAAC. If you do not already have an Earthdata Login account, visit http://urs.earthdata.nasa.gov to register."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Start authenticated session with Earthdata Login to allow for data downloads:\n",
"def setup_earthdata_login_auth(endpoint: str='urs.earthdata.nasa.gov'):\n",
" netrc_name = \"_netrc\" if system()==\"Windows\" else \".netrc\"\n",
" try:\n",
" username, _, password = netrc.netrc(file=join(expanduser('~'), netrc_name)).authenticators(endpoint)\n",
" except (FileNotFoundError, TypeError):\n",
" print('Please provide your Earthdata Login credentials for access.')\n",
" print('Your info will only be passed to %s and will not be exposed in Jupyter.' % (endpoint))\n",
" username = input('Username: ')\n",
" password = getpass('Password: ')\n",
" manager = request.HTTPPasswordMgrWithDefaultRealm()\n",
" manager.add_password(None, endpoint, username, password)\n",
" auth = request.HTTPBasicAuthHandler(manager)\n",
" jar = CookieJar()\n",
" processor = request.HTTPCookieProcessor(jar)\n",
" opener = request.build_opener(auth, processor)\n",
" request.install_opener(opener)\n",
"\n",
"setup_earthdata_login_auth(endpoint=\"urs.earthdata.nasa.gov\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Download data\n",
"\n",
"Download data using the `requests` library. The data will be downloaded directly to this directory in a new Outputs folder. The progress of each order will be reported."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"def request_nsidc_data(API_request):\n",
" \"\"\"\n",
" Performs a data customization and access request from NSIDC's API/\n",
" Creates an output folder in the working directory if one does not already exist.\n",
" \n",
" :API_request: NSIDC API endpoint; see https://nsidc.org/support/how/how-do-i-programmatically-request-data-services for more info\n",
" on how to configure the API request.\n",
" \n",
" \"\"\"\n",
"\n",
" path = str(os.getcwd() + '/Outputs') # Create an output folder if the folder does not already exist.\n",
" if not os.path.exists(path):\n",
" os.mkdir(path)\n",
" \n",
" base_url = 'https://n5eil02u.ecs.nsidc.org/egi/request'\n",
"\n",
" \n",
" r = request.urlopen(API_request)\n",
" esir_root = ET.fromstring(r.read())\n",
" orderlist = [] # Look up order ID\n",
" for order in esir_root.findall(\"./order/\"):\n",
" orderlist.append(order.text)\n",
" orderID = orderlist[0]\n",
" statusURL = base_url + '/' + orderID # Create status URL\n",
" print('Order status URL: ', statusURL)\n",
" request_response = request.urlopen(statusURL) # Find order status \n",
" request_root = ET.fromstring(request_response.read())\n",
" statuslist = []\n",
" for status in request_root.findall(\"./requestStatus/\"):\n",
" statuslist.append(status.text)\n",
" status = statuslist[0]\n",
" while status == 'pending' or status == 'processing': #Continue loop while request is still processing\n",
" print('Job status is ', status,'. Trying again.')\n",
" time.sleep(10)\n",
" loop_response = request.urlopen(statusURL)\n",
" loop_root = ET.fromstring(loop_response.read())\n",
" statuslist = [] #find status\n",
" for status in loop_root.findall(\"./requestStatus/\"):\n",
" statuslist.append(status.text)\n",
" status = statuslist[0]\n",
" if status == 'pending' or status == 'processing':\n",
" continue\n",
" if status == 'complete_with_errors' or status == 'failed': # Provide complete_with_errors error message:\n",
" messagelist = []\n",
" for message in loop_root.findall(\"./processInfo/\"):\n",
" messagelist.append(message.text)\n",
" print('Job status is ', status)\n",
" print('error messages:')\n",
" pprint(messagelist)\n",
" if status == 'complete' or status == 'complete_with_errors':# Download zipped order if status is complete or complete_with_errors\n",
" downloadURL = 'https://n5eil02u.ecs.nsidc.org/esir/' + orderID + '.zip'\n",
" print('Job status is ', status)\n",
" print('Zip download URL: ', downloadURL)\n",
" print('Beginning download of zipped output...')\n",
" zip_response = request.urlopen(downloadURL)\n",
" with zipfile.ZipFile(io.BytesIO(zip_response.read())) as z:\n",
" z.extractall(path)\n",
" print('Download is complete.')\n",
" else: print('Request failed.')\n",
" \n",
" # Clean up Outputs folder by removing individual granule folders \n",
" for root, dirs, files in os.walk(path, topdown=False):\n",
" for file in files:\n",
" try:\n",
" shutil.move(os.path.join(root, file), path)\n",
" except OSError:\n",
" pass\n",
" for name in dirs:\n",
" os.rmdir(os.path.join(root, name))\n",
" return \n",
"\n",
"\n",
"# NOTE: downloads ~ 200MB of CSV files\n",
"request_nsidc_data(api_request)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"___\n",
"## Read in SnowEx data\n",
"\n",
"This SnowEx data set is provided in CSV. A [Pandas DataFrame](https://pandas.pydata.org/pandas-docs/stable/getting_started/overview.html) is used to easily read in data."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"snowex_path = './Outputs/SnowEx17_GPR_Version2_Week1.csv' # Define local filepath\n",
"df = pd.read_csv(snowex_path, sep='\\t') \n",
"df.head()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Extract date values"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The collection date needs to be extracted from the `collection` parameter:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"df['date'] = df.collection.str.rsplit('_').str[-1].astype(str)\n",
"df.date = pd.to_datetime(df.date, format=\"%m%d%y\")\n",
"df = df.sort_values(['date'])\n",
"df.head()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Convert to Geopandas dataframe to provide point geometry\n",
"\n",
"According to the SnowEx documentation, the data are available in UTM Zone 12N so we'll set to this projection to allow for geospatial analysis:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"gdf_utm= gpd.GeoDataFrame(df, geometry=gpd.points_from_xy(df.x, df.y), crs='EPSG:32612')\n",
"gdf_utm.head()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"___\n",
"## Additional data imagery services\n",
"\n",
"### NASA Worldview and the Global Browse Imagery Service\n",
"\n",
"NASA’s EOSDIS Worldview mapping application provides the capability to interactively browse over 900 global, full-resolution satellite imagery layers and then download the underlying data. Many of the available imagery layers are updated within three hours of observation, essentially showing the entire Earth as it looks “right now.\"\n",
"\n",
"Several MODIS snow data products from NSIDC include high-resolution browse imagery available through NASA Worldview, including [\"MODIS/Terra Snow Cover Daily L3 Global 500m SIN Grid, Version 61\"](https://nsidc.org/data/mod10a1). This layer can be downloaded as various image files including GeoTIFF using the snapshot feature at the top right of the page. This link presents the MOD10A1 NDSI layer over our time and area of interest: https://go.nasa.gov/35CgYMd. \n",
"\n",
"Additionally, the NASA Global Browse Imagery Service provides up to date, full resolution imagery for select NSIDC DAAC data sets as web services including WMTS, WMS, KML, and more. These layers can be accessed in GIS applications following guidance on the [GIBS documentation pages](https://wiki.earthdata.nasa.gov/display/GIBS/Geographic+Information+System+%28GIS%29+Usage). "
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.8"
}
},
"nbformat": 4,
"nbformat_minor": 4
}