Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add tutorial notebook for TIKE #149

Closed
wants to merge 9 commits into from
Closed

Add tutorial notebook for TIKE #149

Changes from 7 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
81213f9
Add TIKE tutorial notebook for LCviz
rosteen Oct 21, 2024
16e51f1
More updates
rosteen Oct 21, 2024
199b0f9
Remove template text
rosteen Oct 21, 2024
5e57f5e
Update notebooks/tike_lcviz_tutorial.ipynb
rosteen Oct 24, 2024
f64f33f
Update notebooks/tike_lcviz_tutorial.ipynb
rosteen Oct 24, 2024
776e0b2
Apply suggestions from code review
rosteen Oct 24, 2024
6f6f7e8
Kepler link instead of TESS
rosteen Oct 24, 2024
abd0ab8
Apply suggestions from code review
rosteen Oct 28, 2024
1c83a13
Use public API to get time origin
rosteen Oct 30, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
373 changes: 373 additions & 0 deletions notebooks/tike_lcviz_tutorial.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,373 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a id=\"top\"></a>\n",
"# LCviz Tutorial\n",
"***\n",
"## Learning Goals\n",
"\n",
"\n",
"By the end of this tutorial, you will:\n",
"\n",
"- Understand how to load time series data into LCviz.\n",
"- Apply flattening to a light curve to remove low frequency trends.\n",
"- Use LCviz to create and display a phase-folded light curve."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Table of Contents\n",
"* [Introduction](#Introduction)\n",
"* [Imports](#Imports)\n",
"* [Using LCviz](#Using-LCviz)\n",
" * [Initializing LCviz](#Initializing-LCviz)\n",
" * [Loading Light Curves into LCviz](#Loading-light-curves-into-LCviz)\n",
" * [Flattening](#Flattening)\n",
" * [Phase Folding](#Phase-Folding)\n",
"* [Additional Resources](#Additional-Resources)\n",
"\n",
"## Introduction\n",
"In this notebook we will be exploring the use of the [LCviz](https://github.com/spacetelescope/lcviz) (light curve visualization and analysis) tool for investigating time series observations of a transiting exoplanet. LCviz is built on the [Jdaviz](https://github.com/spacetelescope/jdaviz/) (James Webb data analysis and visualization) tool and additionally relies heavily on the [lightkurve](https://github.com/lightkurve/lightkurve) and [astropy](https://github.com/astropy/astropy) packages.\n",
rosteen marked this conversation as resolved.
Show resolved Hide resolved
"\n",
"Specifically, we will be using LCviz to look at a Kepler long cadence light curve of HAT-P-11, a K4V host to\n",
"a transiting hot Neptune with a 4.8 day period, and a stellar rotation period of 29 days."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Imports\n",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lcviz isn't currently installed on the production version of TIKE – I'm assuming that will be done when this notebook is available?

Also, these imports only work if you're in the kernel named TESS Environment. Should we mention that here?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct, they will install it once 1.0 is released AND they have this notebook. I think the general instructions handle selecting the kernel environment, but that's a good question for @ttdu

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lcviz will be installed in ~December, when we release a new environment for AAS.

we normally specify the default kernel in the notebook metadata, so that TESS Environment is automatically selected. it never hurts to be verbose, but Kyle is correct that the general instructions cover this

"\n",
"- *s3fs* used to access cloud files as though they were local\n",
"- *astropy.units* used to define quantities with units attached\n",
"- *astropy.time Time* used to define the epoch for the ephemeris of the planet\n",
"- *astroquery.mast* used to search for and select data\n",
"- *lightkurve* for reading the light curve file\n",
"- *lcviz* for plotting and analyzing data"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [],
"source": [
"import s3fs\n",
"\n",
"import astropy.units as u\n",
"from astropy.time import Time\n",
"from astroquery.mast import Observations\n",
"import lightkurve\n",
"\n",
"from lcviz import LCviz"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"***"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Using LCviz"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Initializing LCviz\n",
"\n",
"Starting the LCviz application takes two steps: first, we need to create an instance of\n",
"the `LCviz` class, and then we need to show the application in the notebook. The two lines\n",
"below will open an empty instance of LCviz in the output cell of the notebook. \n",
"\n",
"The two main sections of LCviz that you will see below are a viewer for displaying flux vs time\n",
"data on the left, and an open tray with expandable sections giving UI access to data analysis plugins\n",
"on the right. Hovering the cursor over the various buttons in the application will display tooltips\n",
"giving a short description of their purpose.\n",
"\n",
"In the next section, we will actually load data into LCviz to be shown in the flux vs time\n",
"viewer."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"jp-MarkdownHeadingCollapsed": true
},
"outputs": [],
"source": [
"# Create an instance of the LCviz class\n",
"lcviz = LCviz()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now we show the application in the notebook. We keep this in a separate cell because\n",
"it is occasionally useful to refresh the display of the app without overwriting\n",
"it with a new instance:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"lcviz.show() # Show the LCviz app"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"It may be more convenient to open the application in a new tab within your Jupyter\n",
"Lab instance instead of scrolling up and down between the app and the code cells.\n",
"To do this you could un-comment and run the following line:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# lcviz.show(loc=\"sidecar:tab-after\") # Show the app in a new tab within Jupyter Lab"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Loading light curves into LCviz\n",
"\n",
"Because TIKE already runs in the cloud, it is fastest and easiest to load data that is also hosted in the cloud using the `s3fs` package, which allows you to access cloud data as if it were local. Here we will use [astroquery](https://astroquery.readthedocs.io/en/latest/) to get the URI for the file that we want to load. For more information about the code below, see [this TIKE webinar notebook](https://github.com/spacetelescope/tike_content/blob/main/content/notebooks/webinar-series/01-lightcurves/01-Lightcurves.ipynb).\n"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since this mentions TIKE specifically, maybe we don't want to actually merge this PR here?

]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"Observations.enable_cloud_dataset() # use cloud data when possible\n",
"fs = s3fs.S3FileSystem(anon=True) # read cloud data as though local"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what could we do to make this more seamless (maybe not in time for lcviz 1.0, but in the future)? Could we have a switch in lcviz to do both of these? (cc @bmorris3)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might be something it would be nice to do upstream - have a switch in Jdaviz at the app or helper level to turn on cloud support (with optional dependencies) that would then allow load_data to take a cloud URI.

]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [],
"source": [
"# Query for Kepler time series Observations of our target. This query\n",
"# will return data from all quarters.\n",
"kepler_table = Observations.query_criteria(objectname=\"HAT-P-11\",\n",
" obs_collection=\"Kepler\",\n",
" dataproduct_type=\"timeseries\",\n",
" obs_id=\"kplr010748390_lc_Q111111101110111011\"\n",
" ) \n",
"\n",
"# Get associated science products for each Observation\n",
"data_products = Observations.get_product_list(kepler_table)\n",
"\n",
"# Keep only the long cadence light curve science products\n",
"lc_prod = Observations.filter_products(data_products, productType=\"SCIENCE\", productSubGroupDescription = \"LLC\")\n",
"\n",
"# Get the cloud URI for the quarter 9 data. Note that this is the 8th entry because\n",
"# there is no file for quarter 7 in the results.\n",
"lc_uri = Observations.get_cloud_uri(lc_prod[8])"
Comment on lines +197 to +213
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would also be nice to be able to get this directly from lightkurve, but I'm guessing that isn't possible?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't believe so.

Copy link

@ttdu ttdu Oct 28, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you could shorten this by passing everything into get_cloud_uris

e.g. this JWST query

obs = Observations.query_criteria(objectname="NGC 3132", obs_collection="JWST")
prod = Observations.get_product_list(obs)
filt = Observations.filter_products(prod, productType="SCIENCE", calib_level=[3])
cloud_files = Observations.get_cloud_uris(filt)

becomes a single (but more complicated) line of code:
cloud_files = Observations.get_cloud_uris(objectname="NGC 3132", obs_collection="JWST", filter_products={"productType": "SCIENCE", "calib_level": [3]})

]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Now that we have the cloud URI for the data, we can load the light curve\n",
"# into the LCviz app\n",
"light_curve = lightkurve.read(lc_uri)\n",
"lcviz.load_data(light_curve)"
Comment on lines +224 to +225
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bmorris3 - what would it take to be able to extend the upstream URI capability to be able to pass lc_uri to load_data directly?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would require one more little block of logic in the jdaviz URI parser. That's work we need to do for RSP soon anyway. In their notebooks, they use this pattern on AWS:

import s3fs
import roman_datamodels as rdm

asdf_dir_uri = 's3://roman-sci-test-data-prod-summer-beta-test/'
fs = s3fs.S3FileSystem()

asdf_file_uri = asdf_dir_uri + 'ROMANISIM/DENSE_REGION/R0.5_DP0.5_PA0/r0000101001001001001_01101_0001_WFI16_cal.asdf'
with fs.open(asdf_file_uri, 'rb') as f:
    file = rdm.open(f)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've tried running the notebook as-is on TIKE and it doesn't work yet, for reasons I don't understand. One challenge that we're likely to face is that lightkurve has tools for initializing a LightCurve from a file path to a FITS file, but not from an open file stream to an HDU list, which is what fsspec is likely to give us.

]
},
{
"cell_type": "markdown",
"metadata": {
"tags": []
},
"source": [
"## Flattening\n",
"\n",
"There are clearly some long term trends in the data, which we would like to remove\n",
"in order to more clearly see the transits. Here we use `Flatten` plugin to remove\n",
"low frequency trends from the light curves. Note that all of the options set here\n",
"are also available in the UI in the `Flatten` plugin, accessible in the plugin\n",
"tray on the right side of the app."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"flatten = lcviz.plugins['Flatten']\n",
"\n",
"# There is currently only one dataset, so we do not\n",
"# need to set flatten.dataset. If there were multiple\n",
"# light curves loaded, you could see the choices to set\n",
"# this with flatten.dataset.choices\n",
"flatten.window_length = 50\n",
"flatten.flatten();"
]
},
{
"cell_type": "markdown",
"metadata": {
"tags": []
},
"source": [
"## Phase Folding\n",
"\n",
"A common operation when dealing with light curves is to phase fold the data, so\n",
"that all of the transits are lined up with each other, essentially displaying all\n",
"of the data over a single orbit of the planet. Here we use the `Ephemeris` plugin\n",
"to phase fold the data using values from the literature for the epoch and period.\n",
"Running this code will create a second flux vs time viewer in the app to display\n",
"the new phase folded light curve."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"scrolled": true,
"tags": []
},
"outputs": [],
"source": [
"# get the origin of the time axis in LCviz:\n",
"time_coordinates = lcviz.app.data_collection[0].coords\n",
"reference_time = time_coordinates.reference_time\n",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this isn't technically public API... can this be pulled out of lcviz.get_data(...).meta instead?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably, I just pulled this from the LCvizExample notebook. Should probably change it there too if we don't want people using this.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I'd vote to do that if possible to avoid publicizing any internal API if there are alternatives

"\n",
"# literature ephemeris for hot Neptune planet HAT-P-11 b:\n",
"morris2017_epoch = Time(2454605.89146, format='jd')\n",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we avoid using Time entirely or is this needed because reference time is defined differently? Or is it good to show how to do this here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again, I just pulled this from the example notebook, which I think @bmorris3 wrote. Probably a question for him.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why would you want to avoid using Time? Time coordinates are dangerously ambiguous otherwise.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

because currently it needs to be passed to lcviz just as a float anyways, so creating a time object just to do to_value seems to add complication to the notebook. Maybe we just link to docs on Time (whether or not we use it), since we don't want this to be a tutorial on that whole topic 🤷‍♂️

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm working through the notebook, and I think as long as the time system is written in the comment, I'm ok with avoiding astropy.time.Time here. The most likely "gotcha" to worry about is that the units of the reference epoch, the period, and the time axis of the light curve must all be in the same units. If you had period in hours, this wouldn't work correctly. For this example, we're fine without specifying.

"morris2017_period = 4.88780258 # days\n",
"\n",
"# phase-fold the transit light curve in an ephemeris viewer:\n",
"eph = lcviz.plugins['Ephemeris']\n",
"eph.period = morris2017_period\n",
"eph.t0 = (\n",
" (morris2017_epoch - reference_time).to_value(time_coordinates.unit) % eph.period\n",
")\n",
"\n",
"# offset the wrapping phase so the transit (at phase 0) displays at center\n",
"eph.wrap_at = 0.5"
rosteen marked this conversation as resolved.
Show resolved Hide resolved
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Additional Resources\n",
"\n",
"- [LCviz documentation](https://lcviz.readthedocs.io/en/stable/index.html)\n",
"- [LCviz Github](https://github.com/spacetelescope/lcviz)\n",
"- [lightkurve documentation](https://lightkurve.github.io/lightkurve/)\n",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure if this link will break in the future when they fix docs.lightkurve.org 🤔

"- [Kepler Archive Page (MAST)](https://archive.stsci.edu/kepler/)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Citations\n",
"\n",
"If you use `astropy`, `lightkurve`, or `Jdaviz` (via `LCviz`) for published research, please cite\n",
"the authors. Follow these links for more information about citing `astropy` and\n",
"`lightkurve`:\n",
"\n",
"* [Citing `astropy`](https://www.astropy.org/acknowledging.html)\n",
"* [Citing `lightkurve`](http://docs.lightkurve.org/about/citing.html)\n",
"\n",
"And cite Jdaviz through its [Zenodo record](https://doi.org/10.5281/zenodo.5513927).\n"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kecnry – should we make a zenodo record for lcviz? (I vote yes!)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, let's plan to do that with 1.0 release. I'll add it to the ticket.

]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## About this Notebook\n",
"\n",
"**Author(s):** Ricky O'Steen, Kyle Conroy, Brett Morris, Thomas Dutkiewicz <br/>\n",
"**Keyword(s):** Tutorial, LCviz, TESS, introduction <br/>\n",
"**First published:** Oct 2024 <br/>\n",
"**Last updated:** Oct 2024 <br/>\n",
"\n",
"***\n",
"[Top of Page](#top)\n",
"<img style=\"float: right;\" src=\"https://raw.githubusercontent.com/spacetelescope/style-guides/main/guides/images/stsci-logo.png\" alt=\"Space Telescope Logo\" width=\"200px\"/> "
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.7"
}
},
"nbformat": 4,
"nbformat_minor": 4
}