Binder badge

What’s new in Lightkurve 2?

Lightkurve 2 offers improved support for TESS

Lightkurve v1 has been a tremendous success. Owing to the efforts of 67 contributors, Lightkurve grew into a popular tool to analyze the data archive of NASA’s Kepler space telescope. The package has been cited by nearly 200 scientific publications to date.

In recent years, the success revealed a shortcoming: Lightkurve 1 was too focused on supporting Kepler data products. Over the past two years, our community has evolved in important ways:

  • NASA successfully launched TESS, adding a significant new data set which requires Lightkurve-like tools.

  • The TESS data archive started hosting numerous community light curve databases created using specialized pipelines.

  • Many new time domain missions are on the horizon, including NASA’s Roman Space Telescope.

In response to these changes, the TESS GI Office at NASA Goddard and the TESS Data Archive at STScI/MAST supported the modification of Lightkurve and its tutorials to better support TESS and other data sets in three important ways:

  1. LightCurve objects are now specialized kinds of AstroPy Table objects, making it a more generic container for light curves from any telescope.

  2. Data search and download functions now support all TESS and Kepler light curve available at MAST, including those created by community pipelines.

  3. The noise removal tools now allow users to perform custom corrections in a way that is very similar to the official TESS pipeline.

The remainder of this page briefly demonstrates these new features. They are explained in more detail in a new set of entry-level tutorials which are avaiable in the online documentation at docs.lightkurve.org/tutorials.

1. LightCurve objects are now specialized kinds of AstroPy Tables

The most important change in Lightkurve 2 is that light curves are now extensions of AstroPy TimeSeries objects, which in turn are a sub-class of AstroPy Table. Compared to a generic table, the key difference is that each LightCurve object is guaranteed to have time, flux, and flux_err columns. This allows Lightkurve to extend AstroPy with tools that are specific to the analysis of TESS-like time series photometry.

✨ New: LightCurve objects behave like tables

Light curves can still be created as before, but now look and act like tables:

[1]:
from lightkurve import LightCurve
lc = LightCurve(time=[2000.0, 2000.1, 2000.2],
                flux=[1.0, 1.0, 1.0],
                flux_err=[0.1, 0.1, 0.1],
                time_format="btjd")
lc
[1]:
LightCurve length=3
timefluxflux_err
objectfloat64float64
2000.01.00.1
2000.11.00.1
2000.21.00.1

✨ New: Light curves support user-defined columns

Because light curves are now tables, they can contain arbitrary user-defined columns:

[2]:
lc['color'] = ['red', 'green', 'blue']
lc
[2]:
LightCurve length=3
timefluxflux_errcolor
objectfloat64float64str5
2000.01.00.1red
2000.11.00.1green
2000.21.00.1blue

✨ New: Columns and rows can be accessed using index notation

Just like Table objects, columns and rows can be accessed using index notation:

[3]:
lc[1:3]
[3]:
LightCurve length=2
timefluxflux_errcolor
objectfloat64float64str5
2000.11.00.1green
2000.21.00.1blue
[4]:
lc['flux']
[4]:
$[1,~1,~1] \; \mathrm{}$

✨ New: Times are stored as an AstroPy Time object

Lightkurve 2 extends AstroPy Time by adding support for the TESS and Kepler time formats (“BTJD” and “BKJD”). As a result, the time column is now always a Time object, which enables user to leverage this standard AstroPy feature:

[5]:
lc.time
[5]:
<Time object: scale='tdb' format='btjd' value=[2000.  2000.1 2000.2]>
[6]:
lc.time.jd   # Easy conversion to Julian Days
[6]:
array([2459000. , 2459000.1, 2459000.2])
[7]:
lc.time.iso  # Easy conversion to ISO time stamps
[7]:
array(['2020-05-30 12:00:00.000', '2020-05-30 14:24:00.000',
       '2020-05-30 16:48:00.000'], dtype='<U23')

✨ New: Read and write light curves in many file formats

Because light curves act like tables, they now have easy read() and write() methods which allow a variety of data formats to be used. For example, it is now much easier to store a light curve to a human-readable text file:

[8]:
lc.write("/tmp/lightcurve.txt", format="ascii.fixed_width", overwrite=True)
!cat /tmp/lightcurve.txt
|   time | flux | flux_err | color |
| 2000.0 |  1.0 |      0.1 |   red |
| 2000.1 |  1.0 |      0.1 | green |
| 2000.2 |  1.0 |      0.1 |  blue |
[9]:
LightCurve.read("/tmp/lightcurve.txt", format="ascii.fixed_width")
[9]:
LightCurve length=3
timefluxflux_errcolor
objectfloat64float64str5
2000.01.00.1red
2000.11.00.1green
2000.21.00.1blue

2. Search & download data from different pipelines

Lightkurve 2 significantly expands the data sets that can be searched and downloaded from the TESS & Kepler data archives at MAST.

✨ New: Search operations support different TESS pipelines

The search operations now support all the Kepler & TESS light curves available from the data archive at MAST, including community-contributed light curves. A new author column in the search results identifies the pipeline (click on a pipeline name for details). As an added bonus, the search and download operations are now much faster owing to improved caching.

[10]:
import lightkurve as lk
lk.search_lightcurve("Polaris")
[10]:
SearchResult containing 6 data products.
#missionyearauthorexptimetarget_namedistance
sarcsec
0TESS Sector 192019SPOC1203032560750.0
1TESS Sector 192019TESS-SPOC18003032560750.0
2TESS Sector 192019QLP18003032560750.0
3TESS Sector 202019QLP18003032560750.0
4TESS Sector 252020QLP18003032560750.0
5TESS Sector 262020QLP18003032560750.0

✨ New: Download TESS fast cadence data

The TESS mission recently introduced a new 20-second exposure time mode and decreased the exposure time of its Full Frame Images to 10 minutes. In response, Lightkurve now allows user to specify the exact exposure time via the optional exptime argument and column.

[11]:
lk.search_lightcurve("AU Mic", exptime=20)
[11]:
SearchResult containing 1 data products.
#missionyearauthorexptimetarget_namedistance
sarcsec
0TESS Sector 272020SPOC204414202360.0

You can also filter search results after the fact via attribute access, e.g. via search.exptime:

[12]:
search = lk.search_lightcurve("AU Mic")
search[search.exptime.value == 20].download().plot();
[12]:
<AxesSubplot:xlabel='Time - 2457000 [BTJD days]', ylabel='Flux [$\\mathrm{e^{-}\\,s^{-1}}$]'>
_images/whats-new-v2_22_1.png

3. New noise removal tools

✨ New: Remove systematics using Cotrending Basis Vectors

Lightkurve 2 adds a new CotrendingBasisVectors class to provide a convenient interface to work with the official TESS and Kepler basis vector data products. We also re-implemented the CBVCorrector class from scratch to perform the correction in a way that is more similar to the official Kepler/TESS pipeline. This work was led by one of the authors of the official pipeline, Jeff Smith.

[13]:
# Download a light curve and apply the new CBVCorrector
from lightkurve.correctors import CBVCorrector
lc = lk.search_lightcurve("TIC 99180739", author="SPOC", sector=10).download(flux_column="sap_flux")
cbvCorrector = CBVCorrector(lc)
lc_corrected = cbvCorrector.correct()
Optimized Over-fitting metric: 0.8490585861948264
Optimized Under-fitting metric: 0.9698928985743609
Optimized Alpha: 3.136e-01

The corrector now produces detailed diagnostic plots showing the light curve before (grey) and after (black) the noise (blue) was removed:

[14]:
cbvCorrector.diagnose();
[14]:
array([<AxesSubplot:ylabel='Flux [$\\mathrm{e^{-}\\,s^{-1}}$]'>,
       <AxesSubplot:title={'center':'Alpha = 3.136e-01'}, xlabel='Time - 2457000 [BTJD days]', ylabel='Flux [$\\mathrm{e^{-}\\,s^{-1}}$]'>],
      dtype=object)
_images/whats-new-v2_27_1.png

✨ New: Remove systematics by optimizing under- and over-fitting metrics

A lightkurve.correctors.metrics module was added to provide functions which can estimate the degree of over- and under-fitting of a corrected light curve. The new CBVCorrector utilizes these metrics to find an appropriate, user-tuneable tradeoff. You can learn more about this new feature in the new CBVCorrector tutorial.

[15]:
cbvCorrector.goodness_metric_scan_plot();
[15]:
<AxesSubplot:title={'center':'Goodness Metrics vs. L2-Norm Penalty (alpha)'}, xlabel='Regularization Factor Alpha', ylabel='Goodness Metric'>
_images/whats-new-v2_29_1.png

✨ New: Inspect pixel-by-pixel light curves using plot_pixels()

A new TargetPixelFile.plot_pixels() method has been added which enables the light curves and periodograms of individual pixels to be inspected within a pixel file. This is very useful for investigating the location of specific signals. A dedicated tutorial explains this new feature in detail.

[16]:
pixelfile = lk.search_targetpixelfile("KIC 2435971", mission="Kepler", quarter=9).download()
pixelfile.plot_pixels();
[16]:
<AxesSubplot:title={'center':'Target ID: 2435971, 808.52 - 905.93 [BKJD days]'}, xlabel='Time / Column (pixel)', ylabel='Flux / Row (pixel)'>
_images/whats-new-v2_31_1.png

Other changes

  • Support for Python 2 has been removed.

  • The online documentation has been updated to provide numerous new tutorials and a more comprehensive API reference guide.

  • Search and download operations are now cached in a more robust way, significantly speeding up accessing data from MAST.

  • Added a column parameter to LightCurve’s plot(), scatter(), and errorbar() methods to enable any column in a light curve object to be plotted.

  • Added the LightCurve.create_transit_mask(period, transit_time, duration) method to conveniently mask planet or eclipsing binary transits.

  • Added the LightCurve.search_neighbors() method to search for light curves around an existing one.

  • Modified TargetPixelFile.plot() to use a more clear hatched style when visualizing the aperture mask on top of pixel data.

  • Added SparseDesignMatrix and modified RegressionCorrector to enable systematics removal methods to benefit from scipy.sparse speed-ups.

In addition, Lightkurve 2 contains numerous smaller improvements and bugfixes. To see a detailed list of all changes, please see the Full Changelog.

FAQ

How do I install Lightkurve 2?

You can upgrade Lightkurve using pip:

python -m pip install --upgrade lightkurve

How do I keep using Lightkurve 1?

You can continue to use Lightkurve 1 by pinning the version number using pip, for example:

python -m pip install lightkurve==1.11

Will existing Lightkurve code break?

While we tried to maximize backwards compatibility, Lightkurve 2 does contain a small number of changes that may break existing code for the following reasons:

  1. lc.time is now always an Astropy Time object. Unfortunately, many NumPy and SciPy functions do not deal seamlessly with Time objects yet. Existing code may have to be modified to pass lc.time.value to NumPy/SciPy functions where lc.time was passed before.

  2. Other numerical columns are now Astropy Quantity objects. While Quantity objects are largely compatible with NumPy/SciPy, situations do arise where users may wish to perform operations on lc.flux.value instead of lc.flux.

  3. All metadata is now stored in the lc.meta dictionary. This includes any header values of the FITS file from which a light curve was loaded. For convenience, we continue to support attribute access, e.g. lc.sector is available as a shortcut for lc.meta['sector'].

  4. LightCurveFile classes have been deprecated. Instead, light curve files can now be loaded using LightCurve.read(filename, format="tess").

  5. lc.time can no longer contain NaN values. This is because AstroPy Time objects do not currently support NaN values. As a result, light curve reading functions will tend to ignore rows with NaN time values. All other columns in a light curve continue to support NaNs values however.

  6. lc.bin() now takes time_bin_size and time_bin_start arguments. This is for consistency with astropy.timeseries.aggregate_downsample. This significantly alters the behavior of binning, because bins are now defined in time instead of by number of data points.

  7. lc.fold() now returns un-normalized phase values by default. This is for consistency with TimeSeries.fold(). For the same reason, the t0 argument has been deprecated in favor of epoch_time, and the method now accepts the extra epoch_phase, wrap_phase, and normalize_phase arguments.

  8. Several attributes and methods have been deprecated. They still work but will yield a warning and may be removed in the future:

  • lc.astropy_time => lc.time

  • lc.time_format => lc.time.format

  • lc.time_scale => lc.time.scale

  • lc.flux_quantity => lc.flux

  • lc.flux_unit => lc.flux.unit

  • search_lightcurvefile() => search_lightcurve()

Contributors

The people who contributed to Lightkurve 2 include:

Thomas Barclay, Geert Barentsen, Keaton Bell, Zachory K. Berta-Thompson, Colin J. Burke, Nuno Ramos Carvalho, Ann Marie Cody, Isabel Coleman, Kaiming Cui, Guy Davies, Jessie Dotson, Stephanie Douglas, Scott Fleming, Daniel Foreman-Mackey, Michael Gully-Santiago, Oliver Hall, Christina Hedges, Daniel Hey, Michael Higgins, Derek Homeier, Rebekah Hounsell, Sam Lee, Jose A. Lerma III, Ken Mighell, Susan Mullally, Szabo Pal, Sheila Sagear, Nicholas Saunders, Jeff Smith, Anand Sundaram, Emma Turtelboom, Andrew Vanderburg, Brennan Vincello, José Vinícius de Miranda Cardoso, Peter Williams, Johnny Zhang.