{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Calculating rest-frame photometric properties\n", "\n", "While SED fitting offers a fantastic avenue to explore galaxy properties, there are often significant assumptions required. For example, the IMF, SFH, metallicity, dust content/geometry, all must be assumed even though we do not have the appropriate photometric data to properly constrain them at this point. This may lead to inaccurate measurements of galaxy properties, and the associated errors are nearly always underestimated.\n", "\n", "As seen in Austin et al 2024 and Cullen et al 2024, UV continuum slope measurements of blue SFGs identified by JWST are biased red by SED fitting, making it more appropriate to measure these directly from the photometric data without the assumptions that are required by the template inputs to Bayesian SED fitting.\n", "\n", "In this notebook, we will demonstrate the available `Rest_Frame_Property_Calculator` classes that are available in galfind. These calculate galaxy properties in the rest-frame UV and optical directly from the photometric data and can be applied on either a `Catalogue`, `Galaxy`, or `Photometry_rest` level.\n", "\n", "Of course, and as always, we start by instantiating our JOF NIRCam v11 `Catalogue` object. This time we only need to load in the SED results of interest; we use the redshift free EAZY run as an example." ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Reading GALFIND config file from: /nvme/scratch/work/austind/GALFIND/galfind/../configs/galfind_config.ini\n", "\"Important: Gaia archive will be intermittently unavailable due to scheduled maintenance on 10-12-2024 from 08:00 to 10:00 (CET)\"\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "WARNING:galfind:Aperture corrections for VISTA not found in /nvme/scratch/work/austind/GALFIND/galfind/Aperture_corrections/VISTA_aper_corr.txt\n", "WARNING:galfind:Aperture corrections for VISTA not found in /nvme/scratch/work/austind/GALFIND/galfind/Aperture_corrections/VISTA_aper_corr.txt\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Failed to `import dust_attenuation`\n", "Install from the repo with $ pip install git+https://github.com/karllark/dust_attenuation.git\n" ] } ], "source": [ "import astropy.units as u\n", "from copy import deepcopy\n", "from galfind import Catalogue, EAZY\n", "from galfind.Data import morgan_version_to_dir" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "INFO:galfind:Loaded aper_diams= for F277W+F356W+F444W\n", "INFO:galfind:Combined mask for NIRCam/F277W+F356W+F444W already exists at /raid/scratch/work/austind/GALFIND_WORK/Masks/JOF/combined/JOF_F277W+F356W+F444W_auto.fits\n", "WARNING: hdu= was not specified but multiple tables are present, reading in first available table (hdu=1) [astropy.io.fits.connect]\n", "WARNING:astroquery:hdu= was not specified but multiple tables are present, reading in first available table (hdu=1)\n", "WARNING:galfind:Aperture correction columns already in /raid/scratch/work/austind/GALFIND_WORK/Catalogues/v11/NIRCam/JOF/(0.32)as/JOF_MASTER_Sel-F277W+F356W+F444W_v11.fits\n", "Calculating depths: 0%| | 0/15 [00:00\n", "WARNING:galfind:cat_aper_diams not in kwargs.keys()=dict_keys(['ZP', 'min_flux_pc_err'])! Setting to aper_diams=\n", "WARNING:galfind:cat_aper_diams not in kwargs.keys()=dict_keys([])! Setting to aper_diams=\n", "INFO:galfind:Made /raid/scratch/work/austind/GALFIND_WORK/Catalogues/v11/NIRCam/JOF/(0.32)as/JOF_MASTER_Sel-F277W+F356W+F444W_v11.fits catalogue!\n", "INFO:galfind:Making .in file for EAZY_fsps_larson_zfree SED fitting for JOF v11 NIRCam\n", "INFO:galfind:Made .in file for EAZY_fsps_larson_zfree SED fitting for JOF v11 NIRCam. \n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Running SED fitting took 0.1s\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:galfind:Loading EAZY_fsps_larson property PDFs into JOF v11 NIRCam\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Loading properties and associated errors took 1.4s\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "Constructing redshift PDFs: 100%|██████████| 16335/16335 [00:00<00:00, 30159.02it/s]\n", "INFO:galfind:Finished loading EAZY_fsps_larson property PDFs into JOF v11 NIRCam\n", "INFO:galfind:Loading EAZY_fsps_larson SEDs into JOF v11 NIRCam\n", "Constructing SEDs: 100%|██████████| 16335/16335 [00:01<00:00, 13781.28it/s]\n", "INFO:galfind:Finished loading EAZY_fsps_larson SEDs into JOF v11 NIRCam\n", "INFO:galfind:Updating SED results in galfind catalogue object\n", "Updating galaxy SED results: 100%|██████████| 16335/16335 [00:00<00:00, 155779.74it/s]\n" ] } ], "source": [ "survey = \"JOF\"\n", "version = \"v11\"\n", "instrument_names = [\"NIRCam\"]\n", "aper_diams = [0.32] * u.arcsec\n", "forced_phot_band = [\"F277W\", \"F356W\", \"F444W\"]\n", "min_flux_pc_err = 10.\n", "\n", "JOF_cat = Catalogue.pipeline(\n", " survey,\n", " version,\n", " instrument_names = instrument_names, \n", " version_to_dir_dict = morgan_version_to_dir,\n", " aper_diams = aper_diams,\n", " forced_phot_band = forced_phot_band,\n", " min_flux_pc_err = min_flux_pc_err\n", ")\n", "\n", "SED_fit_params_arr = [{\"templates\": \"fsps_larson\", \"lowz_zmax\": None}]\n", "for SED_fit_params in SED_fit_params_arr:\n", " EAZY_fitter = EAZY(SED_fit_params)\n", " EAZY_fitter(JOF_cat, aper_diams[0], load_PDFs = True, load_SEDs = True, update = True)\n", "SED_fit_label = EAZY_fitter.label" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Example 1: Photometry_rest, Galaxy, and Catalogue level calculations\n", "\n", "In this example, we will work through the calculation of these properties excluding any associated errors. The main advantage of this is that it will quickly produce results for large surveys or simulations to, for example, determine sample completeness/contamination. Firstly, we need an example `Rest_Frame_Property_Calculator` object; we will create an `UV_Beta_Calculator` object in this instance, although in principle any of the other `Rest_Frame_Property_Calculator` child classes will do the trick. The `UV_Beta_Calculator` class is a subclass of `Rest_Frame_Property_Calculator` and, as with all `Rest_Frame_Property_Calculator` objects, must be instantiated with an explicit `aper_diam` and `SED_fit_label`. The `rest_UV_wav_lims` given here are the default values." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "from galfind import UV_Beta_Calculator\n", "beta_calculator = UV_Beta_Calculator(\n", " aper_diam = aper_diams[0],\n", " SED_fit_label = SED_fit_label,\n", " rest_UV_wav_lims = [1_250., 3_000.] * u.AA\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Calling this object while inserting either a `Photometry_rest`, `Galaxy`, or `Catalogue` as the first argument will perform the calculation. Let's first create a `Photometry_rest` object corresponding to the z=14.63 photometric candidate from Robertson et al. 2023." ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "****************************************\n", "PHOTOMETRY_REST: z = 14.662217140197754\n", "----------\n", "****************************************\n", "\n" ] } ], "source": [ "phot_rest_z14 = deepcopy(JOF_cat[717].aper_phot[aper_diams[0]].SED_results[SED_fit_label].phot_rest)\n", "print(phot_rest_z14)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In this example, we will run with `n_chains=1` (i.e. without the Monte Carlo). In the codeblock below this `n_chains` parameter is the only argument that is changed from the default value, which is 10,000." ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "beta_calculator(\n", " phot_rest_z14,\n", " n_chains = 1, \n", " output = False,\n", " overwrite = False,\n", " n_jobs = 1\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Using the overloaded __str__ operator, we can see the impact on the properties stored in the `phot_rest_z14` object." ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "****************************************\n", "PHOTOMETRY_REST: z = 14.662217140197754\n", "----------\n", "****************************************\n", "\n", "{'beta_[1250,3000]AA': }\n" ] } ], "source": [ "print(phot_rest_z14)\n", "print(phot_rest_z14.properties)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now let's pass the entire galaxy through and see what happens." ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "****************************************\n", "Galaxy(718, [53.10763,-27.86013]deg)\n", "****************************************\n", "PHOTOMETRY:\n", "----------\n", "Photometry_obs(NIRCam, 0.32 arcsec, EAZY_fsps_larson_zfree)\n", "----------\n", "SELECTION FLAGS:\n", "----------\n", "bluewards_Lya_SNR<2.0_EAZY_fsps_larson_zfree_0.32as: True\n", "redwards_Lya_SNR>5.0,5.0_widebands_EAZY_fsps_larson_zfree_0.32as: True\n", "ALL_redwards_Lya_SNR>2.0_EAZY_fsps_larson_zfree_0.32as: True\n", "red_chi_sq<3.0_EAZY_fsps_larson_zfree_0.32as: True\n", "chi_sq_diff>4.0,dz>0.5_EAZY_fsps_larson_zfree_0.32as: True\n", "zPDF>60%,|dz|/z<0.1_EAZY_fsps_larson_zfree_0.32as: True\n", "unmasked_F090W: True\n", "unmasked_F115W: True\n", "unmasked_F150W: True\n", "unmasked_F162M: True\n", "unmasked_F182M: True\n", "unmasked_F200W: True\n", "unmasked_F210M: True\n", "unmasked_F250M: True\n", "unmasked_F277W: True\n", "unmasked_F300M: True\n", "unmasked_F335M: True\n", "unmasked_F356W: True\n", "unmasked_F410M: True\n", "unmasked_F444W: True\n", "unmasked_NIRCam: True\n", "bluest_band_SNR<2.0_0.32as: True\n", "sex_Re_F277W>45.0mas: True\n", "sex_Re_F356W>45.0mas: True\n", "sex_Re_F444W>45.0mas: True\n", "sex_Re_F277W+F356W+F444W>45.0mas: True\n", "EPOCHS_NIRCam_EAZY_fsps_larson_zfree_0.32as: True\n", "EPOCHS_lowz_NIRCam_EAZY_fsps_larson_zfree_0.32as: True\n", "----------\n", "****************************************\n", "\n", "****************************************\n", "Galaxy(718, [53.10763,-27.86013]deg)\n", "****************************************\n", "PHOTOMETRY:\n", "----------\n", "Photometry_obs(NIRCam, 0.32 arcsec, EAZY_fsps_larson_zfree)\n", "----------\n", "SELECTION FLAGS:\n", "----------\n", "bluewards_Lya_SNR<2.0_EAZY_fsps_larson_zfree_0.32as: True\n", "redwards_Lya_SNR>5.0,5.0_widebands_EAZY_fsps_larson_zfree_0.32as: True\n", "ALL_redwards_Lya_SNR>2.0_EAZY_fsps_larson_zfree_0.32as: True\n", "red_chi_sq<3.0_EAZY_fsps_larson_zfree_0.32as: True\n", "chi_sq_diff>4.0,dz>0.5_EAZY_fsps_larson_zfree_0.32as: True\n", "zPDF>60%,|dz|/z<0.1_EAZY_fsps_larson_zfree_0.32as: True\n", "unmasked_F090W: True\n", "unmasked_F115W: True\n", "unmasked_F150W: True\n", "unmasked_F162M: True\n", "unmasked_F182M: True\n", "unmasked_F200W: True\n", "unmasked_F210M: True\n", "unmasked_F250M: True\n", "unmasked_F277W: True\n", "unmasked_F300M: True\n", "unmasked_F335M: True\n", "unmasked_F356W: True\n", "unmasked_F410M: True\n", "unmasked_F444W: True\n", "unmasked_NIRCam: True\n", "bluest_band_SNR<2.0_0.32as: True\n", "sex_Re_F277W>45.0mas: True\n", "sex_Re_F356W>45.0mas: True\n", "sex_Re_F444W>45.0mas: True\n", "sex_Re_F277W+F356W+F444W>45.0mas: True\n", "EPOCHS_NIRCam_EAZY_fsps_larson_zfree_0.32as: True\n", "EPOCHS_lowz_NIRCam_EAZY_fsps_larson_zfree_0.32as: True\n", "----------\n", "****************************************\n", "\n" ] } ], "source": [ "gal_z14 = deepcopy(JOF_cat[717])\n", "print(gal_z14)\n", "beta_calculator(\n", " gal_z14,\n", " n_chains = 1\n", ")\n", "print(gal_z14)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Of course we can loop through these galaxies to update the catalogue, or we can simply pass the entire catalogue in to calculate these properties." ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "WARNING: hdu= was not specified but multiple tables are present, reading in first available table (hdu=1) [astropy.io.fits.connect]\n", "WARNING:astroquery:hdu= was not specified but multiple tables are present, reading in first available table (hdu=1)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "****************************************\n", "CATALOGUE(JOF,v11,NIRCam):\n", "----------\n", "CAT PATH = /raid/scratch/work/austind/GALFIND_WORK/Catalogues/v11/NIRCam/JOF/(0.32)as/JOF_MASTER_Sel-F277W+F356W+F444W_v11.fits\n", "TOTAL GALAXIES = 16335\n", "RA RANGE = [53.01070689 53.11059594] deg\n", "DEC RANGE = [-27.91226173 -27.83206063] deg\n", "----------\n", "****************************************\n", "MULTIPLE_FILTER\n", "----------\n", "FACILITY: JWST\n", "INSTRUMENT: NIRCam\n", "FILTERS: ['F090W', 'F115W', 'F150W', 'F162M', 'F182M', 'F200W', 'F210M', 'F250M', 'F277W', 'F300M', 'F335M', 'F356W', 'F410M', 'F444W']\n", "****************************************\n", "\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "Calculating beta_[1250,3000]AA: 100%|██████████| 16335/16335 [00:31<00:00, 510.85it/s]\n", "WARNING: hdu= was not specified but multiple tables are present, reading in first available table (hdu=1) [astropy.io.fits.connect]\n", "WARNING:astroquery:hdu= was not specified but multiple tables are present, reading in first available table (hdu=1)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "****************************************\n", "CATALOGUE(JOF,v11,NIRCam):\n", "----------\n", "CAT PATH = /raid/scratch/work/austind/GALFIND_WORK/Catalogues/v11/NIRCam/JOF/(0.32)as/JOF_MASTER_Sel-F277W+F356W+F444W_v11.fits\n", "TOTAL GALAXIES = 16335\n", "RA RANGE = [53.01070689 53.11059594] deg\n", "DEC RANGE = [-27.91226173 -27.83206063] deg\n", "----------\n", "****************************************\n", "MULTIPLE_FILTER\n", "----------\n", "FACILITY: JWST\n", "INSTRUMENT: NIRCam\n", "FILTERS: ['F090W', 'F115W', 'F150W', 'F162M', 'F182M', 'F200W', 'F210M', 'F250M', 'F277W', 'F300M', 'F335M', 'F356W', 'F410M', 'F444W']\n", "****************************************\n", "\n" ] } ], "source": [ "print(JOF_cat)\n", "JOF_cat_copy = deepcopy(JOF_cat)\n", "beta_calculator(\n", " JOF_cat_copy,\n", " n_chains = 1\n", ")\n", "print(JOF_cat_copy)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Fantastic! We have now computed the UV beta slopes for every galaxy in our JOF catalogue. Some, of course, will fail and produce NaN's, namely when there are fewer than 2 photometric filters entirely within the rest frame UV wavelength range used. This will mean that depending on the filterset used, some redshift ranges will have more/less precise UV continuum slope measurements, and in some ranges it will be impossible to calculate this quantity." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Example 2: Running Monte Carlo to produce PDFs\n", "\n", "Instead of a single chain, as done in the previous example, we will now compute the UV continuum slope using a Monte Carlo technique to incorporate the photometric errors. We scatter each photometric data point about its error profile, which is assumed to be Gaussian, before re-computing the beta slope for each scattered photometry. This creates an array of beta slopes from which we extract the median, and upper and lower 1σ errors.\n", "\n", "In the below codeblock, we calculate this Monte Carlo using 10,000 chains, which is the default." ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "Calculating beta_[1250,3000]AA: 0%| | 0/16335 [00:00